Some thoughts on the effectiveness of the Fraunhofer Society
The Fraunhofer Society is a german research organization that focuses on applying science to develop new technologies. With over 28000 employees and over three billion dollar annual budget the Fraunhofer Society is the biggest research organization in Europe. Therefore, I find it surprising that very little is written about the organization from independent sources. Having worked at Fraunhofer for over one year, I want to share some thoughts and insights on the Fraunhofer organization.
The Fraunhofer Society was founded in 1949 as an attempt to accelerate the growth of West Germany. Today, the Fraunhofer Society says on its website: With its focus on developing key technologies that are vital for the future and enabling the commercial exploitation of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future [1] . In this sentence no goal of the Society is mentioned. That is because the goal of the society has never been postulated clearly, or even vaguely. The Fraunhofer Society is a non-profit that is uniquely funded receiving a public basic funding of thirty percent of its budget (about one billion dollar annually). Seventy percent of the funding comes from contract research for industry and public sector (more details on that later). The Fraunhofer Society receives only negligible amounts of donations from private donors. Despite that fact, the Fraunhofer Society is a non-profit with a budget that (at least in part) comes
straight from all tax-payers. This budget can be increased or decreased. As a result, the Fraunhofer Society is under pressure to work effectively. If it is no longer a organization that german citizen find worth investing in, it will cease to exist.
This obviously raises the question: How effective is the Fraunhofer Society ? Surprisingly (or maybe not), you will find no answer googling this question. Having worked at Fraunhofer for a bit now I have gained the following insights:
1. The Fraunhofer Society has a very ill-faded incentive structure
This begins with the basic funding of the Society. The Society states, that the basic funding is in place so that research projects can be undertaken whose payoffs lie too far in the future to be of interest for private companies [2]. Although this particular explanation does not make a lot of sense, it is true that the basic financing gives absolute freedom to pursue research that is for whatever reason (market failures etc.) not economical for companies, but could be extremely valuable. However, an unrestricted basic funding also allows for interventions that are not within this space. Furthermore, since the Fraunhofer Society also does contract research for the industry it provides an unfair comparative cost advantage. If you were to provide contract research privately you would have to beat the Fraunhofer Society by far more than 30% in efficiency since you have to pay your competitor. Surprisingly (maybe not), you probably will be able to do that. At Fraunhofer I learned that, despite their advantage, rather rarely teams have the performance and efficiency to challenge private sector companies for contract research. Almost all of the contract research is done for public projects, often in joint-ventures with companies. That way, most of the funding comes from public sources.
Although the funding structure of the Fraunhofer Society is troubling in and of itself, it actually still allows for the Fraunhofer Society kicking ass and being an extremely effective research organization. Now here is why it is not: In order to undertake effective interventions the Society has to be either forced and restricted to effective projects or has to have some built-in form of goal-oriented behavior that incentivizes effectiveness. Since there is not even a goal, the latter is not the case. There is some effort by the public sector to only fund projects that are effective. As stated before, besides the basic funding most of the funding comes from public projects. The public sector in Germany always invites for tenders in a broad range of areas or causes. The institutes (and other companies, NGOs etc.) provide project descriptions of the projects that they offer to carry out to improve on those causes. How the decision which projects get funded and which do not is made is very intransparent. The thing is, the project choice is very, very poor. I see this as the most fundamental system flaw. If there were an intact system that would prohibit the worst projects from being realized, that would be huge.
I will try to quickly give an idea about how bad the situation is. I have worked only on a handful of projects so my sample size is limited to say the least. One project that is supposed to mitigate the “plastic problem” relies on inventing a thermoplastic material that can be mechanically repurposed infinitely often. Not having a particular background in chemistry, I know that this is almost certainly physically impossible, not speaking of whether this intervention would be effective if it were. The project budget for this one is around 3 million dollars. An other project focused on developing a new technology that was supposed to be used to safeguard products. After about 2 million dollars and 3 years of development, there is no single product known that can benefit from this protection. The value of these interventions is downright zero. These were fully publicly funded projects. That is around million dollars flushed. I did not cherry-pick these projects. I have spent most of my time at Fraunhofer with these two projects. They make up most of my sample size. It gets worse. I have read a couple of project descriptions. The project descriptions are really terrible. A couple of distinct features are:
There is never any total metric of impact (like this intervention raises y by x). Instead the benefits of the intervention are described very buzzwordy and non-quantitative (like this intervention helps transition to industry 4.0, protect workplaces in the region b and improves the digitalization of x)
The project descriptions are full of logical errors and unproven statements, most often when describing the benefits of the intervention (like stating that a intervention improves x, but there is no obvious or stated correlation or causation). For example, it was stated that a technology that was developed would basically end counterfeiting of pharmaceuticals, without giving a hint why this assumption has been made. It took me a couple hours of internet research to find out that there is no way of even applying the technology to most pharmaceuticals. This should have come up during the decision making.
Deception and deliberately lying about the teams capabilities and equipment is normal. I have been in a team that manipulated their measurement results to exhibit a correlation (that they knew was not there) in order to land a project, stating that they were able to reproduce and apply the effect. If there are quantitative estimates in project descriptions they rely on physical boundaries (like this antenna will be able to broadcast 10 meters far). In the end, results from demonstrators often lie orders of magnitude below the estimates made in the project description.
Of course, it is not that the employees at Fraunhofer want to do harmful things. Many are cognitively dissonant, actually thinking that they do tremendous good. But many are aware of the problematic situation they are in. The dilemma is: Not having any goal-oriented incentive system, the Fraunhofer Society is dominated by the personal incentive of its members: Job security. At Fraunhofer people do have a little less job security than the average worker. If your team has not enough funding you will find yourself working elsewhere within months. Therefore many groups and institutes are desperate for funding and just send in a lot of trashy project offers. I have the feeling that given enough project offers, someone is going to decide to fund a project, which is almost the definition of non-scientific.
2. Fraunhofer has the image of being a place where the brightest minds work to achieve innovations that the average person can not conceive of
This is the feeling that I have when I speak about Fraunhofer with friends, family and strangers. You can watch this german image movie to get a sense of what I am referring to. I think the creation of this image is done to help justify the huge public expenses of Fraunhofer. People are supposed to think Fraunhofer is a place where smart scientists in lab coats shoot with quantum-lasers at mutated plants to retrieve a pharmaceutical that heals all cancer. Fraunhofer is not the place where the brightest minds work. To make this a quick one, a couple of citations from three different senior scientists and team or project leaders:
“I have contacted the colleagues from the Institute B. I just want another Institute on the title page of our project description to give the project more weight (importance)”
“If you are not a scientist, than you do not have to work scientifically”
When I told a senior scientist about CoolEarth, she replied:
“When it comes to climate change, we have to stop thinking in numbers”
When I asked her why, she said : “Because you can´t just throw a couple of dollars at the ground and ask mother nature to do it one more year”
(This has not been previously suggested by anyone)
To end on a more positive note: Almost all of my colleagues are nice and pleasant people with whom I would really like to work on something else. Especially as a student or PhD student Fraunhofer is worth considering as a investment in your career capital. You can very easily get into a decently paid position at Fraunhofer where you are learning a lot quickly because the set of tasks is often very diverse. Working at Fraunhofer can be very fun and pleasant if you are actually working on something meaningful or when you are distanced from the economic reality of the Society and unsuspecting. Sorry about that one. I feel as if the Fraunhofer Society was (similar to the catholic church) founded with good intentions but the underengineered structure took on a life of its own and has deviated from the intended path.
-------------------------------------------------------------------
Concluding, the effectivity of the Fraunhofer Society is likely much lower than competing alternatives. The main problem seems to be that public funding is not provided based on evidence.
My guess is that great improvement in this area comes with considerable effort since multiple parts of the system are broken and all have to be fixed to mitigate the problem. However, I can think of some “quick fixes” that might be worth investing in, yielding greater result per unit of effort. One possible measure would be to just demand a quantitative assessment of impact in the project descriptions. This would force a calculation which I think is easier to detect as flawed and allows for easier cross-project comparison by grant-makers. But I am not very sure about that. An additional measure could be restricting the project funding or future funding to actual results (in essence to the track record of the organization, ideally using a gittins index backoff). This would create incentive to estimate realistic project outcomes.
------------------------------EDIT_Nov20---------------------------
After reading “Inadequate Equilibria” by Eliezer Yudkowsky, I come to the conclusion that the Fraunhofer-Public-Funding-System is in a state of inadequacy.
1. Lack of sensible incentive-structures
Decision makers do not benefit from better judgment. Nobody gets fired when the money is allocated to the wrong project OR when the most important project does not get funded.
2. Asymmetrical information
The Institutes hold way more information about the potential project than grant-makers. That way they can easily create a false impression (e.g. estimated likelihood of project success) .
3. Nash-equilibria that is not the best possible outcome
Everybody does what is best for them. Employees at Fraunhofer secure their job by securing funds, lying about their capabilities and the expected impact of the project if necessary. Decision-makers in charge of allocating funds do not look closely for the most efficient projects since this would mean investing a lot of time and thought without extra pay.
Kudos for the brutal honesty.
Great post! Do either you or other commentators here have a sense of how Fraunhofer compares to publicly funded research groups in other countries, like NASA, NIH, etc?
Also, have there been unusually strong success stories from other groups in that reference class?
I do not know much about any other publicly funded research group. It could be possible that only few projects have an impact and that that impact is tremendous (unusually strong success). Whilst that would justify continuing the funding it clearly would not justify sticking with the current system since improvements can be made. I highly encourage further analysis of their operations since I would not be surprised to see similar issues. If someone is interested, do not hesitate to contact me.
I regard Australia’s Commonwealth Scientific and Industrial Research Organisation (CSIRO) as having been quite successful. From Wikipedia:
And the items listed in the Innovation section. Still, I’m sure they have had (at least) a few research projects that didn’t go anywhere.
This was a really interesting article on a subject I’d never heard of before, thanks very much. I assume similar issues affect government research organisations in other countries as well.
It seems from your description that part of the problem is that the same body invents projects for itself to work on. Do you think things would be significantly improved if, after coming up with a research project, they had to invite external bids for the project, and only do it in-house if they won the tendering process? Perhaps this would be prohibitively hard to implement in practice.
Sorry if this is misleading in the text. Publicly funded projects are open for bids from almost everyone. Private Companies and other non-profit organizations do do public projects. It is just that not always the best project candidate wins because decisions are not rational. Organizations like Fraunhofer feast on that. There are huge resources devoted to spamming grantmakers with project proposals.
It would be an interesting case study on organisational effectiveness to compare the Fraunhofer Society to the Max Planck Society. Although they focus on different stages of research (applied innovation vs. basic science) they both German non-profit research organizations and relatively similar in size (quick google on MPS gives around 24 thousand staff and $2.1 billion budget for 2018). Yet MPS is a world-renowned research organization and its researchers have been awarded numerous Nobel prizes. I’m not sure if MPS has specific goals, but nonetheless, it seems to be achieving much more impact than Fraunhofer. Some of this difference is probably just in appearances as basic research tends to get more recognition and publicity than applied work, but it still seems like MPS is systematically doing better. Why is that?
---
This is the same general trend I observed amongst a lot of University researchers, but it sounds like it’s progressed much further where you work. Careerism seems to kill the integrity of researchers.
---
This reminded me of The value of a life from the Minding Our Way sequence.
A comparison to the Max Planck Society in regards to effectiveness would be very interesting indeed. Especially since the Max Planck Society is almost fully funded through unrestricted basic funding.
I was quite surprised to hear how large the Fraunhofer Society is given I’ve never heard of it before! I think in and of itself this is a kind of evidence against their effectiveness, although I could also imagine they’ve turned out some winning innovations as parts of contracts and so their involvement gets lost because I think of it as a thing that company X did.
Like NunoSempere, I appreciate the brutal honesty. It’s good and refreshing to see someone recognize that the lies in the thing that a) their society views as high-status and good and b) they personally have a vested interest in believing is really good.
I think this is an important virtue in EA, and we should applaud it in most situations where we see it.
I think your review for post of the decade is very generous.
Like your review now, the other original comments are credulous and warm. This makes perfect sense and should be the default when welcoming someone new who is sharing a risky and confrontational story.
But a more prosaic explanation is that they are a frustrated person who hates the org they are in.
What makes me skeptical of the poster is the gears-level content in the OP tries to express in writing.
This is describing basic research, and there is no reason to be skeptical of this, as the OP is.
It’s true that this can be dysfunctional but the OP provides no evidence.
It’s unclear what the poster means. People outsource to research firms all the time.
This seems ungenerous. Moonshots are a thing.
It is possible that this was ill advised, but we have to take the OP’s word for it, again.
But credentials and references are a thing. Also, scientists don’t just add co-authorship costlessly, there’s a process of vouching or gatekeeping that implies trust. This is a real signal.
Without more context, it’s unclear why the OP believes this is a problem.
So the poster doesn’t like being warned against a myopic focus on measurable results and being promoted some potent but illegible alternative. Doesn’t this sound a little familiar ?
Like you, I rounded all of these in favor of the OP because the poster is not an English speaker. I’ve never had to elaborate in a foreign language how an institution was really a shell of rent seeking hacks coasting on warm feelings and technobabble. But whatever the truth is, it probably shouldn’t be going into the same queue with the post of the decade.
You say:
I thought you said you worked in tech?
I like your critique! I must admit that I was very unhappy with the org but already left it at the time when I wrote this post.
However, I want to push back on some of your points. The Fraunhofer Society indeed conducts somewhat basic research, although the results are much more predictable than in the case of say foundational physics research. There is absolutely nothing wrong with this and I am not sceptical of it. In the post I wanted to point out that the organisation has an unhealthy relationship with grantmakers. This leads to a situation where there is no incentive for the institution to actually provide any valuable research results. I have a better understanding of game theory now and would improve upon the post, if that would be considered helpful.
I think this would be really valuable! Linch really liked your post and note that he has good judgement and is well respected.
I’m no one important, and I could be the only person who has this opinion:
I find that if I didn’t agree with someone’s judgement on a deep issue like institutions, when they add applied math (graph theory, control theory, bayesian statistics, game theory, etc.) this often won’t change my opinion. I could be wrong about this, and I would like to learn if I am. I just like to state this up front.
I think you should post a new post, or write the changes as a comment.
Almost no one will notice an old post being changed.
Ah yeah you’re right I was probably being overly credulous.
That said, this is a supposedly large basic research group that I’ve never heard of before, which I feel like is a bit of evidence against them actually being really impressive?
Their “notable projects” section also feels a bit underwhelming to me, given that they have a comparable research budget to a large American research university.
Could you please explain that further? Looking at this document, page 13, it says that almost 50% of the proceeds from contract research is from economic sources (“Wirtschaftserträge”), and only 41 percent of the contract research money comes from public sources (“EU” and “Bund/Länder”). If my reading is correct, then it would be misleading to say that “almost all” of the research is done for public projects. Or does the category “Wirtschaftserträge” also contain public projects somehow?
https://www.bundestag.de/resource/blob/365560/21b0bb5898655935c70791e438c95e2c/Stellungnahme_FG-data.pdf
Yes, about 30-35% of the total funding comes from private companies. However, these are often not direct research contracts (Company pays Fraunhofer to develop x) but publicly funded projects in which Fraunhofer works together with private companies (notably KMUinnovativ for example). But this assumption is based on my observation in different teams/institutes. I could be wrong.