Should We Limit the Number of Grants Per Researcher?
The NIH recently released a proposal to limit the number of grants per researcher at some level—perhaps 4, 3, or even just 2 grants. The rationale:
As stewards of the taxpayers’ investments in biomedical research and in support of the NIH’s Unified Funding Strategy, the NIH must ensure that it uses funding approaches that maximize scientific productivity and innovation. One way to do this is through policies that allow the NIH to support a greater number of investigators, allowing more ideas to be explored and eventually leading to more breakthroughs (see, for example, 1). Supporting this proposal, a number of studies have observed diminishing marginal returns as grant funding for individuals increases (e.g., 2, 3, 4, 5, 6). Studies have also indicated that larger and more complex research teams tend to produce less innovative and transformational work than smaller teams and are associated with worse career outcomes for the junior researchers in them (7, 8, 9, 10). All of these effects could be due, at least in part, to PIs being unable to effectively oversee and manage research teams once they become too large and complex (11). . . .
Limiting the number of [grants] per PI would have many benefits for U.S. science, including:
Broadening the distribution of funding among geographic regions and institutions to ensure different health needs can be addressed and more talent can be brought into research careers. Broadening the distribution of funding among geographic regions and institutions to ensure different health needs can be addressed and more talent can be brought into research careers.
Supporting more PIs and institutions, which would allow more ideas to be tested.
Strengthening the biomedical workforce by increasing support for early-stage investigators and retaining talented mid-career researchers.
Improving oversight of research projects, which would strengthen mentoring of trainees and increase rigor and reproducibility.
We agree with all of these aims. But it isn’t clear that a limitation like this would be the best way to accomplish them. There might be a better way.
The Evidence
Let’s start by looking more closely at the cited evidence. It is true that several studies suggest there are “diminishing marginal returns as grant funding for individuals increases.” But the cited studies have a couple of problems:
they amount to correlation not causation, and
they measure outcomes like number of papers or number of citations (or similar metrics like Relative Citation Ratio), which are a highly imperfect measure of things we might actually care about.
It would be nice to have an actual experiment here. At the scale of NIH, how about an experiment in which we randomly assign proposals into one of two tracks, one in which there is a thumb on the scale for new investigators or those with only 1 grant, and one in which there is not? Then over time we could see if spreading the money more widely actually led to different outcomes.
After all, the goal of NIH is not to maximize the number of papers or citations. Those are only (very rough) proxies for the things that matter, such as the number of genuine discoveries or the amount of actual progress against cancer and Alzheimer’s.
When we make “number of papers/citations” the metric, just because those are the easiest things to quantify, we are arguably distorting all of science. See Goodhart’s Law.
Next, NIH’s notice says that “larger and more complex research teams tend to produce less innovative and transformational work than smaller teams and are associated with worse career outcomes for the junior researchers in them.”
The first paper cited there isn’t about team size per se (let alone the number of NIH grants per investigator), but about the “hierarchy” of the authors on a given paper as suggested by the use of words like “conceive” or “design” versus words like “assisted.” They do briefly say that larger teams can be more hierarchical, but that doesn’t seem like the most powerful reason to sharply limit the number of grants per scientist.
The second paper cited did find that “larger teams generate an unintended side effect: individuals who finish their PhD when the average team in their field is larger have worse career prospects.” But the reason isn’t that larger teams are bad per se. Instead, they point to the fact that “academic science has not adjusted its reward structure, which is largely individual, in response to team science.”
In other words, larger teams may be doing great work in an age where we need many specialized skills for some scientific inquiries, but academia is behind the times, and only rewards people who were “lead authors” and the like.
Academia’s failure to reward all of the members of large teams who contributed to great science isn’t a reason to stop funding those teams! To the contrary, maybe funders like NIH should use their substantial power to nudge academia towards better norms and reward systems!
The other two papers (here and here) do find that smaller teams are, on average, more “disruptive” or innovative than larger teams (which tend to be more incremental). Even taking those findings at face value, that isn’t a reason to impose a blanket national rule on all scientific teams. When we think about the many tens of billions of dollars spent on biomedical research each year, the last thing we need is one uniform rule for how everyone has to work.
We need more breakthrough innovations, yes, but we also need lots of incremental work to help those breakthroughs turn into reality. Alexander Fleming’s discovery of penicillin was a huge breakthrough, but it took years of work by Howard Florey and others to carry out clinical trials and then figure out how to manufacture it at scale. Without the latter, the discovery of penicillin would have been an academic curiosity.
The point is, even if the balance needs to shift towards smaller teams, that still doesn’t mean that everything funded by NIH needs to look the same.
Funding Younger Researchers
We do think it would be good to empower younger people in science. The most creative scientific years can often occur then:
James Watson was 25 when he co-published the famous paper on the structure of DNA.
Albert Einstein was 25-26 when he had his most productive year in 1905 (the “annus mirabilis,” or miracle year, when he published papers on the photoelectric effect, Brownian motion, special relativity, and E=mc2).
Marshall Nirenberg won the Nobel Prize in 1968 (at age 41) for work he had done by age 34 or so.
Lawrence Bragg co-won a Nobel Prize at age 25 for working to develop X-ray crystallography.
Isaac Newton started developing calculus in his early 20s, and later wrote in a letter that “in those days, I was in the prime of my age for invention and minded Mathematics and Philosophy, more than at any time since.”
Robert Oppenheimer was the ripe old age of 38 when he was chosen to lead the Los Alamos Laboratory for the Manhattan Project.
Marie Curie was 36 when she won a Nobel Prize for discovering radium at the age of 31.
To be sure, many older scientists have done groundbreaking work as well. Here’s a chart that Ben Jones made about the age at which Nobel Prize winners and great inventors made their biggest discoveries:
The problem is that our current scientific system puts the people on the left side of that graph (i.e., under age 40) at a disadvantage.
The NIH is well aware of the age problem, and has been making modest efforts to steer grants towards what they call “early career investigators” (i.e., people who are within 10 years of their PhD).
But the solutions to date have barely made a dent in the problem. Moreover, it isn’t enough just to set aside a relative handful of grants for younger folks—after all, what happens when those same folks get a few years older and find themselves at a huge disadvantage compared to the early-career folks and the older folks with a long track record?
Political Realities
Whenever any policymaker proposes an idea where the chief messaging is “we want take away resources from popular and influential people,” the idea is likely to fail.
Look at what happened to the disastrous slogan, “defund the police.” Messaging like “we should put more funding towards mental health” would probably have been far more effective in raw political terms, even if the ultimate outcome was the same (i.e., putting relatively more spending on mental health vs. the police).
Indeed, as many folks well remember, NIH floated the same sort of policy nearly 10 years ago. NIH internal analyses had shown that the number of publications and citations did not have a constant return to scale. In other words, if you give more and more grants to the same lab at one time, the number of publications and citations didn’t grow in proportion to the extra grants. NIH concluded that “supporting a greater number of investigators at moderate funding levels is a better investment strategy than concentrating high amounts of funding in a smaller number of researchers.”
In early 2017, the NIH leadership announced a new plan that they called the Grant Support Index. It assigned points to a wide variety of NIH grants, and then limited the points, such that no one researcher could get more than 3 R01-equivalents at once. The NIH estimated that this policy would only affect about 3% of researchers, but it would enable NIH to make some 900 new grants.
To say that the plan was controversial would be an understatement. Famous researchers and universities were outraged, and one top researcher at Harvard gave a somewhat-controversial quote to the Boston Globe: “If you have a sports team, you want Tom Brady on the field every time. You don’t want the second string or the third string.” The policy was abandoned within about a month.
Other Options to Accomplish the Same Goal
First, instead of trying to impose an artificial limit on all grants, why not expand NIH funding while dedicating the expanded funding to newer and younger scientists?
The current proposal has a chart showing how much money and how many grants would be affected:
It seems more politically feasible to expand NIH funding by $2-3 billion dedicated to younger/newer investigators, than to propose cutting funding by over 12 percent while directly taking on the most powerful and influential scientists.
Second, here’s another option that is very tentative (Stuart wrote this up last year, but put a pause on it after feedback from folks like Robert Langer): put a lifetime limit on the number of grants [or the amount of money] that any single person can receive as the PI, but let them continue to serve as co-Investigator on grants where someone else is the PI.
Say you’re a famous and highly successful researcher. You may hit the lifetime limit at age 60 or even 50. No problem. That doesn’t mean you have to retire or shut your lab down. Instead, it just means that for future grants, you aren’t the PI any more.
That is, you can still get research/salary support from the grant, but you have to step aside and let someone else take the lead so that they can launch their career rather than laboring in the shadows until they’re middle-aged.
To make administration as simple as possible, and to avoid potential unintended consequences like trapping young researchers into smaller and/or shorter awards, all non-SBIR/STTR mechanisms should count towards the lifetime total. Similarly, for the sake of simplicity and to discourage gaming, both single PI and MPI awards, as well as project leader roles on large multicomponent grants, should count towards the total.
Since an earlier year of first award is roughly correlated with the total number of awards received, this should have the net effect of shifting grantmaking toward younger scientists.
Figure. Summary award history over the last twenty five years for 7,162 principal investigators (PIs) who won a competing R01, U01, R37, R35, DP1, or DP2 award in fiscal year 2024. Training, equipment, and construction grants are excluded; subprojects are included.
Or, instead of a harsh limit, we could simply give priority (in terms of a fund/no-fund decision, and in terms of dollar amounts) to renewal grants with 1) an investigator who hit the lifetime cap, who 2) steps aside and lets someone else be the principal investigator for their own first time. That is, give a thumb on the scale to well-established researchers who hand the starring role to someone just starting their career.
[We could do this on a sliding scale, of course, so as not to create a totally binary threshold.]
The attraction of a sliding scale is that it provides a potential way to manage outliers. The need for such a mechanism is clear; although many investigators increasingly draw on older work as they age, a scientist’s most influential paper can appear at any point in their career. Cutting off PIs whose best ideas happen to arrive late in their professional life penalizes everyone.
There are other objections, as well. First, a senior investigator who originates an idea may feel a loss of ownership and become disengaged if sidelined. Second, researchers might simply game the system by signing on more junior “principal investigators” who are principal in name only. Third, it might be the case that the occasional large lab (e.g., Robert Langer, David Baker, Jennifer Doudna) is actually a great thing. As one well-informed friend said, “I think the NIH push against big labs was a bad idea. Big labs are sometimes what you need to create a bubble that siphons off a group for a period of time from the ambient institutional pressures.”
And fourth, it remains an uncomfortable fact that despite the present of insider and reputational privilege, some scientists really are more creative and more productive than others. It would be a shame if we inadvertently stunted those scientists’ efforts, and the main point is that a rule about giving preferential PI status to younger investigators might be far less disruptive than a blanket limit to something like 2 grants at once.
Third, another possibility would be capping the cumulative number of grants for a single line of investigation. In the olden days, this would have been nearly impossible, but new techniques developed by one of us (Kris Willis) might make it possible.
To take a step back, let’s look more closely at the rationale for limiting the number of awards to a single PI.
One concern is that the reputation of some investigators may provide them with an insurmountable competitive advantage, even when their proposals only incrementally advance existing knowledge; the canonical example is the edge senior investigators often have over younger members of their field.
Another concern is that access to substantial resources can enable an investigator of any career stage to keep getting even more funding for work that may be only incrementally different from their existing projects.
In both cases, regulating the number of awards is just a proxy for the more difficult task of managing their substance.
Obviously, it can be difficult to figure out (across all possible scientific sub-fields) what is only an incremental advance. Many experts may have different, equally legitimate opinions, and deep familiarity with a field may lead specialists to see small differences as more significant than history ultimately proves them to be.
However, our new ability to computationally define the borders of scientific topics substantially lowers the difficulty of this task. Using networks to define topics, as Kris and her colleagues have done in their recent paper on predicting transformative breakthroughs in biomedicine, has two important advantages over using the large language models that power AI chatbots.
First, networks based on human curation can link topics that are related but don’t have any explicit linguistic commonalities.
Second, relying too heavily on AI can lead to scientific tunnel vision. By contrast, a network approach preserves and surfaces niche communities so that no area of research is overlooked.
Once the scientific universe is divided into topics, it’s straightforward to map new applications onto this landscape. We can then measure how closely each new proposal resembles the awards and papers that came before it. This arrangement makes it possible to flag proposals that are not explicit replication studies but are nonetheless very similar to prior work as potentially incremental, whereas those that are sufficiently distinct could be considered novel and reviewed by staff as candidates for a new award, even if the PI had met their lifetime cap.
If we could strengthen this sort of topic-based management of the NIH portfolio, we could do far more than merely assist in the mitigation of cumulative advantage and identification of potentially incremental work. Instead, we could help maximize the return on research investments by promoting scientific productivity and innovation more broadly.
One obvious synergy would be with an ARPA-H or Common Fund program to fast-track funding to predicted breakthroughs. Although there is still much to learn about how a breakthrough spreads through the scientific community, the trajectories of past advances suggest that as a theory or method gains traction, it also gains new adherents who apply it in new ways. Objectively classifying applications according to a common scheme would allow administrators to monitor and manage investments in breakthrough topics as they grow and diffuse out from a dedicated incubator and into other programs and administrative units.
Check out the graph below, which shows the potential of this approach. The graph shows the progression of a particular field (protein engineering and the directed evolution of enzymes), which eventually was the subject of a Nobel Prize in 2018. What you see in the graph is that as of the black asterisk on the far left (in 1997), it was possible with just the data and publications at the time to predict that a future breakthrough might be coming. The year 2005 is when the breakthrough actually occurred (indicated by the blue asterisk). An 18-year head start on predicting a future breakthrough isn’t too shabby!
Then over time, the field became more popular (indicated by the size of the bubbles) but it also became much more stagnant (indicated by the color of the bubbles – more red equals a greater percentage of the field that isn’t doing anything new).
Robust topic-based management would also help ensure that the full range of work building on a breakthrough discovery is identified and supported, including efforts to translate the advance into clinical practice, develop and patent related technologies, and maintain the community resources needed for deeper investigation. At the same time, improved tracking and classification of investments at the level of entire fields would help to minimize expenditures on unnecessarily duplicative or substantially overlapping projects.
***
People would eventually find a way to game any of the above ideas, like they do everything else. But maybe we should be evaluating the effectiveness of agency policies often enough, and be willing enough to adjust as needed, that no single approach to gaming is going to be successful over time.
 | A guest post by
|