I read the other day somewhere that the science peer review system has its origins in the jury system of English Common Law. Two thoughts come to mind: first that the jury system makes mistakes and second that not a lot of folks look eagerly towards their jury duty. The same is true of scientific peer review of course.
The problem is what might work better? While the Howard Hughes Medical Institute and other elite entities pride themselves on funding individuals instead of projects (ideas), it’s not at all clear to me that having a few powerful scientists or tech billionaires make the choices would be immune to making mistakes. Consequential scientific discoveries are tricky things: they emerge often where least expected. I remember one story where the Institution had to hurriedly award tenure after a Nobel Prize was announced.
Some folks look towards the X-prize model or even towards predictive markets. Once again, the former involves a choice of what is an important challenge, the latter has notoriously imperfect predictive powers.
When I was asked about this problem at any one of the many NSF panels that I visited to do Q&A, I would point out that one alternative to these not so good methods, would be to simply divide the NSF budget equally among all US PI’s–roughly $5K. A typical minimum value for a science grant is $1M/3 years. A small number of the panelists let me know that the 5K would be just fine–which surprised me.
We have a similar problem with journal articles that we have with grants. Today, I heard it suggested that we simply dispense with peer review of manuscripts and allow the bad ones in– figuring that eventually, only the good ones would be cited and the rest would wither away in the cybersphere.
Drop me a line, if you have a way to solve this problem.
Discoverers and Discoveries 