How to reform NIH…

Recently, I’ve mostly written in this respect about the NSF, but I also spent six years at the NIH, as a staff fellow in the intramural program (the biomedical medical center in Bethesda Maryland). When most folks think about the NIH, they are not really focussing on the intramural program. Rather, it’s the extramural program that gives out grant awards to biomedical researchers at US Colleges and Medical Centers that gets the attention. And I guess that’s fine because the extramural program represents about 90% of the NIH budget.

But, if I were going to magically reform the agency, I would focus on the intramural program. That’s because it has so much potential. With an annual budget north of $4B/year, America’s largest research medical center and thousands of young researchers from all over the world, it has so much potential. If Woods Hole is a summer nexus for life sciences during the summer, the NIH Bethesda campus is that thing on steroids year round.

The special sauce for the intramural program is that ideas can become experiments and then discoveries without the usual intermediate step of writing a proposal and waiting to see if it was funded. When I was at NIH, I could literally conceive of a new experiment, order the equipment and reagents and publish the results several months later. Hence, the intramural program has the structure in place to be a major science accelerator.

But, for some reason, when we think of such science accelerators, we generally consider private institutions like HHMI, the Allen Institutes and perhaps the Institute for Advanced Study in Princeton. What about NIH? On the criteria of critical mass, it dwarfs those places.

To my mind the problem lies in NIH’s ‘articles of confederation’ nature: it’s really 27 (or so) different Institutes and other units that are largely quite independent (especially the NCI), with a relatively weak central leadership. And this weak confederation organization plays out, not only on the Hill or in the awarding of extramural awards, but crucially also on the Bethesda campus, where intramural institute program directors rule fiefdoms that are more insular than academic units on a college campus. And this weak organizational architecture acts in the opposite direction of the science accelerator advantage that I wrote about above.

So here’s a big idea: let’s make the intramural program it’s own effective NIH institute. And have Congress authorize it and fund it separately, as a high risk, high payoff biomedical research program for the country. Does that sound like ARPA-H? Ooops. Well, then maybe we should just give the Bethesda campus to ARPA-H.

What is going on with global politics?

Photo by Pixabay on Pexels.com

I spend most of my time thinking either about global issues somewhat removed from politics, or the politics of science and the academy. But it’s clear that geopolitically, something much larger and emergent is going on that is fostering an increase in violent conflict and political anger. And this is happening not just here in the United States, but globally with hot wars going on in Europe and the Middle East, to say nothing of new threatened conflict in South America.

Apart from the micro-details of each particular conflict, the bigger picture, from my point of view, is that the pandemic and climate disruption are global shocks that have led to this emergent manifestation of large-scale human-on-human violence. Apart from a complexity science approach, how would we study this notion? Are there any testable hypotheses here?

Asking for a friend…

Reproducibility redux…

The crisis of reproducibility in academic research is a troubling trend that deserves more scrutiny. I’ve blogged and written about this before, but as 2024 begins, it’s worth returning to the issue. Anecdotally, I’ve noticed that most of my scientist colleagues have experienced the inability to reproduce published results on at least one occasion. For a good review of the actual numbers, see here. Why are the findings from prestigious universities and journals seemingly so unreliable?

There are likely multiple drivers behind the reproducibility meme. Scientists face immense pressure to publish groundbreaking positive results. Null findings and replication studies are less likely to be accepted by high-impact journals. This incentivizes scientists to follow flashier leads before they are thoroughly vetted. Researchers must also chase funding, which increasingly goes to bold proposals touting novel discoveries over incremental confirmations. The high competition induces questionable practices to get an edge.

The institutional incentives seem to actively select against rigor and verification. But individual biases also contaminate research integrity. Remembering back to my postdoctoral experiences at NIH, it was clear even then that scientists get emotionally invested in their hypotheses and may unconsciously gloss over contrary signals. Or they may succumb to confirmation bias, doing experiments in ways that stack the deck to validate their assumptions. This risk seems to increase as the prominence of the researcher increases. It’s no surprise that findings thus tainted turn out to be statistical flukes unlikely to withstand outside scrutiny.

More transparency, data sharing, and independent audits of published research could quantify the scale of irreproducibility across disciplines. Meanwhile, funders and academics should alter incentives to emphasize rigor as much as innovation. Replication studies verifying high-profile results deserve higher status and support. Journals can demand stricter methodological reporting to catch questionable practices before publication. Until the institutions that produce, fund, publish and consume academic research value rigor and replication as much as novelty, the problem may persist. There seem to be deeper sociological and institutional drivers at play than any single solution can address overnight. But facing the depth of the reproducibility crisis is the first step.

Happy New Year!

Cognition, Neuroscience, Biology, Chemistry, Physics…

There is a continuum here. We believe that the same fundamental rules of nature (life) apply and that if we were clever enough, we could explain the first (cognition) from the rules of the last (physics). But we aren’t–at least yet. And so there is a gap which is filled by humans in a number of ways that may be comforting if not scientific.

And yet, there is the question of emergence, in which order arises from complexity in ways that may be rule-based locally, but not globally. Seizures and hurricanes are relatively simple examples of such emergence, but learning and memory may fall into that category also.

And so, while there are definitely quarks which obey rules, the phenomenon of Shakespeare’s plays may not be deduced from quark-laws. In fact, at an intuitive level, I very much doubt it. But my intuition is not science and therefore we may in the end be able to recreate the Bard from first principles. And if so, there goes free will.

Space-based solar

Peggy Hollinger’s fine piece behind the paywall in today’s FT Big Read, here. Space-solar was the raison d’etre for Gerard K O’Neill’s vision for huge space colonies back in the 1970s. The colonies would support themselves by exporting virtually unlimited electrical power to Earth. No need for fusion!

I’m amazed at how our global ambitions have shrunk into our iPhones over the years. About 15 years ago, Nicholas Carr wrote a piece in the Atlantic about how the ‘newish’ internet was changing our brains (and making us stupid). I remember he interviewed me for that article and I was a bit skeptical of the notion that our minds could be dumbed down by the likes of Google.

Today, not so skeptical. Our lack of an ambitious global investment strategy for a sustainable planet that improves lives, keeps the peace and gets us out in the Solar System is palpable.

David Leonhardt nails it…

In today’s NYT on-line, here. I remember those first passenger jets shrinking distances. And I remember driving coast-to-coast on America’s interstates in 52 hours during my twenties.

What could our country look like if we went long and big with the kind of investments that he’s talking about?

An AI wrote the ‘Broader Impacts’

As artificial intelligence systems grow more advanced, they are being applied in novel ways that raise ethical questions. One such application is using AI tools to assist in preparing grant proposals for submission to funding agencies like my old agency, the National Science Foundation (NSF). On the surface, this may seem innocuous – after all, AI is just a tool being used to help draft proposals, right? However, utilizing these systems raises several ethical concerns that must be considered.

First, there is the issue of authorship and originality. If an AI system generates significant portions of a grant proposal, can the researchers truly claim it as their own original work? And does it really matter? After all, the currency of science is discovery and the revealing of new knowledge, not filling out another post-doctoral management plan. My own sense is that AI in an assistive role is fine. And at some point in the not too distant future, the AI’s may act more as a partner, as they now do in competitive chess.

Relatedly, the use of AI grant writing tools risks distorting the assessment of the researchers’ true capabilities. Grant reviews are meant to evaluate the creativity, critical thinking, and scientific merit of the applicants. If parts of the proposal are artificially generated, it prevents a fair judgement, undermining the integrity of the review process. Researchers who utilize extensive AI assistance are gaining an unfair advantage over their peers. But what can we do to stop this. It seems to be that the horse has left the barn on that one. Perhaps it’s fairer to assess a scientist in a post-hoc fashion, based on what they’ve accomplished scientifically.

There are also concerns that AI-aided grant writing exacerbates social inequalities. These systems can be costly to access, disproportionately benefiting researchers at elite universities or with ample funding. This widens gaps between the research “haves” and “have nots” – those who can afford the latest AI tools versus those relying solely on their own writing. The playing field should be as even as possible.

Additionally, the use of grant-writing AI poses risks to the advancement of science. If the systems encourage generic proposals repeating current trends, rather than groundbreaking ideas, it could stifle innovation. Researchers may become over-reliant on the technology rather than developing their own critical thinking capacity. Shortcuts to generating proposals can inhibit big picture perspective.

AI is a powerful emerging technology, but it must be utilized carefully and ethically. Researchers adopting these tools have a duty to consider these ethical dimensions. Honesty and transparency about process, maintaining fairness, and upholding originality are critical to preserving research ethics. The ends do not necessarily justify the means – progress must not come at the cost of our values.

What if our understanding of transformer AI models is only marginally better than our understanding of biological brains?

At some level we do understand biological brains–we understand action potentials, synapses and some circuit networks. We understand neurotransmitters, plasticity, and signal transduction cascades. We even can look at the entire functional map of animal brains in real time as they behave. Unfortunately, the mapping function that goes from neuron to behavior or thought remains beyond us.

My question for my computer science friends is this: at what level do we understand how specific ‘neuron’ activations cause AI output that resembles a robot passing the Turing Test? I’m worried that it’s only marginally more than what we have for neurobiology and that’s why explainable AI doesn’t work.

Am I wrong?