“Science relies on the careful collection and analysis of facts. Science also benefits from human judgment, but that intuition isn't necessarily reliable. A study finds that scientists did a poor job forecasting whether a successful experiment would work on a second try,” writes Richard Harris in their recent NPR article entitled “Scientists Are Not So Hot At Predicting Which Cancer Studies Will Succeed.”
According to Harris’s article, “That matters, because scientists can waste a lot of time if they read the results from another lab and eagerly chase after bum leads.”
"There are lots of different candidates for drugs you might develop or different for research programs you might want to invest in. What you want is a way to discriminate between those investments that are going to pay off down the road, and those that are just going to fizzle," says Jonathan Kimmelman, an associate professor of biomedical ethics at McGill University in Montreal, in the NPR piece.
“Kimmelman has been studying scientific forecasting for that reason. He realized he had a unique opening when other researchers announced a multi-million dollar project to replicate dozens of high-profile cancer experiments. It's called the Reproducibility Project: Cancer Biology. Organizers have written down the exact protocols they would be using and promised not to deviate,” Harris continues.
Harris notes in their NPR article, "This was really an extraordinary opportunity,’ according to Kimmelman because so often scientists change their experiment as they go along, so it's hard to know whether a poor forecast was simply because the experiment had changed along the way.”
Now, while you’re not forecasting results of cancer studies, we found a lesson to be learned here.
Do you see it?
