Not so long ago I came across with an interesting article in Nature about how researchers overvalue the concept of statistical significance (p-value) as the “gold standard”.
Its plot thread goes around p-hacking, data dredging and other evil analysis “techniques” hinting at some unprofessional researchers who seem to spend their time in multiple comparison procedures hoping that at least one of them turns out to be “significant enough” to make their article publishable even though the conclusions are still meaningless anyway.
Far from putting the outcry I firmly believe that, despite the possibility truly exists, no serious researcher would really go so far and pay the price of such imprudence and scientific negligence… or at least I like to think so…
Nonetheless, I bet that everyone wants to be more efficient and that means, among other things, having better analysis tools in our labs.
But not everything is lost…
The good news is that there is a balanced solution to reach the goal: exploratory data analysis.
The article closes with the contribution of the renowned political scientist and statistician Andrew Gelman of Columbia University in New York City which I quote:
“…exploratory and confirmatory analysis are approached differently and clearly labelled. Instead of doing four separate small studies and reporting the results in one paper, for instance, researchers would first do two small exploratory studies and gather potentially interesting findings without worrying too much about false alarms. Then, on the basis of these results, the authors would decide exactly how they planned to confirm the findings …”
Assuming an exploratory phase is required before good old confirmation tools take over, we are meant to change the strategy when designing experiments.
Enhancing our lab’s toolbox with specific applications to do that work would help us to understand better the story that our data is telling us and thus to identify the most promising "hot spots" and predictors in your data to focus on.
Efficiency. That’s all. Isn’t it?