Why I don't think publishing more negative results is worth any effort at all

This is part II of my reaction to the standard litany of well-intentioned ideas to improve science.

One general refrain I hear everywhere is the need to avoid the file drawer effect, publication bias, and more generally the large dark matter of unpublished results floating in the center of the universe that, were it published, would save everyone lots of time.

While I am equally opposed to the file drawer effect, publication bias, and the agony of re-following someone else's doomed footsteps, I dont think the solution is as easy as publishing negative results. And definitely not requiring publication of negative results.

[edit: Obviously many negative results are published currently and Im not opposed to that. Im talking about requiring publication of all negative results.]

Back in my undergraduate days, in physical chemistry, we learned the reason for the 2nd Law of Thermodynamics. There is only one way to be ordered and many many ways to be disordered.

The same principle applies to science. There is only one reason your experiment worked: you did it right and your hypothesis was right*. There are many many reasons it didn't work: most of them are the huge number of ways you did the experiment wrong. One of them is that you did everything perfectly but it just happened your hypothesis was wrong. But that's just one.

[* It's also possible it worked by chance, but we already deal with that possibility through hypothesis testing.]

Im not just talking about making basic mistakes. There are many levels at which problems could creep in. I won't enumerate them. Suffice it to say, science is hard. Even those of us who have been doing it for 12 years (=me) still screw up more often than we get it right. I have oodles of negative results in my file drawer (technically in folders on my laptop). Some of them are probably negative results because I did silly stupid sloppy things I would be embarrassed for anyone to know about.

I suppose I COULD take the time to do everything right, but maybe I used a task that is slightly wrong. So I should try all slightly different tasks. Or maybe I was recording a few millimeters away from the hotspot. So I could go explore all the adjacent areas. I COULD do that, but I don't have all the time in the universe. And until I do, my negative results are not interpretable by anyone.

We have a current system for publication of negative results. It's called word of mouth. I recently got interested in Exciting New Brain Area "X". I casually asked a couple people what they had heard about it. They told me they heard persons Y and Z were recording there. I casually asked Y and Z for their reactions. They both told me they got negative results. Their experiences gave me some guidance on how to proceed. They did not have to spend a month preparing manuscripts summarizing their findings. They just had to talk to me for a minute or two. Much less effort for them!

This is why I am all in favor of grad students travelling more and talking to other scientists. I think that informal communication conveys as much information as published papers and talks. But that's a topic for another post.