Why do we have all these issues with statistics nowadays?

It's not just voodoo correlations in social neuroscience.

There's all the problems with genome wide association studies, and there's lack of replications in medicine and other fields, and of course there's the rest of neuroimaging, and other ones. I predict that any day they will start to realize it's time for statistical re-education in old fashioned fields like single neuron physiology.

More and more there are people getting uptight about statistics. I see more and more calls for changes in the way statistics are done. I have heard that Nature Neuroscience for one has started doing stats reviews independently of the rest of the review, possibly with other reviewers.

I think most people can agree that statistics is pretty rotten and that it's time for improvements.

But I wonder, why now? Has this always been a problem but no one noticed until now?

My hunch is that this really is a new problem. There's always been an underlying low-level problem, sure. But the fact that the problem is enormous nowadays, I think is new.

And I think the problem reflects a major change in the way data are collected and stored and analyzed. Without exactly realizing it, we are living in the era of Big Data. But we are using the statistical methods invented to handle Small Data.

When you have huge amounts of data, you can very quickly search for patterns, and you can even make your computer search for patterns for you. You can make it search all night and it will do as much in one night as it used to take computers centuries to do. With this computing power comes the possibility of false positives, and with that comes the need for multiple comparison tests.

And that's never really been something that statisticians have worried too much about before, because it was never an issue.

The common sense test

Walter Mischel has done some very influential experiments showing that self-control at age 4 is insanely predictive of success later in life. SAT scores, income, addiction, PFC activity decades later...

They put a 4 year old child in front of a marshmallow, and ask him/her to not eat the marshmallow while they leave the room - if they abstain they get 2 when the experimenter comes back. Then they time how long it takes the kid to give up and eat it.

I'm not saying this is fraud. I'm just saying, that's a really fucking powerful test. Does that level of prediction even make sense? Can you decide that much about someone at age 4 in 15 minutes? Not just 15 minutes, but with a single scalar variable measured in those 15 minutes??


I dunno, it kinda feels too good to be true. 


Maybe it measures something else, like parental income, which is insanely predictive of lifetime success. Or maybe there's something fishy going on with the study, which AFAIK has not been replicated (at least the longitudinal part). 

If Ariely is right there's a lot more fraud in science going on

Dan Ariely has a new book out. It comes highly recommended from me even though (better be 100% honest in a post about fraud]) I haven't read it. But I've seen him give the talk many times and I've read some book reviews.

The short version is that cheating in our society works like this: most people cheat a little bit, a few people cheat a lot.

Quantitatively, most of the problems in society, I assume he argues, come from the large amount of small cheating, rather than the Madoffs and Barry Bonds.

If Ariely is right, how does this affect our understanding of scientific fraud?

Well, most people get truly exercised about the Hausers and the Stepels of the world. They are the funnest to talk about, certainly. But what if they are the just tip of the iceberg?

Well, first, big fraudsters are potentially more important in science than in other fields because of the non-linear nature of scientific progress. Big names are attractor states, and if they are totally wrong, lots of resources can get redirected to them. Especially the most valuable resource, the minds of the next generation.

But of course, those big fraudsters also get more scrutiny and presumably get caught sooner. Their status is more precarious (Im assuming).

And what about the small fish?

First of all, it make me worry about any p between 0.04 and 0.05. I mentally replace it with a 0.06.

More generally, it makes me more worried about fields where p's close to those values are the norm. [In my field, p's tend to me lower than that, for whatever reason]. Those fields include animal psychology, for example. Also, medicine.

It also makes me worry about fields where scientists can run slightly different experiments quickly and easily and get slightly different data. So for example, when a scientist has to travel to Mauna Loa to gather some astronomical observations about a once-in-a-lifetime comet, it's hard for her to go replicate that experiment under slightly different conditions. Or to go new analyses, which is basically replicating the experiment once the data is collected.

If these people are interested in not getting caught, they are probably publishing results that generally agree with the broader paradigm. They are not sticking their necks out publishing counterintuitive findings. This creates a false sense of validation for the dominant view, makes it even harder to publish contradictory findings, and increases biases in the literature.

If cheating is pretty common, it also would explain why meeting people is so important in science. Having a personal relationship makes it easier to judge whether someone is one of those mild fraudsters or whether they are one of the truly upright people. Also, being friends with someone makes it easier to forgive them if one suspects they are cheating a little bit.

I don't have any conclusions or suggestions. This has just been on my mind lately. I just think of the posters at SFN and wonder if maybe half of them are slight frauds.

Departmental Grants Guru

•  How much does a program officer with 5 years of service make?
•  How much extra money, in overhead, would such a person bring in if hired by my department to help us write grants full time?

I assume the second number is larger.

If my department has 20 faculty, and each is typically funded with one or two R01s per year, and this Departmental Grants Guru could nudge two $250K applications from  unfunded to funded, this would generate about $260K/yr in overhead. I assume that's enough to pay this person's salary. If not, then 3 R01's should be enough.

Even more importantly, think of the time saved. Young faculty spend measurably large portions of their time on grant writing. If average youngster could take 10% of her total effort and transfer it from grant-grubbing to other stuff, this would be $7500 in value gained (assuming a $75000 annual salary). This effect is even larger for older faculty, but presumably they don't need as much help.

As an economic proposition, having an in-house Departmental Grants Guru would seem to be wise. I imagine this person being a former PO. This person can help guide study section selection, can help with the writing at all stages, including shaping the ideas. Thinking of a good set of Aims. Brainstorming. Giving writing tips, etc, etc. 


Maybe this person even takes care of calling the actual PO, and the two of them chat using PO-language, and get the messages across more efficiently. This person also knows the faculty research interests and scans the RFA's and things.

This person will essentially assume the mentorship role that older faculty often provide pro bono (but not when they are busy).


This person would also, in my imagination, be in charge of the training grant, and helping to formulate or maintain a core grant. This would save a ton of time from the senior faculty. If the person has extra time, they could help the junior faculty with foundation applications. 

I'm not advocating for or against the creation of this position. But I am asking why it's not something that is done everywhere. So many of the other functions faculty used to do themselves is being farmed out to administrators. I'm not against this; it's division of labor, and it's probably more efficient. Why are grants left in the hands of people who are really supposed to be doing research?


(I guess one counterargument is that this essentially IS the job of the professor nowadays. The actual research is done by the grad students and the post-docs.)

Beans and rice with vegetables

I remember one time in college my roommate said he wished there was 'human food' kind of like dog food. Food that was tasty, had all the necessary vitamins, was healthy for you. A balanced meal you could eat without feeling guilty and without having to think too hard about it. A default option.

Right now there's a default food option for most people, but it's McDonalds. And that's just about the least healthy thing imaginable. But I do want to emphasize the good thing about MicDonalds, the thing that is so often ignored in the debate about healthy food:

It takes almost no mental effort to choose it.

McDonalds is ridiculously efficient, and lets you focus on your life with a minimum of distraction. And some times that's what you want. Not sometimes, lots of times. We are busy people, and we are thinking about important things. Not just professors either. Everyone.

What we need, or at least, what I would love, would be a counterpart to McDonalds that sells zero effort food that you know is perfectly fine for you. Im not looking for mineral rich superfoods. Don't need antioxidants. Doesn't even need to be terribly cheap. (Although if this is going to benefit people poorer than me, maybe that should be a factor.)

Like human food, it doesn't have to to be super delicious, just reasonably yummy. Im not a nutritionist, but like, a $6, a bowl of beans and rice with fried vegetables on it, and spices like garlic, pepper, and cumin.

There are healthy foods and there are foods you can eat without thinking. It would be great if there were something in the middle there. Ubiquitous, easy, fast, healthy food.

That's kind of what cereal is. That's why cereal is so great. Breakfast is a solved problem, as long as you stick to healthy ones. Now let's extend that solution to lunch. I love nice food, and I like making my own food. But I can only do that 1 meal a day max.

Discretion

When they say "Viewer Discretion Advised" I always imagine that they mean the primary definition of discretion. So: "Please don't tell anyone what you see here."

Signs I might have mild Asperger's Syndrome

Based on this wikipedia article and this one.

Intense pre-occupation with a narrow subject?
Yes. For example, I frequently read articles about the Interstate highway system or about geography. At other times in my life, world records, facts about the Olympics, French grammar, math, certain comic books.

Restricted prosody?
Yes. According to my sister (and others) I have no prosody.

Physical clumsiness?
Kind of. I am particularly bad at dancing, and at all sports, aside from running. I know everyone says this, but when I took a ballroom class, I was clearly an outlier in how slow I was to learn, despite really really trying.

Failure to develop friendships?
True until college. Not so true since then.

Lack of interest in fiction. Exclusive interest in non-fiction?
Yes. I have tried fiction many times, but have never really gotten into it, at least, since starting college. When I was younger I read everything, but even then more non-fiction. Now it's hard to see the point. Kavalier and Clay is still on my to-read list. Unlike a lot of nerdy people I have never been interested in fantasy or sci-fi books.

Stereotyped and repetitive motor behaviors?
Yes. Mild Tourette-like symptoms, especially during adolesence, and during times of stress.

Unusually large vocabulary at young age?
Yes.

Called "little professor" as a child?
Yes.

Obsession with routines and rituals?
Yes. From young age until present.

Pedantic, idiosyncratic or formal speech?
Somewhat. Moreso before college.

Extreme shyness, unwillingness to talk to unfamiliar people?
7 lonely years at summer camp say yes.

Difficulty understanding social norms?
Somewhat true until college.

Close affinity to animals?
Yes. Especially dogs. Also, my username is a polarbear. I wonder if that's meaningful.

Male?
Yes.

Other ones not so much:  Lack of empathy? Not really true at all. Sudden affective outbursts? Not true. Feeling of being detached from the world around them? Not really.

My Catch 22

Whenever I am worked up and trying to go to sleep, I put away my laptop and won't let myself open it.

My thoughts usually turn to the two dozen or so emails I absolutely need to write instantly.
My obsessive worries say that if I don't write them right now, I will forget them by morning.

By forbidding myself from writing them I prevent myself from surfing and wind up getting to sleep sooner.

But the downside is, in the morning, I have forgotten all the emails I was supposed to send.

Going away and coming back

I went away for a week and got back today. When you include me checking out to finish my R01 submission, I was kinda gone for 3 weeks. Things carried on as they would have had I stayed there. I probably have micromanagerial tendencies, so I deliberately did not communicate with the lab while I was gone, or at least resisted the urges more than I might otherwise have.

In the 1960's, Noam Chomsky was running a linguistics 'lab' at MIT. His students and post-docs included future heavyweights in the field. His ideas were in the process of changing linguistics, and are still influential today.

In 1966, he went to Berkeley for his sabbatical and stayed for the summer. It was 1966 and Chomsky was becoming more and more interested in leftist politics. Meanwhile, his star pupils, including George Lakoff and Paul Postal, among others, continued his regular twice weekly journal club. At this club, they continued to explore his ideas and apply them to new problems. In a practical sense these guys were interested in staking out their own territory so they could get jobs in academia. They tried to move his ideas beyond syntax to prosody, word structure, even the mind - the structure of concepts. They came up with ideas they were proud of and wanted him to support and publicize.

When Chomsky returned in the fall, he was horrified by these ideas and repudiated them. He rejected all of their ideas. They in turn rejected his rejection, and went on to promulgate their ideas elsewhere. Personally, he had a falling out with his students, who went on to found what could be called an anti-Chomskian school. Chomsky's time away led to the founding of two opposed schools of linguistics that have really defined a large part of how linguistics works as an academic field for the past 50 years.

Basically, this kind of thing is my biggest fear whenever I go away.