They put a 4 year old child in front of a marshmallow, and ask him/her to not eat the marshmallow while they leave the room - if they abstain they get 2 when the experimenter comes back. Then they time how long it takes the kid to give up and eat it.
I'm not saying this is fraud. I'm just saying, that's a really fucking powerful test. Does that level of prediction even make sense? Can you decide that much about someone at age 4 in 15 minutes? Not just 15 minutes, but with a single scalar variable measured in those 15 minutes??
I dunno, it kinda feels too good to be true.
Maybe it measures something else, like parental income, which is insanely predictive of lifetime success. Or maybe there's something fishy going on with the study, which AFAIK has not been replicated (at least the longitudinal part).
Seems like expecting replication of a decades-long longitudinal study is a lot to ask for so soon, doesn't it?
ReplyDeleteAlso, the original data from the 4-year olds was published, so the researchers would have to have been pretty clever to somehow predict how to alter the data to make it more predictive of an outcome they wouldn't know for 20 years.
I'm also pretty sure they controlled for parental income, but you'd have to read the paper to be sure -- which would be a good thing to do before accusing people of conducting fishy research that's too good to be true.
The flip side of the fraud problem is that we're also quick to dismiss solid findings which don't line up with our preconceptions. I don't mean to sound harsh but this was a pretty unfair critique with accusations of misconduct sprinkled in with literally nothing to back them up.
Who cares if it's too much to ask for? That's irrelevant. The question is whether we should trust this study that has not (to my knowledge) been replicated. [And im not an expert, maybe it has been replicated]
DeleteSecond, they would not have to be very clever to fudge the data. They could have just used the blue dot method, for example (http://news.sciencemag.org/scienceinsider/2012/06/rotterdam-marketing-psychologist.html?ref=hp)!
Not that I am saying Mischel et al did anything resembling fraud. I suspect they did not. I think this is a case where the results are just so strong that they don't really pass my personal common sense test. And I think "sounds too good to be true" is a legitimate critique, at least on the internet. If I were writing a critical review of the study for a journal, I'd spend more time looking into it.
This comment has been removed by the author.
Deleteoops, apparently deleted my comment... will return to rewrite if i get a chance.
ReplyDelete