Saturday, January 12, 2013

Science and its #overlyhonestmethods

This week, a hashtag trended on Twitter that will be of particular interest to historians, sociologists, and other students of science: #overlyhonestmethods. One site has called it science's PostSecret; 75 of the best examples have been curated at Storify. Here's what they look like:


Frank, humorous admissions (or inventions?) of how protocols are cobbled together and assumptions are papered over, the #overlhonestmethods meme makes for great reading, especially for those of us interested in the gap between representation and reality in scientific practice and publishing.

Bloggers at The Guardian, Scientific American, the Public Library of Science, and elsewhere have weighed in, most seeing #overlyhonestmethods as a refreshing peek inside the black box of science. Here, I'll inventory a few of the strands in the conversation, and offer some thoughts on what it all means.

The Guardian's headline captures what's at stake: "Scientists take to Twitter to reveal their less than scientific methods." By adopting the passive voice of science papers while subverting standard claims to precision and objectivity, the posts lay bare the exigencies of everyday scientific practice.

The meme began (and thrived) in the world of laboratory practices. A neuropharmacologist (and blogger) who Tweets as @dr_leigh kicked things off* on Monday evening:


And then it went viral. Targets included reaction times ("We incubated this for however long lunch was," @pedmills), precision techniques ("Brains were removed and dissected in, on average, 58 seconds. We know precisely due to a long running lab competition," @SciTriGrrl), and experimental set-up ("Plants were grown at temperature and light conditions last set by the only person understanding the incubator set-up," @AnneOsterrieder).

It didn't take long for topics to spread beyond the lab. From literature reviews ("We didn't read half of the papers we cite because they are behind a paywall," @devillesylvain) to writing up ("We used jargon instead of plain English to prove that a decade of grad school and postdoc made us smart," @eperlste) to submission ("Our paper lacks post-2010 references as it's taken the co-authors that long to agree on where to submit the final draft," @drd1983), the whole scientific process is in there.


As the meme breached the lab, it also spread beyond the lab sciences. One site has aggregated posts relevant to political science, asserting that what connects them to the natural sciences is the need to replicate results—and highlighting how difficult, and comical, such efforts can be.

The upshot, for most, is that this is "how science is really done." Indeed, that's the subtitle of many comments on #overlyhonestmethods, to which the Scientific American post added another one: #sogladwe'rehavingthisconversation. Coverage like this rests on the idea that the meme is poking holes in an otherwise ironclad objectivism in the public perception of scientific methodology—something many see as a good thing.

Others, however, find it troubling—either because of the money being squandered on sloppy research, or out of fear that #overlyhonestmethods will fuel the anti-science campaigns of creationists and climate-deniers. Some think that the tweets might "represent the very beginnings of a 'revolution' in how scientists and the public alike, view and talk about the scientific method"—for better or for worse.

We'll have to wait and see whether either celebration or worries are warranted. For now, let me underscore a connection others have pointed out between #overlyhonestmethods and science studies: Latour and Woolgar's Laboratory Life has been mentioned, and Lynch's Art and Artifact, Pickering's Constructing Quarks, and Galison's How Experiments End are in the air, too.

The "Crackling" 1980s!
In the 1980s, scholars of all sorts converged on the tangled world of science-in-practice, and found then (as here) that reports of their methods were greatly exaggerated. Science was human, and—as in everything else—the humans doing it muddled along as best they could. The difference, as Lynch put it, was in how they account for that process:
Science is perhaps unique as an occupation for the way in which its shop work is made extensively accountable in written reports, and examined for its rational, logical and systematically achieved 'methods' as formulated in those accounts. (Lynch 1986: 56-7)
The tension between rhetoric and reality is part of what's fun (and funny) about #overlyhonestmethods. But there's a deeper irony, too. Were reproducing results the central part of science it's claimed to be, then admitting these sorts of things would actually improve the overall process. That replication isn't paramount—as one particular author has noted—is only underlined by the humor inherent in this warts-and-all episode.

All these Tweets have emphasized (or given new meaning to) one of the concluding thoughts in Galison's first book, on how experimentation evolved in twentieth-century physics:
But laboratory work also exists amid practical constraints that may have little in the way of theory to support them: beliefs in instrument types, in programs of experimental inquiry, in the trained, individual judgments about very local behavior of pieces of apparatus or the tracks, pulses, and counts recorded every day. (Galison 1987: 277)
If we take these Tweets seriously—and, while there's much humor here, we might as well—then there's a lot more going on in experimentation than even Galison's seemingly expansive redefinition suggests. What's more, scientists' self-awareness suggests they'd be more or less on board with 1980s efforts to pry open the black box—or that they've always already been there, laughing amongst themselves.

More opaque—and more interesting—is what "the public" will make of all this, and what (if any) consequences the #overlyhonestmethods meme will have for the wider conversation about the nature and authority of science.
------------------------------------------------------------------------
*Correction: a previous version misleadingly linked to Latour's Laboratory Life in order to emphasize a parallel between that book's early role in laboratory studies and @dr_leigh's in #overlyhonestmethods.

9 comments:

  1. Thanks for this post, Henry. I'd totally missed the trend. It's interesting to see how admissions of not-very-scientific laboratory events (timing trials to lunch, dealing with undergrad errors, forgetting procedures) dovetail with grumbling at the bureaucratic and other structures that constrain and shape academic research (paywalls, funding deadlines, citation tracking, competing for priority, intellectual property concerns) and mocking certain long-standing conventions (passive voice writing, lab hierarchy). As you point out in your reflection on Galison, the #overlyhonestmethods tweets seem to unite all sorts of important threads of science studies into one meta-comment on scientific life.

    I wonder whether, in reflecting on what the trend says about practicing scientists, it's also important to consider the career-stage range of the posters. The ones I've read have a very young-researcher vibe to them. Do you think the aggregation of comments would look different if there were more from well established scientists?

    ReplyDelete
  2. Thanks, Helen! I think the comments *would* look different—and that further connects your points about hierarchy and bureaucracy.

    PIs are often so busy away from "the bench" (writing grants, for example) that they're dialed out of the tinkering these tweets describe. That might explain one like this, and it's encapsulated in this cartoon.

    That cartoon, though, also suggests that the relationship between Tweeting and the hierarchy might run the other way, too. The one that reads "The real expert on this paper was the tech who showed us all how to do it but isnt an author because they're the tech" is an example of this.

    I'd just also add (since it came up on Twitter) that I don't think most of these should be mistaken for actual admissions of guilt (or creativity), nor do I think what I've written above assumes or depends on that. Conversations with a few friends (and some of my own distant memories) suggest that what's fun about this is precisely how it reminds folks of the contingent, creative, sometimes sloppy innovations that happen in lab but don't make it into the write-up.

    ReplyDelete
  3. Do you really think Alexander Fleming intended to grow fungi? Everyone has the same data, it's how you look at it that counts.

    --bks

    ReplyDelete
  4. I wonder whether, in reflecting on what the trend says about practicing scientists, it's also important to consider the career-stage range of the posters.

    ReplyDelete
  5. I think that the tweets might "represent the very beginnings of a 'revolution' in how scientists and the public alike, view and talk about the scientific method"—for better or for worse.
    Investing in Property

    ReplyDelete
  6. Thanks for this post, Henry. I'd totally missed the trend.

    ReplyDelete
  7. I wonder whether, in showing on what the pattern says about exercising researchers, it's also essential to consider the career-stage variety of the images. The ones I've study have a very young-researcher feel to them. Do you think the gathering or amassing of feedback would look different if there were more from well recognized scientists?
    Property Investment

    ReplyDelete
  8. Thank you for making the effort to go over this, Personally i think strongly about this and love learning more about this subject. If at all possible, while you gain expertise, can you mind upgrading your site with increased information? It's very useful for me personally.

    ReplyDelete
  9. The information is very useful!Thank you for sharing the information!Keep up the great works !
    Recycling of Waste in Bangalore
    Biogas Plant in Bangalore

    ReplyDelete