The High Quality Research Act: A Blast from the Past?

Melinda Baldwin, a historian of science interested in the development of peer review, has written a guest post about some interesting parallels between the High Quality Research Act and an older controversy about peer review at the NSF. You can learn more about her work here.

A few months ago, Hank and Lee shared some thoughts about the discussion surrounding the "High Quality Research Act," a bill drafted by Rep. Lamar Smith (R-TX), the current head of the House Committee on Science, Space and Technology. The bill would require the NSF director to pledge that funded projects are "high-quality" and benefit the American people, and it seems to be grounded in Smith's concern that the NSF is funding "questionable" projects. Shortly before a draft of the HQRA leaked, Smith had called Presidential science advisor John Holdren and acting NSF Director Cora Marrett before Congress to justify the NSF's spending decisions. 

Smith's repeated statement that he wanted to "improve" on the NSF's grant-awarding process raised hackles in the scientific community. Currently, the NSF relies on reports from referees—i.e., peer review—to choose which applications will be funded. What many observers found really alarming was the letter Smith wrote to Marrett, requesting copies of the referee reports related to five NSF grants that he felt were suspicious, all in the social sciences. 

http://www.nsf.gov/news/mmg/media/images/nsf_bldg_f1.jpg

Reaction from the scientific community, and from politicians, was swift. Most of it played on the same theme: that peer review is a sacrosanct part of the scientific process and that Congressional interference would have dire consequences for the quality of research in the United States. Eighteen former NSF Assistant Directors signed a letter arguing that requiring the NSF to circulate peer review reports might "severely damage a merit review system that is the envy of the world." Smith's fellow Texan Eddie Bernice Johnson (D-TX) said Smith was "sending a chilling message to the entire scientific community that peer review may always be trumped by political review." And President Obama himself, in his April 29 address to the National Academy of Sciences on its 150th anniversary, assured listeners that he would work to protect "the integrity of the scientific process" from "political maneuvers." 

I recently began a project on the development of peer review in the twentieth century, so I am always interested when peer review pops up in the news. But the main reason the HQRA debate piqued my interest is that it's almost identical to a controversy about NSF funding from 1975. 

See if this sounds familiar. In 1975, Senator William Proxmire (D-WI) expressed concern about whether NSF spending was benefitting the American public. Proxmire named five NSF-funded grants that he said were "at best, of nominal value to the American taxpayer who foots the bill." He then called NSF director H. Guyford Stever before the Senate to defend the NSF's spending decisions.

Unlike Smith, whose proposals don't seem to have attracted much support, Proxmire found Congressional allies in Rep. John Conlan (R-AZ) and Rep. Robert Bauman (R-MD). Bauman proposed that the NSF should submit all grants for Congressional approval before promising any funding. Conlan, like Proxmire, believed that the NSF's grants were of minimal benefit to most Americans and that they were disproportionately awarded to elite private universities in the Northeast. Conlan quickly became one of the NSF's harshest critics.

Here's where things get interesting. One of the criticisms Conlan lobbed at the NSF was that the organization was ignoring referee opinions when it made funding decisions. In other words, Conlan set Congress forth as the defender of peer review.

Inside the review process (Source: Wikimedia Commons)

Conlan's criticisms appeared to strike a nerve with the NSF. In the early days of the organization, referees were seen as advisors to the division directors, but the directors retained the power to fund projects with lukewarm reports or reject proposals with enthusiastic ones. This was typical of peer review processes in the 1940s and 1950s, which tended to give editors and grant organization employees the power to accept or ignore the referees' advice as they saw fit.

But by the 1970s, the attitude towards peer review in the United States had changed. Peer review was increasingly being linked with scientific legitimacy. (Figuring out how and why this happened is the goal of my new project.) The idea that NSF directors might award a grant to a proposal with lukewarm referee reports was less acceptable to the scientific community—and to the public. NSF officials responded to the 1975 criticisms by placing more responsibility for decision-making on referee reports. A new audit office at the NSF was created to ensure that directors placed appropriate weight on positive and negative reports.

Director Stever then used the peer review reforms to justify rejecting Proxmire, Bauman, and Conlan's other proposals. Stever and other NSF officials argued that having proposals reviewed by experts was the best and only way to decide which projects should be funded. No further scrutiny was needed to guarantee that good science was funded and poor science was not, especially scrutiny from non-expert reviewers such as members of Congress.

The strategy was successful; the suggestion of Congressional review for NSF grants was dropped. Essentially, the NSF and Congress agreed to place their trust in peer review in order to determine how the NSF's chunk of taxpayer dollars would be spent.

http://strange-matter.net/screen_res/nz060.jpg (with permission)

Are the parallels between Smith's ideas and the 1975 proposals just an amusing case of déjà vu? Maybe, but there are some differences that are worth noting. In 1975, the criticisms of NSF spending led to Congress placing more trust in the NSF's peer review process. Thirty-eight years later, Smith seems to think that this trust might have been misplaced. In a May 3 interview with Science, an anonymous Science Committee aide explained that the HQRA was designed to add an extra step in between the referee reports and approval of the grants. As he put it: "There is a step between peer review and the awards being made, and somewhere in there, Congress is saying, 'We think an additional step is needed to solve the problem of so many questionable grants being awarded.'"

In other words, Smith's office seems to think that NSF reviewers can't be trusted to approve good projects and reject inadequate ones. An extra step is needed to make sure nothing "questionable" receives funding.

Does the HQRA signal that people outside the academy are losing their trust in scientific peer review? Actually, I think it's just the opposite. The speed with which the HQRA appears to have died on the vine suggests that public faith in peer review is still quite robust. Notably, in the interview I linked above, the Science Committee aide went out of his way to convince the reporter that the HQRA was not interfering with peer review itself. No one seems to think that attacking peer review is going to be a winning strategy.

In fact, trust in peer review might just be stronger outside the academy than within it at the moment. Scholars in many fields have written volumes on whether peer review actually works the way we want it to. I will be interested to see if the furor surrounding the HQRA dampens these kinds of critiques. When "trust in peer review!" has been such an effective rallying cry for the pro-NSF crowd, will scientists and other scholars want to criticize their best defense against Congressional interference?