Vast Literatures and the Two Paper Rule for Peer-Review

About two weeks ago, Noah Smith (@noahpinion) wrote an extensive blog post about the problem of academics referencing “vast literatures” to rebut an argument. As Smith tells it, simply admonishing another academic that he/she/they ignored extant vast literature is at best impractical and at worst pointless. In the first case, not only is it impractical to go back and read vast amounts of articles but it is also imprudent to jump into a vast literature willy-nilly. As to the latter case, well, the vast literature rebuttal may well point toward a line of research, however vast, that is as baseless as phrenology. As such, Smith properly puts the onus on the referencer of the vast literature to provide two exemplars in the literature. According to Smith:

what you need to assess quickly is not what the literature claims to find, but whether those claims are generally credible. And that is why you need to see the best examples the literature has to offer. Hence the Two Paper Rule.

I find the Two Paper Rule interesting not just as a nascent scholar but as an editorial board member for an academic journal. As such, I am an initial gatekeeper to what gets published in one of the oldest peer-reviewed journals in education. While I can’t guarantee the quality of the peer-reviews themselves, I can redouble my efforts to strengthen my link in the peer-review chain. No one wants to contribute to a controversy like what happened over at Hypatia, but following the Two Paper Rule could directly streamline and improve academic publishing and arguments between scholars.

Like any general rule there are limits and potential abuses, as noted by Krugman and Cowen, and I too have my own quibbles with the Two Paper Rule. However, I believe everyone in the peer-review chain has an obligation to use his/her/their hard-won knowledge to improve both the published article and the process. For instance, if I decide to reject a submission or ask that it be revised and resubmitted in order to account for a missing area of literature, then I need to provide, to the best of my ability, examples or authors that the submitting scholar should read. After all, if I know enough about the topic to notice the discussion is missing an entire area of research, then I am familiar with at least one prominent scholar or one well-written article in the discipline.

Anyway, that’s what I would want from feedback on a paper I submit for publication.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s