A Proposal for Open Peer Review

[1] Throughout its history, Music Theory Online has leveraged new digital technologies to increase access to high-level scholarship and facilitate discussions of published materials inside the journal itself. (See, for example, the commentaries in Volumes 0.2, 1.1, 13.3, and 16.4.) New technologies and publishing practices have developed in the past few years that can carry these practices even further, in great service to the academic community. One such publishing practice is open peer review, exemplified by publications like the Journal of Digital Humanities, Digital Humanities Now, Digital Humanities This, American History Now, and the book Web Writing: Why and How for Liberal Arts Teaching and Learning (Dougherty et al. 2014). While multiple practices exist that can be considered manifestations of open peer review, open peer review in its fullest sense takes the scholarly discussion that traditionally follows publication—such as the discussion threads contained in the early issues of MTO—and moves it pre-publication, rendering them part of the review process. Open peer review ensures high visibility for the best work, extensive vetting by the scholarly community pre-publication, and a timely publication process, all the while maintaining high standards for peer-reviewed publication.

[2] Music theorists would benefit from having an open peer-review journal, and MTO is best situated to be that journal. MTO is also well situated to experiment with open peer review without committing its entire future to such a model. In this article, I will explain open peer review in more detail by following an example article through the process of review and publication, commenting on the potential benefits of this process along the way. I conclude with a proposal for how MTO might experiment with the open peer-review model in order to gauge its potential for our field more precisely.

Following the Process

[3] One of the first articles to appear in the Journal of Digital Humanities is Trevor Owens’s “Defining Data for Humanists: Text, Artifact, Information or Evidence?” This article began as a chapter for the book, Writing History in the Digital Age, itself an open peer-review project (Nawrotski and Dougherty, 2013). Writing History in the Digital Age was undertaken by co-editors Jack Dougherty (Trinity College, Connecticut) and Kristen Nawrotzki (Pädagogische Hochschule Heidelberg), in collaboration with the University of Michigan Press. This project is available from UMP as a print book, a downloadable ebook, and an open-access web book as part of their series of digitalculturebooks that explore novel publication models like open peer review and simultaneous print and open-access publication. For that project, Owens co-authored a chapter with Frederick W. Gibbs called “The Hermeneutics of Data and Historical Writing.” The first version of this chapter was posted to the project website in the fall of 2011. On this website, readers can comment on the chapter as a whole, or on specific paragraphs in the chapter. The chapter received 35 total comments—12 on the chapter as a whole, and 23 on specific passages. Some of these comments we would recognize as typical peer-review summaries (accept/reject/revise with comments on how best to revise); others were directed at improving specific elements in the chapter.

[4] Owens and Gibbs replied to some comments directly, and ultimately composed a revision of this chapter, which appeared on the project website in the spring of 2012. At this stage in the publication process, the chapter moved from review mode to copyediting mode. The chapter on the website includes a link to a document in Google Drive where any reader can comment (though comments ended up limited to the two authors and two editors), but only the document owners can change the text in response to those comments. The final version of this document was submitted to the publisher for inclusion in the print and electronic book editions (Gibbs and Owens 2013). However, the open peer-review process for Owens’s and Gibbs’s chapter did not end with the publication of Writing History in the Digital Age.

[5] In December 2011, the middle of the open peer-review process for this chapter, Owens wrote on his blog:

We were asked to clarify what we saw as the difference between data and evidence. We will help to clarify this in the paper, but it has also sparked a much longer conversation in my mind that I wanted to share here and invite comments on. As I said, this is too big of a can of worms to fit into that paper, but I wanted to take a few moments to sketch this out and see what others think about it.

[6] What followed in that blog post was the first version of Owens’s essay, “Defining Data for Humanists: Text, Artifact, Information or Evidence?” This essay sparked the interest of digital humanists on the internet, and on February 2, 2012, it was re-posted on Digital Humanities Now, an open-access website that aggregates content on the open web related to the field of digital humanities, or DH. In addition to simply aggregating or re-posting the blog post, DHNow’s editors labeled the post “Editor’s Choice,” meaning that their team of regular editors and temporary “editors-at-large” agree that it is worth particular attention from the DH community. Though DHNow does include some form of peer review (albeit without comments, or even the idea of “submission”), it is not a journal, simply a quality-controlled aggregator that attempts to help DH scholars find good content on the open web. Comments and responses to items posted on DHNow are posted on the original sites (controlled by the authors), and journal publication is left to others.

[7] DHNow is one of several publications produced by the Roy Rosenzweig Center for History and New Media. Another is the Journal of Digital Humanities. As stated on the journal’s website,

The Journal of Digital Humanities selects content from the Editors’ Choice pieces from Digital Humanities Now, which highlights the best scholarship—in whatever form—that drives the field of digital humanities field [sic] forward. The Journal of Digital Humanities provides three additional layers of evaluation, review, and editing to the pieces initially identified by Digital Humanities Now.

[8] Owens’s blog post that appeared in DHNow’s Editor’s Choice stream was picked up by JDH in just such a manner. After a round of revisions, it appeared in Volume 1.1 of JDH in Winter 2011 (Owens 2011)—before the book chapter completed its editorial process. Though this final editing process did not take place on the open web, we can compare the blog post to the version published in JDH and see that most of the changes made during this final review stage were small. The most substantial changes were additions: a new introduction to fit the new medium of publication, a new paragraph engaging existing research not cited in the original, and a conclusion—something often missing from a blog post, but essential to a polished publication.

[9] This open peer-review process produced several unique benefits: (1) It produced two publications in highly respected media instead of one. Having two publications is an obvious benefit for the author, but since the extra publication allowed Owens to address a significant issue that he and Gibbs could not address fully in the original chapter, the extra publication benefits the field as well. (2) The open peer-review process allowed a greater number of scholars to provide refining input to the authors before final publication. (3) It generated a collaborative environment in which to make the chapter and article better. (4) In addition to the advantages of the open review process, both the book and the journal processes included the biggest advantages of traditional publishing: working with peer reviewers, editors, and publishers to ensure high quality content and to make the work available to a wide readership.

[10] Open peer review and its advantages are not exclusive to the field of digital humanities. For instance, Empirical Musicology Review employs a “public peer review” model and publishes commentaries alongside articles in the journal, with the goal of “allowing readers to witness a scholarly conversation.” (See this description of Empirical Musicology Review’s peer-review process.) Open peer review can benefit music theorists, as well, and MTO is well positioned to incorporate it into a viable publication model for music theory. With the advantages of open peer review in mind, I will now outline a proposal for an open peer-review experiment for MTO.

A Proposal for Open Peer Review in MTO

[11] For the past year, the SMT Networking Committee (of which I am a member) has been discussing the possibility of creating a website that will curate content of interest to music theory scholars available on the open web. We have built a prototype for this curation site using open-source content management software called Pligg. Pligg allows a community to register its members, and all registered members can nominate content for inclusion on the site. Once a member submits a link to an online resource and a brief description of the resource, it appears on a page available to members that contains the most recently submitted content. Members can follow the link to the original resource, comment on the original page, comment within Pligg’s members area regarding its merit for inclusion, and vote thumbs-up or thumbs-down on Pligg. Once a submitted resource reaches a certain threshold—five more thumbs up than down, for example—it is automatically moved to the front page, which is viewable by anyone, members and non-members alike. This proposed curation site is based on the model of DHThis, which is similar to DHNow, but with a more transparent and community-driven review process.

[12] Such a curation site would be a valuable tool in itself. As SMT members have moved more of their online discussion and resource sharing off of smt-talk, discussions have become more fragmented. This curation site could serve as a valuable portal to help music scholars find good content and online discussions, a portal that is flexible enough to deal with the changing landscape of online social interaction. As more music scholars are blogging about their research, and especially their teaching, and more music instructors are posting course materials on the open web, this curation site could serve as a valuable portal to good research, research-in-progress, and pedagogical tips and materials. Such a site would also allow, potentially, for a greater diversity of voices within the music theory community, as a small sub-community could easily share their materials. (If that community is active and enthusiastic enough, they could readily help each other’s work reach the main page. And if a community abused the process, algorithms for front-page inclusion could be adjusted or users could be suspended.)

[13] However, just as DHNow is not the final stage in a peer-review process, this music theory curation site need not be the final stage in a peer-review process. I propose that MTO take on a function similar to the Journal of Digital Humanities. MTO’s editors can periodically mine the front page of this curation site, as well as the comment threads, for articles of interest, to find blog posts and essays that are of great interest to the music theory community, that receive significant positive feedback, and that have improved as a result of the feedback received. MTO’s editors can seek out simply the best content, or the best content related to a pre-identifed theme. MTO’s editors (as well as organizers of conferences and symposia) can also identify themes that emerge on the curation site, and use those themes to organize special issues.

[14] Once content from the curation site has been identified for potential inclusion in MTO, I envision one of two primary options for the final stage of review. The first option would follow JDH: accept the article pending revision, and assign a small team of editors and/or referees to work collaboratively, and non-anonymously, with the author to bring the article up to publication standard. The second option would follow something similar to the Writing History in the Digital Age model: accept the article pending revision, and make the article available for public comment pre-publication, so that the author can revise the article based on (good) comments received, followed by a final copyediting pass before publication on MTO. In both cases, the editor would retain the final decision about inclusion in the journal.

A Community Mind-shift

[15] Such a publication model would both produce and require a mind-shift from the music theory community. It would require a greater comfort than our field has traditionally shown with sharing ideas publicly that may not be camera-ready. It would require individuals to overcome a greater technological hurdle than with traditional publishing. Since this open peer-review model would be based around pre-published material on the web, it would generally lead to shorter publications than traditional publishing. Articles that work well for open peer review tend to be shorter, much more targeted, and have a much shorter bibliography than traditional articles, even when they represent the same level of research quality. Lastly, just as authors would need to be more comfortable sharing ideas that may not be camera-ready, reviewers need to be more comfortable sharing their comments, including critical ones, on the open web for such a project to work. Blind peer review protects reviewers as much as authors. Open peer review may be a non-starter for those used to providing their comments anonymously. Regardless, open peer review does require a major shift in thought here, and it may only work in sub-communities (such as the FlipCamp community, or within a department or a school of thought) where a strong collegiality already exists.

[16] Open peer review would also produce a mind-shift among music theory scholars: scholarship would be less about individuals working in isolation, and more about collaboration. That, in turn, means that working and talking openly with other scholars becomes a primary means of participating in scholarly discourse. (That is the spirit behind Empirical Musicology Review’s review process and open access policy.) For my part, that would be a welcome and exciting shift in the way we as music theorists work.

[17] Open peer review has much to offer our scholarly community. While the proposal I have outlined may not be the ultimate destination for SMT or MTO, it is a process that has proven valuable enough to other humanistic disciplines that it is worth attempting in ours. Regardless of the end result, I hope that we as a scholarly society can be among the leaders of those increasing access to good research and teaching materials and facilitating meaningful scholarly discourse about music, both within our society and without.

Kris P. Shaffer
University of Colorado–Boulder
College of Music
301 UCVB
Boulder, CO 80309
kris.shaffer@gmail.com

Works Cited

Dougherty, Jack, Jason B. Jones, Dina Anselmi, and Tennyson O’Donnell, eds. 2014. Web Writing: Why and How for Liberal Arts Teaching & Learning. Ann Arbor, Mich.: University of Michigan Press.

Gibbs, Frederick W. and Trevor J. Owens. 2013. “The Hermeneutics of Data and Historical Writing.” In Writing History in the Digital Age, ed. Kristen Nawrotzki and Jack Dougherty. Ann Arbor, Mich.: University of Michigan Press.

Nawrotzki, Kristen and Jack Dougherty, eds. 2013. Writing History in the Digital Age. Ann Arbor, Mich.: University of Michigan Press.

Owens, Trevor. 2011. “Defining Data for Humanists: Text, Artifact, Information or Evidence?” Journal of Digital Humanities 1, no. 1 (accessed January 10, 2014).

Copyright Statement

Copyright © 2014 by the Society for Music Theory. All rights reserved.

[1] Copyrights for individual items published in Music Theory Online (MTO) are held by their authors. Items appearing in MTO may be saved and stored in electronic or paper form, and may be shared among individuals for purposes of scholarly research or discussion, but may not be republished in any form, electronic or print, without prior, written permission from the author(s), and advance notification of the editors of MTO.

[2] Any redistributed form of items published in MTO must include the following information in a form appropriate to the medium in which the items are to appear:

This item appeared in Music Theory Online in [VOLUME #, ISSUE #] on [DAY/MONTH/YEAR]. It was authored by [FULL NAME, EMAIL ADDRESS], with whose written permission it is reprinted here.

[3] Libraries may archive issues of MTO in electronic or paper form for public access so long as each issue is stored in its entirety, and no access fee is charged. Exceptions to these requirements must be approved in writing by the editors of MTO, who will act in accordance with the decisions of the Society for Music Theory.

This document and all portions thereof are protected by U.S. and international copyright laws. Material contained herein may be copied and/or distributed for research purposes only.