UC System, Carnegie Mellon Pilot Tool for Sharing Research Methods

Universities hope widespread use of protocols.io will increase research reproducibility.

• By David Raths
• 06/13/19

The University of California system and Carnegie Mellon University are both piloting the use of a platform called protocols.io in an effort to bring down a major barrier to reproducible research: the creation and sharing of detailed methods in published articles. As part of the larger open access movement, the universities hope to facilitate collaborative method development and to increase research reproducibility.

The UC system is at the forefront of open access publishing. For example, its recent and high-profile split with publishing giant Elsevier over costs and default open access publication for UC authors sent shock waves across scholarly publishing. (UC is hosting a Washington, D.C., workshop in August to help other institutions that want to reconsider commercial publisher agreements to support a sustainable open access transformation.)

The California Digital Library in the UC Office of the President recently announced a three-year pilot for unlimited protocols.io subscriptions for each of the 10 UC campuses. The pilot will remove all cost barriers and allow UC researchers to test the use of protocols.io for private collaboration around method development and for use in classrooms. (More than 10 percent of the researchers already registered on protocols.io are from one of the UC campuses.)

John Chodacki is responsible for overseeing the strategic planning, development and operational management of California Digital Library's digital curation group, UC3. He works to ensure that UC3 services meet the emerging needs of the scholarly community, including digital preservation, data management and reuse. In a recent interview, he discussed the goals of the protocols.io pilot.

Chodacki noted that scholarly articles are really just containers for information. "A common piece of that puzzle is the methods and protocols section, which is an explanation of how a researcher did their experiments and how they want to show their work," he said. "When we talk about open access, we are trying to unlock not only the licensing of the content but also the knowledge inside the article. Protocols and methods are one of the main pieces of information we are trying to unlock."

In published articles, the methods sections are usually quite terse. Publishers are looking to cut down on word count, and researchers know that, he said. During the writing and editing process, there is this pressure to keep things short. "Reproducibility is a very complex issue, but one of the main ways we are trying to push that forward from a research perspective is by unlocking the methods and protocols sections of articles and showcasing that, so people can have a better understanding of how the research is done and also create more efficiency when they are reproducing another researcher's methods."

Chodacki said that protocols.io co-founder Lenny Teytelman realized that researchers were focused on publishing activities as defined by publishers. "He took a piece of that, and said if we could publish protocols and methods and design a platform for that use case instead of relying on other platforms, we could move research forward. It is not just about sharing, but also about the speed of research. It became his pet project and he has been working on it ever since."

The protocols.io platform allows researchers to record their methods and the way they ran experiments. "The interface is configured to have parameters and the right kind of controls to have meaningful descriptions, and they are not just human-readable but machine-actionable as well," Chodacki said. "That is a more comprehensive way for people to record methods and share them. You can do that privately within your lab or share them publicly. The whole goal is around public dissemination, but it is very upstream in the research lifecycle, so it makes sense for you to be able to craft those methods privately before you publish them out."

He noted that publishing methods on protocols.io is a free service, but researchers pay to have private collaboration spaces. "In our pilot we are hoping that the ability to have access to that private collaboration space will help drive adoption," Chodacki said.

All content created in protocols.io is licensed CC-BY. Public content in protocols.io is archived in CLOCKSS, and there is a free public API allowing a snapshot of the entire protocols.io public content at any time. PDFs and JSON files of all public protocols are also mirrored in the open and searchable GitHub repository. For private protocols, every researcher can export all of their protocols and records in bulk as PDF or JSON files; there is also integration with Google Drive, Dropbox and Box for single-click export from protocols.io.

Carnegie Mellon put out a statement saying open access is a priority, benefiting researchers in their dual roles as authors and readers — and that protocols.io joins a suite of tools and service offerings at its libraries that support the university's commitment to open access.