Big Data Could Pose Unsustainable Challenges to Universities

• By Dian Schaffhauser
• 12/08/21

Big data research operations in higher education could hit a wall. While universities are "meeting many current needs," according to a new research project, big data work is taxing institutional technology, human and financial resources. On many campuses, the infrastructure supporting big data is highly decentralized, running in individual labs, and dependent more on personal interactions than structured and coordinated programs. Considering the value of research for those schools, both financially and as a major ingredient of their brands, the stakes for creating sustainable research infrastructure and practices are high.

That's the conclusion of a new report from Ithaka S+R, which teamed up with librarians from 20 colleges and universities to understand how well schools can support their research efforts now and into the future. Participants interviewed more than 200 faculty members, exploring how researchers work with big data and identifying the challenges they faced.

According to "Big Data Infrastructure at the Crossroads: Support Needs and Challenges for Universities," the challenges are many:

• There's a tension between disciplinary and interdisciplinary mentalities. While big data is primarily an interdisciplinary enterprise, "divergent incentive structures, cultures and unequal access to funding can affect disciplinary participation in big data research projects."
• Managing complex data isn't easy. As the report noted, "the work of acquiring, cleaning, and organizing data is typically the most labor-intensive aspect of big data projects."
• The structure for collaboration often emphasizes "local, lab-based" IT over the centralized IT operations.
• There's confusion about sharing of data, both formal and informal.
• The ethical aspects of big data research are still in flux, creating uncertainty about what the best practices are.
• Researchers favor informal training for those involved in projects over "formal training in big data methods." That leaves "the potential for blind spots" in their research efforts.

The report also offered numerous recommendations useful to university research leaders, libraries, computing centers, IT and information professionals, faculty and staff who engage in big data research, along with the publishers, funders and others with stakes in research infrastructures.

For example, the authors suggested that institutions create protocols for regular assessment of on-campus big data infrastructure, including mapping resources and assembling working groups across IT, libraries, high-performance computing, research offices and other relevant divisions, "to coordinate support services, identify gaps and reduce redundancies." The report also suggested that universities produce a formal catalog of data services and resources for circulation to researchers.

Individual departments were encouraged to hire people who could be embedded into research teams, to provide data science, data management, statistical and computational expertise.

Libraries could create and update guides to datasets that would be of interest to their research communities, perhaps in collaboration with other academic libraries; and also host events for researchers, to enable them to share their work across fields.

"As big data grows, the difficulty of supporting the research mission of universities — already a substantial challenge for administrators — will increase," the authors noted in their conclusion. "Making big data sustainable, if that is possible (its carbon costs are daunting), will require coordinated action by universities, something that is difficult to accomplish at institutions with decentralized bureaucracies and cultures."

The report is openly available on the Ithaka S+R website.