Data Analytics

When Thinking About Data, What Keeps You Up at Night?

• By Suzanne Carbonaro
• 07/09/24

With great technology comes great responsibility.

The proliferation of technology in education means we have more data about how, what and if students are learning than ever before. The question is, how do we ensure that data gets into the hands of the people who can use it to improve teaching and learning, without invading a student or educator's privacy?

What we hear from the 1EdTech community is that, ideally, educational institutions want their data centralized, as opposed to a patchwork of homegrown tools and supplier solutions. They want to access their data in one place, when they need it, so they can spend time consuming and using the data rather than formatting it to make it actionable. This also gives them more time to collaborate internally and externally with stakeholders.

"If we can have the data from our entire portfolio available in a standardized format, it allows us to focus on supporting student success and gaining a better understanding of our learners, rather than making the data interoperable at the campus level," said Steven Williams, principal product manager for UCLA.

These ideas were discussed in several sessions at the recent 2024 1EdTech Learning Impact Conference in Salt Lake City between 1EdTech member suppliers, higher education institutions, and K-12 districts. So, what questions keep both educators and suppliers up at night as they consider ways to make these needs a reality?

How Do Educators Get the Data?

Right now, most ed tech applications capture metadata; but accessing that data, ensuring student privacy, or understanding how that data is stored is often unclear. Some of these applications provide educators with data that could support student success through dashboards. Dashboards can be helpful to an extent, but they don't pull multiple tools' data together in one location. So, if an educator wants a full picture of a student, they need to log into multiple tools to get the various data points available. And that data is likely to export into different formats, making it difficult to consume and use for improvement.

In other cases, institutional leaders leverage data standards, such as those created by 1EdTech or other standards organizations, which cull data together into one standard format, across all applications, so they can review all the data in one place, in one single format. Dashboards allow educators to slice and dice the data, and analyze the data across applications longitudinally. This effective use of data for improvement has led 1EdTech community members to want more from their ed tech suppliers. They need more suppliers to upgrade their tools to meet industry standards, so they can access this data in consumable formats for improvement.

So, What Data Do Educators Need?

As Warren Goetzel, director of academic technology and engagement at Georgia Institute of Technology, points out, "To best understand what data educators need we should be asking them directly. At Georgia Tech we interviewed a wide range of stakeholders including faculty to get a baseline understanding of their needs related to learning data and learning analytics. It is also important to ascertain not just what data they need, but what problems would data help solve."

To that end, ed tech suppliers point out that there is a lot of data available, and not every institution is ready to store all of that data itself. This is the harsh reality; data warehouses and the expertise in maintaining them are expensive, and not every institution has the funds or infrastructure to do so.

If suppliers are going to go through the work and, frankly, the cost of creating data-sharing solutions, they need to know what data educators actually want and are capable of consuming. Until multiple institutions ask for data in the same format, suppliers won't invest in the standards.

The 1EdTech community has started taking steps in this direction with the Caliper Analytics® standard and LTI™ Advantage Data initiative. Caliper allows institutions to better understand and visualize learning activity and product usage and present the information in meaningful ways to students, instructors, and advisors. LTI Advantage Data boils the Caliper standard down to the basics, providing suppliers with an easier starting point for providing the data in smaller bites.

When provided in a standardized form, data can also be used to collaborate across an institution or even with other institutions, breaking down silos while also enabling future technologies to advance using standards.

When Institutions Have the Data, Who Gets to See It?

Perhaps the biggest question is once educational institutions have the data, who has permission to access it?

Data and insights can be valuable, but information about students and instructors should not necessarily be available for everyone to see. There are of course privacy laws, policies, and practices to consider, all of which can vary between states and individual institutions.

Suppliers point out that there can't be an infinite number of roles and permissions for each application. At some point, there will need to be a basic understanding of what roles can access what data and what that data can be used for.

It's an ongoing debate, but both institutional leaders and suppliers agree: It is about the learner and improving their outcomes.

Where Does AI Fit in?

As it seems with every technology-based conversation these days, AI was considered as a possible solution and accelerator for making decisions for improvement. Can AI sort through the data and alert the necessary parties when that data needs to be acted upon? Maybe, but more work needs to be done around trust before we can rely on AI.

1EdTech's community is working together to tackle some of the big AI questions. It has released the 1EdTech AI Preparedness Checklist, which provides guiding prompts for establishing protocols, policies, and best practices for using AI in teaching and learning; and the first part of the TrustEd Apps™ Generative AI Data Rubric, a self-assessment tool that provides a foundation for the ethical, productive, and safe use of gen AI data that can be employed by institutions and ed tech suppliers alike.

Working Toward Solutions

There is no question that data can help us get where we want to go in terms of supporting student success. Data can be used to trigger learning support, inform curricular revisions, and design personalized learning. But data needs to be handled with fidelity, protecting student privacy and security. And data needs to be consumable and accessible to stakeholders in a consistent reliable format, so time can be spent on analyzing the data rather than cleansing and formatting it for consumption. Then there is the data repository consideration: Where do we put all of this data? The solutions are in front of us as long as we can continue to have honest conversations and balance innovation with best practices.

Conversations and collaborations will continue in the 1EdTech community, including the Innovative Leadership Network dedicated to Success and Data Analytics, where members from across the country discuss and design resources to support data collection and use. And, of course, save the date for the 2025 Learning Impact Conference, June 2-5.