Coursera Adds A/B Testing to MOOC Platform

an example of A/B testing in a Coursera course

An example of A/B testing in a Coursera course. Source: Coursera.

Coursera is now allowing for A/B testing on its MOOC platform. This lets instructors and researchers test a single variable to see how it affects some aspect of the course, such as participation. As a Coursera blog article explained, the testing is done by "randomizing enrollees into different versions of the same course" and then seeing what impact the change-up makes on the "learning outputs."

For one recent experiment, for example, an instructor wanted to examine the impact on interaction in the MOOC by gender-based "situational background cues." As Christopher Brooks, director of learning analytics and research in the Office of Digital Education & Innovation and a research assistant professor at the University of Michigan, explained in a video, he teaches a large number of introductory Python and data science courses. He wanted to find out whether different course imagery can cause "a difference in [students'] sense of belonging." A single condition that he tested displayed women working in a data science environment in the background of a lecture, while another used men doing the same.

The variation in conditions influenced the levels of interaction, according to the data found in the clickstream logs and discussion forums, but no difference in retention or completion, which, according to Brooks, was what he and his research team "anticipated based on [the research] literature."

The benefit of tackling this kind of testing in a massive open online course can't be overstated, he said. "On campus if I were going to do A/B testing, I'd generally have to use a couple of different sections and maybe I'd get 100, 150 or even 200 students. But here on the Coursera platform, I get tens of thousands, and these learners are from a global subject pool. They're not just our regular students on campus but they are more representative of a world population."

Other kinds of testing referenced by Coursera include the use of videos vs. slideshows vs. text to determine the optimal content format; the use of formative assessments vs. summative testing to understand the impact on learning outcomes; and the impact of using motivational efforts, such as stating goals upfront, for improving course progression for certain groups of students.

According to Coursera, instructors are conducting "more than a dozen experiments live on the platform today." "By empowering our partner community to experiment, we hope to accelerate the pace of pedagogical innovation on the platform," the organization added.

About the Author

Dian Schaffhauser is a former senior contributing editor for 1105 Media's education publications THE Journal, Campus Technology and Spaces4Learning.

Featured

  • an online form with checkboxes, a shield icon for security, and a lock symbol for privacy, set against a clean, monochromatic background

    Educause HECVAT Vendor Assessment Tool Gets an Upgrade

    Educause has announced HECVAT 4, the latest update to its Higher Education Community Vendor Assessment Toolkit.

  • illustration of a football stadium with helmet on the left and laptop with ed tech icons on the right

    The 2025 NFL Draft and Ed Tech Selection: A Strategic Parallel

    In the fast-evolving landscape of collegiate football, the NFL, and higher education, one might not immediately draw connections between the 2025 NFL Draft and the selection of proper educational technology for a college campus. However, upon closer examination, both processes share striking similarities: a rigorous assessment of needs, long-term strategic impact, talent or tool evaluation, financial considerations, and adaptability to a dynamic future.

  • university building surrounded by icons for AI, checklists, and data governance

    Improving AI Governance for Stronger University Compliance and Innovation

    AI can generate valuable insights for higher education institutions and it can be used to enhance the teaching process itself. The caveat is that this can only be achieved when universities adopt a strategic and proactive set of data and process management policies for their use of AI.

  • DeepSeek on AWS

    AWS Offers DeepSeek-R1 as Fully Managed Serverless Model, Recommends Guardrails

    Amazon Web Services (AWS) has announced the availability of DeepSeek-R1 as a fully managed serverless AI model, enabling developers to build and deploy it without having to manage the underlying infrastructure.