LMS | Research

Study Ranks Accessibility of Top Learning Management Systems

• By Dian Schaffhauser
• 05/13/13

A University of Illinois at Urbana-Champaign study published in March comparing learning management systems has concluded that the latest crop of learning management systems evaluated have made "significant progress" since the assessments started in 2005. However, one of the report's authors noted, "We believe much more needs to be done."

The review examined four major LMSes: Blackboard Learn version 9.1 Service Pack 6 and 8, Desire2Learn version 10, Moodle version 2.3, and Sakai version 2.8. Each of these organizations runs an interest group to guide improvements in accessibility of their products. They were also part of previous evaluations done by the same group in 2010 and 2012.

The disabilities tested for encompassed visual, mobility, learning, and cognitive. Testing and evaluation categories covered functional accessibility in nine broad areas:

• Testing and evaluation;
• Login, configuration, and compatibility testing;
• Personalization and customization;
• Navigation;
• Forms;
• Help and documentation;
• Common student-facing modules and tools such as announcements and discussions;
• Authoring tools and content creation, such as the gradebook and multimedia content handling; and
• Features unique to each LMS that affect accessibility.

For each category each LMS was given a weighted score that varied from one to five, depending on the relative necessity of the task. For example, the ability for users to interact with the login page and submit necessary credentials with certainty could receive a maximum of five points; the ability for a user to set a global session timeout length was worth a maximum of one point.

The project, which was headed by Hadi Rangin, software development specialist for assistive communication and IT at U Illinois, didn't give a final ranking for the products, preferring to let the scores speak for themselves. For example, Desire2Learn was the top scorer for the four tasks in the login, configuration, and compatibility testing set; and Sakai and Moodle both received the top scores for the six help and documentation tasks. Blackboard was the top contender in none of the categories; however, the testing team noted in its disclaimers that Blackboard's discussion board, as an example, had been updated in a subsequent release to address deficiencies in accessibility.

The evaluation didn't assess Web-based solutions such as Instructure Canvas or Pearson LearningStudio. However, Rangin said that because the evaluation team has developed a set of "well defined functional tasks, and a testing/evaluation methodology...it would be very easy to expand the evaluation to other LMS." He added that his group would be willing to "work with other professionals who know these LMSs and can provide a testing environment for these systems."

While the accessibility of LMS applications is improving, Rangin noted that "we will never likely reach 100 percent accessibility."

The reason? Product innovation.

"LMS vendors and open source developers have been competing to introduce new features to their systems, and they, of course, want to be the first one with their ideas," he said. "While some vendors and open source developers have been trying to design the new features/modules with accessibility in mind, they often fail to get them right in the first attempt; it usually takes a few releases until they get all accessibility features implemented correctly. Furthermore, most new features/modules still depend on legacy code, which makes it difficult to implement some accessibility features without changing the core code." He added that Desire2Learn and Moodle, in particular, "are more proactive and have started to make the necessary changes in the legacy code to enhance the accessibility features of their systems."

"The good news is that we have achieved a respectable degree of public awareness for accessibility on all levels," conclude the authors of the study. "Many IT administrators are now considering accessibility features in their product selection or at least ask about accessibility in the [request for proposal] processes. LMS vendors and open-source developers have realized that by considering accessibility in their design and implementation processes, they can make their products more accessible and usable for everyone, while increasing the likelihood that their products will be more readily adopted."

The complete study is freely available online.