Carnegie Mellon Students Debut Ed Software for Deaf and Hard of Hearing

Carnegie Mellon University students have launched two new, free educational software products designed to help learners who are deaf or hard of hearing. SignBook is an open source program that allows teachers to create custom video dictionaries of local sign language. Speak Up! is a set of games that help children learn how to vocalize sounds. Both sets of programs come out of the university's TechBridgeWorld, which is dedicated to helping people around the world with technology.

SignBook is a platform for teachers to create a local database of animations depicting a sign. It was specifically launched for teachers at Mathru Center's Free School for Hearing Impaired Children in Bangalore, India. Although ample reference material exists for teaching American Sign Language, developer Erik Pintar noted that there are more than 300 other documented sign languages without such supports.
SignBook is an open source program that allows teachers to create custom video dictionaries of local sign language.

This fifth-year senior in human-computer interaction and electrical and computer engineering worked with the Mathru teachers to create a program that would allow them to make a custom video dictionary of the local sign language, Kannada, in order to support new teachers and staff in learning the language too.

SignBook includes the ability to capture sign videos and relevant pictures for each entry and categorize entries by topic and is usable by any teacher who wants to create a dictionary for any sign language.

Speak Up!, created by a student team, is a series of games intended to motivate students to make sounds and to give them visual feedback on the volume and pitch of the sounds. The idea is to prevent teachers working with students in grades 1-4 from having to make loud sounds themselves as prompts for their students, which can be hard on vocal cords.

As explained by Amal Nanavati, a sophomore computer science major who worked on the team, a game might cause a picture to appear and move on the screen if the child successfully makes a noise. In one game, the sound will cause a rickshaw to move; to keep it climbing on a hill, the student has to sustain the sound. Another game causes a bird to fly at different altitudes, depending on the pitch of the sound. In a third, a fish swims up or down depending on volume.

"Teachers would use these in their classes to keep students more engaged," said Nanavati.

Speak Up! includes 10 games that can be used by teachers with little or no speech training and that can be adapted to fit a given curriculum or the local culture.

Both programs were created during the summer by Carnegie Mellon students participating in TechBridgeWorld's Student Technology ExPerience (iSTEP). iSTEP allows the students to conduct technology-oriented research abroad to fill unmet needs in the communities where they work.

"By providing teachers with the tools to create their own signs and come up with their own games, these solutions have the potential to be relevant for different communities and their unique needs," said M. Bernardine Dias, associate research professor of robotics and TechBridgeWorld founder and director. "By releasing the software open source, our hope is that other educators will find our tools useful and that the open source community will build on the work we started."

About the Author

Dian Schaffhauser is a former senior contributing editor for 1105 Media's education publications THE Journal, Campus Technology and Spaces4Learning.

Featured

  • simplified, abstract illustration focusing on the negative side of generative AI misuse, balancing the concepts of cybersecurity and human impact

    Researchers Provide Breakdown of Generative AI Misuse

    In an effort to clarify the potential risks of GenAI and provide "a concrete understanding of how GenAI models are specifically exploited or abused in practice, including the tactics employed to inflict harm," a group of researchers from Google DeepMind, Jigsaw, and Google.org recently published a paper entitled, "Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data."

  • clock with gears and digital circuits inside

    Report Pegs Cost of AI at Nearly $300K Per Minute

    A new report from cloud-based data/BI specialist Domo provides a staggering estimate of the minute-by-minute impact of today's generative AI boom.

  • stylized illustration of a college administrator lying awake in a cozy bed, looking thoughtful

    When Thinking About Data, What Keeps You Up at Night?

    The proliferation of technology in education means we have more data about how, what and if students are learning than ever before. The question is, how do we ensure that data gets into the hands of the people who can use it to improve teaching and learning, without invading a student or educator's privacy?

  • a glowing gaming controller, a digital tree structure, and an open book

    Report: Use of Game Engines Expands Beyond Gaming

    Game development technology is increasingly being utilized beyond its traditional gaming roots, according to the recently released annual "State of Game Development" report from development and DevOps solutions provider Perforce Software.