A group of University of Washington has developed a software that allows for the first time deaf and hard of hearing Americans to use sign language in a mobile phone. UW engineers have the phones working together this spring, and recently received a National Science Foundation for granting a person of 20 field projects to be launched next year in Seattle.
This is the first time in two-way real-time video communication has been demonstrated over cell phones in the United States. Since the publication of the prototype of a video on YouTube, deaf people across the country have been written on a daily basis.
“A lot of people are excited about this,” said lead researcher Eve Riskin, a UW professor of electrical engineering.
For mobile communications, now deaf people communicate by cell phone using text messages. “But the point is you want to be able to communicate in their native language,” said Riskin. “For the deaf American Sign Language.”
Video is much better than that of text messaging because it is faster and is better at conveying emotion, said Jessica DeWitt, a UW undergraduate in psychology who is deaf and is a partner in the project MobileASL. She said that much of their communication with facial expressions, which are transmitted via video phones.
Low rates of data transmission on mobile networks in the U.S., coupled with limited processing power in mobile devices, have so far prevented real-time streaming video with enough frames per second that could be used to convey the language of signs. Communication on rates of U.S. cellular networks allow about one tenth of the speed of data transmission common in places like Europe and Asia (sign language on mobile phones is already possible in Sweden and Japan).
Even faster than the networks are increasingly common in the United States, there is still a need for phones that operate on systems of the slowest.
“The faster networks are not available around the world,” said PhD student Anna Cavender. “They also cost more. I do not think it is fair for someone deaf to have to pay more for your cell phone from someone the audience.”
The team tried different ways to get into understandable language for signs of low-resolution video. They found that the most important part of the image to transmit a high resolution is around the face. This is not surprising, since the eye tracking studies have shown that people spend most of their time looking for a person in the face while signing.
The current version of MobileASL uses a standard video compression tool to stay on the edge of data transmission. Future versions will incorporate customized tools to get better quality. The team developed a system to transmit to the person’s face and hands in high resolution, and background in low resolution. They are now working on another feature that identifies where people are moving their hands, to reduce battery consumption and processing power when the person was not signed.
The team is using the phones imported from Europe, which alone could find that would be compatible with the software and have a camera and video screen located on the same side of the phone so people can film themselves while watching the screen.
Mobile video sign language will not be widely available until the service is provided through a commercial cell phone manufacturer, said Riskin. The team already has been in talks with a major supplier of cellular network that has expressed interest in the project.
The team includes MobileASL Richard Ladner, a UW professor of computer science and engineering; Sheila Hemami, a professor of electrical engineering at Cornell University; Wobbrock Jacob, an assistant professor at UW’s Information School, and UW graduate student Neva Cherniavsky, Chon and Jaehong Rahul Vanam.
Related posts:
