Learning to read is hard work for any kid. For deaf and hard-of-hearing children, it is even more challenging.
Chinese mobile giant Huawei recently launched StorySign, a free mobile app that translates children's books into sign language, so as to enrich family story times and enhance learning experiences for kids with hearing problems.
StorySign "allows parents and children to learn to sign and read together, at their own pace," Huawei said in a statement.
When users open the app and hold the smartphone over the page, an animated girl named Star signs the text on screen in real time as parents and kids flip the pages. And each printed word is highlighted as she goes.
According to the company, the app uses a combination of augmented reality and AI technologies. Image recognition enables StorySign to detect words even the phone is positioned at an angle and optical character recognition (OCR) increases accuracy.
OCR refers to the electronic conversion of images of handwritten or printed text into machine-encoded text.
The app became available to download for free on both Google Play and Huawei's own AppGallery in 10 European markets on Dec. 4. The company did not say whether it would be available on iPhone.
Huawei said the app does not require a Huawei smartphone although it is optimized for its own AI-infused phones, such as the Huawei Mate 20 Pro.
StorySign asks users to choose their preferred sign language variant. So far, it supports British, French, German, Italian, Spanish, Dutch, Portuguese, Irish, Belgian Flemish and Swiss-German sign languages.
Currently, each language only has one book. Where's Spot, a classic children's story published in 1980, is available for English users. Huawei is working with publishing partner Penguin Random House to bring more titles to the app library.
The World Health Organization (WHO) said in March that around 466 million people worldwide have disabling hearing loss, including 34 million children. It noted that children with deafness should be given the opportunity to learn sign language along with their families to minimize the impact of hearing loss on their development and education.
However, few people know sign language outside the deaf community as 90 percent of deaf children are born to hearing parents.
StorySign is an example of AI technologies with the potential to bridge the gap between the hearing and non-hearing.
"We believe AI can make a positive difference to the world," Huawei said in a statement.
StorySign was supported by charities including the European Union of the Deaf and the British Deaf Association.
"We're very hopeful that it will make a significant impact in the deaf community, helping more deaf children learn how to read at the same level as hearing children," said Mark Wheatley, executive director for European Union of the Deaf in a statement. "We also hope the launch of StorySign will support a wider conversation about ensuring equality in every aspect of their lives for deaf people across Europe."
