A small, lightweight, arm device designed by a 17-year-old scientist to support communication for deaf and mute people could cost just a fraction of the price of existing technology - and be developed to be even more accurate.
Inspired by his cousin, a young man with cerebral palsy who cannot speak, London-based A-Level pupil Tom Partridge designed and made a device to transform interactions between people with different communication needs.
His cousin’s experience prompted him to think about how important communication is to quality of life, and how advanced technology could be harnessed to help disabled people.
Existing products on the market such as BrightSign – a sign language to text and speech translation glove – have had a huge impact on improving communication. But the cost of these devices (around £2,500) can be prohibitive.
They also rely entirely on finger-movement, so key parts of the grammar of British Sign Language, which can include eyebrow and mouth movements are not translated into speech or text.
Tom wanted to design something smaller, less bulky, cheaper and more user-friendly than a glove: his device keeps virtually the whole hand free, so users can go about daily tasks, using a phone or eating, for example.
He used complex techniques involving muscle censors, machine learning, coding and 3D printing to build the device.
His prototype features two EMG sensors placed along the forearm to detect electrical activity generated by muscles, and measure muscle activity.
A Raspberry Pi Zero 2W (a tiny low-cost computer) running a custom machine learning model allows portable, real-time translation of sign gestures into text and speech.
The program incorporates an autocorrect library called Symspell, which has access to a full dictionary. This allows it to recognise almost any word and correct small errors automatically. As a result, Tom’s testing found that the effective accuracy is nearly 100%.
The design also incorporates an accelerometer and gyrometer to help the model better distinguish between similar signs. Much of the device hardware is stored in a small 3D-printed case on an adjustable wrist strap.
Tom’s project has been awarded a Gold CREST Award from the British Science Association.
The charity’s flagship science education awards scheme aims to empower students to explore science through flexible, hands-on projects. A Gold Award typically takes around 70 hours of work, with students completing it alongside their daily school lessons and homework.
Tom has a lot of ideas for how his design could continue to develop: through integration into smartphones or smartwatches for more portability, whilst also being able to make use of the greater processing power of these devices.
“Expanding the gesture set to cover the full British Sign Language (BSL) vocabulary would allow for more natural, faster conversations,” he says.
He would like the design to ultimately incorporate more facial or torso movement:
This could be achieved by adding extra sensors around the body to capture a wider range of movements, for example, an additional EMG sensor to detect facial muscle activity, enabling the device to account for the full range of contextual cues that BSL relies on.
The device could also be adapted for other use, such as controlling devices through muscle signals for people with mobility impairments.
Having taught himself coding during the COVID-19 lockdown, Tom threw himself into learning new skills for this project, from 3D printing to accessing, reading and understanding academic research papers.
From conception to testing and writing up a report, Tom’s project took seven weeks to complete.
His next step is to study engineering at university, and he has a particular interest in robotics.
Tom says:
By improving the way deaf and mute people can communicate with those who don’t know sign language, we can help make day-to-day situations - like asking for help in public, chatting with friends, or ordering food - much easier.
Better communication tools can also increase access to education and jobs, as people would be more able to express themselves clearly and quickly without needing a translator or writing everything down. When people can understand and connect with each other more easily, it naturally brings them closer together. This project is all about making that possible with affordable and practical technology that could actually work in real life.