Students at the Amrita Robotics Research Lab (ARRL) have developed a “smart glove” called MUDRA that converts Indian-language handwriting gestures into spoken English, helping people with speech disorders communicate more effectively with others. This was achieved by a team of four BTech students – Abhidjit Bhaskarana, Anupa J. Naira, Deepak Rama and Krishna Anantananarayana.
Abhidjit says: “Gloves are much cheaper compared to similar gesture sensing products available today. It took 16 weeks to build the prototype and it cost £ 7,500. Currently, the glove can recognize numbers from 1 to 10 and Indian gestures in sign language that match words such as morning, night, goodbye, thank you, etc. It can identify four different states of each finger and can customize up to 70 gestures. Now the glove is at an advanced stage of the production cycle. We started testing its social feasibility. Preliminary results are encouraging. ” The team intends to conduct field trials after developing custom experiments with all possible conditions and permutations.
During the development of the glove, students faced many challenges. Initially, they intended to use a camera-based device, but it proved cumbersome and expensive. After extensive research, the flexibility sensors have been tested, improved and integrated with the glove to recognize four different positions of each finger. The design of the glove was crucial as rigid fixation on the fingers was required. The range of values was calibrated accurately for each specific finger position, and the rest were filtered.
The movement of the hand created another problem. Although inertial units (IMUs) offered values, they were not accurate due to noise, so filtering techniques were adopted for accuracy. Because distinguishing different orientations and hand movements with just one sensor proved difficult, students developed a new method of assessing condition.
“According to the 2011 census, 12 million Indians have some kind of speech or hearing impairment. They face many problems in society, unable to speak properly or express themselves effectively. They communicate through sign language and hand gestures, which are often ridiculed by others, creating in them social insecurity. Students have developed MUDRA to help bridge this gap, ”said Dr. TSB Sudarshan, head of research at Amrita.
HR Nandi Vardhan, who mentored the students, said: “MUDRA can be reprogrammed for a number of applications in which motion sensor technology plays an important role, such as gaming stations, remote control devices, robotics and the medical industry. It has great potential because of its simplicity and powerful algorithm. ” MUDRA is lightweight and can be worn as a riding glove. It recognizes hand gestures in all directions and angles using flexible resistors, accelerometer and gyroscope. The output is transmitted as speech through built-in speakers. The unique feature of the glove is cost-effectiveness without compromising on quality and efficiency.