You are here

MILE Lab at IISc: Developing technologies to enable the specially abled

A braille embosser, which prints out text in braille pattern, costs a few Lakhs as compared to a regular printer, which costs around a few thousand rupees. Can anything be done so that printed text can be easily digitized and made available in forms useable by people with visual disability? Or, can text be made available in audio format to the visually impaired ? Indeed yes! Medical Intelligence and Language Engineering (MILE) Laboratory at IISc seeks to provide solutions to challenges faced by the specially abled.

Located in the sprawling campus of the Indian Institute of Science, Bangalore, MILE Lab (http://mile.ee.iisc.ernet.in/mile/index.html) is part of the Department of Electrical Engineering (http://www.ee.iisc.ernet.in/). The researchers in the lab work on cutting edge technologies -- like Machine Listening, Medical Image Processing, Document Image Analysis and Recognition, Online Handwriting Recognition, Text to Speech Synthesis in Indian languages and Natural language processing -- to develop solutions for the specially abled. The head of the laboratory, Prof A. G. Ramakrishnan, won the Manthan award (http://manthanaward.org/index.asp) given by the Digital Empowerment Foundation (http://www.livemint.com/Specials/53C6ZuiulkTbXt70TNhxnM/Enabling-the-bli...) in December 2014 for his project Gift of new abilities. This is in recognition of his leading role in developing optical character recognition (OCR) technologies to convert printed text into computer readable and braille format and reaching it to the people.

Though technology has advanced by leaps and bounds in recent years, a significant section of people with visual and speech impairment in India have not benefited by these developments. According to Census of India 2011, the total population of people afflicted with different disabilities is 26 lakh. Of this 26 lakh, visually impaired constitute more than 5 lakh and another 5 lakh is constituted by speech impaired. Some of them are not well versed in English, and some lack the ability to read because of the blindness. Moreover, the printing costs of Braille text is sky high, making it inaccessible to this group of people. Developing accessible technologies to recognize texts in Indian languages (both handwritten and printed) has been the motto of the lab members. The group works with Tamil and Kannada, two widely spoken south Indian languages. Different technologies are used to convert the written text into editable digitized format and also to audio format.

The process of online handwriting recognition (OHWR) may seem simple for the users, but making a machine to recognize the complex character set of Indian languages with the added complexity of different writing styles is no trivial task. There are challenges posed by the language itself; there are technology constraints, interface issues; achieving a convenient and comfortable user experience is by itself an arduous task. As Prof. Ramakrishnan says, a lot of engineering goes into developing these tools: “Unlike English and Hindi, Indian languages like Kannada, Tamil and Telugu are morphologically rich. [They are also] agglutinative or polysynthetic, meaning [they have] a huge number of derived or compound words [formed] by putting together constituents, each of which expresses a single definite meaning”. This complexity is one of the reasons why much work has not been done on Indian languages.

Furthermore, lack of standard handwritten databases for Tamil and Kannada makes the process of 'training' the computer to recognise handwritten Indian texts much more challenging. The team has been collecting data for nearly a decade now, resulting in a database of more than 2 lakh words. More than 10 educational institutions participated in the data collection exercise. Supported by Technology Development in Indian Languages, Department of Information Technology, Prof. Ramakrishnan and his team of researchers embarked on a mission to develop tools for OHWR. Today -- more than a decade after the technology was envisioned -- it can recognize Kannada and Tamil handwriting with about 90 and 95% character level accuracy.

The OCR technology developed by Prof. Ramakrishnan, Mr. Shiva Kumar H R and the team converts printed text in minimal time to digital format (e-text). The main difference between OHWR and OCR is that OHWR detects handwriting, whereas OCR recognizes printed text. Now, as handwriting is a very personal thing, the software needs to be so advanced and trained with many more writing styles so that it can recognize the character irrespective of the handwriting difference.

The output data (digitized form of printed text) of the 'Gift of New Abilities' project can be converted to speech by another tool called Text To Speech or TTS. The software has been developed for Tamil and Kannada, and the demo is available on the laboratory website (http://mile.ee.iisc.ernet.in:8080/tts_demo/). In this process, the printed text is scanned as an image and then converted into e-text by the OCR software. After correction and verification of e-text, the data can be either fed to TTS software for converting it to speech or it is converted into braille codes for printing. The braille coded text is then printed by a braille embosser and finally verified by a visually impaired reader before binding it. A challenging task this team has accomplished is the Indian accent in their text to speech engine. Based on their interaction with a group of visually impaired people working in Bangalore, it was observed that some of them were not able to relate to foreign accent in another TTS software (e-speak) and would prefer Indian accent and toning pattern. Working on improving the speech quality and naturalness is the future task. Dr. T.V Ananthpadmanabha, an expert on speech signal processing, is working together with the MILE lab to make further progress in this domain.

The lab has 4 members who are working exclusively on checking the integrity of the data collected, its detailed annotation for using it as training and testing data, etc. Jerrin and Pratibha, technical project staff members in the lab, mentioned “The process is long term, right from data collection, cleaning and verifying the collected data and then training the engine and finally testing the engine for accuracy”. Currently, the team is also working on automatic form filling of census data in Tamil. This will enable enumerators enter information directly in digital format, and can potentially save a lot of time and energy. During the 2010 census, entering the handwritten data to the servers took more than two years, while the actual data collection lasted six months. The lab is working with Center for Development of Advanced Computing (C-DAC), Pune, to extend this application for other Indian languages as well. In future, the research would be focused on feeding the OHWR output to Text to Speech software which would be of immense use to the people who cannot speak (patients who have throat issues, people who have had laryngectomy, etc.).

Recently, OCR technology has been used by Worth Trust, Chennai, for converting printed Tamil textbooks into computer readable text and Braille format. Over 500 books have been converted with benefit to many visually impaired students. This has proved to be useful for blind students because texbooks were not easily available in Braille. There are many hurdles to be overcome in both OHWR and OCR to reach the highest possible accuracy. For instance, OCR for old printed text books is difficult because of the degradation (lack of visibility) of the text. Personal handwriting differences is definitely the most important challenge OHWR faces.

Prof. Ramakrishnan, though, is not the one to be deterred by these challenges. His next task is to conduct research and develop a tool for multilingual speech recognition, which is still in the nascent stage. “The language we use currently is interspersed with English, Hindi and whichever languages we are comfortable with. I wish to develop a technology which will recognize speech involving simultaneous use of more than one language.”

Both the tools, OHWR/OCR and TTS, developed within the lab have immense market potential. “Though we have met many organizations working with visually impaired, and are ready to share our technology, very few have shown interest. Our real achievement would be when the majority of people with reading difficulties make use of this tool and enjoy its benefits”, adds Prof. Ramakrishnan.

About the MILE lab: Prof. A G Ramakrishnan heads the MILE laboratory, which is housed under Electrical Engineering department of Indian Institute of Science, Bangalore. Seven technical staff and two Ph D students of MILE lab have contributed to the above projects at different levels.

Prof. Ramakrishnan can be reached at ramkiag@ee.iisc.ernet.in