How voice computing App aids reading
Conventional reading is currently accessible only to the traditionally literate person. Many school learners face barriers to reading, and consequently learning, due to early literacy difficulties, visual impairment, dyslexia and other special educational needs.
The Council for Scientific and Industrial Research (CSIR) is helping to change this with the development of a user-friendly application (app) that adds and synchronises human-narrated or computer-generated audio to text.
The audio can be added to text that is written in any of the 11 official languages of South Africa. “The app targets both the day-to-day content of educators and the textbooks of publishers. It adds value to the publishing industry by breaking down these barriers to turn non-readers into readers and increase the industry’s addressable market size”, says CSIR voice computing research group leader Karen Calteaux.
The meaning of ISinkwe
The app is named iSinkwe, which is the isiZulu word for a bushbaby. The big eyes and ears of the bushbaby simbolise how iSinkwe empowers school learners to read through a rich audio-visual experience. The app allows users to create more accessible textbooks, class notes and other documents, opening up an exciting, interactive world of reading and learning to everyone.
ISinkwe functions by adding audio, which is either recorded manually by a voice artist or generated automatically by a computer, to a standard Electronic Publication Version Three (EPUB3) document. It automates the synchronisation of the audio to the text of the document at word, sentence and paragraph levels, allowing users to switch between these levels when they read or navigate through the document using audio and highlighting. For clarification, iSinkwe does not translate the content from one language to another, but simply adds audio to documents that are already written in the language of the user’s choice.
“If users only have a Microsoft DOCX or PDF document, they can convert those documents automatically to the standard EPUB3 format using iSinkwe Convert. They simply upload the document to the iSinkwe website, and it will provide them with an EPUB3 version that they can download,” says product leader Georg Schlünz.
How the app works
ISinkwe Synchronise allows users to upload a standard EPUB3 document and, optionally, the pre-existing human-narrated audio files, to the iSinkwe website. When no human-narrated audio is available, computer-generated audio is added instead. It also allows for a combination of human-narrated and computer-generated audio. Users can then download the EPUB3 document with the synchronised text and audio included.
To read any of the EPUB3 documents with synchronised text and audio, users need to install the iSinkwe Read app, which is currently available for Android, with iOS and Windows versions coming soon.
Once the EPUB3 document is added to the library and opened, users can listen to and navigate through the audio on word, sentence and paragraph levels. Text highlighting on these levels follow the audio automatically as it is played. Users can customise the reading experience further by changing the font type, font size and background colour, as well as the playback speed of the audio and associated text scrolling behaviour.
“Bushbaby” app available
ISinkwe is now available for licensing by resellers. The resellers will be able to sell iSinkwe to B2B customers like schools, publishers and accessible format providers who want to integrate iSinkwe into their operations, as well as to B2C customers for private use.
Producers who add the audio pay for the ISinkwe Convert and Synchronise web services. Consumers who read the audio are free to download the branded iSinkwe Read Android app from the Google Play store. B2B customers who want to use iSinkwe Read in their own distribution channels can obtain a white-label version directly from the CSIR.
The CSIR piloted iSinkwe to teachers and learners at Jan Kriel School, Langerug School and Tafelberg School. The engagements with the schools were made possible by the Western Cape Education Department.
“The schools have already requested additional features! The reading assistance provided by the app must further convert text, math and other visuals in images to audio. The app must also give writing assistance in the form of automated dictation. We are excited to include these features in the product roadmap of the next three years,” says Schlünz.
The app in its current form is ready to be licensed by interested resellers. They can contact commercialisation manager Minal Soni at msoni1@csir.co.za.
In the meantime, schools, publishers and others can visit www.isinkwe.com or contact the iSinkwe team at info@isinkwe.com for more information and updates on the licensing progress.
Source: The Council for Scientific and Industrial Research (CSIR) – Issued by CSIR Strategic Communications
