Ebook: Computational Approaches to Assistive Technologies for People with Disabilities

Over a decade of research into computational approaches for assistive technology has resulted in this manuscript. A number of graduate theses successfully completed over the past dozen or so years supervised by Kanlaya Naruedomkul and Nick Cercone form the basis of the chapters in this volume. Most, but not all, of the former students who have contributed to this manuscript were enrolled in either Mathematics or the Institute for Innovative Technology at Mahidol University in Bangkok, Thailand where Professor Naruedomkul teaches. These students include S. Dangsaart, S. Sonamthiang, P. Pattaraintakorn, N. Ditcharoen, W. Wongkia, and W. Supap. A current student contributor, E. Chaowicharat is visiting York University from Mathematics at Mahidol University. One student, L. Narupiyakul was a dual PhD awardee from Dalhousie University and King Mongkut's University if Technology, Thonburi. Another student, O. Poobrasert, completed her PhD at Dalhousie University. All of them have spent at least a year (normally their third year of a PhD program) in Canada with Nick Cercone, either at the University of Waterloo, Dalhousie University or York University where he has spent the last 15 years of his career. Finally, V. Keselj completed his PhD at the University of Waterloo and N. Yakovets and A. Agrawal are current PhD students at York University.

Some applications in the chapters use Thai language examples but the techniques employed are not restricted to any single language. This volume is not a comprehensive treatise on assistive technology nor is it by any means broad. Each chapter is based on the PhD work of a former or current student, suitably updated and presented for interested readers. The software was developed and tested in each case and the results of the evaluations are reported. References are presented chapter by chapter so as to direct the interested reader to more literature on the topic of the chapter.

We voice a special thanks to all of our former students who have contributed to this manuscript. We also would like to give a special thank you to Dr. Barbara Whitmer, the project manager of the Centre for Innovation in Visualization and Data Driven Design (CIVDDD) for her kind proof reading of the manuscript.

Nick Cercone, Kanlaya Naruedomkul

April 2013

Toronto

We explain our use of computational approaches to assistive technology for persons with disabilities and introduce the four parts of this manuscript, including sixteen chapters and 2 appendices. The four parts include: Part I – Introduction and Underpinnings; II – Applications; Part III – Theories and Systems; and Part IV – Conclusions and Future.

We survey a range of assistive technology products for the visually and hearing impaired and for those with mild cognitive disabilities.

The use of machine translation (MT) is becoming much more pervasive and a quality translation is required. Constraint application is one key feature that makes a generated translation candidate in Generate and Repair Machine Translation (GRMT) very close to a perfect translation. Constraints represent syntactic differences between the source language (SL) and the target language (TL). These constraints are used to ameliorate syntactic differences between the SL and that of the TL, to refine the scope of translation choices of each input word, and to complete the syntax of the translation language.

One set of constraints can be applied to languages, e.g., Chinese, Japanese and Thai which share common significant (syntactic) features. Our constraints are used with GRMT to translate back and forth between English and Thai. Examples are provided throughout the paper to illustrate the uses of the constraints. Some constraint applications on Chinese and Japanese are also provided. Other considerations on ordering are presented as well.

A speaker's utterance may convey different meanings to a hearer than what the speaker intended. Such ambiguities can be resolved by emphasizing accents at different positions. In human communication, the utterances are emphasized at a focus part to distinguish the important content and reduce ambiguity in the utterance. In our Focus-to-Emphasize Tone (FET) system, we determine how the speaker's utterances are influenced by focus and speaker's intention. The relationships of focus information, speaker's intention and prosodic phenomena are investigated to recognize the intonation patterns and annotate the sentence with prosodic marks. We propose using the Focus to Emphasize Tone (FET) analysis, which includes: (i) generating the constraints for foci, speaker's intention and prosodic features, (ii) defining the intonation patterns, and (iii) labelling a set of prosodic marks for a sentence. We also design the FET structure to support our analysis and to contain focus, speaker's intention and prosodic components. An implementation of the system is described and the evaluation results on the CMU Communicator (CMU–COM) dataset are presented.

Most deaf persons experience difficulties in mastering their reading skills, and are often faced with comprehension problems in reading text. Thus, for assisting the deaf to overcome such problems and to be able to communicate in a world of spoken languages, we present the Thai Text-Thai Sign Translation Application (T³STA). T³STA is able to translate Thai text into Thai Sign language simply and conveniently anytime, anywhere. Thai Sign language is the language of the deaf in Thailand. In the translation process, the distinction between Thai text and Thai Sign Language in both grammar and vocabulary are concerned in each processing step to ensure the accuracy of translation. It provides the meaning of each word and describes the structure formation and word order of the translated sentence. With T³STA, the deaf are able to enhance their communication ability and to improve their knowledge and learning skills. In our initial experiment, T³STA was implemented to translate sentences/phrases collected from different sources including textbooks, bedtime stories, and the public labels. T³STA was tested and evaluated in terms of translation accuracy and user satisfaction. Evaluation results show translation accuracy acceptable and satisfy users' needs.

We present the SIGNMT, Thai Sign to Thai machine translation system, which is able to translate from Thai sign language into Thai text. In the translation process, SIGNMT takes into account the differences between Thai and Thai sign language in terms of both syntax and semantic to ensure the accuracy of translation. SIGNMT was designed to be not only an automatic interpreter but also a language learning tool. It provides the meaning of each word in both text and image forms which is easy to understand for the deaf. The grammar information and the order of the sentence are presented in order to help the deaf in learning Thai, their second language. SIGNMT was designed to support individual learning. With SIGNMT, deaf students are less dependent on a teacher, have more freedom to experiment with their own language and improve their knowledge and learning skills. The SIGNMT system was tested and evaluated in terms of translation accuracy and user satisfaction. The evaluation results show that the translation accuracy is acceptable and it satisfies the users' needs.

We propose an intelligent accessible mathematics approach, called i-Math for people with visual impairment on reading math expressions in Thai. Math expressions in various forms were investigated for developing a set of rules to correctly read the math expressions. Our proposed approach has revealed it is possible to construct a practically automatic math expressions reading system by suitable technologies of speech processing. The i-Math system prototype was tested to generate the accurate speech (intelligibility and overall speech quality) for a math problem sample set and its usage (user satisfaction) was determined by voluntary participants, both students and teachers. The intelligibility indicated that i-Math can generate the understandable pronunciations for math text. Overall speech quality showed that the utterances read by i-Math are good quality and understandable with slight effort. The user satisfaction questionnaire can be included that the teachers and students had positive perceptions toward the use of i-Math.

We proposed the MATHMASTER approach that was designed to help students to gradually develop their skills required in solving math word problems. MATHMASTER automatically performs the translation from math word problem into an appropriate equation by taking into account the relationship between natural language and math concepts both in structure and semantics. The MATHMASTER system was implemented based on the MATHMASTER approach to translate word problems collected from many published mathematics practice books and primary school textbooks. MATHMASTER was designed to support individual learning and understanding of word problem solving through step by step instructions. Students can practice word problem solving at their own pace and enhance their word problem solving skill. The MATHMASTER system was tested and evaluated in terms of translation accuracy, usability of MATHMASTER in education, and user satisfaction. The evaluation results show that the translation accuracy is acceptable, MATHMASTER can be used to facilitate student's learning, and it satisfies the users' needs.

Sentence simplification aims to reduce the reading complexity of a sentence by incorporating more accessible vocabulary and sentence structure. In this chapter we examine the process of lexical substitution and particularly the role that word sense disambiguation plays in this task. Most previous work substitutes difficult words using a predefined dictionary. We present the challenges faced during lexical substitution and how it can be improved by disambiguating the word within its context. We provide empirical results which show that our method creates simplifications that significantly reduce the reading difficulty of the input text while maintaining its grammaticality and preserving its meaning.

TTS system initially transforms text input into a sequence of sound symbols. The complexity of Thai requires about several hundred of rules including main and specific rules to derive most pronunciations for the rule-based approach. Our proposed rule-based system is based on the syllable structure analysis. To increase the quality of TTS, an exception dictionary to cover anomalous pronunciations and using rule reference engine to determine sentence structure are included in the system. The phonetic generation and speech generation by mapping table are sequential steps to transform a set of phonetic symbols to speech waveform.

Current methods of assessing dementia of Alzheimer type (DAT) rely on structured interviews, which attempt to capture the complex nature of deficits suffered. One of the most significant areas affected by the disease is the capacity for functional communication as linguistic skills break down. These methods often do not capture the true nature of language deficits in spontaneous speech. This issue is addressed by exploring novel automatic and objective methods for diagnosing patients through analysis of spontaneous speech. We detail several lexical approaches to the problem of detecting and rating DAT. The approaches explored rely on character n-gram-based techniques, which are shown to perform successfully in a different, but related task of automatic authorship attribution. We also explore the correlation of usage frequency of different parts of speech and DAT. We achieve a high 95% accuracy of detecting dementia when compared with a control group, we achieve 70% accuracy in rating dementia in two classes, and 50% accuracy in rating dementia into four classes. These results show that purely computational solutions offer a viable alternative to standard approaches to diagnosing the level of impairment in patients, and they present a significant step forward toward automatic and objective means to identifying early symptoms of DAT in older adults.

There is no commercial software that supports Thai handwriting. Thai handwritten character recognition is needed to convert handwritten text written on mobile and tablet devices into computer encoded text. We propose a novel method that joins three curve signatures. The first signature is the normalized tangent angle function (TAF), which provides rough classification. The other two novel curve signatures are the relative position matrix (RPM), used to compare global curve features, and the straightened tangent angle function (STAF), used to compare the tangent angle along the cumulative unsigned curvature domain. In the recognition process, an input curve is extracted for these three signatures and the similarity against each character in the handwriting templates is measured. Then, the similarity scores are weighted and summed for ranking. Our experiment is done on 48 handwriting samples (44 Thai consonants appear in each set, and there are 4 sets per handwriting). Our methods yield an accuracy of 93.89% for personal handwriting, and 91.33% for general handwriting.

This chapter presents an innovative approach to discover granular learning behavior patterns of students learning interactions with an intelligent tutoring system (ITS). The approach is domain independent and able to manage learning behavior uncertainty. An N-gram analysis is used to model the learning behavior from the learning action streams to obtain regular and irregular learning behavior patterns. Then, the N-gram models are clustered into a hierarchy using a rough set-based map granule. The hierarchical pattern can be used to improve the domain knowledge of an ITS in predicting student's actions, sequencing problems to be solved, and adjusting hint mechanisms.

Survival analysis is a traditional statistical approach to medical data that involves an event of interest and time. Historical data will be the target of the study and statistical models can be inductively derived. These models try to provide statistical explanation to patients and aim to prevent or prolong undesirable events. We provide our view and our experience of using rough set theory, a relatively young but well-known data mining technique, to medical survival data. We focus on widening the technique and model from a deductive approach to an inductive approach, or a combination that is more flexible and efficient.

Multimedia technology, applied to education, typically involves the use of computer assisted instruction in conjunction with other media and is very helpful when incorporated in an intelligent tutoring system. The use of computer animation and colorful graphics may also aid in motivation and attention processes for children. Although the computer cannot replace a teacher in the classroom, when used as supplement, multimedia use in the classroom provides undeniable benefits for both teachers and students. In addition, multimedia is interactive and synthesizes sound, images, and text. Therefore it is also known that multimedia are an excellent technology when using with students with disabilities as an assistive technology. Assistive technology is used by individuals with disabilities in order to perform functions that might be difficult or impossible. It also refers to products, devices, or equipment, whether acquired commercially, modified or customized, used to maintain, increase or improve functional capabilities of individuals with disabilities. Hence, findings from this study emphasize multimedia support systems such as our program extended teachers' reach for hearing impaired students in Thailand. Additionally, study evidence results indicated our program assists students to learn more successfully when they are learning difficult words.

We observe some conceptions and misconceptions about assistive technology with some concluding remarks regarding our use of assistive technology for persons with disabilities. We speculate briefly about the future of assistive technologies.