Robot- assisted speech and language therapy for children with hearing impairment

Abstract

Robots become very popular in research on learning, education and rehabilitation especially for children with disabilities. Our paper presents a study of a robot-assisted speech and language therapy session with eleven hearing-impaired children who use cochlear implant(s). The study explores the opportunity of using a humanoid robot for the development of speech, language and listening skills. NAO cannot articulate, therefore, children cannot do lip-reading. We considered that this special characteristic of the robot would provoke the participating children to use actively their listening. Results showed how NAO assisted speech and language therapy sessions with hearing-impaired children who use two different communication modalities – total communication and auditory-oral.

References:

Arendsen, J., Janssen, J. B., Begeer, S.& Stekelenburg F. C. (2010). The use of robots in social behavior tutoring for children with ASD. In: Proceedings of the 28th Annual European Conference on Cognitive Ergonomics (pp. 371-372). New York: ACM.

Bacciu, A., Pasanisi, E., Vincenti, V., Ormitti, F., Di Lella, F., Guida, M., Berghenty, M. & Bacciu, S. (2009). Cochlear implantation in children with cerebral palsy. A preliminary report. International journal of pediatric otorhinolaryngology, 73 (5), 717-721.

Belpaeme, T., Baxter, P., De Greeff, J., Kennedy, J., Read, R., Looije, R., Neerincx, M., Baroni, I. & Zelati, M. C. (2013). Child-robot interaction: Perspectives and challenges. In: International Conference on Social Robotics (pp. 452-459). Cham: Springer.

Cifuentes, C.A., Pinto, M.J., Céspedes, N.& Munera, M. (2020). Social Robots in Therapy and Care. Current Robotics Report. https://doi.org/10.1007/s43154-020-00009-2.

Cooper, H., & Craddock, L. (2006). Cochlear implants: a practical guide. John Wiley & Sons.

Crowe, K., McLeod, S., McKinnon, D. H., & Ching, T. Y. C. (2014). Speech, sign, or multilingualism for children with hearing loss: Quantitative insights into caregivers' decision-making. Language, Speech, and Hearing Services in Schools, 45, 234-247. doi:10.1044/2014_LSHSS-12-0106.

Estabrooks, W., Maclver-Lux, K., & Rhoades, E.A. (2016). Auditory-verbal therapy: For young children with hearing loss and their families, and the practitioners who guide them. San Diego, CA: Plural Publishing.

Estabrooks, W., Morrison, H.M.C. & MacIver-Lux, K. (2020). Auditory-Verbal Therapy: Science, Research, and Practice. Plural Publishing, Incorporated.

Fitzpatrick, E. M., Hamel, C., Stevens, A., Pratt, M., Moher, D., Doucet, S. P., Neuss, D.,.... Na, E. (2016). Sign Language and Spoken Language for Children With Hearing Loss: A Systematic Review. Pediatrics, 137(1), 10.1542/peds.2015-1974. https://doi.org/10.1542/peds.2015-1974.

Flaherty, M. (2015). What We Can Learn from Hearing Parents of Deaf Children. Australasian Journal of Special Education, 39(1), 67-84. doi:10.1017/jse.2014.19.

Hamzah, M. S. J., Shamsuddin, S., Miskam, M. A., Yussof, H., Hashim, K. S. (2014). Development of Interaction Scenarios Based on pre-school Curriculum in Robotic Intervention for Children with Autism. International Conference on Robot PRIDE 2013-2014 - Medical and Rehabilitation Robotics and Instrumentation, ConfPRIDE 2013-2014, Procedia Computer Science, 42, 214–221, Elsevier.

Homburg, D., Thieme, M. S., Völker, J., Stock, R. (2019). RoboTalk – Prototyping a Humanoid Robot as Speech-to-Sign Language Translator. Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 118839, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).

Hummes, L. E, Bess, F. H. (2008). Audiology and Communication Disorders: An Overview. Lippincott Williams & Wilkins.

Ioannou, A., & Andreeva, A. (2019). Play and Learn with an Intelligent Robot: Enhancing the Therapy of Hearing-Impaired Children. In: IFIP Conference on Human-Computer Interaction, (pp. 436-452). Cham: Springer.

Ioannou, A., Kartapanis, I. & Zaphiris, P. (2015). Social Robots as Co-Therapists in Autism Therapy Sessions: A Single-Case Study. In: Social Robotics, vol. 9388 of the series Lecture Notes in Computer Science, 255-263. Springer International Publishing, Switzerland.

Kitani, M., Hara, T., & Sawada, H. (2011). Voice articulatory training with a talking robot for the auditory impaired. International Journal on Disability and Human Development, 10(1), 63-67. https://doi.org/10.1515/IJDHD.2011.012.

Kitani, M., Hayashi, Y., & Sawada, H. (2008). Interactive training of speech articulation for hearing impaired using a talking robot.

Kose-Bagci, H. & Yorganci, R. (2011). Tale of a robot: Humanoid Robot Assisted Sign Language Tutoring. In: 11th IEEE-RAS International Conference on Humanoid Robots, pp. 105-111. IEEE, Bled Slovenia.

Lee, H. W., & Hyun, E. J. (2015). The Intelligent Robot Contents for Children with Speech-Language Disorder. Educational Technology & Society, vol.18, No 3, p. 100–113.

Malik, N. A., Yussof, H., & Hanapiah, F. A. (2014). Development of imitation learning through physical therapy using a humanoid robot. Procedia Computer Science, 42, 191-197.

Marshark, M. & Hauser, P. C. (2012). How Deaf Children Learn: What Parents and Teachers Need to Know. Oxford University Press.

Nalin, M., Baroni, I., Kruijff-Korbayová, I., Cañamero, L., Lewis, M., Beck, A., Cuayáhuitl, H., & Sanna, A. (2012). Children's adaptation in multi-session interaction with a humanoid robot. 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication (pp. 351-357).

Paul, P. V., & Whitelaw, G. M. (2010). Hearing and deafness. Jones & Bartlett Publishers.

Polycarpou, P., Andreeva, A., Ioannou, A., & Zaphiris, P. (2016). Don’t read my lips: assessing listening and speaking skills through play with a humanoid robot. In: International Conference on Human-Computer Interaction (pp. 255-260), Springer, Cham.

Robins, B., Dautenhahn, K., Boekhorst, R., & Billard, A. (2005). Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? In: Universal Access in the Information Society, 4, 105–120. Springer, Heidelberg.

Sawada, H., Kitani, M., & Hayashi, Y. (2008). A robotic voice simulator and the interactive training for hearing-impaired people. BioMed Research International, 1, [768232]. https://doi.org/10.1155/2008/768232.

Srinivasan, S. M., Park, I. K., Neelly, L. B., & Bhat, A. N. (2015). A comparison of the effects of rhythm and robotic Interventions on Repetitive Behaviors and affective States of children with Autism Spectrum Disorder (ASD). Research in Autism Spectrum Disorders, 18, 51 – 63, Elsevier.

Uluer, P., Akalin, N., Kose, H. (2015). A New Robotic Platform for Sign Language Tutoring Humanoid Robots as Assistive Game Companions for Teaching Sign Language. International Journal of Social Robotics, 7, 571–585. Springer, Netherlands.

Vo Nhu, T. & Sawada, H. (2017). Text-to-speech of a talking robot for interactive speech training of hearing impaired. 10th International Conference on Human System Interactions (HSI), Ulsan, 2017, pp. 166-171, doi: 10.1109/HSI.2017.8005021.

Welling, D. R., & Ukstins, C. A. (2017). Fundamentals of Audiology for the Speech-language Pathologist. Jones & Bartlett Learning.

Wood, L. J., Robins, B., Lakatos, G., Syrdal, D. S., Zaraki, A., & Dautenhahn, K. (2019). Developing a protocol and experimental setup for using a humanoid robot to assist children with autism to develop visual perspective taking skills. Paladyn, Journal of Behavioral Robotics, 10(1), 167-179.

World Health Organization (2020). http://www.who.int.

Zhanatkyzy, A. Turarova, A., Telisheva, Z., Abylkasymova, G. & Sandygulova, A. (2019). Robot-Assisted Therapy for Children with Delayed Speech Development: a Pilot Study. 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India, 2019, pp. 1-5, doi: 10.1109/RO-MAN46459.2019.8956257.