A Learning Approach for Adaptive Multi-Modal Augmentative and Alternative Communication

Topic Description

Communication is a fundamental human right and a critical determinant of an individual’s overall well-being within a community. Yet, children with complex disabilities due to conditions such as Cerebral Palsy (CP) can struggle to effectively communicate even their essential needs. The challenge of communication associated with young, non-verbal CP children, makes this an area of need. While there has been significant research and development for Augmentative and Alternative Communication (AAC) systems, some users' communication needs re-main unmet. Many children who are unable to verbalize or manipulate technology, still express themselves through idiosyncratic combinations of sounds, motions, expressions, and gestures. While currently only their family members or close carers may understand these highly individualized vocabularies, these patterns open opportunity for a new paradigm of assistive technologies.

The aim of this PhD project is to develop a hybrid adaptive personalised approach based on AI-based learning techniques to support non-verbal CP children. More specifically, the project will build on our A3CP project (http://a3cp.org.uk), and our initial multimodal adaptive mobile communication system that adapts to individual abilities, context, and personal variation (Zisman et al, 2024). The PhD student will investigate extending our initial work to expand the communication abilities of non-verbal CP children based on the identification of new communication patterns from sounds, motions, expressions and gestures and generate new vocabularies and conversations.

Skills Required

Background in software development and/or artificial-intelligence.
Willingness to interact with domain experts (e.g., speech and language, physio, and occupational therapists; neurologists; educators), caregivers, and children.

Background Reading

Zisman, Andrea; Katz, Dmitri; Bennasar, Mohamed; Alrimawi, Faeq; Price, Blaine and Johnston, Anthony (2024).Towards Adaptive Multi-modal Augmentative and Alternative Communication for Children with CP. In: Computers Helping People with Special Needs. 19th International Conference, ICCHP 2024, 8-12 Jul 2024, Linz, Austria, pp. 159–167. DOI: https://doi.org/10.1007/978-3-031-62849-8_20