Context
Context
Dyslexia is a specific learning disability that is neurobiological in origin, characterized by difficulties with accurate and/or fluent word recognition and poor spelling and decoding abilities. This condition affects a significant portion of the population, making it crucial to develop early and effective screening tools to support children's educational development. Traditional screening methods often fail to engage children fully, leading to inaccurate assessments and delayed intervention.
Dyslexia is a specific learning disability that is neurobiological in origin, characterized by difficulties with accurate and/or fluent word recognition and poor spelling and decoding abilities. This condition affects a significant portion of the population, making it crucial to develop early and effective screening tools to support children's educational development. Traditional screening methods often fail to engage children fully, leading to inaccurate assessments and delayed intervention.
The DyslexiAR project was conceived to address these challenges by leveraging Augmented Reality (AR) technology. The goal was to create an interactive and playful screening tool that could make the assessment process more engaging for children. By integrating Child-Computer Interaction principles, the project aimed to provide a more accurate and enjoyable way to assess learning patterns and cognitive skills in young children.
The DyslexiAR project was conceived to address these challenges by leveraging Augmented Reality (AR) technology. The goal was to create an interactive and playful screening tool that could make the assessment process more engaging for children. By integrating Child-Computer Interaction principles, the project aimed to provide a more accurate and enjoyable way to assess learning patterns and cognitive skills in young children.
Project Objectives
Enhance Engagement: Utilize AR to design interactive games that align with children's preferences and sustain their attention, making screenings enjoyable.
Improve Accessibility: Ensure the tool is easy to use and understand, following Child Computer Interaction guidelines to cater to young users.
Optimize Diagnosis: Integrate cognitive assessment techniques within the AR environment to accurately identify signs of dyslexia early on.
Enhance Engagement: Utilize AR to design interactive games that align with children's preferences and sustain their attention, making screenings enjoyable.
Improve Accessibility: Ensure the tool is easy to use and understand, following Child Computer Interaction guidelines to cater to young users.
Optimize Diagnosis: Integrate cognitive assessment techniques within the AR environment to accurately identify signs of dyslexia early on.
Background
Background
Dyslexia is often diagnosed through various screening tools, and in India, the most prominent of these is the Dyslexia Assessment for Languages of India (DALI). Developed to cater to the multilingual context of Indian schools, DALI is designed to assess reading, writing, and comprehension abilities in children. It is a comprehensive tool that evaluates different aspects of language processing and literacy skills. However, while DALI and similar tools are effective in identifying dyslexia, they often rely on traditional methods of assessment, such as paper-based tests and one-on-one evaluations with a professional.
Dyslexia is often diagnosed through various screening tools, and in India, the most prominent of these is the Dyslexia Assessment for Languages of India (DALI). Developed to cater to the multilingual context of Indian schools, DALI is designed to assess reading, writing, and comprehension abilities in children. It is a comprehensive tool that evaluates different aspects of language processing and literacy skills. However, while DALI and similar tools are effective in identifying dyslexia, they often rely on traditional methods of assessment, such as paper-based tests and one-on-one evaluations with a professional.
These conventional methods, although reliable, have several limitations. First, they can be time-consuming, requiring extended sessions with each child. Additionally, they may induce anxiety in children, having been framed like an examination, potentially leading to results that do not accurately reflect the child’s abilities.
Our goal with this project was to make the screening test fun and easy to conduct. We picked AR as a medium for this to increase immersiveness and enable gamification.
These conventional methods, although reliable, have several limitations. First, they can be time-consuming, requiring extended sessions with each child. Additionally, they may induce anxiety in children, having been framed like an examination, potentially leading to results that do not accurately reflect the child’s abilities.
Our goal with this project was to make the screening test fun and easy to conduct. We picked AR as a medium for this to increase immersiveness and enable gamification.
Methodology
Methodology
Deciding What to Test
The process began with identifying key skills associated with dyslexia that could be effectively tested using AR technology. Our focus was on phonological processing, visual-spatial skills, and motor coordination, which are critical in early childhood development and often present challenges for children with dyslexia. We consulted existing screening methods like the DALI test, and reviewed literature on dyslexia, aiming to pinpoint areas where AR could offer significant advantages. This informed our selection of the specific skills and knowledge areas to target in our games.
The process began with identifying key skills associated with dyslexia that could be effectively tested using AR technology. Our focus was on phonological processing, visual-spatial skills, and motor coordination, which are critical in early childhood development and often present challenges for children with dyslexia. We consulted existing screening methods like the DALI test, and reviewed literature on dyslexia, aiming to pinpoint areas where AR could offer significant advantages. This informed our selection of the specific skills and knowledge areas to target in our games.
Shortlisting the Games
After determining the key areas to test, we brainstormed and designed several game concepts, each intended to engage and assess a particular skill. These concepts were then evaluated based on their potential effectiveness, engagement level, and feasibility within an AR environment. We narrowed down the list to a set of games that were both enjoyable and capable of providing meaningful insights into the children’s abilities. The final selection included games that were interactive, required active participation, and provided immediate feedback, ensuring a robust and immersive screening experience.
After determining the key areas to test, we brainstormed and designed several game concepts, each intended to engage and assess a particular skill. These concepts were then evaluated based on their potential effectiveness, engagement level, and feasibility within an AR environment. We narrowed down the list to a set of games that were both enjoyable and capable of providing meaningful insights into the children’s abilities. The final selection included games that were interactive, required active participation, and provided immediate feedback, ensuring a robust and immersive screening experience.
Games
Games
Spell Check: AR flashcards for pattern recognition and spelling.
Spell Check is an AR flashcard game that tests spelling and pattern recognition skills. Players rearrange flashcards with virtual letters in this interactive game to form correct words. Upon successful arrangement, an audio confirmation plays, reinforcing the learning experience.
The game leverages Unity's advanced collision detection between 3D models, ensuring the words are spelled correctly. This engaging and immersive approach evaluates critical aspects of dyslexia, explicitly focusing on spelling and pattern recognition skills. It assesses their ability to recognize and sequence letters, a fundamental skill often challenged in dyslexia. By integrating interactive and multisensory elements, the game tests and aids in developing essential reading and spelling skills for early literacy.
Spell Check is an AR flashcard game that tests spelling and pattern recognition skills. Players rearrange flashcards with virtual letters in this interactive game to form correct words. Upon successful arrangement, an audio confirmation plays, reinforcing the learning experience.
The game leverages Unity's advanced collision detection between 3D models, ensuring the words are spelled correctly. This engaging and immersive approach evaluates critical aspects of dyslexia, explicitly focusing on spelling and pattern recognition skills. It assesses their ability to recognize and sequence letters, a fundamental skill often challenged in dyslexia. By integrating interactive and multisensory elements, the game tests and aids in developing essential reading and spelling skills for early literacy.
Our earlier prototype would have all the letters be attached to a single Image Target and be dragged by finger. However, this caused a lot of Usability Issues in 3-dimensions leading us to scrap this approach.
Multiple flash cards would be placed, each corresponding to a letter. Unity would check the collisions between each letter, and when a word was correctly formed, auditory cues would indicate success.
Rapid Automatized Naming Test: Timed object identification using AR.
Our earlier prototype would have all the letters be attached to a single Image Target and be dragged by finger. However, this caused a lot of Usability Issues in 3-dimensions leading us to scrap this approach.
Multiple flash cards would be placed, each corresponding to a letter. Unity would check the collisions between each letter, and when a word was correctly formed, auditory cues would indicate success.
Rapid Automatized Naming (RAN) is a process used in dyslexia assessments. It measures how quickly individuals can name aloud a series of familiar items like letters, numbers, colors, or objects. This task assesses the speed of processing, a skill often found to be weaker in individuals with dyslexia.
In our game, scanning a letter like 'C' triggers a 3D model of a cat to appear with a 10-second countdown. This design avoids the anxiety of a reverse countdown. RAN is considered a strong predictor of reading ability, as it reflects the efficiency of accessing and retrieving phonological information, a crucial component in reading development.
Rapid Automatized Naming (RAN) is a process used in dyslexia assessments. It measures how quickly individuals can name aloud a series of familiar items like letters, numbers, colors, or objects. This task assesses the speed of processing, a skill often found to be weaker in individuals with dyslexia.
In our game, scanning a letter like 'C' triggers a 3D model of a cat to appear with a 10-second countdown. This design avoids the anxiety of a reverse countdown. RAN is considered a strong predictor of reading ability, as it reflects the efficiency of accessing and retrieving phonological information, a crucial component in reading development.
The counter is triggered only when the image target is detected. To deal with accidental stutters, we ensured that the timer would only pause when the image target is lost.
Object Identification with Distractors: Selecting the correct object among distractors in AR.
Object Identification, the third game in our AR dyslexia screening tool, provides an immersive way to assess a child's ability to process visual information and their word-object recognition skills. Upon scanning an image target, a 3D model of an object appears, encircled by four words, with only one accurately describing the thing. The others serve as intentional distractors. Accompanied by an audio narration of the object's name, the game invites the child to drag the correct word to the model.
This exercise, enhanced by Unity's ability to detect the right match and provide auditory confirmation, is pivotal in screening for dyslexia. It evaluates the child's selective attention and ability to filter out irrelevant information — essential reading and comprehension skills often found challenging by individuals with dyslexia. This game serves as a diagnostic tool and enriches the child's learning experience, combining technology with essential cognitive skill development in a fun, engaging format.
Object Identification, the third game in our AR dyslexia screening tool, provides an immersive way to assess a child's ability to process visual information and their word-object recognition skills. Upon scanning an image target, a 3D model of an object appears, encircled by four words, with only one accurately describing the thing. The others serve as intentional distractors. Accompanied by an audio narration of the object's name, the game invites the child to drag the correct word to the model.
This exercise, enhanced by Unity's ability to detect the right match and provide auditory confirmation, is pivotal in screening for dyslexia. It evaluates the child's selective attention and ability to filter out irrelevant information — essential reading and comprehension skills often found challenging by individuals with dyslexia. This game serves as a diagnostic tool and enriches the child's learning experience, combining technology with essential cognitive skill development in a fun, engaging format.
When an image target corresponding to a HAT is scanned, a 3D model of a hat surrounded by four words, one of which correctly spells HAT and three that resemble HAT phonologically appear.
The user is then tasked with dragging and colliding the correct word with the model as seen in the picture on the right.
Letter Tracing with Machine Learning Validation: Tracing letters in AR with machine learning accuracy checks.
Letter Tracing is an innovative game in our dyslexia screening toolkit. When the user scans an image target, a 3D model of a letter appears on-screen. The user must trace the letter in the space provided and click submit. Then, using a machine learning model, we check and predict the letter drawn, comparing it to the 3D letter on the screen to show an on-screen prompt showing whether they are right or wrong.
We trained a CNN on the EMNIST/letters dataset for the model and achieved a 91% accuracy. We connected the model to Unity via a WebSocket connection using this library.
This exercise evaluates the child's selective attention and ability to filter out irrelevant information — essential skills in reading and comprehension, often found challenging by individuals with dyslexia. This game not only serves as a diagnostic tool but also enriches the child's learning experience, combining technology with essential cognitive skill development in a fun, engaging format.
Letter Tracing is an innovative game in our dyslexia screening toolkit. When the user scans an image target, a 3D model of a letter appears on-screen. The user must trace the letter in the space provided and click submit. Then, using a machine learning model, we check and predict the letter drawn, comparing it to the 3D letter on the screen to show an on-screen prompt showing whether they are right or wrong.
We trained a CNN on the EMNIST/letters dataset for the model and achieved a 91% accuracy. We connected the model to Unity via a WebSocket connection using this library.
This exercise evaluates the child's selective attention and ability to filter out irrelevant information — essential skills in reading and comprehension, often found challenging by individuals with dyslexia. This game not only serves as a diagnostic tool but also enriches the child's learning experience, combining technology with essential cognitive skill development in a fun, engaging format.
When an image target corresponding to a HAT is scanned, a 3D model of a hat surrounded by four words, one of which correctly spells HAT and three that resemble HAT phonologically appear. The user is then tasked with dragging and colliding the correct word with the model as seen in the picture on the right.
When an image target corresponding to a HAT is scanned, a 3D model of a hat surrounded by four words, one of which correctly spells HAT and three that resemble HAT phonologically appear.
The user is then tasked with dragging and colliding the correct word with the model as seen in the picture on the right.
Support Document
To ensure a comprehensive evaluation, a support document was created alongside the AR games. This document guides guardians, teachers, or parents in assessing children's performance during the games. It includes scoring instructions and detailed indicators for each game, such as recognizing letters, pattern recognition, phonological retrieval, and selective attention. The support document emphasizes considering the child's home and social environment when evaluating their performance.
To ensure a comprehensive evaluation, a support document was created alongside the AR games. This document guides guardians, teachers, or parents in assessing children's performance during the games. It includes scoring instructions and detailed indicators for each game, such as recognizing letters, pattern recognition, phonological retrieval, and selective attention. The support document emphasizes considering the child's home and social environment when evaluating their performance.
User Testing and Future Works
User Testing and Future Works
Although we were unable to conduct user testing during the project's timeline, a comprehensive testing plan has been developed to ensure that the DyslexiAR tool meets the needs of its intended users effectively. This plan outlines the potential users we aim to engage with, the insights we hope to gain from them, and the challenges we may face during the process.
By identifying key user groups, including pre-readers, parents, and teachers, the plan focuses on understanding their unique needs and the difficulties they encounter in supporting children with dyslexia. This feedback will be crucial in refining the tool, enhancing its usability, and ensuring it delivers meaningful outcomes in real-world settings.
Although we were unable to conduct user testing during the project's timeline, a comprehensive testing plan has been developed to ensure that the DyslexiAR tool meets the needs of its intended users effectively. This plan outlines the potential users we aim to engage with, the insights we hope to gain from them, and the challenges we may face during the process.
By identifying key user groups, including pre-readers, parents, and teachers, the plan focuses on understanding their unique needs and the difficulties they encounter in supporting children with dyslexia. This feedback will be crucial in refining the tool, enhancing its usability, and ensuring it delivers meaningful outcomes in real-world settings.
As we move forward, the insights gathered from this testing phase will guide the future iterations of DyslexiAR, allowing us to further tailor the experience to the needs of children with dyslexia and their educators.
As we move forward, the insights gathered from this testing phase will guide the future iterations of DyslexiAR, allowing us to further tailor the experience to the needs of children with dyslexia and their educators.
That's the end of this one.
Wanna see another?
That's the end of this one.
Wanna see another?