Eye Tracking in Neuropsychological Research of Visual Gnosis Features in Children with Hearing Impairment

The article is devoted to the problem of assessing the state of visual gnosis in children with hearing impairment in order to identify possible mechanisms for compensating for uneven development of mental functions and assessing the possibilities of remedial education. We combined the research potential of the classical neuropsychological approach with the eye-tracking methodology in order to study in detail through eye movements the uneven development of auditory, visual and verbal functions in information processing in children with hearing impairment. The task was to conduct not only a classical neuropsychological assessment of visual gnosis in children with hearing impairment, but also to obtain new data on eye tracking during neuropsychological tests. The study involved cochlearly implanted preschoolers with hearing impairment (N=15) and typically developing preschoolers (N=16) aged 5 to 7 years (M=6.1; SD=0.5). At the first stage, classical neuropsychological tests for visual gnosis were conducted, at the second stage, eye tracking was used to record eye movements during neuropsychological tests by preschoolers. Our study confirms the weakness of visual information processing processes during neuropsychological tests in children with hearing impairment. Analysis of eye movement strategies recorded by the eye tracker during neuropsychological tests allowed us to identify a number of features of visual information processing in children with hearing impairment: chaotic search, changes in the recognition sequence, decreased noticeability (“recognizability”) of images or decreased interest (duration of fixations and viewing), weakness of the holistic strategy and synthesis, fragmentation of perception. The duration of the scanning path, spatial density of fixations, scanning regularity, and scanning direction during image recognition differ. Features of the visual search strategy are observed in the number of switches, fixation time, and patterns of gaze transitions between areas of interest. In children with impairments, when performing tests for visual gnosis, an increase in the time of fixations is recorded, which indicates the cognitive complexity of perception. The combination of weakness in the processing of auditory and visual information leads to diffuseness, incorrectness of the correlation of the image of an object and its verbal designation due to the poverty of the vocabulary of children with hearing impairments.

1. Anosova L.V., Krylova O.V., Shashukova E.A., Levin S.V. Dinamika psikhicheskogo razvitiya detei s sensonevral'noi tugoukhost'yu 4-i stepeni posle kokhlearnoi implantatsii na fone neiroprotektornoi terapii [Dynamics of mental development of children with sensorineural hearing loss of the 4th degree after cochlear implantation against the background of neuroprotective therapy]. Rossiiskaya otorinolaringologiya = Russian Otorhinolaryngology, No. 2 (75), pp. 13–17. (In Russ.)
2. Akhutina T.V., Pylaeva N.M. Diagnostika zritel'no-verbal'nykh funktsii u detei 5–7 let [Diagnostics of visual-verbal functions in children 5–7 years old]. Shkol'nyi psikholog = School psychologist, No. 15 (87), pp. 36–43. (In Russ.)
3. Akhutina T.V., Pylaeva N.M. Diagnostika razvitiya zritel'no-verbal'nykh funktsii: Ucheb. posobie dlya stud. vyssh. ucheb. zavedenii [Diagnostics of the development of visual-verbal functions: Textbook for students of higher educational institutions]. Moscow: Publ. centre “Akademiya”, 2003. 64 p. (In Russ.)
4. Barabanshchikov V.A., Zhegallo A.V. Aitreking: Metody registratsii dvizhenii glaz v psikhologicheskikh issledovaniyakh i praktike [Eye Tracking: Methods of Recording Eye Movements in Psychological Research and Practice]. Moscow: Cogito-centre, 2014. 128 p. (In Russ.)
5. Voprosy vospitaniya slepykh, glukhonemykh i umstvenno otstalykh detei [Issues of raising blind, deaf-mute and mentally retarded children]. L.S. Vygotsky (Eds.). Moscow: Izdanie otdela sotsial'no-pravovoi okhrany nesovershennoletnikh Glavsotsvosa Narkomprosa RSFSR, 1924. 158 p. (In Russ.)
6. Ishankhodzhaeva G.T. Neiropsikhologicheskoe issledovanie kognitivnoi deyatel'nosti u detei s neirosensornoi tugoukhost'yu [Neuropsychological study of cognitive activity in children with sensorineural hearing loss]. Zhurnal nevrologii i neirokhirurgicheskikh issledovanii = Journal of Neurology and Neurosurgical Research, Vol. 4, no. 3, pp. 27–30. (In Russ.)
7. Karaush I.S., Dashieva B.A., Kupriyanova I.E., Stoyanova I.Ya. Osobennosti psikhologicheskoi diagnostiki detei s sensornymi narusheniyami [Features of psychological diagnostics of children with sensory impairments]. Sibirskii psikhologicheskii zhurnal = Siberian psychological journal, No. 51, pp. 132–140. (In Russ.)
8. Kaffemanas R.B. Issledovanie osyazatel'nogo vospriyatiya anomal'nykh detei [Study of tactile perception of abnormal children]. Defektologiya = Defectology, No. 2, pp. 21–28. (In Russ.)
9. Kornienko A.A. Obzor zarubezhnykh issledovanii o vliyanii kokhlearnoi implantatsii na kachestvo zhizni lits s narusheniyami slukha [Review of foreign studies on the impact of cochlear implantation on the quality of life of persons with hearing impairment]. Sovremennaya zarubezhnaya psikhologiya= Journal of Modern Foreign Psychology, Vol. 10, no. 2, pp. 79–85DOI: 10.17759/jmfp.2021100208 (In Russ., abstr. in Engl.)
10. Krivonogova K.D., Razumnikova O.M. Osobennosti kognitivnogo statusa studentov s narusheniem slukha: znachenie dlya inklyuzivnogo obrazovaniya [Specificity of the Cognitive Status of Students with Hearing Impairment: Implications for Inclusive Education]. Klinicheskaya i spetsial'naya psikhologiya = Clinical Psychology and Special Education, Vol. 8, no. 2, pp. 38–52. DOI: 10.17759/cpse.2019080203 (In Russ., abstr. in Engl.)
11. Matveev V.M. Psikhicheskie narusheniya pri defektakh zreniya i slukha [Mental disorders in visual and hearing defects]. Moscow: Meditsina, 1987. 184 p. (In Russ.)
12. Pivnenko V.V., Skorobogatova N.V. Sostoyanie sensorno-pertseptivnykh mekhanizmov grammatikalizatsii u detei s tyazhelymi narusheniyami rechi [The state of sensory-perceptual mechanisms of grammaticalization in children with severe speech disorders]. Spetsial'noe obrazovanie = Special Education, Vol. 53, no. 1, pp. 54–63. DOI: 10.26170/sp19-01-05 (In Russ., abstr. in Engl.)
13. Rozanova T.V. Razvitie pamyati i myshleniya glukhikh detei [Development of memory and thinking in deaf children]. Moscow: Pedagogika, 1978. 232 p. (In Russ.)
14. Senchenko G.V., Lazaryan V.V., Senchenko A.Yu. Neiropsikhologicheskaya korrektsiya pervoklassnikov s diagnozom neirosensornaya poterya slukha, obuchayushchikhsya po sisteme inklyuzivnogo obrazovaniya [Neuropsychological correction of first-graders diagnosed with neurosensory hearing loss, studying in the system of inclusive education]. Russian Journal of Education and Psychology, Vol. 12, no. 2, pp. 95–116. DOI: 10.12731/2658-4034-2021-12-2-95-116 (In Russ., abstr. in Engl.)
15. Skuratova K.A., Shelepin E.Yu., Shelepin K.Yu. Programmnye vozmozhnosti primeneniya metoda aitrekinga v issledovaniyakh zritel'nogo vospriyatiya [Software capabilities for using the eye tracking method in visual perception studies]. Rossiiskii psikhologicheskii zhurnal = Russian Psychological Journal, Vol. 19, no. 4, pp. 173–185. DOI: 10.12731/2658-4034-2021-12-2-95-116 (In Russ., abstr. in Engl.)
16. Khvattsev M.E Shabalin S.N. Osobennosti psikhologii glukhogo shkol'nika [Features of the psychology of a deaf schoolchild]. Moscow: Uchpedgiz,1961. 215 p. (In Russ.)
17. Shipitsyna L.M. Neiropsikhologicheskie aspekty diagnostiki detei v protsesse korrektsionno-razvivayushchego obucheniya [Neuropsychological aspects of diagnosing children in the process of correctional and developmental education]. In: V. Zashchirinskaya (Ed.). Psikhologiya detei s zaderzhkoi psikhicheskogo razvitiya: khrestomatiya = Psychology of children with mental retardation: a textbook. Saint Petersburg: Rech, 2004. Pp. 152–155. (In Russ.)
18. Bavelier D., Dye M.W., Hauser P.C. Do deaf individuals see better? Trends in Cognitive Science, 2006. Vol. 10 (11), pp. 512–518. DOI: 10.1016/j.tics.2006.09.006
19. Becker S., Küchemann S., Klein P. et al. Gaze patterns enhance response prediction: More than correct or incorrect. Physical Review Physics Education Research, 2022. Vol. 18, art. 020107. DOI: 10.1103/PhysRevPhysEducRes.18.020107
20. Chettaoui N., Atia A., Bouhlel M.S. Student performance prediction with eye-gaze data in embodied educational context. Education and Information Technologies, 2023. Vol. 28, pp. 833–855. DOI: 10.1007/s10639-022-11163-9
21. Conway C.M., Pisoni D.B., Kronenberger W. The importance of sound for cognitive sequencing abilities: The auditory scaffolding hypothesis. Current directions in psychological science, 2009. Vol. 18 (5), pp. 275–279. DOI: 10.1111/j.1467-8721.2009.01651.x
22. Conway C.M., Karpicke J., Anaya E.M. et al. Nonverbal cognition in deaf children following cochlear implantation: Motor sequencing disturbances mediate language delays. Developmental Neuropsychology, 2011. Vol. 36 (2), pp. 237–254. DOI:1080/87565641.2010.549869
23. Dye M., Hauser P., Bavelier D. Visual attention in deaf children and adults: Implications for learning environments. In: Marschark, P.C. Hauser (Eds.). Deaf cognition: Foundations and outcomes. New York: Oxford University Press, 2008. Рp. 250–263. DOI: 10.1093/acprof:oso/9780195368673.003.0009
24. Gremp M.A., Deocampo J.A., Walk A.M., Conway C.M. Visual sequential processing and language ability in children who are deaf or hard of hearing. Journal of Child Language, 2019. Vol. 46 (4), pp. 785–799. DOI: 10.1017/S0305000918000569
25. Harris M., Terlektsi E. Reading and spelling abilities of deaf adolescents with cochlear implants and hearing aids. Journal of Deaf Studies and Deaf Education, 2011. Vol. 16 (1), pp. 24–34. DOI: 10.1093/deafed/enq031
26. Li Q., Xu S., Chen Y. et al. Detecting preservice teachers’ visual attention under prediction andnonprediction conditions with eye-tracking technology. Physical Review Physics Education Research, 2022. Vol. 18 (1), art. 010134. DOI:1103/PhysRevPhysEducRes.18.010134
27. Marschark M., Edwards L., Peterson C. et al. Understanding theory of mind in deaf and hearing college students. Journal of Deaf Studies and Deaf Education, 2019. Vol. 24 (2), pp. 104–118. DOI: 10.1093/deafed/eny039
28. Merabet L.B., Pascual-Leone A. Neural reorganization following sensory loss: The opportunity of change. Nature Reviews Neuroscience, 2010. Vol. 11 (1), pp. 44–52. DOI: 10.1038/nrn2758
29. Monroy C., Chen C.H., Houston D., Yu C. Action prediction during real-time parent-infant interactions. Developmental Science, 2021. Vol. 24 (3), art. e13042. DOI: 10.1111/desc.13042
30. Sarant J.Z., Harris D.C., Benneta L.A. Academic outcomes for school-aged children with severe–profound hearing loss and early unilateral and bilateral cochlear implants. Journal of Speech, Language, and Hearing Research, 2015. Vol. 58 (3), pp. 1017–1032. DOI: 10.1044/2015_JSLHR-H-14-0075
31. Sharma K., Giannakos M., Dillenbourg P. Eye-tracking and artificial intelligence to enhance motivation and learning. Smart Learning Environments, 2020. Vol. 7 (1), pp. 1–13. DOI: 10.1186/s40561-020-00122-x
32. Yurkovic-Harding J., Lisandrelli G. et al. Children with ASD establish joint attention during free-flowing toy play without face looks. Current biology: CB, 2022. Vol. 32 (12), pp. 2739–2746. DOI: 10.1016/j.cub.2022.04.044