Eye-Tracking Is a Study of the Features of the Perceptual Activity of Preschool Children with Hearing Impairment When Interacting With Visual Educational Material in the Learning Process

The article is devoted to the analysis of oculomotor activity observed during the perception of visual educational material in children with hearing impairment. The need to revise traditional methods and develop new approaches to the rehabilitation and education of children with hearing impairment is associated with the increasing technological capabilities of cochlear implantation. Previous research demonstrates that children with hearing impairments have visual attentional characteristics during a learning task compared to their hearing peers. It becomes relevant to identify how visual clarity can be built into the process of teaching children with hearing impairment and perceived by them as representing the material being presented. With the help of eye-tracking, it became possible to track the change in the nature of perceptual activity in children with hearing impairment with different forms of visual stimulus educational material. The question is raised about the influence of visual cues on the success of solving problems in children with hearing impairment and how they direct the attention of children with hearing impairment, reducing cognitive load, providing depth of information processing. In order to fix the restructuring of the perceptual activity of preschool children with hearing impairment under the influence of learning influence during the experiment, we monitored the dependence of the specificity of the child's eye movements on the form of visual presentation of educational material. The study sample consisted of 9 cochlear-implanted preschool children with sensorineural hearing loss from 4 to 6 years. Eye movement was recorded using a portable eye tracker in the form factor of Pupil Labs glasses. 4 series of experiments were proposed using the educational task "classification of objects". In 4 series of the experiment, the presence explicit and hidden prompts changed, the black-and-white and color solution of the most stimulating educational material changed. It has been shown that the use of different types of cue stimuli and color scheme leads to a shift in visual attention to the target area, facilitates processing, leads to more intensive processing of stimuli, and reduces the time we need to respond to information occurring in this given space.

1. Andreeva L.V. Surdopedagogika [Surdopedagogy]. Moscow, Izdatel'skij centr «Akademija» Publ., 2005. 576 p. (In Russ.).
2. Bozhovich E.D. Nekotorye voprosy obuchenija po obrazcam [Some questions of teaching by samples]. Voprosy psihologii [Questions of psychology], Vol. 2, pp. 135—139. (In Russ.).
3. Ershova V.V., Popova M.R. Psihologicheskaja adaptacija detej s narushenijami sluha posle kohlearnogo implantirovanija [Psychological adaptation of children with hearing impairments after cochlear implantation]. Vestnik Adygejskogo gosudarstvennogo universiteta. Serija 3: Pedagogika I psihologija [Bulletin of the Adyghe State University. Series 3: Pedagogy and psychology], Vol. 2, p. 178. (In Russ.).
4. Zankov L.V. Nagljadnost' i aktivizacija uchashhihsja v obuchenii [Visibility and activation of students in learning]. Moscow, GUPI MP RSFSR Publ., 1960. 312 p. (In Russ.).
5. Korneev A.A., Ahutina T.V., Matveeva E.Ju. Osobennosti chtenija tret'eklassnikov s raznym urovnem razvitija navyka: analiz dvizhenij glaz [Features of reading third-graders with different levels of skill development: analysis of eye movements]. Vestnik Moskovskogo universiteta. Serija 14. Psihologija [Bulletin of the Moscow University. Series 14. Psychology], Vol. 2, pp. 64—87. (In Russ.).
6. Koroleva I.V. Nauchno-metodologicheskie osnovy reabilitacii rano oglohshih detej posle kohlearnoj implantacii. Chast' 1. «Sluhovoj metod» [Scientific and methodological foundations for the rehabilitation of deaf children after cochlear implantation. Part 1. "Auditory method"]. Rossijskaja otorinolaringologija [Russian otorhinolaryngology], 2011. Vol. 2, pp. 88—98. (In Russ.).
7. Leont'ev A.H. Dejatel'nost'. Soznanie. Lichnost' [Activity. Consciousness. Personality]. Moscow, Politizdat Publ., 1975. 304 p. (In Russ.).
8. Pelymskaja T.V., Shmatko N.D. Formirovanie ustnoj rechi doshkol'nikov s narushennym sluhom [Formation of oral speech of preschool children with impaired hearing]. Moscow, Vlados Publ., 2003. 223 p. (In Russ.).
9. Rozanova T.V. Kliniko-psihologicheskoe izuchenie gluhih detej so slozhnym defektom (Soobshhenie 1) [Clinical and psychological study of deaf children with a complex defect (Message 1)]. Defektologija [Defectology], Vol. 2-3. (In Russ.).
10. Rulenkova L.I. Kak nauchit' gluhogo rebenka slushat' i govorit' na osnove verbotonal'nogo metoda [How to teach a deaf child to listen and speak using the verbo-tonal method]. Moscow, Paradigma Publ., 2010. 91 p. (In Russ.).
11. Sokolovskaja T.A. Osobennosti rechevogo razvitija detej posle kohlearnoj implantacii [Features of speech development of children after cochlear implantation]. Special'noe obrazovanie [Special education], 2013. 2, pp. 91—97. (In Russ.).
12. Solov'eva V.A., Venig S.B., Belyh T.V. Analiz glazodvigatel'noj aktivnosti studentov pri chtenii s jekrana PK . [Analysis of oculomotor activity of students when reading from a PC screen]. Integracija obrazovanija [Integration of education], 2021. Vol. 25, no. 1, pp. 91—109. (In Russ.).
13. Talyzina N.F. Upravlenie processom usvoenija znanij (psihologicheskie osnovy) Izdanie 2-e, dopolnennoe, ispravlennoe [Management of the process of assimilation of knowledge (psychological foundations) Edition 2, supplemented, corrected]. Moscow, Izdatel'stvo Moskovskogo universiteta Publ., 1984. 345 p. (In Russ.).
14. Andrade A., Danish J.A., Maltese A.V. A measurement model of gestures in an embodied learning environment: Accounting for temporal dependencies. Journal of Learning Analytics, № 4(3). P. 18—46.
15. Antonietti A., Colombo B., di Nuzzo C. Metacognition in self-regulated multimedia learning: integrating behavioural, psychophysiological and introspective measures. Learning Media and Technology, 2015. 40, no. 2, pp. 187—209.
16. Becker S., Küchemann S., Klein P. Andreas Lichtenberger, and Jochen Kuhn Gaze patterns enhance response prediction: More than correct or incorrect. Rev. Phys. Educ. Res., 2022. Vol. 18, pp. 020107.
17. Berthold K., Eysink T.H.S., Renkl A. Assisting Self-Explanation Prompts Are More Effective than Open Prompts When Learning with Multiple Representations. Instructional Science, 2009. 37, pp. 345—363.
18. Boucheix J.-M., Lowe R. An eye tracking comparison of external pointing cues and internal continuous cues in learning with complex animations. Learning and Instruction, Vol. 20, pp. 123—135.
19. Chettaoui N., Atia A., Bouhlel M.S. Student Performance Prediction with Eye-Gaze Data in Embodied Educational Context. Educ Inf Technol., Vol. 28, pp. 833—855.
20. Conway C., Pisoni D., Kronenberger W. The Importance of Sound for Cognitive Sequencing Abilities: The Auditory Scaffolding Hypothesis. Current directions in psychological science, Vol. 18, pp. 275—279.
21. Conway C.M., Karpicke J., Anaya E.M., Henning S.C., Kronenberger W.G., Pisoni D.B. Nonverbal cognition in deaf children following cochlear implantation: Motor sequencing disturbances mediate language delays. Developmental Neuropsychology, Vol. 36, pp. 237—254.
22. Conway C.M., Pisoni D.B., Anaya E.M., Karpicke J., Henning S.C. Implicit sequence learning in deaf children with cochlear implants. Developmental Science, 2011. No. 14, pp. 69—82.
23. Day E., Tang H., Kendhammer L., Pienta N. Sequence Analysis: Use of Scanpath Patterns for Analysis of Students. Problem-Solving Strategies, 2018. Pp. 73—97.
24. Dye M., Hauser P., Bavelier D. Visual Attention in Deaf Children and Adults /In M. Marschark & P.C. Hauser (Eds.). Deaf cognition: Foundations and outcomes. Oxford University Press, 2008. Pp. 250—263.
25. Emerson A., Sawyer R., Azevedo R., Lester J. Gaze-enhanced student modeling for game-based learning. In Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, 2018. Pp. 63—72.
26. Engbert R., Kliegl R. Microsaccades uncover the orientation of covert attention. Vision Research, Vol. 43, no. 9, pp. 1035—1045.
27. Fennell C.T., Waxman S.R. What paradox? Referential cues allow for infant use of phonetic detail in word learning. Child Dev., Vol. 81, no. 5, pp. 1376—1383.
28. Glaser M., Schwan S. Explaining pictures: How verbal cues influence processing of pictorial learning material. Journal of Educational Psychology, Vol. 107, no. 4, pp. 1006—1018.
29. Grant E., Spivey M. Eye Movements and Problem Solving: Guiding Attention Guides Thought. Psychological science, Vol. 14, pp. 462—466.
30. Gremp M.A., Deocampo J.A., Walk A.M., Conway C.M. Visual sequential processing and language ability in children who are deaf or hard of hearing. J Child Lang, Vol. 46, no. 4, pp. 785—799.
31. Holšánová J, Holmberg N., Holmqvist K. Reading Information Graphics: The Role of Spatial Contiguity and Dual Attentional Guidance. Applied Cognitive Psychology, Vol. 23, pp. 1215—1226.
32. Jacob R., Karn K. Eye Tracking in Human-Computer Interaction and Usability Research: Ready to Deliver the Promises / In book: The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research. Publisher: Elsevier Science, 2003. Pp. 573—605.
33. Jarodzka H., Holmqvist K., Grube H. Eye tracking in Educational Science: Theoretical frameworks and research agendas. Journal of Eye Movement Research, Vol. 10, no. 1, pp. 1—18.
34. Just M.A., Carpenter P.A. A theory of reading: From eye fixations to comprehension. Psychological Review, Vol. 87, no. 4, pp. 329—354.
35. Klein P.J. Viiri Kuhn J. Visual cues improve students’ understanding of divergence and curl: evidence from eye movements during reading and problem solving. Physical Review Physics Education Research, Vol. 15, no. 1, pp. 010126—010121.
36. Knoors H., Marschark M. Teaching Deaf Learners: Psychological and Developmental. Publisher: Oxford University Press, 304 р.
37. Krejtz I., Nezlek J., Michnicka A., Paweł Holas Rusanowska M., Krejtz I., Michnicka Á., Nezlek J., Holas P. Counting One's Blessings Can Reduce the Impact of Daily Stress. Journal of Happiness Studies, Vol. 17, pp. 25—39.
38. Kruger J.-L., Doherty S. Measuring cognitive load in the presence of educational video: Towards a multimodal methodology. Australasian Journal of Educational Technology, Vol. 32, no. 6, pp. 19—31.
39. Laubrock J., Kliegl R., Rolfs M., Engbert R. When do microsaccades follow spatial attention? Attention, perception & psychophysics, 2010. 72, no. 3, pp. 683—694.
40. Lee-Cultura S., Sharma K., Giannakos M. Children’s play and problem-solving in motion-based learning technologies using a multi-modal mixed methods approach. International Journal of Child-Computer Interaction, Vol. 31, pp. 100355.
41. Li Q., Xu S., Chen Y., Lu C., Zhou S. Detecting preservice teachers’ visual attention under prediction and nonprediction conditions with eye-tracking technology. Rev. Phys. Educ. Res., 2022. Vol. 18, pp. 010134.
42. Mayer R.E. Using multimedia for e-learning. Journal of Computer Assisted Learning, 2017. 33, pp. 403—423.
43. Monroy C., Chen C.H., Houston D., Yu C. Action prediction during real-time parent-infant interactions. Developmental science, Vol. 24, no. 3, pp. e13042.
44. Pecher D., Zeelenberg R., Barsalou L. Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation. Psychonomic bulletin & review, 2004. 11, pp. 164—167.
45. Posner M.I., Snyder C.R., Davidson B.J. Attention and the detection of signals. Journal of Experimental Psychology: General, Vol. 109, no. 2, pp. 160—174.
46. Rasch T., Schnotz W. Interactive and non-interactive pictures in multimedia learning environments: Effects on learning outcomes and learning efficiency. Learning and Instruction - LEARN INSTR, Vol. 19, pp. 411—422.
47. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, Vol. 124(3), pp. 372—422.
48. Ruf V., Küchemann S., Kuhn J., Klein P. Comparison of Written and Spoken Instruction to Foster Coordination between Diagram and Equation in Undergraduate Physics Education". Human Behavior and Emerging Technologies, 2022. 13, pp. 1—13.
49. Sass T.R., Hannaway J., Xu Z., Figlio D.N., Feng L. Value added of teachers in high-poverty schools and lower poverty schools. Journal of Urban Economics, Vol. 72, no. 2—3, pp. 104—122.
50. Scheiter K., Eitel A. The effects of signals on learning from text and diagrams: how looking at diagrams earlier and more frequently improves understanding / In A.K. Goel, M. Jamnik, & N.H. Narayanan (Eds.). Diagrammatic representation and inference. Heidelberg: Springer, 2010. Pp. 264—270.
51. Scheiter K., Gerjets P., Heise E. Distraction during learning with hypermedia: Difficult tasks help to keep task goals on track. Frontiers in psychology, Vol. 5, pp. 268.
52. Schmidt-Weigand F., Scheiter K. The role of spatial descriptions in learning from multimedia. Computers in Human Behavior, Vol. 27(1), pp. 22—28.
53. Sharma K., Giannakos M., Dillenbourg P. Eye-tracking and artificial intelligence to enhance motivation and learning. Smart Learning Environments, Vol. 7, pp. 1—13.
54. Shvarts A., Stepanov A., Chumachenko D. Automatic Detection of Gaze Convergence in Multimodal Collaboration: A Dual Eye-Tracking Technology. The Russian Journal of Cognitive Science, Vol. 5, no. 3, pp. 4—17.
55. Sungkur R.K., Antoaroo M.A., Beeharry A. Eye tracking system for enhanced learning experiences. Educ Inf Technol., Vol. 21, pp. 1785—1806.
56. Van Gog T., Kester L., Nievelstein F., Giesbers B., Paas F. Uncovering cognitive processes: Different techniques that can contribute to cognitive load research and instruction. Computers in Human Behavior, Vol. 25, pp. 325—331.
57. Yokoyama H., Taguchi T., Sugiyama H., Sato H. Committee for the Standardization of Renal Pathological Diagnosis and for Renal Biopsy and Disease Registry in the Japanese Society of Nephrology. Membranous nephropathy in Japan: analysis of the Japan Renal Biopsy Registry (J-RBR). Clin Exp Nephrol., Vol. 16, no. 4, pp. 557—563.
58. Yurkovic-Harding J., Lisandrelli G., Shaffer R. C., Dominick K. C., Pedapati E. V., Erickson C. A., Yu C., Kennedy D. P. Children with ASD establish joint attention during free-flowing toy play without face looks. Current biology: CB, Vol. 32, no. 12, pp. 2739—2746.e4.