Oculomotor activity parameters of the operator in the P300 brain–computer interface with variating stimulus situations

We tested the hypotheses about the correlation of visual environment properties in the BCI P300 with oculomotor activity and operator efficiency. We varied level of stimulus intensification and the frame surrounding the stimulus elements. So we had four situation: 1) low contrast, without frame; 2) low contrast, with frame; 3) high contrast, without frame; 4) high contrast, with frame. 12 subjects participated. Our study showed that visual environment which provides lowest level of operator’s errors and so the highest efficiency of the BCI P300 workflow combined with lowest fixation dispersion and highest fixation duration. However, various subjects demonstrated the highest level of the efficiency at the different visual environments. We did not define the best type of the visual environment for the most efficient BCI P300 workflow. This results demonstrate the opportunity to use the eyetracking for optimization visual environment of the BCI P300 for most efficient and comfort operator’s workflow. The study was funded by RFH, grant 15-36-01386 “Consistent pattern of organization oculomotor activity in an environ- ment of brain-computer interface”.

1. Barabanschikov V.A. Okulomotomye struktury vospriyatiya [Oculomotor structures of the perception]. Mos- cow, Institute of psychology RAS Publ., 1997. 383 p. (In Russ.).
2. Barabanschikov V.A., Zhegallo A.V. Aitreking: metody registratsii dvizhenii glaz v psikhologicheskikh issle- dovaniyakh i praktike [Eyetracking: registration methods for eye movements in psychological studies and prac- tice]. Moscow, Cogito-Centr Publ., 2014. 128 p. (In Russ.).
3. Barabanschikov V.A., Zhegallo A.V. Registratsiya i analiz napravlennosti vzora cheloveka [Registration and analysis of the human gaze]. Moscow, Institute of psychology RAS Publ., 2013. 323 p. (In Russ.).
4. Basyul I.A. Elektroehncefalograficheskie pokazateli i okulomotornaya aktivnost’ pri rabote v interfejse mozg–komp’yuter na volne R300 [EEG characteristics and oculomotor activity in the BCI-P300]. In Bara- banschikov V.A. (ed.), Procedury i metody ehksperimental’no-psihologicheskih issledovanij [The procedures and methods of the experimental psychological studies]. Moscow, Institute of Psychology RAS Publ., 2016, pp. 438–443 (In Russ.).
5. Basyul I.A., Kaplan A.Ya. Izmeneniya N200 i P300 komponentov potentsialov, svyazannykh s sobyti- yami, pri var’irovanii uslovii vnimaniya v sisteme Brain Computer Interface [Changes in the N200 and P300 Components of Event-Related Potentials on Variations in the Conditions of Attention in a Brain- Computer Interface System]. Zh Vyssh  New Deiat  IP Pavlova,  Moscow, 2014, no. 2 (64), pp. 159–166 (In Russ., abstract in Engl.).
6. Bianchi L., Sami S., Hikkerbrand A., Fawcett I.P., Quitadamo L.R, Seri S. Which physiological compo- nents are more suitable for visual ERP based brain-computer interface? A preliminary MEG/EEG study. Brain Topogr, 2010, no. 23, pp. 180–185. doi: 10.1007/sl0548-010-0143-0
7. Blankertz B., Tangermann M., Vidaurre C., Fazli S., Sannelli C., Haufe S., Maeder C., Ramsey L., Sturm I., Curio G., Muller K.R. The Berlin Brain-Computer Interface: Non-Medical Uses of BCI Technology. Front Neurosci, 2010, no. 4, p. 198. doi: 10.3389/fnins.2010.00198
8. Brunner P., Joshi S., Briskin S., Wolpaw J.R., Bischof H., and Schalk G. Does the “P300” Speller De- pend on Eye Gaze? J Neural Eng, 2010, vol. 7, no. 5, pp. 056013. doi: 10.1088/1741-2560/7/5/056013
9. Cipresso R, Meriggi P., Carelli L., Solca E, Meazzi D., Poletti B., Lule D., Ludolph A.C., Giuseppe R., Silani V. The combined use of Brain Computer Interface and Eye-Tracking technology for cognitive assess- ment in Amyotrophic Lateral Sclerosis. Pervasive Computing Technologies for Healthcare (PewasiveHealth), Dublin, Irland, 23–26 May 2011, pp. 320–324.
10. Donfinguez-Marrinez E., Parise E., Strandvall T., Reid V.M. The Fixation Distance to the Stimulus In- fluences ERP Quality: An EEG and Eye Tracking N400 Study. PLoS ONE. 2015, vol. 10, no. 7, pp. e0134339. doi:   10.1371/journal.pone.0134339
11. Farwell L.A., Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event- related brain potentials. EEG a. Clin. Neurophysiol, 1988, no. 70, pp. 510–523.
12. Frisoli A., Loconsole C., Leonardis D., Banno E, Barsotti M., Chisari C., Bergamasco M. A New Gaze- BCI-Driven Control of an Upper Limb Exoskeleton for Rehabilitation in Real-World Tasks. IEEE Transac- tions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2012, no. 42, pp. 1169–1179.
13. Ganin I.P., Shishkin S.L., Kochetova A.G., Kaplan A.Ya. Interfeis mozg-komp’yuter «па  volne Р300»: issledovanie effekta nomera stimulov v posledovatel’nosti ikh pred”yavleniya [The P300-based brain-computer interface: the effect of the stimulus position in a stimulus train], Fiziologiya cheloveka [Hu- man Physiology], 2012, no. 38 (2), pp. 5–13 (In Russ., abstract in Engl.).
14. Gneo M., Severini G., Conforto S., Schmid M., D’Alessio T. Towards a brain-activated and eye-con- trolled wheelchair. Inter. J. of Bioelectromagnetism, 2011, vol. 13, no. 1, pp. 44–45. doi: 10.1186/1743-0003-11-7
15. Kaplan A.Ya., Kochetova A.G.,  Shishkin  S.L.,  Basyul  I.A.,  Ganin  I.P.,  Vasil’ev  A.N.,  Liburki- na S.P. Eksperimental’no-teoreticheskie osnovaniya i prakticheskie realizatsii tekhnologii interfeis mozg- komp’yuter [Experimental and theoretical foundations and practical implementation of brain-computer interface technology]. Bulleten Sibirskoy Meditsini [Bulletin  of  Siberian  medicine],  2013,  no.  12  (2), pp. 21–29 (In Russ.)
16. Kaplan A.Ya., Lim J.J., Jin K.S., Park B.W., Byeon J.G., Tarasova S.U. Unconscious operant condi- tioning in the paradigm of brain-computer interface based on color perception. Intern. J. Neurosci, 2005, no. 115, pp. 781–802.
17. Kaplan A.Ya., Shishkin S.L., Ganin I.P., Basyul I.A., Zhigalov A.Y. Adapting the РР300-based brain-computer interface for gaming: a review. IEEE Trans, on Comput. Intelligence and Alin Games, 2013, vol. 5, no. 2, pp. 141–149. doi: 10.1109/TCIAIG.2012.2237517
18. Kaufmann T., Hammer E. M., Kubler A. ERPs Contributing to Classification in the “P300” BCI. Pro- ceedings of the Fifth International BCI Conference, Graz, Austria, 22-24 September 2011, pp. 136–139.
19. Kim B.H., Kim M., Jo S. Quadcopter flight control using a low-cost hybrid interface with EEG- based classification and eye tracking. Computers in Biology and Medicine, 2014, vol. 51, pp. 82–92. doi:   10.1016/j.compbiomed.2014.04.020
20. Kleih S.C., Kaufmann T., Zickler C., Haider S., Leotta E, Cincotti E, Aloise E, Riccio A., Herbert C., Mattia D., Kubler A. Out of the frying pan into the fire – the Р300-based BCI faces real-world challenges. Prog. Brain Res, 2011, vol. 194, pp. 27–46. doi: 10.1016/B978-0-444-53815-4.00019-4
21. Krusienski D.J., Sellers E.W., McFarland D.J., Vaughan T.M., Wolpaw J.R. Toward enhanced P300 spel- ler performance. J Neurosci. Methods, 2008, vol. 167, pp. 15–21. doi: 10.1016/j.jneumeth.2007.07.017
22. Lee E.C., Woo J.C., Kim J.H., Whang M., Park K.R. A brain-computer interface method combined with eye tracking for 3D interaction. J Neurosci Methods, 2010, vol. 190, no. 2, pp. 289–298. doi: 10.1016/j.jneu- meth.2010.05.008
23. Mak J.N, Arbel Y., Minett J.W., McCane L.M., Yuksel B., Ryan D., Thompson D., Bianchi L., Erdog- mus D. Optimizing the P300-based brain-computer interface: current status, limitations and future direc- tions. J Neural Eng, 2011, vol. 8, pp. 025–033. doi: 10.1088/1741-2560/8/2/025003
24. McCullagh P., Galway L., Lightbody G. Investigation into a Mixed Hybrid Using SSVEP and Eye Gaze for Optimising User Interaction within a Virtual Environment. In C. Stephanidis and M. Antona (eds.), UAHCI/HCI, 2013, Part I, LNCS 8009, pp. 530–539. doi: 10.1007/978-3-642-39188-0_57
25. Mikhailova E.S., Chicherov V.A., Ptushenko I.A., Shevelev I.A. Prostranstvennyi gradient volny P300 zritel’nogo vyzvannogo potentsiala mozga cheloveka v modeli neirokomp’yuternogo interfeisa [Spatial Gradient of P300 Area in the Brain-Computer Interface Paradigm], Zh Vyssh Nerv Deiat Im IP Pavlova, 2008, no. 58 (3), pp. 302–308 (In Russ.),
26. Nicolelis M.A. Brain-machine interfaces to restore motor function and probe neural circuits. Nat. Rev. Neurosci, 2003, vol. 4, no. 5, pp. 417–422.
27. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Com- puting, Vienna, Austria, 2015. URL: http://www.R-project.org/.
28. Sellers E.W., Vaughan T.M., Wolpaw J.R. A brain-computer interface for long-term independent home use. Amyotroph. Lateral Scler, 2010, vol. 11, pp. 449–455. doi: 10.3109/17482961003777470
29. Shishkin S.L., Ganin I.P., Basyul I. A., Zhigalov A.Y., Kaplan A.Y. N1 wave in the P300 BCI is not sensi- tive to the physical characteristics of stimuli. J Integr Neurosci, 2009, vol. 8, no. 4, pp. 471–485.
30. Vidal J.J. Real-time detection of brain events in EEG. IEEE Proc, 1977, vol. 65, pp. 633–641. doi: 10.1109/ PROC.  1977.10542
31. Wolpaw J.R., Birbaumer N., McFarland D.J., Pfurtscheller G., Vaughan T.M. Brain-computer interfac- es for communication and control. Clin. Neurophysiol, 2002, vol. 113, pp. 767–791.
32. Wolpaw J.R., McFarland D.J., Neat G.W., Fomeris C.A. An EEG-based brain-computer interface for cur- sor control. EEG a. Clin. Neurophysiol, 1991, vol. 78, no. 3, pp. 252–259.
33. Zander T.O, Gaertner M., Kothe C., Vilimek R. Combining Eye Gaze Input with a Brain-Compu- ter Interface for touchless Human-Computer Interaction. International journal of human-computer interac- tion, 2011, vol. 27, no. 1, pp. 38–51. doi: 10.1080/10447318.2011.535752