Perception of Natural and Artificial Dynamic Facial Expressions During Stroboscopic Presentation

In an experimental study, we explored the role of the natural or artificial character of expression and the speed of its exposure in the recognition of emotional facial expressions during stroboscopic presentation. In Series 1, participants identified emotions represented as sequences of frames from a video of a natural facial expression; in Series 2 participants were shown sequences of linear morph images. The exposure speed was varied. The results showed that at any exposure speed, the expressions of happiness and disgust were recognized most accurately. Longer presentation increased the accuracy of assessments of happiness, disgust, and surprise. Expression of surprise, demonstrated as a linear transformation, was recognized more efficiently than frames of natural expression of surprise. Happiness was perceived more accurately on video frames. The accuracy of the disgust recognition did not depend on the type of images. The qualitative nature of the stimuli and the speed of their presentation did not affect the accuracy of sadness recognition. The categorical structure of the perception of expressions was stable in any type of exposed images. The obtained results suggest a qualitative difference in the perception of natural and artificial images of expressions, which can be observed under extreme exposure conditions.

1. Barabanshchikov V.A., Zhegallo A.V., Korolkova O.A. Pertseptivnaya kategorizatsiya vyrazhenii litsa [Perceptual categorization of facial expressions]. Moscow: Kogito-Tsentr Publ., 2016. 360 p. (In Russ.)
2. Barabanshchikov V.A., Korolkova O.A. Vospriyatie ekspressii «zhivogo» litsa [Perception of expressions of a “living” face]. Eksperimental’naya psikhologiya [Experimental Psychology (Russia)], 2020. Vol. 13, no. 3, pp. 55—73. (In Russ.; abstr. in Engl.) DOI:10.17759/exppsy.2020130305
3. Barabanshchikov V.A., Korolkova O.A., Lobodinskaya E.A. Vliyanie mikropauzy na raspoznavanie bazovykh ekspressii pri stroboskopicheskoi ekspozitsii litsa [Influence of micropause on recognition of basic expressions during stroboscopic exposure of the face]. Litso cheloveka v prostranstve obshcheniya [Human face in the space of communication]. Moscow: MIP; Kogito-Tsentr Publ., 2016. P. 339—353. (In Russ.)
4. Barabanshchikov V.A., Korokova O.A., Lobodinskaya E.A. Vospriyatie mikroekspressii litsa v usloviyakh kazhushchegosya dvizheniya i maskirovki [Perception of facial micro-expressions in conditions of apparent movement and masking]. Moscow: Kogito-Tsentr Publ., 2021. 178 p. (In Russ.)
5. Barabanshchikov V.A., Korolkova O.A., Lobodinskaya E.A. Vospriyatie emotsional’nykh ekspressii litsa pri ego maskirovke i kazhushchemsya dvizhenii [Perception of emotional expressions of the face during its disguise and apparent movement]. Eksperimental’naya psikhologiya [Experimental Psychology (Russia)], 2015. Vol. 8, no. 1, pp. 7—27. (In Russ.; abstr. in Engl.)
6. Barabanshchikov V.A., Korolkova O.A., Lobodinskaya E.A. Raspoznavanie emotsii v usloviyakh stupenchatoi stroboskopicheskoi ekspozitsii vyrazhenii litsa [Recognition of emotions in the conditions of stepped stroboscopic exposure of facial expressions]. Eksperimental’naya psikhologiya [Experimental Psychology (Russia)], 2018. Vol. 11, no. 4, pp. 50—69. (In Russ.; abstr. in Engl.) DOI:10.17759/ exppsy.2018110405
7. Korolkova O.A., Lobodinskaya E.A. Induktsiya estestvennykh emotsional’nykh vyrazhenii litsa [Induction of natural emotional facial expressions]. In P.N. Ermakov, V.A. Labunskaya, G.V. Serikov (eds.), Vneshnii oblik v razlichnykh kontekstakh vzaimodeistviya: materialy Vserossiiskoi nauchnoi konferentsii, 28— 30 oktyabrya 2019 goda [Appearance in various contexts of interaction: materials of the All-Russian Scientific Conference, October 28—30, 2019]. Moscow: KREDO Publ., 2019. P. 41—43. (In Russ.)
8. Alves N.T. Recognition of static and dynamic facial expressions: a study review. Estudos de Psicologia (Natal), 2013. Vol. 18, no. 1, pp. 125—130. DOI:10.1590/S1413-294X2013000100020
9. Ambadar Z., Schooler J.W., Cohn J.F. Deciphering the Enigmatic Face: The Importance of Facial Dynamics in Interpreting Subtle Facial Expressions. Psychological Science, 2005. Vol. 16, no. 5, pp. 403— 410. DOI:10.1111/j.0956-7976.2005.01548.x
10. Baltrusaitis T., Zadeh A., Lim Y.C., Morency L.-P. OpenFace 2.0: Facial Behavior Analysis Toolkit. 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Xi’an, China: IEEE, 2018. P. 59—66. DOI:10.1109/FG.2018.00019
11. Bould E., Morris N. Role of motion signals in recognizing subtle facial expressions of emotion. British Journal of Psychology, 2008. Vol. 99, no. 2, pp. 167—189. DOI:10.1348/000712607X206702
12. Bruce V., Young A.W. In the eye of the beholder: the science of face perception. Oxford: Oxford University Press, 1998.
13. Calder A.J., Rhodes G., Johnson M.H., Haxby J. V. The Oxford Handbook of Face Perception. In A.J. Calder, G. Rhodes, M.H. Johnson, J. V Haxby (eds.), Oxford: Oxford University Press, 2011. 916 p.
14. Calvo M.G., Avero P., Fernández-Martín A., Recio G. Recognition thresholds for static and dynamic emotional faces. Emotion, 2016. Vol. 16, no. 8, pp. 1186—1200. DOI:10.1037/emo0000192
15. Cunningham D.W., Wallraven C. The interaction between motion and form in expression recognition. Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization (APGV 2009). New York, New York, USA: ACM Press, 2009. P. 41—44. DOI:10.1145/1620993.1621002
16. Delis I., Chen C., Jack R.E., Garrod O.G.B., Panzeri S., Schyns P.G. Space-by-time manifold representation of dynamic facial expressions for emotion categorization. Journal of Vision, 2016. Vol. 16, no. 8, pp. 1—20. DOI:10.1167/16.8.14
17. Ekman P., Friesen W.V. Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press, 1978.
18. Ekman P., Rosenberg E.L. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Y.: Oxford University Press, 2005. 486 p. DOI:10.1093/ acprof:oso/9780195179644.001.0001
19. Fusar-Poli P., Placentino A., Carletti F., Landi P., Allen P., Surguladze S., Benedetti F., Abbamonte M., Gasparotti R., Barale F. [et al.]. . Functional atlas of emotional faces processing: A voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. Journal of Psychiatry and Neuroscience, 2009. Vol. 34, no. 6, pp. 418—432. DOI:10.1016/S1180-4882(09)50077-7
20. Irtel H. PXLab: The Psychological Experiments Laboratory [online]. Version 2.1.11. Mannheim (Germany): University of Mannheim, 2007.
21. Kamachi M., Bruce V., Mukaida S., Gyoba J., Yoshikawa S., Akamatsu S. Dynamic properties influence the perception of facial expressions. Perception, 2001. Vol. 30, no. 7, pp. 875—887. DOI:10.1068/p3131
22. Kätsyri J., Sams M. The effect of dynamics on identifying basic emotions from synthetic and natural faces. International Journal of Human-Computer Studies, 2008. Vol. 66, no. 4, pp. 233—242. DOI:10.1016/j. ijhcs.2007.10.001
23. Korolkova O.A. The role of temporal inversion in the perception of realistic and morphed dynamic transitions between facial expressions. Vision Research, 2018. Vol. 143, pp. 42—51. DOI:10.1016/j. visres.2017.10.007
24. Krumhuber E.G., Kappas A., Manstead A.S.R. Effects of Dynamic Aspects of Facial Expressions: A Review. Emotion Review, 2013. Vol. 5, no. 1, pp. 41—46. DOI:10.1177/1754073912451349
25. Lander K., Butcher N.L. Recognizing Genuine From Posed Facial Expressions: Exploring the Role of Dynamic Information and Face Familiarity. Frontiers in Psychology, 2020. Vol. 11, pp. 1—6. DOI:10.3389/ FPSYG.2020.01378/BIBTEX
26. Langner O., Dotsch R., Bijlstra G., Wigboldus D.H.J., Hawk S.T., Knippenberg A. van. Presentation and validation of the Radboud Faces Database. Cognition & Emotion, 2010. 24, no. 8, pp. 1377—1388. DOI:10.1080/02699930903485076
27. Namba S., Makihara S., Kabir R.S., Miyatani M., Nakao T. Spontaneous Facial Expressions Are Different from Posed Facial Expressions: Morphological Properties and Dynamic Sequences. Current Psychology, 2017. Vol. 36, no. 3, pp. 593—605. DOI:10.1007/s12144-016-9448-9
28. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria., 2020.
29. Recio G., Schacht A., Sommer W. Classification of dynamic facial expressions of emotion presented briefly. Cognition & Emotion, 2013. Vol. 27, no. 8, pp. 1486—1494. DOI:10.1080/02699931.2013.794128
30. Sowden S., Schuster B.A., Keating C.T., Fraser D.S., Cook J.L. The role of movement kinematics in facial emotion expression production and recognition. Emotion, 2021. DOI:10.1037/emo0000835
31. The Psychology of Facial Expression. In J.A. Russell, J.M. Fernández-Dols (ed.), Cambridge: Cambridge University Press, 1997. 415 p. DOI:10.1017/CBO9780511659911
32. Wegrzyn M., Riehle M., Labudda K., Woermann F., Baumgartner F., Pollmann S., Bien C.G., Kissler J. Investigating the brain basis of facial expression perception using multi-voxel pattern analysis. Cortex, 2015. Vol. 69, pp. 131—140. DOI:10.1016/j.cortex.2015.05.003
33. Wingenbach T.S.H., Ashwin C., Brosnan M. Sex differences in facial emotion recognition across varying expression intensity levels from videos. PLOS ONE, 2018. Vol. 13, no. 1, pp. 1—18. DOI:10.1371/journal. pone.0190634
34. Young A.W., Rowland D., Calder A.J., Etcoff N.L., Seth A., Perrett D.I. Facial expression megamix: Tests of dimensional and category accounts of emotion recognition. Cognition, 1997. Vol. 63, no. 3, pp. 271—313.