DOI: 10.3724/SP.J.1042.2018.00423

Advances in Psychological Science (心理科学进展) 2018/26:3 PP.423-432

A comparison for machine processing of emotional body language and facial expression

Both body language and facial expression recognition provide sensitive clues about individual emotion recognition. Facial expression and bodily expression can be detected to a similar extent in the early stages of visual processing, where reflects the subcortical processing pathway and the dorsal route structures associated with unconscious threat information cues for automated processing. Both body language and facial expression recognition have similar N170 component, which reflect early visual coding in the temporal occipital visual cortex. Nonetheless, the N170 component of body language and facial expression involve different neural processes. Moreover, the N170 component and vertex positive potential (VPP) of facial expression processing are superior to the responses to body language processing. The early posterior negativity (EPN) component reflects visual coding in late attentional processing. The P3 and the late positive component (LPC) represent higher cognitive processing in the fronto-parietal cortices, and facial expression require more cognitive resources than physical actions. Furthermore, there is a special N190 response to body language based in the extrastriate body area (EBA). It is sensitive to emotions expressed by body language and information from physical actions. This special visual coding may represent a social adaptation mechanism effectively for the perception of the emotions and intentions of others. Future research should focus on the influence of emotional body language on the processing of facial expressions, the integration of facial and bodily cues during emotional change perception and others.

Key words:bodily expression,facial expressions,P1,N170,N190

ReleaseDate:2018-03-29 16:57:01

陈丽, 李伟霞, 张烨, 张庆林. (2015). 躯体知觉的认知神经机制. 心理科学, 38(1), 85-92.

丁小斌, 康铁君, 赵鑫. (2017). 情绪识别研究中被"冷落"的线索:躯体表情加工的特点、神经基础及加工机制. 心理科学, 40(5), 1084-1090.

方平, 李洋, 姜媛. (2009). 情绪躯体语言研究进展. 心理科学, 32(5), 1155-1158.

何振宏, 赵婷, 张丹丹. (2016). 正立和倒立的威胁性身体表情的识别. 心理科学, 39(1), 158-165.

王丽丽, 贾丽娜, 罗跃嘉. (2016). 情绪自动化加工的证据与争议. 心理科学进展, 24(8), 1185-1197.

侠牧, 李雪榴, 叶春, 李红. (2014). 面部表情加工的ERP成分. 心理科学进展, 22(10), 1556-1563.

张丹丹, 赵婷, 柳昀哲, 陈玉明. (2015). 恐惧情绪面孔和身体姿势加工的比较:事件相关电位研究. 心理学报, 47(8), 963-970.

Gazzaniga, M. S., Lvry, R. B., & Mangun, G. R. (2013). 认知神经科学:关于心智的生物学 (周晓林, 高定国等 译, 3版). 北京:轻工业出版社.

Adolphs, R. (2002). Recognizing emotion from facial expressions:Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews, 1(1), 21-62.

Ales, J. M., Yates, J. L., & Norcia, A. M. (2010). V1 is not uniquely identified by polarity reversals of responses to upper and lower visual field stimuli. NeuroImage, 52(4), 1401-1409.

Avenanti, A., Annella, L., Candidi, M., Urgesi, C., & Aglioti, S. M. (2013a). Compensatory plasticity in the action observation network:Virtual lesions of STS enhance anticipatory simulation of seen actions. Cerebral Cortex, 23(3), 570-580.

Avenanti, A., Candidi, M., & Urgesi, C. (2013b). Vicarious motor activation during action perception:Beyond correlational evidence. Frontiers in Human Neuroscience, 7(4), 185-192.

Aviezer, H., Trope, Y., & Todorov, A. (2012). Body cues, not facial expressions, discriminate between intense positive and negative emotions. Science, 338(6111), 1225-1229.

Borhani, K., Borgomaneri, S., Làdavas, E. & Bertini, C. (2016). The effect of alexithymia on early visual processing of emotional body postures. Biological Psychology, 115, 1-8.

Borhani, K., Làdavas, E., Maier, M. E., Avenanti, A. & Bertini, C. (2015). Emotional and movement-related body postures modulate visual processing. Social Cognitive and Affective Neuroscience, 10(8), 1092-1101.

Borgomaneri, S., Gazzola, V., & Avenanti, A. (2012). Motor mapping of implied actions during perception of emotional body language. Brain Stimulation, 5(2), 70-76.

Borgomaneri, S., Gazzola, V., & Avenanti, A. (2015). Transcranial magnetic stimulation reveals two functionally distinct stages of motor cortex involvement during perception of emotional body language. Brain Structure and Function, 220(5), 2765-2781.

Borgomaneri, S., Vitale, F., Gazzola, V., & Avenanti, A. (2015). Seeing fearful body language rapidly freezes the observer's motor cortex. Cortex, 65, 232-245.

Calvo, M. G., & Beltrán, D. (2014). Brain lateralization of holistic versus analytic processing of emotional facial expressions. NeuroImage, 92(10), 237-247.

Campanella, S., Quinet, P., Bruyer, R., Crommelinck, M., & Guerit, J.-M. (2002). Categorical perception of happiness and fear facial expressions:An ERP study. Journal of Cognitive Neuroscience, 14(2), 210-227.

Chen, X., Pan, Z., Wang, P., Yang, X., Liu, P., You, X., & Yuan J. (2016). The integration of facial and vocal cues during emotional change perception:EEG markers. Social Cognitive and Affective Neuroscience, 11(7), 1152-1161.

de Gelder, B. (2006). Towards the Neurobiology of Emotional Body Language. Nature Review Neuroscience, 7(3), 242-249.

de Gelder B., & de Borst A. W. (2015). Body perception. In:A. W. Toga (Ed.), Brain mapping:An encyclopedic reference (pp. 107-114). Amsterdam:Elsevier.

de Gelder, B., de Borst, A. W., & Watson, R. (2015). The perception of emotion in body expressions. Wiley Interdisciplinary Reviews:Cognitive Science, 6(2), 149-158.

de Gelder, B., Hortensius, R., & Tamietto, M. (2012). Attention and awareness each influence amygdala activity for dynamic bodily expressions-a short review. Frontiers in Integrative Neuroscience, 6, 54-63.

de Gelder, B., Snyder, J., Greve, D., Gerard, G., & Hadjikhani, N. (2004). Fear fosters flight:A mechanism for fear contagion when perceiving emotion expressed by a whole body. Proceedings of the National Academy of Science of the United States of America, 101(47), 16701-16706.

Downing, P. E., & Peelen, M. V. (2011). The role of occipitotemporal body-selective regions in person perception. Cognitive Neuroscience, 2(3-4), 186-203.

Downing, P. E., & Peelen, M. V. (2016). Body selectivity in occipitotemporal cortex:Causal evidence. Neuropsychologia, 83, 138-148.

Eimer, M. (2000). Effects of face inversion on the structural encoding and recognition of faces:Evidence from event-related brain potentials. Cognitive Brain Research, 10(1-2), 145-158.

Eimer, M., Holmes, A., & McGlone, F. P. (2003). The role of spatial attention in the processing of facial expression:An ERP study of rapid brain responses to six basic emotions. Cognitive, Affective, & Behavioral Neuroscience, 3(2), 97-110.

Feuerriegel, D., Churches, O., Hofmann, J., & Keage, H. A. D. (2015). The N170 and face perception in psychiatric and neurological disorders:A systematic review. Clinical Neurophysiology, 126(6), 1141-1158.

Flaisch, T., Häcker, F., Renner, B., & Schupp, H. T. (2011). Emotion and the processing of symbolic gestures:an event-related brain potential study. Social Cognitive and Affective Neuroscience, 6(1), 109-118.

Flaisch, T., & Schupp, H. T. (2013). Tracing the time course of emotion perception:The impact of stimulus physics and semantics on gesture processing. Social Cognitive and Affective Neuroscience, 8(7), 820-827.

Flaisch, T., Schupp, H. T., Renner, B., & Junghöfer, M. (2009). Neural systems of visual attention responding to emotional gestures. NeuroImage, 45(4), 1339-1346.

Foley, E., Rippon, G., Thai, N. J., Longe, O., & Senior, C. (2011). Dynamic facial expressions evoke distinct activation in the face perception network:A connectivity analysis study. Journal of Cognitive Neuroscience, 24(2), 507-520.

Gliga, T., & Dehaene-Lambertz, G. (2005). Structural encoding of body and face in human infants and adults. Journal of Cognitive Neuroscience, 17(8), 1328-1340.

Gu, Y. Y., Mai, X. Q., & Luo, Y.-J. (2013). Do bodily expressions compete with facial expressions? Time course of integration of emotional signals from the face and the body. PLoS One, 8(7), e66762.

Hadjikhani, N., & de Gelder, B. (2003). Seeing fearful body expressions activates the fusiform cortex and amygdala. Current Biology, 13(24), 2201-2205.

Hietanen, J. K., & Astikainen, P. (2013). N170 response to facial expressions is modulated by the affective congruency between the emotional expression and preceding affective picture. Biological Psychology, 92(2), 114-124.

Hietanen, J. K., Kirjavainen, I., & Nummenmaa, L. (2014). Additive effects of affective arousal and top-down attention on the event-related brain responses to human bodies. Biological Psychology, 103, 167-175.

Ishizu, T., Amemiya, K., Yumoto, M., & Kojima, S. (2010). Magnetoencephalographic study of the neural responses in body perception. Neuroscience Letters, 481(1), 36-40.

Lu, Y., Zhang, W.-N., Hu, W., & Luo, Y.-J. (2011). Understanding the subliminal affective priming effect of facial stimuli:An ERP study. Neuroscience Letters, 502(3), 182-185.

Luo, Q., Holroyd, T., Jones, M., Hendler, T., & Blair, J. (2007). Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. NeuroImage, 34(2), 839-847

Luo, Q., Holroyd, T., Majestic, C., Cheng, X., Schechter, J., & Blair, R. J. (2010). Emotional automaticity is a matter of timing. Journal of Neuroscience, 30(17), 5825-5829.

Luo, W. B., Feng, W. F., He, W. Q., Wang, N.-Y., & Luo, Y.-J. (2010). Three stages of facial expression processing:ERP study with rapid serial visual presentation. NeuroImage, 49(2), 1857-1867.

Joyce, C., & Rossion, B. (2005). The face-sensitive N170 and VPP components manifest the same brain processes:The effect of reference electrode site. Clinical Neurophysiology, 116(11), 2613-2631.

Meeren, H. K. M., de Gelder, B., Ahlfors, S. P., Hämäläinen, M. S., & Hadjikhani, N. (2013). Different cortical dynamics in face and body perception:An MEG study. PLoS One, 8(9), e71408.

Meeren, H. K. M., Hadjikhani, N., Ahlfors, S. P., Hämäläinen, M. S., & de Gelder, B. (2016). Early preferential responses to fear stimuli in human right dorsal visual stream-A Meg Study. Scientific Report, 6, 24831.

Meeren, H. K. M., van Heijnsbergen, C. C. R. J., & de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 102(45), 16518-16523.

Minnebusch, D. A., & Daum, I. (2009). Neuropsychological mechanisms of visual face and body perception. Neuroscience & Biobehavioral Reviews, 33(7), 1133-1144.

Minnebusch, D. A., Keune, P. M., Suchan, B., & Daum, I. (2010). Gradual inversion affects the processing of human body shapes. NeuroImage, 49(3), 2746-2755.

Morel, S., Beaucousin, V., Perrin, M., & George, N. (2012). Very early modulation of brain responses to neutral faces by a single prior association with an emotional context:Evidence from MEG. NeuroImage, 61(4), 1461-1470.

Moro, V., Urgesi, C., Pernigo, S., Lanteri, P., Pazzaglia, M., & Aglioti, S. M. (2008). The neural basis of body form and body action agnosia. Neuron, 60(2), 235-246.

Oliver, L. D., Mao, A., & Mitchell, D. G. V. (2014). "Blindsight" and subjective awareness of fearful faces:Inversion reverses the deficits in fear perception associated with core psychopathic traits. Cognition and Emotion, 29(7), 1256-1277.

Pitcher, D., Walsh, V., Yovel, G., & Duchaine, B. (2007). TMS evidence for the involvement of the right occipital face area in early face processing. Current Biology, 17(18), 1568-1573.

Pichon, S., de Gelder, B., & Grèzes, J. (2008). Emotional modulation of visual and motor areas by dynamic body expressions of anger. Social Neuroscience, 3(3-4), 199-212.

Pichon, S., de Gelder, B., & Grèzes, J. (2009). Two different faces of threat. Comparing the neural systems for recognizing fear and anger in dynamic body expressions. NeuroImage, 47(4), 1873-1883.

Pourtois, G., Dan, E. S., Grandjean, D., Sander, D., & Vuilleumier, P. (2005). Enhanced extrastriate visual response to bandpass spatial frequency filtered fearful faces:Time course and topographic evoked-potentials mapping. Human Brain Mapping, 26(1), 65-79.

Pourtois, G., Schettino, A., & Vuilleumier, P. (2013). Brain mechanisms for emotional influences on perception and attention:What is magic and what is not. Biological Psychology, 92, 492-512.

Recio, G., Wilhelm, O., Sommer, W., & Hildebrandt, A. (2017). Are event-related potentials to dynamic facial expressions of emotion related to individual differences in the accuracy of processing facial expressions and identity? Cognitive, Affective, & Behavioral Neuroscience, 17(2), 364-380.

Righart, R., & de Gelder, B. (2007). Impaired face and body perception in developmental prosopagnosia. Proceedings of the National Academy of Sciences of the United States of America, 104(43), 17234-17238.

Righart, R, & de Gelder, B. (2008). Rapid influence of emotional scenes on encoding of facial expressions:An ERP study. Social Cognitive and Affective Neuroscience, 3(3), 270-278.

Rossion, B., & Jacques, C. (2008). Does physical interstimulus variance account for early electrophysiological face sensitive responses in the human brain? Ten lessons on the N170. NeuroImage, 39(4), 1959-1979.

Sadeh, B., Podlipsky, I., Zhdanov, A., & Yovel, G. (2010). Event-related potential and functional MRI measures of face-selectivity are highly correlated:A simultaneous ERP-fMRI investigation. Human Brain Mapping, 31(10), 1490-1501.

Schönwald, L. I., & Müller, M. M. (2014). Slow biasing of processing resources in early visual cortex is preceded by emotional cue extraction in emotion-attention competition. Human Brain Mapping, 35(4), 1477-1490.

Schupp, H. T., Flaisch, T., Stockburger, J., & Junghöfer, M. (2006). Emotion and attention:Event-related brain potential studies. Progress in Brain Research, 156(2), 31-51.

Sinke, C. B. A., Sorger, B., Goebel, R., & de Gelder, B. (2010). Tease or threat? Judging social interactions from bodily expressions. NeuroImage, 49(2), 1717-1727.

Spapé, M. M., Harjunen, V., & Ravaja, N. (2017). Effects of touch on emotional face processing:A study of event-related potentials, facial EMG and cardiac activity. Biological Psychology, 124, 1-10.

Stekelenburg, J. J., & de Gelder, B. (2004). The neural correlates of perceiving human bodies:An ERP study on the body-inversion effect. Neuroreport, 15(5), 777-780.

Tamietto, M., & de Gelder, B. (2010). Neural bases of the non-conscious perception of emotional signals. Nature Reviews Neuroscience, 11(10), 697-709.

Taylor, J. C., Roberts, M. V., Downing, P. E., & Thierry, G. (2010). Functional characterisation of the extrastriate body area based on the N1 ERP component. Brain and Cognition, 73(3), 153-159.

Taylor, J. C., Wiggett, A. J., & Downing, P. E. (2010). fMRI-adaptation studies of viewpoint tuning in the extrastriate and fusiform body areas. Journal of Neurophysiology, 103(3), 1467-1477.

Theeuwes, J. (2010). Top-down and bottom-up control of visual selection. Acta Psychologica, 135(2), 77-99.

Thierry, G., Pegna, A. J., Dodds, C., Roberts, M., Basan, S., & Downing, P. (2006). An event-related potential component sensitive to images of the human body. NeuroImage, 32(2), 871-879.

Thom, N., Knight, J., Dishman, R., Sabatinelli, D., Johnson, D. C., & Clementz, B. (2014). Emotional scenes elicit more pronounced self-reported emotional experience and greater EPN and LPP modulation when compared to emotional faces. Cognitive, Affective, & Behavioral Neuroscience, 14(2), 849-860.

van de Riet W. A., Grèzes, J., & de Gelder, B. (2009). Specific and common brain regions involved in the perception of faces and bodies and the representation of their emotional expressions. Social Neuroscience, 4(2), 101-120.

van den Stock, J., Tamietto, M., Sorger, B., Pichon, S., Grézes, J., & de Gelder, B. (2011). Cortico-subcortical visual, somatosensory, and motor activations for perceiving dynamic whole-body emotional expressions with and without striate cortex (V1). Proceedings of the National Academy of Science of the United States of America, 108(39), 16188-16193.

van den Stock, J., Vandenbulcke, M., Sinke, C. B. A., & de Gelder, B. (2012). Affective scenes influence fear perception of individual body expressions. Human Brain Mapping, 35(2), 492-502.

van Heijnsbergen, C. C. R. J., Meeren, H. K. M., Grèzes, J., & de Gelder, B. (2007). Rapid detection of fear in body expressions, an ERP study. Brain Research, 1186(1), 233-241.

Williams, M. A., Morris, A. P., McGlone, F., Abbott, D. F., & Mattingley, J. B. (2004). Amygdala responses to fearful and happy facial expressions under conditions of binocular suppression. The Journal of Neuroscience, 24(12), 2898-2904.

Yovel, G., & Kanwisher, N. (2005). The neural basis of the behavioral face-inversion effect. Current Biology, 15(24), 2256-2262

Zhu, X.-R., & Luo, Y.-J. (2012). Fearful faces evoke a larger C1 than happy faces in executive attention task:An event-related potential study. Neuroscience Letters, 526(2), 118-121.