DOI: 10.3724/SP.J.1041.2017.01172

Acta Psychologica Sinica (心理学报) 2017/49:9 PP.1172-1183

The effect of emotional scene and body expression on facial expression recognition

Traditionally, the recognition of facial expression was studied by using isolated faces, which was affected by the categorical theory of emotion. In fact, in real life, facial expression always occurs in context. The result of facial expression recognition could be ambiguous when observer typically viewed face in isolation without any other cues. Recently many researches investigated how context may influence the facial expression processing. Numerous studies found context effect in recognition of facial expression, namely visual scene, body expression, emotional concept and surrounding other faces all shaped the facial expression processing. Evidence from ERPs and eye movements showed that facial expression was combined with both of emotional visual scene and body expression during the early stage of processing, and to some degree, was automatic. But none of these studies investigated the role they played simultaneously. Moreover, studies of context effect in recognition of facial expression seldom investigated the recognition of visual scene directly by using memory paradigm. Given that emotional concept may provide a top-down constraint in facial expression recognition, in the current study, we increased number of emotion labels for a better understanding of the context effect.
Two experiments were conducted in this study, both of them adopted a 2 (facial expression:disgust, fear)×2(emotion congruent between face and context:congruent, incongruent) within-subject design. Stimuli were made up by images of disgust and fear (low level of perceptual similarity) facial expression, visual scene and body expression. All participants were required to label an emotion word for the face which was shown against backgrounds of visual scene (Exp1) or both visual scene and body expression (Exp2). Experiment 1 aimed to study the effect of emotion congruent between facial expression and visual scene on facial expression recognition. On the basis of experiment 1, experiment 2 added congruent or incongruent body expression to investigate whether body expression disturbed the effect of visual scene on facial expression recognition.
Results of the two experiments indicate that:(1)The influence of visual scene on facial expression recognition is still significant when increased the number of emotion labels; (2)Participants would rely more on visual scene and remember the scene more often when emotional facial expression is shown against an incongruent visual scene; (3)The effect of visual scene on facial expression recognition can be influenced by body expression, but the emotion of visual scene still play an important role in recognition of negative facial expression.

Key words:facial expression recognition,context effect,visual scene,body expression

ReleaseDate:2017-10-20 02:10:45

Adolphs, R. (2002). Recognizing emotion from facial expressions:Psychological and neurological mechanisms. Behavioral and Cognitive Neuroscience Reviews, 1, 21-61.

Aviezer, H., Bentin, S., Dudarev, V., & Hassin, R. R. (2011). The automaticity of emotional face-context integration. Emotion, 11(6), 1406-1414.

Aviezer, H., Hassin, R. R., Bentin, S., & Trope, Y. (2008). Putting facial expressions back in context. In N. Ambady & J. Skowronski (Eds.), First impressions (pp. 255-286). New York, NY:Guilford Press.

Aviezer, H., Hassin, R. R., Ryan, J., Grady, C., Susskind, J., Anderson, A.,... Bentin, S. (2008). Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science, 19(7), 724-732.

Aviezer, H., Trope, Y., & Todorov, A. (2012). Holistic person processing:Faces with bodies tell the whole story. Journal of Personality and Social Psychology, 103(1), 20-37.

Bai, X. J., & Yue, P. F. (2013). Affect labeling can reduce negative emotions:Evidences from autonomic nervous activity. Acta Psychologica Sinica, 45(7), 715-724.[白学军, 岳鹏飞. (2013). 情绪标注对负性情绪的抑制:来自自主神经活动的证据. 心理学报, 45(7), 715-724.]

Barrett, L. F., & Kensinger, E. A. (2010). Context is routinely encoded during emotion perception. Psychological Science, 21(4), 595-599.

Barrett, L. F., Lindquist, K. A., & Gendron, M. (2007). Language as context for the perception of emotion. Trends in Cognitive Sciences, 11(8), 327-332.

Barrett, L. F., Mesquita, B., & Gendron, M. (2011). Context in emotion perception. Current Directions in Psychological Science, 20(5), 286-290.

de Gelder, B. (2006). Towards the neurobiology of emotional body language. Nature Reviews Neuroscience, 7(3), 242-249.

de Gelder, B., & van den Stock, J. (2011). The bodily expressive action stimulus test (BEAST). Construction and validation of a stimulus basis for measuring perception of whole body expression of emotions. Frontiers in Psychology, 2, 181.

Ekman, P. (1992). Facial expressions of emotion:New findings, new questions. Psychological Science, 3(1), 34-38.

Ekman, P., & Friesen, W. V. (1976). Pictures of facial affect. Palo Alto, CA:Consulting Psychologists Press.

Flowe, H., & Bessemer, A. (2011). The effects of target discriminability and criterion placement on accuracy rates in sequential and simultaneous target-present lineups. Psychology, Crime & Law, 17(7), 587-610.

Fugate, J. M. B., Gouzoules, H., & Barrett, L. F. (2010). Reading chimpanzee faces:Evidence for the role of verbal labels in categorical perception of emotion. Emotion, 10(4), 544-554.

Goodenough, F. L., & Tinker, M. A. (1931). The relative potency of facial expression and verbal description of stimulus in the judgment of emotion. Journal of Comparative Psychology, 12(4), 365-370.

Hassin, R. R., Aviezer, H., & Bentin, S. (2013). Inherently ambiguous:Facial expressions of emotions, in context. Emotion Review, 5(1), 60-65.

Kensinger, E. A., Garoff-Eaton, R. J., & Schacter, D. L. (2007). Effects of emotion on memory specificity:Memory trade-offs elicited by negative visually arousing stimuli. Journal of Memory and Language, 56(4), 575-591.

Kret, M. E., Stekelenburg, J. J., Roelofs, K., & de Gelder, B. (2013). Perception of face and body expressions using electromyography, pupillometry and gaze measures. Frontiers in Psychology, 4, 28.

Lindquist, K. A., Barrett, L. F., Bliss-Moreau, E., & Russell, J. A. (2006). Language and the perception of emotion. Emotion, 6(1), 125-138.

Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012). The brain basis of emotion:A meta-analytic review. Behavioral and Brain Sciences, 35(3), 121-143.

Liu, J. Y., Huang, X. T., & Yang, S. (2013). Three components of attentional bias in social anxiety disorder. Advances in Psychological Science, 21(4), 664-670.[刘静怡, 黄希庭, 杨帅. (2013). 社交焦虑障碍中的注意偏差三成分研究述评. 心理科学进展, 21(4), 664-670.]

Liu, W. J., Shen, M. Q., Li, Y., & Wang, R. M. (2016). The interaction between emotional concept processing and emotional face perception. Acta Psychologica Sinica, 48(2), 163-173.[刘文娟, 沈曼琼, 李莹, 王瑞明. (2016). 情绪概念加工与情绪面孔知觉的相互影响. 心理学报, 48(2), 163-173.]

Masuda, T., Ellsworth, P. C., Mesquita, B., Leu, J., Tanida, S., & van de Veerdonk, E. (2008). Placing the face in context:Cultural differences in the perception of facial emotion. Journal of Personality and Social Psychology, 94(3), 365-381.

Meeren, H. K. M., van Heijnsbergen, C. C. R. J., & de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 102(45), 16518-16523.

Mondloch, C. J., Nelson, N. L., & Horner, M. (2013). Asymmetries of influence:Differential effects of body postures on perceptions of emotional facial expressions. PLoS ONE, 8(9), e73605.

Righart, R., & de Gelder, B. (2006). Context influences early perceptual analysis of faces-an electrophysiological study. Cerebral Cortex, 16(9), 1249-1257.

Righart, R., & de Gelder, B. (2008a). Rapid influence of emotional scenes on encoding of facial expressions:An ERP study. Social Cognitive and Affective Neuroscience, 3(3), 270-278.

Righart, R., & de Gelder, B. (2008b). Recognition of facial expressions is influenced by emotional scene gist. Cognitive, Affective, & Behavioral Neuroscience, 8(3), 264-272.

Russell, J. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102-141.

Schwarzlose, R. F., Baker, C. I., & Kanwisher, N. (2005). Separate face and body selectivity on the fusiform gyrus. The Journal of Neuroscience, 25(47), 11055-11059.

Susskind, J. M., Littlewort, G., Bartlett, M. S., Movellan, J., & Anderson, A. K. (2007). Human and computer recognition of facial expressions of emotion. Neuropsychologia, 45(1), 152-162.

Xiao, Y. L., Yin, G. E., & Yang, W. (2015). The interaction of different face parts in the similarity judgment of the face part. Studies of Psychology and Behavior, 13(6), 729-736.[肖艳丽, 阴国恩, 杨文. (2015). 面孔部件相似性判断中不同部件的相互影响研究. 心理与行为研究, 13(6), 729-736.]

Yin, G. E., Yang, W., Xiao, Y. L., & Wang, Z. G. (2013). The weight distribution of the face part in the face similarity judgment. Journal of Psychological Science, 36(3), 553-557.[阴国恩, 杨文, 肖艳丽, 王正光. (2013). 面孔相似性判断中各部件的权重分配. 心理科学, 36(3), 553-557.]

Yovel, G., Pelc, T., & Lubetzky, I. (2010). It's all in your head:Why is the body inversion effect abolished for headless bodies?. Journal of Experimental Psychology:Human Perception and Performance, 36(3), 759-767.

Yue, P. F., Du, W. W., Bai, X. J., & Xu, Y. L. (2015). When does inhibition of affect labeling occur:ERP study. Acta Psychologica Sinica, 47(9), 1124-1132.[岳鹏飞, 杜婉婉, 白学军, 许远理. (2015). 情绪标注对情绪的抑制发生在何时:一项ERPs研究. 心理学报, 47(9), 1124-1132.]

Zhang, D. D., Zhao, T., Liu, Y. Z., & Chen, Y. M. (2015). Comparison of facial expressions and body expressions:An event-related potential study. Acta Psychologica Sinica, 47(8), 963-970.[张丹丹, 赵婷, 柳昀哲, 陈玉明. (2015). 恐惧情绪面孔和身体姿势加工的比较:事件相关电位研究. 心理学报, 47(8), 963-970.]