DOI: 10.3724/SP.J.1041.2018.00483

Acta Psychologica Sinica (心理学报) 2018/50:5 PP.483-493

Visual and auditory verbal working memory affects visual attention in the semantic matching

Previous studies have showed that information held in working memory (WM) can guide or capture attention during visual search in a relatively automatic way, even when it is irrelevant and detrimental to current task performance. Some researchers have proposed that the semantic match between WM contents and distractors could also capture attention, as well as the perceptual match. As we known, the verbal WM contents can be stored in the visual and auditory inputs. Even though the automatic influence of visual verbal WM on visual attention have been demonstrated, it remains unknown whether the auditory verbal WM could automatically capture attention. Therefore, it is necessary to investigate the attention guidance by the verbal WM contents.
The present study included two experiments to explore the questions presented above. In Experiment 1, the memory item was a verbal Chinese character that presented visually, denoting a color, such as "红". The participants were instructed to remember the word and avoid the potential distractors. Subsequently, they completed a visual search task, in order to test whether the verbal WM contents could guide attention. The results showed that, compared with the control condition, the visual search RTs were longer in the perceptual-matching and semantic-matching conditions, and the same as the RTs in the fastest trials. With the memory item that never matched the target in the search task, we suggested that the verbal WM contents that were presented visually (vis-VWM) could capture attention at perceptual and semantic levels automatically. In Experiment 2, the memory item was presented by the auditory inputs via the headphones (audi-VWM). The results showed that the visual search RTs in the semantic-matching condition were shorter than RTs in the control and perceptual-matching conditions, and there was no significant difference in the other conditions. Meanwhile, compared the shortest RTs across the different conditions, the results showed that the RTs in the semantic-matching condition were longer than in the control condition, which suggested that the aurally presented verbal WM could capture attention at the semantic level in the fastest response trials.
In conclusion, the present study demonstrated that the verbal working memory that presented visually could automatically capture attention at both perceptual and semantic levels, and also verified the hypothesis that the attention capture effect would occur at the early stages of attention. However, the contents of verbal working memory would always capture attention at the earlier processing stage and could only be rejected at the later processing stage when the contents were aurally presented. Due to the modality specificity, attentional resources would be distributed to different sensory modalities. The memory-matching distractors could be rejected at the later processing stage because of there were the sufficient cognitive resources.

Key words:attention capture,verbal working memory,visual search,semantic matching

ReleaseDate:2018-07-02 16:16:01

Baddeley, A. D. (2000). The episodic buffer:A new component of working memory? Trends in Cognitive Sciences, 4(11), 417-423.

Baddeley, A. D. (2003). Working memory and language:An overview. Journal of Communication Disorders, 36, 189-208.

Baddeley, A. D., & Larsen, J. D. (2007). The phonological loop:Some answers and some questions. The Quarterly Journal of Experimental Psychology, 60(4), 512-518.

Belke, E., Humphreys, G. W., Watson, D. G., Meyer, A. S., & Telling, A. L. (2008). Top-down effects of semantic knowledge in visual search are modulated by cognitive but not perceptual load. Perception & Psychophysics, 70(8), 1444-1458.

Calleja, M. O., & Rich, A. N. (2013). Guidance of attention by information held in working memory. Attention, Perception, & Psychophysics, 75(4), 687-699.

Campoy, G., Castellà, J., Provencio, V., Hitch, G. J., & Baddeley, A. D. (2015). Automatic semantic encoding in verbal short-term memory:Evidence from the concreteness effect. The Quarterly Journal of Experimental Psychology, 68(4), 759-778.

Carlisle, N. B., & Woodman, G. F. (2011). Automatic and strategic effects in the guidance of attention by working memory representations. Acta Psychologica, 137(2), 217-225.

Chambers, C. D., Stokes, M. G., & Mattingley, J. B. (2004). Modality-specific control of strategic spatial attention in parietal cortex. Neuron, 44(6), 925-930.

Crottaz-Herbette, S., Anagnoson, R. T., & Menon, V. (2004). Modality effects in verbal working memory:Differential prefrontal and parietal responses to auditory and visual stimuli. NeuroImage, 21(1), 340-351.

Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193-222.

Donohue, S. E., Liotti, M., Perez, R., Ⅲ., & Woldorff, M. G. (2012). Is conflict monitoring supramodal? Spatiotemporal dynamics of cognitive control processes in an auditory Stroop task. Cognitive, Affective, & Behavioral Neuroscience, 12(1), 1-15.

Eimer, M. (2014). The neural basis of attentional control in visual search. Trends in Cognitive Science, 18(10), 526-535.

Gil-Gómez de Liaño, B., Potter, M. C., & Rodríguez, C. (2014). Working memory effects in speeded RSVP tasks. Psychological Research, 78, 124-135.

Greene, C. M., Kennedy, K., & Soto, D. (2015). Dynamic states in working memory modulate guidance of visual attention:Evidence from an n-back paradigm. Visual Cognition, 23(5), 546-560.

Hollingworth, A., & Hwang, S. (2013). The relationship between visual working memory and attention:Retention of precise colour information in the absence of effects on perceptual selection. Philosophical Transactions of the Royal Society B:Biological Sciences, 368(1628), 20130061.

Hu, Y. M., & Zhang, M. (2016). Electrophysiological evidence for memory-based attentional capture and memory-based attentional rejection effects. Acta Psychologica Sinica, 48(1), 12-21.[胡艳梅, 张明. (2016). 基于记忆的注意捕获和注意抑制效应:ERP证据. 心理学报, 48(1), 12-21.]

Hu, Y. M., Zhang, M., Xu, Z., & Li, B. Q. (2013). Guidance of working memory on attention:The effects of inhibition incentive. Acta Psychologica Sinica, 45(2), 127-138.[胡艳梅, 张明, 徐展, 李毕琴. (2013). 客体工作记忆对注意的导向作用:抑制动机的影响. 心理学报, 45(2), 127-138.]

Huang, L. Q., & Pashler, H. (2007). Working memory and the guidance of visual attention:Consonance-driven orienting. Psychonomic Bulletin & Review, 14(1), 148-153.

Kawashima, T., & Matsumoto, E., (2017). Cognitive control of attentional guidance by visual and verbal working memory representations. Japanese Psychological Research, 59(1), 49-57.

Kim, S. Y., Kim, M. S., & Chun, M. M. (2005). Concurrent working memory load can reduce distraction. Proceedings of the National Academy of Sciences of the United States of America, 102(45), 16524-16529.

Moores, E., Laiti, L., & Chelazzi, L. (2003). Associative knowledge controls deployment of visual selective attention. Nature Neuroscience, 6, 182-189.

Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture:Visual working memory content affects visual attention. Journal of Experimental Psychology:Human Perception and Performance, 32(5), 1243-1265.

Pan, Y. (2010). Effects of verbal memory on visual selection:Dimension-based automatic guidance of attention. Acta Psychologica Sinica, 42(12), 1118-1127.[潘毅. (2010). 言语记忆对视觉选择的影响:基于维度的自动注意引导. 心理学报, 42(12), 1118-1127.]

Santangelo, V., Fagioli, S., & Macaluso, E. (2010). The costs of monitoring simultaneously two sensory modalities decrease when dividing attention in space. NeuroImage, 49, 2717-2727.

Sasin, E., Morey, C. C., & Nieuwenstein, M. (2017) Forget Me if You Can:Attentional capture by to-Be-remembered and to-Be-forgotten visual stimuli. Psychonomic Bulletin & Review, 24, 1643-1650.

Sasin, E., & Nieuwenstein, M. (2016). Memory-driven attentional capture reveals the waxing and waning of working memory activation due to dual-task interference. Psychonomic Bulletin & Review, 23, 1891-1897.

Soto, D., Heinke, D., Humphreys, G. W., & Blanco, M. J. (2005). Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology:Human Perception and Performance, 31(2), 248-261.

Soto, D., Hodsoll, J., Rotshtein, P., & Humphreys, G. W. (2008). Automatic guidance of attention from working memory. Trends in Cognitive Sciences, 12(9), 342-348.

Soto, D., & Humphreys, G. W. (2007). Automatic guidance of visual attention from verbal working memory. Journal of Experimental Psychology:Human Perception & Performance, 33(3), 730-737.

Soto, D., Humphreys, G. W., & Heinke, D. (2006). Working memory can guide pop-out search. Vision Research, 46(6-7), 1010-1018.

Soto, D., Mannan, S. K., Malhotra, P., Rzeskiewica, A., & Humphreys, G. W. (2011). Distinguishing non-spatial from spatial biases in visual selection:Neuropsychological evidence. Acta Psychologica, 137, 226-234.

Soto, D., Rotshtein, P., Hodsoll, J., Mevorach, C., & Humphreys, G. W. (2012). Common and distinct neural regions for the guidance of selection by visuoverbal information held in memory:Converging evidence from fMRI and rTMS. Human Brain Mapping, 33(1), 105-120.

Sun, S. Z., Shen, J., Shaw, M., Cant, J. S., & Ferber, S. (2015). Automatic capture of attention by conceptually generated working memory templates. Attention, Perception, & Psychophysics, 77(6), 1841-1847.

Telling, A. L., Kumar, S., Meyer, A. S., & Humphreys, G. W. (2010). Electrophysiological evidence of semantic interference in visual search. Journal of Cognitive Neuroscience, 22(10), 2212-2225.

Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3, 159-177.

Wolfe, J. M., Horowitz, T. S., Kenner, N., Hyle, M., & Vasan, N. (2004). How fast can you change your mind? The speed of top-down guidance in visual search. Vision Research, 44(12), 1411-1426.

Woodman, G. F., & Luck, S. J. (2007). Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology:Human Perception and Performance, 33(2), 363-377.

Zhang, B., Hu, C. L., & Huang, S. (2016). What do eye movements reveal about the role of cognitive control in attention guidance from working memory representation. Acta Psychologica Sinica, 48(9), 1105-1118.[张豹, 胡岑楼, 黄赛. (2016). 认知控制在工作记忆表征引导注意中的作用:来自眼动的证据. 心理学报, 48(9), 1105-1118.]

Zhang, B., & Huang, S. (2013). Mechanisms underlying the attentional guidance form working memory representations. Advances in Psychological Science, 21(9), 1578-1584.[张豹, 黄赛. (2013). 工作记忆表征对视觉注意的引导机制. 心理科学进展, 21(9), 1578-1584.]

Zhang, B., Huang, S., & Hou, Q. X. (2014). The priority of color in working-memory-driven ocular capture. Acta Psychologica Sinica, 46(1), 17-26.[张豹, 黄赛, 候秋霞. (2014). 工作记忆表征捕获眼动中的颜色优先性. 心理学报, 46(1), 17-26.]

Zhang, B., Huang, S., & Qi, L. (2013). Working memory representation does guide visual attention:Evidence from eye movements. Acta Psychologica Sinica, 45(2), 139-148.[张豹, 黄赛, 祁禄. (2013). 工作记忆表征引导视觉注意选择的眼动研究. 心理学报, 45(2), 139-148.]

Zhang, B., Shao, J. Y., Hu, C. L., & Huang, S. (2015). Attentional guidance from activated and inhibitory states of working memory representations. Acta Psychologica Sinica, 47(9), 1089-1100.[张豹, 邵嘉莹, 胡岑楼, 黄赛. (2015). 工作记忆表征的激活与抑制状态对注意引导效应的影响. 心理学报, 47(9), 1089-1100.]

Zhang, M., & Wang, A. J. (2012). Working memory content-based attentional capture and suppression in the visual search. Advances in Psychological Science, 20(12), 1899-1907.[张明, 王爱君. (2012). 视觉搜索中基于工作记忆内容的注意捕获与抑制. 心理科学进展, 20(12), 1899-1907.]