search

An Embodied Account of Visual Working Memory

Download

Please note! This essay has been submitted by a student.

Download PDF

Abstract

Traditional models of visual memory strictly incorporate internal memory storage and ignore our reliance on the external visual world to maintain information. Experiments on visual working memory generally use paradigms that are designed to maximally load internal memory storage, although these situations do not necessarily translate to the actual use of visual working memory in daily life. Here, I discuss an embodied view of visual memory in which there is a continuous decision about which information to internalize and which information to leave in the external world for (possible) access later in time. In this view, the known limited capacity of visual working memory is not a problem in daily life, as the external world typically remains readily available and can be accessed relatively easily by executing eye movements to relevant locations. Whenever you walk in a forest or stroll through a downtown area, you experience a richly visual world. You enjoy the various shades of green of the trees or are overwhelmed by all of the details that a busy city center offers. Although you might have the impression of a rich visual world, we now know that your brain only represents very little of this visual world at each moment in time. For instance, remarkably large changes in the environment generally go undetected , indicating that the external visual world is only partly represented internally.

Essay due? We'll write it for you!

Any subject

Min. 3-hour delivery

Pay if satisfied

Get your price

Current memory models refer to ‘visuospatial working memory’ as the memory system responsible for the internal representation of the visual world . Visuospatial working memory is divided into a visual and a spatial component, with spatial working memory maintaining relevant locations in the visual world and visual working memory (VWM) maintaining visual features of objects . VWM is generally regarded as capacity-limited, effortful storage for visual information that is no longer available (hence the term ‘memory’).

The recent rise of interest in VWM has resulted in lively debates and important findings on the nature of its capacity . One of the most intriguing issues has been the maximum capacity of VWM. This question has resulted in a fierce debate about whether the capacity limits of VWM should be interpreted in terms of slots as discrete units or in terms of available resources .

Although this is an interesting theoretical discussion, the corresponding experiments might not translate to the actual use of VWM in daily life. It is actually quite difficult to come up with a task in daily life that involves holding multiple visual items in memory, besides perhaps complex visual imagery. Simply look at the effort that participants have to deliver to perform our experiments in the laboratory and it becomes clear why we prefer not to maintain multiple items in working memory. First, maintaining an item in VWM is expensive as this process requires attentional resources . Second, internal representations are fragile, prone to decay or disturbance due to incomplete or incorrect encoding .

Most experiments on VWM study memory performance for visual information that is no longer physically present. For instance, in change blindness experiments, the observer must identify a change from one visual scene to another, when the initial scene is no longer present . To correctly perform this task, information has to be stored in VWM. Furthermore, the traditional paradigm for studying the neural correlates of VWM is a task in which an array of random items is presented and removed, and recall is tested a few seconds later . By enforcing a strategy in which observers have to store information in VWM to correctly perform the task, researchers have ignored the fact that outside of the lab, our external visual world typically remains available and is relatively stable.

So, although the maximum capacity of VWM might be considered to be about 3 to 4 items when interpreting the limits in terms of discrete slots, this capacity might not be used in daily life when we interact with physically present information. In this situation, humans can depend on the external world to access visual information in their environment. There is no need for an internal representation of multiple objects as long as the visual information is readily available in the external world. The focus on the maximum capacity in experiments on VWM is similarly present in the neuropsychological tests that are available to assess a person’s maximum capacity of VWM . If we hardly ever use the maximum capacity in daily life, this sort of assessment will have little predictive value of a patient’s functioning during daily activities requiring VWM, such as navigation and visual search.

So, how are we able to survive with such a small capacity to internally store information from the external visual world? Luckily, we have a system in place that allows us to internalize very little of the external visual world in internal memory: the eye movement system. We do not passively perceive the world but interact with our environment. One of these interactions is the execution of eye movements to relevant locations in our visual world. Because these locations contain objects, we can access these objects by moving our eyes to these locations. Eye movements are so efficient that they allow us to use the world as an ‘external memory. Eye movements have even been claimed to be ‘cheap’ as they are executed extremely quickly and are associated with low effort . We are generally not aware of the many eye movements we execute, even though previous research has unraveled that the selection of where to execute the next eye movement is far from trivial . Despite the multitude of processes necessary to execute eye movements, we generally do not come home after a day of work complaining about how tiring it has been to execute the thousands of eye movements during an average day.

This efficiency of eye movements also underlies our subjective impression of a complete internal representation of the visual world. Because eye movements and attention are tightly coupled , any saccade will result in a shift of attention to the fixated location. The impression of consciously seeing everything is due to these shifts of attention. Whenever you want to scrutinize an object in your visual world, the resulting saccade will be accompanied by a mandatory shift of visual attention to this object to allow further inspection . Everything you check on will therefore be immediately available for scrutiny through a quick shift of visual attention, creating the impression of a complete representation of the visual world. The only requirement for such a system to be efficient is an internal memory of where important information is positioned. The use of the world as an external visual memory is therefore enabled by the human ability to make rapid eye movements to relevant locations.

Our brain can internalize information from the external visual world by directing the eyes and transferring the information at the fixated location to VWM (‘sampling information’). In line with this, we recently concluded that the overlap between VWM and the eye movement system is even stronger than previously thought : every time we move our eyes, the saccade target is automatically transferred into VWM . The role of eye movements in the functioning of VWM goes beyond simply the resampling of visual information. Even when the external world does not provide the relevant visual information, studies still find that participants make eye movements to the locations of previously relevant visual information . This suggests that visual memoranda in VWM are linked to locations in the external world .

A complete understanding of visual memory requires an embodied approach which embraces the external visual world as actual memory storage that should be incorporated in traditional memory models. We can recruit our environment for achieving our goals with a minimum expenditure of our scarce mental resources. Current memory models ignore an important property of the human brain: our brain is an energy-efficient system that aims to minimize its load. Instead of using the energy-consuming internal memory, our brain can rely on the external visual world to maintain important visual information. The concept of external memory is therefore in line with the extended mind thesis, which claims that a person’s mind and associated cognitive processing are not body-bound but extend into the external world.

As we are embedded in our environment, mental processes, such as visual memory, extend beyond the body to include aspects of the environment . The idea that our cognition is intimately coupled with the outside world is not a new one. Philosophers that consider our mind as inseparable from the body and the environment we inhabit date back to Kant and Heidegger. ‘Embodied cognition in the context of psychological research has been accepted and expanded upon during the past three decades. According to embodiment theory, the brain’s cognitive capacities are uniquely shaped by the body’s biological makeup and sensorimotor capacities. This idea of offloading our cognitive capacity by engaging the outside world as a form of an external cognitive resource offers a new perspective from which we can investigate the nature of VWM.

Traditional models of VWM strictly incorporate internal memory storage and ignore our interaction with the external visual world to maintain information. This is surprising because it is already known that we frequently adopt the external world as a memory resource in our daily life. For instance, think of the use of the term ‘memory’ for a mobile phone: we store information about phone numbers in the external memory that is our phone, safe and secure, without any load on our internal memory. Similarly, we write information down in external memory to offload our internal memory system or remind ourselves not to forget our keys by placing them at a salient position in the external world. In my view, the same energy-efficient principle holds for the visual memory of the world around us. An embodied approach to visual working memory extends the definition of what is memory. Memory should refer to storage, irrespective of whether it is internal or external memory storage.

How to Measure

If we indeed use the world as an external memory, there is a continuous decision about whether or not to store an item internally or externally. This decision is then based on a tradeoff between the costs associated with the execution of a saccade and the costs of storing an object in visual working memory. It automatically follows from this assumption that internal visual working memory will be used in those situations in which the costs of a saccade exceed a certain threshold. An example of a task to measure external memory use is the ‘copying task’, in which the observer has to copy a complicated figure constituting simple colored shapes (‘the template’) using the mouse. Previous research has shown that participants perform many eye movements between the template and the workspace while performing a copying task, indicating that the template is not fully memorized in internal visual memory and that observers rely on the objects in the external world {Ballard, 1995 #4150}. In this task, participants have the freedom to choose their own task parameters. For instance, they can choose not to rely on internal visual working memory and store the features of the template in external memory. Minimizing the use of internal visual working memory will therefore result in a high proportion of eye movements between the template and the workspace. In this situation, the reluctance to use internal visual working memory can be explained by the fact that such memory is expensive to use compared to a strategy in which the information is stored in external memory.

In a recent study, we successfully observed a reluctance for the use of “expensive” memory when time costs associated with a saccade are increased, confirming the presence of a trade-off between storing information in VWM and making saccades . We influenced the trade-off between eye movements and VWM utilization by introducing a cost to a saccade. If there is an adaptive trade-off between using the external visual world and VWM, the tradeoff should be influenced by increasing the cost associated with using external information. Higher costs were created by adding a delay in stimulus availability to a copying task (i.e. the time between the landing of the saccade on the template and the appearance of the template). By removing the template from the screen at the start of a saccade towards the template and delaying its presentation after the saccade, the (time) cost of a saccade is increased. This experiment included three different delays after which the template was revealed (250, 1500, and 3000 ms). Results showed that increased saccade cost results in fewer saccades towards the model and an increased dwell time on the model. These results suggest a shift from making eye movements towards taxing internal VWM. Our findings reveal that the trade-off between executing eye movements and building an internal representation of our world is based on an adaptive mechanism, governed by cost-efficiency.

These results were reminiscent of the findings by Droll and Hayhoe . These authors manipulated the cognitive cost of the task by increasing the number of visual features or unpredictably changing block sorting rules. The behavioral and eye-tracking results showed that the memory/gaze tradeoff is highly dynamic and heavily depends on the number of task-relevant features that need to be tracked as well as the unpredictability of the task. Specifically, if a task can be easily resolved with limited usage of working memory (i.e. at low capacity), then visual working memory is the preferred mechanism as it offsets the cost of fixating and re-fixating, and externalizing mental computations, which itself carries a cognitive cost. Conversely, when visual working memory is taxed by a high number of features or the task is unpredictable, whereby encoding and maintenance of information are inefficient and inaccurate, participants rely on just-in-time saccades and re-fixations as their strategy of choice due to the lower cognitive cost. Droll and Hayhoe concluded that the assumption of object invariance also heavily dictates usage of visual working memory; if an object is simple and invariant and the visual task predictable, it will be encoded and executed in working memory as that is the least cognitively costly strategy.

There is already some research on the neural correlates of our interaction with objects still within view. As mentioned, previous research on VWM was conducted in situations in which stimuli disappear, mostly in the context of static, flat 2d shapes and objects disappearing from a computer screen. In a recent study, Tsubomi and colleagues examined neural responses when viewing targets that remained visible until response. They used the CDA component, an especially strong neural marker that corresponds to not only the number of active representations maintained in the working memory but also the rate of their decay and discarding . When they compared the neural activation pattern with that found in a normal change detection task, they found that the two patterns were identical. This finding suggests that visual working ‘memory’ is not a memory in the traditional sense of the word. Rather, working memory could be interpreted as a powerful but limited cognitive resource deployed according to visual task demands, regardless of the object’s presence or absence.

Conclusion

Because the brain only stores a limited amount of visual information in VWM, there is a continuous decision about which information to internalize and which information to leave in the external world for (possible) access later in time. Current theories do not take into account the fact that our visual world is relatively stable, and that we may not need to store information when it remains externally available. Traditionally, experiments on VWM use visual stimuli that are presented briefly or change features rapidly and the participant’s response is entirely contingent on their ability to use their working memory capacity effectively. This is the correct approach if we are mostly interested in the memory aspect of visual working memory, however, we have seen so far that the memory component of visual working memory is perhaps not its most crucial one. A more natural approach to visual tasks is required, with the ability to create the external and stable visual reference.

Our brain aims for an optimal balance between storing information inexpensive and vulnerable VWM and leaving information in the outside world without internal storage (i.e. an internal mental economy). The outcome of this trade-off is determined by the costs of storage and the cost of (re-)acquiring the visual information in the external world. To fully understand human cognition, it is not enough to focus only on what goes on inside the skull, because our abilities are supported by our environment.

72
writers online
to help you with essay
banner clock
Clock is ticking and inspiration doesn't come?
We`ll do boring work for you. No plagiarism guarantee. Deadline from 3 hours.

We use cookies to offer you the best experience. By continuing, we’ll assume you agree with our Cookies policy.