Institute For The Future

Embodied Learning

Immersive media's educational potential

In 1990, Steve Jobs famously promoted the computer as “the equivalent of a bicycle for our minds.” In some ways, that’s how computers are still understood by most people today: as devices that we use primarily with our brains. But that’s about to change. Advances in haptics, mixed and virtual reality, and full-body motion capture are turning computers into devices for the whole body. And this has some important implications for learning and education.

Much has already been made about the potential for mixed-reality learning. Many have pointed out that augmented reality (AR) enables us to overlay helpful educational information onto places and objects in the physical world, and virtual reality (VR) could allow students to drop into any learning environment on the planet. But the true potential of this technology may lie in something else entirely.

That’s because, at the same time that computers are shifting from tools for the mind alone to tools for the whole body, we’re experiencing a shift in how we understand the mind itself.

For centuries, the Western world has conceived of the mind and body as separate. The mind decides, feels, interprets, and thinks. And the body serves as a fleshy transportation device, sending sensory information to the body and carrying out commands. “Embodied cognition” theory changes all that. It asserts that the mind often thinks through the body, using sensing, movement, and physicality as a form of processing. And this way of thinking about thinking suggests that we often learn better by using our whole body to understand, experience, and interact with learning content. Over the next decade, as body tracking and haptic feedback technologies get more precise, immersive media will leverage our embodied cognitive processes in novel ways to supercharge the way we learn everything from playing the piano to coding. Here are some examples of how this approach is already being applied with today’s technologies.

 

Leveraging Spatial Awareness

Altar Virtual is one of a handful of companies aiming to leverage embodied learning by connecting early tools for physicalizing VR to cognitive learning techniques, like the “memory palace” trick. The popular technique for memorization uses a form of imagined physicality for learning. In it, you imagine walking room-to-room in your home and putting the information you want to remember on bits of the walls and counters. The very act of imagining walking through the space helps you recall that information with astounding accuracy. Altar and other companies are creating new virtual environments tailor-made to leverage our natural propensity to learn and remember through spatial relationships.

Tapping Muscle Memory

There are several projects that use gesture interfaces for teaching to exploit research findings that indicate we learn better by engaging more than one sense (i.e., having students physically move numbers to solve mathematical equations). For example, the Virtual ENVironment Interaction software was designed to teach kids programming by having them input code by performing dance moves, instead of typing on a keyboard. This both creates a more fun and engaging learning experience and taps into the link between memory and physical movement.

Embodying Avatars

Researchers Saadia Khan and John Black have pioneered what they call “surrogate embodied cognition,” having students use avatars in virtual worlds to enhance learning. Though the students are only manipulating an avatar on a screen, because they identify with the avatar, they imagine themselves physically moving which, neuroscience shows, activates the same parts of the brain as if the students were actually moving in real life. Going forward, we could see the “surrogate” removed from the techniques pioneered for “surrogate embodied cognition” as students go from controlling avatars with their hands to moving through virtual space with their whole bodies.

Understanding Unconscious Aspects of Mastery

Oftentimes, when an experienced performer in any given craft is trying to describe to a novice the process of executing a task they have mastery over, they are not consciously aware of some of their most important actions. But full-body capture technology could reveal some of these techniques. For instance, the creators of a specialized set of eye-tracking glasses conducted a study that mapped where a master pianist looks when they are playing a piano as a way of better understanding the unconscious activities that master practitioners engage in. Going forward, machines will increasingly track not just our movements and conscious expressions, but also track things our body is doing that we might not be aware of, uncovering new insights into the learning process.

Rewiring the Brain

Researchers at Duke University conducted an experiment using VR to help paraplegics control robotic exoskeleton legs. After the study ended, researchers were surprised to discover that seven out of eight of their paraplegic subjects (who had been using VR mostly to train the brain-computer interface to the robotic legs) had actually begun to regain motor control of their real legs. This points to the potential to use VR to actually rewire neural pathways in the human body, opening opportunities for learning that we can scarcely begin to conceive.