Why is novelty important in VR/AR?

When you are in a novel situation, your brain processes information differently than when you are in your humdrum routine.  For example, can you remember your commute home last night?  Or what you ate for lunch last Wednesday?  In a normal routine, attention and memory don't encode as well compared to when you are doing something new and novel.  This is why some neuroscientists insist on driving home via a new route every night.  

Being able to change attention and memory processing has important consequences when you are making a VR/AR experience.  What things do you want your user to pay attention to?  What do you want them to remember after the experience?  Here's a model of cognitive processing that you can use to consider how new information is going to be presented to users called "Top-Down Bottom-Up."

Excerpted from: Mind: Journey to the Heart of Being Human’ by Daniel J Siegel:

In the view we will be using here, top-down refers to ways we have experienced things in the past and created generalised summaries or mental models, also known as schema, of those events. For example, if you’ve seen many dogs, you’ll have a general mental model or image of a generic dog. The next time you see a furry canine strolling by, your top-down processing might use that mental model to filter incoming visual input, and you won’t really see the uniqueness of this dog in front of you. You have overlaid your generalised image of ‘dog’ on top of the here-and-now perceptual stream of energy that creates the neural representation of ‘dog’. What you actually have in awareness is that amalgam of the top-down filtering of your experience.

So here, ‘top’ means that prior experience is activated, making it difficult to notice the unique and vibrant details of what is happening here and now. The top-down generalised notion of dog will shade and limit your perception of the actual animal in front of you. The benefit of top-down is that it makes your life more efficient. That’s a dog, I know what it is, I don’t need to expend any more energy than needed on insignificant, non-threatening things, so I’ll take my limited resources and apply them elsewhere. It saves time and energy, and therefore is cognitively efficient. That’s top-down processing.

On the other hand, if you’ve never seen a spiny anteater before, the first time you come across one on the trail, it will capture all of your attention, engaging your bottom-up processing so that you are seeing with beginner’s eyes. These are eyes leading to circuitry in the brain, not shaping and altering ongoing perception through the top-down filters of prior experience. You’ll be taking in as much pure sensation from eyesight as possible, without the top-down filter altering and limiting what you see now based on what you’ve seen before.

These screenshots are from UpLoadVR's playthrough of the game "Accounting" by Squanchtendo.  The game uses different environments, such as a windowless accounting office (image left) to contrast with fantasy worlds where there are talking trees (image right) and other bizarre characters.  When you are in the fantasy world of "Accounting," you are likely using "bottom up" cognitive processing.  People feel excited when they are trying something new.  They take information from the exterior world, move it up a level, add language, move it up, analyze, and the user experience shifts.  Novel experiences tend to be more pleasurable because they rely on sensation and not cognition. 

The most vivid part of the mind bubbles up through sensation and new experience when unencumbered by analytical thought.
— Daniel J. Siegel

More on sensation from Siegel:

Sensation might be as bottom-up as we get. Since we live in a body, our within-mind experience is shaped by the physical apparatus that lets us take in energy flow from the outside world. We have our first five senses of sight, hearing, smell, taste and touch; we have our proprioceptive sense of motion; and we have our ‘interoceptive’ sense of the signals from the interior of the body. These perceptual capacities to sense the outer world and internal bodily world are built upon the physical neural machinery that enables energy to flow. Information is created with these energy patterns, generated as ions flow in and out of membranes and chemicals are released in the pathways of neural activity. As energy flows into the brain from our external sense organs, such as our eyes and ears, or from internal receptors of our body’s muscles, bones and internal organs, we move from sensation to perception – with pure sensation as close as we get to being fully present in the world.

Key Takeaways for VR/AR experience designers

  • Novelty increases awareness, which influences attention and memory
  • Top-down is the analytical system, which relies on prior experience
  • Bottom-up is the sensation-driven system, which looks with the eyes of a beginner 
  • Take a new route home on your commute tonight to experience novelty yourself.  Then look for ways to integrate that type of novelty into your experience.  


Further reading: 

Siegel, Daniel J. The Open Mind.  Aeon Magazine.  October 25, 2016

UploadVR full play through of "Accounting"