The use of eye tracking in virtual reality for training and development or to research human behavior and decision-making processes is highly effective because visual attention is so strongly correlated with cognition and performance. The study discussed by Dr Tim Holmes below, explores how behaviors that are observed in VR environments mirror those in real-world ones. This validates the use of VR as a tool to aid professional training and development, and expand research possibilities relating to product design, wayfinding, safety and much more.
For the past 14-years I have been researching how to best use technologies like eye tracking to understand and predict people’s responses to art and design. This is a challenging field of research because the behavioral responses we capture with an eye tracker are affected by the context in which we observe them.
We have seen for example, that the viewing patterns of large artworks are quite different when viewed on a small screen as opposed to a full-size one, and that environmental factors such as lighting, proximity to other artworks, and even the presence of a title or not can all change the way in which the artwork is viewed. Virtual reality presents a unique opportunity for research in this field since it allows eye-movement data to be collected whilst participants view works at scale from a comfortable viewing position in a controlled environment.
Of course, VR can do more than simply mimic real-world environments, it can also facilitate research into real-world scenarios without any dependence on the presence or existence of a physical environment - allowing design alternatives to be tested in context, non-longer existing works to be recreated, and even completely original and sometimes fanciful environments to be presented - as is the case in an ongoing project at Royal Holloway, University of London, being led by my former PhD supervisor Professor Johannes Zanker.
You are probably already familiar with the artist Piet Mondrian, who is best known for his abstract artworks comprising horizontal and vertical lines forming rectangles which are filled in white, grey, black, red, yellow, and blue.
However, you might be less familiar with his one, and only, foray into three-dimensional architecture, and for good reason - the design he created for Ida Bienart for an entire room in this same style was never actually realized in his lifetime. The reasons for this are debated but might include the way orthogonal lines no longer appear to form rectangles when viewed in three-dimensional perspective. This of course, is not the case once immersed in such an environment, where the perspective is no longer forced and changes with movement.
A few museums attempted to realize Mondrian’s design after his death, but these attempts were met with limited success. In fact, the creation of a 3D model based on his original plans requires some difficult decisions to be made - since Mondrian was no architect, his walls do not join up!
Perhaps more surprisingly, whilst the relative size and color of each rectangle is clear in the plans, the precise tint of the red, yellow and blue “paint” is not, and as any designer knows, color REALLY matters. It certainly did to Mondrian, since he believed his designs used color and shape to create a “universal harmony”.
It is here that research in VR really comes into its own, because the Unity 3D model was developed and tested iteratively to create a final design that felt both natural when viewed from ‘within’ it and also remained true to Mondrian’s aesthetic.
The research confirmed several expected results, such as:
Results like these confirm aspects of behavior we see in more natural environments such as supermarkets for example, and highlight the potential for crossover research in the art and retail sectors.
Other data collected in this environment has confirmed some eye-movement behaviors that are well reported outside of VR, such as a reduction in fixations when performing a targeted search task such as counting the colored the squares, and an increase in fixation duration when participants are allowed to ‘free-view’ in the Mondrian Room. Such results provide strong support for the use of VR to study naturalistic behavior, since similar results from real-world research can be seen within VR.
By comparing the physical build of the Mondrian Room in Dresden with the VR version, we observed an important result for anyone looking to use VR as a proxy for the real world. Overall viewing duration, fixation duration, and fixation counts were all slightly higher in VR than in the real world, and this is something we need to research further to fully understand. This might turn out to be an artefact of VR itself since it is still a novel technology for most participants and so there is a tendency to look around in wonder! For researchers, this highlights the need to familiarize participants with VR to allow the “wow” factor to fade before commencing data collection.
The flexibility of VR is one of its biggest advantages because we can easily change elements whilst keeping everything else constant. Moreover, the behavioral evidence confirms the value of eye tracking within any VR environment where the behavioral impact of design is of interest, be it architectural, educational, or retail.
Perhaps more importantly, the results described here confirm that eye movements recorded in VR are as informative as those we have been recording and analyzing for years using screen-based, and more recently, mobile eye trackers. This is important because it paves the way for VR applications that use the gaze data to learn about the viewer and adapt the VR experience to them.
VR is still a relatively new methodology for research using eye tracking, but results like those reported here, show that it is a good proxy for real-world research or training environments since the behaviors exhibited are so similar. For those looking to avail themselves of the scalability, cost, and time saving that VR can provide, the results from these studies should provide the confidence to dive-in and take a closer look at eye tracking within VR.
Dr Tim Holmes is a visual neuroscientist who researches the role that environment and design play in decision making and behavior. He is recognized as a leading authority on eye tracking and visual attention and has worked with brands, retailers, architects, content creators, and sports teams to educate on and develop behavioral interventions. Tim also works with many academic institutions, and is an award winning educator and public speaker on the application of neuroscience to behavioral influence.
Read more about the Mondrian Room VR project here.