In the search for truth at the bottom of human behavior and motivation, you may find the retrospective think-aloud (RTA) protocol a useful tool. RTA involves participants watching a video replay of their gaze, recorded while they were performing a specified task, and explaining the thoughts and feelings which accompanied carrying out this task. This method helps preserve natural behavior in the moment and gives access to a person's motivations with more accuracy than plain reflection.
This article explores the use of the RTA interview alongside eye tracking and other data collection methods as a means to get more valuable insights from your research.
If you ask ten researchers to name the most valuable output of eye tracking, you'll probably get ten different answers. Heat maps or engagement metrics or impact zones or visibility benchmarks – the variety of pathways from raw data to actionable findings is nearly endless. And yet, there is one powerful benefit that I rarely see mentioned: the ability to enhance qualitative interview results. Most of the research reports you'll read from the field of eye tracking are all about fixation data, those handy X and Y coordinates that allow us to build a quantified aggregate understanding of the visual experience. The value of these kinds of results is beyond dispute, but it's also worthwhile to consider how attention data can be used to enhance more subjective, perception-based, messy methodologies.
The traditional interview dilemma: If you're a qualitative moderator executing a study, your job is to elicit true and accurate feedback as a means of answering critical questions on a topic of interest. It doesn't sound that hard – talking to normal people about their thoughts, experiences, and motivations – but in reality it's incredibly challenging, mostly because people are challenging. Interview participants can be forgetful. They can be biased. They can be eager to please and fuzzy on details and unaware of their blind spots, and they can even be untruthful at times, without even realizing it. All of these natural pitfalls of qualitative research make it hard to collect accurate data, especially when the interview is often far removed from the context that the interviewee is asked to discuss (e.g. sitting in a bland, taupe focus group room trying to vividly describe the experience of filling your cart at the grocery store).
The think-aloud dilemma: Part of the dilemma described above has been solved by incorporating "Think-Aloud" interview protocol, in which the moderator elicits feedback in real-time as the interviewees complete the activities that they are being asked about (e.g. ditch the focus group and ask the shopper what they are doing while they are actually shopping for groceries.) This type of concurrent interviewing cuts down on forgetting and captures more detailed contextual insights, but there is one major downside. The Observer Effect is a concept from physics that suggests that the mere act of observing a phenomenon changes the phenomenon being observed. Admittedly, qualitative research is a long way from physics, and yet the Observer Effect is an apt consideration in this case. For a participant, the act of verbalizing an experience to a moderator during a task (i.e. thinking-aloud) will always make the observed task less natural. So in effect this technique changes the behavior of interest while simultaneously capturing feedback on that unnaturally changed behavior. Obviously, this is not ideal for building results that may be submitted to a journal or used to inform critical business decisions.
As described above, two of the main drawbacks to qualitative research are (1) out-of-context traditional interviewing can be unreliable and (2) in-context "Think-Aloud" interviewing can be misleading. But what if there was a way to capture all of the detailed, second-by-second insights from a "Think-Aloud" while maintaining the unobtrusiveness of a traditional interview? This is where eye tracking swoops in and saves the day. The Retrospective Think-Aloud Interview (RTA) is a technique that utilizes post-interaction replay of gaze video as a visual cue during a qualitative interview. This may sound complex at first, but actually it's a simple, straightforward method:
Moderating an RTA requires some training, but you don't need to be a vision scientist or computer wizard to run this type of interview. Once learned, the technique quickly begins to feel automatic, even for old-schoolers and technophobes. The learning curve is quite steep, which makes perfect sense when you think about it. Humans are intensely visual creatures. Our experience is built sensation by sensation, and none of our senses account for more of these building-blocks than our sense of sight. For a qualitative moderator charged with understanding people as deeply as possible, the ability to see what they see is a huge advantage. The further step of closing the loop and engaging in a detailed retrospective discussion of an activity feels quite natural to both the participant and the moderator because – to paraphrase William James – attention is experience.
That's why the learning curve is steep. The RTA is not an example of technology disconnecting researcher from subject; it's quite the opposite, in fact. The use of eye tracking in qualitative interviewing creates a level of connection between moderator and participant that is not achievable through any other means. As researchers, the closer we can get to living the experience of real people, the better we will be able to understand people, explain behavior, and reveal truth.
About the author: Mike Bartels is the director of marketing research and user experience for Tobii Pro in North America. He has a master’s degree in experimental psychology and 12 years of experience within the field of eye-tracking research and attention measurement, across a range of different areas of commercial and academic fields. His publications include eye tracking–related articles for several marketing research magazines including Quirks, QRCA Views, MRA Alert, and a book chapter in Eye Tracking in User Experience Design (published in 2014 by Schall and Bergstrom). He has presented at several scientific conferences including the Human-Computer Interaction International Conference, IIEX and the Eye Tracking Research and Applications Conference.