Digging into intervals and Times of Interest

Tobii Pro Lab

Times of Interest (or simply TOIs) is a Pro Lab concept that provides an amazing degree of analytical flexibility. When used properly, it allows researchers and analysts to organize the recording data according to intervals of time during which meaningful behaviors and events take place.

Examples of TOI use:

  • selecting data for visualizations such as heat maps and gaze plots
  • selecting and aggregating data to calculate metrics associated with a coded task or subtask periods in your recording (recording task or trial analysis)
  • selecting and aggregating data to calculate metrics associated with coded subject behaviors or actions (behavioral coding)
  • selecting and aggregating data associated with media exposure and snapshot coverage (media or snapshot restricted task or trial analysis)

Times of Interest are created by specifying events and intervals, either of which can be system-generated or user-generated.    

Events

Events are multipurpose markers that can be used to locate important occurrences on the eye tracking timeline. Events as markers can be generated automatically by Pro Lab (RecordingStart, ImageStimulusStart, sync events), the participant (Keypress), or the researcher/analyst (Custom events). Events have an associated timestamp, the exact time the event marker was applied. They can be counted but they do not have any duration since they simply mark a meaningful point in time. The terminology we will use is that the creation of a named event creates a class of events and any instance thereafter is referred to simply as an “event.” Once you have a few event markers, though, you can do more than just count them and that brings us to the next key concept, intervals.

Intervals

Intervals are spans of time on the recording timeline that have a start and end point. Just as with events, intervals can be generated automatically by the system or the experiment (using the corresponding system-generated events). For example, there will always be an interval that corresponds to the entire duration of the recording from start to finish. And if you are running a screen-based study where images as stimuli are shown to the participant, then the system will automatically generate an interval in the timeline of each experiment that spans the period of time when an image appears on the screen. A helpful aspect of how Pro Lab implements intervals is that data on all intervals are available in the metrics output. This means that statistics on the duration and start/end point time stamps of all intervals can be obtained just by checking a box. 

Putting the concepts together

If this is your first time reading about these new concepts, there is likely to be some degree of confusion. To help pull these ideas together, we have put them all on a representative study timeline that is a schematic representation of what you would find in Pro Lab (see image below).

This diagram presents a study recording that lasts 64 seconds. Thus, the timeline starts at 0 and ends at 64. The corresponding system-generated events at 0 and 64 are RecordingStart and RecordingEnd (blue and green event markers respectively) and they automatically create a TOI entitled "Entire Recording". Since those events only occur once in the recording, this TOI is composed of a single interval that spans the whole Recording. Additionally, there are two user-created events StartEvent and EndEvent (purple and orange event markers respectively). This pair of events occurred three times on the timeline - at 5 and 7, 12 and 18, and 37 and 52 seconds (Recording time). Next, we create a custom TOI, by setting the StartEvent as the start point of an interval and EndEvent as the respective end point. Now whenever StartEvent is followed by EndEvent, Pro Lab will automatically create an interval (here, Intervals 1, 2, 3) and assign it to the TOI.

Keep in mind that while the study or timeline clock runs continuously from start to end (Recording time in this example), the custom time of interest "clock" (cTOI time) only runs, adding to the total duration of the TOI, whenever an interval as we have defined it is created. In this example, the custom time of interest "clock" starts when interval 1 starts, then pause when it ends (cTOI time 0-2s), restarting and pausing again when the 2nd interval starts and ends (cTOI time 2-8s), and restarting and ending when the 3rd interval starts and ends (cTOI time 8-23s). The intervals belonging to the cTOI also have an independent "clock" that resets every time an interval ends and begins (Interval time). These different "clocks" will play a role in the calculation of the different metrics included in the Metrics Export, i.e. when exporting 'Excel reports', the calculations are done using the cTOI "clock", while when exporting the 'Interval based TSV file' the metrics are calculated using the Interval "clock".    

intervals and Times Of Interest

Types of Times of Interest

According to how they are generated, we can divide TOIs into two general types -  System-generated TOIs (based on recording, media and snapshot events), and custom TOIs created manually by the user. 

System-generated TOIs

Depending on the type of project you are running - Screen, Glasses and Scene Camera, Pro Lab will automatically generate different TOIs. On the table below shows you which types of system-generated TOIs are created in recordings of different project types.  

 Project type
System-generated TOI IntervalsEventsScreen VR360 GlassesScene Camera
Recording Whole recording RecordingStart, RecordingStop yes yes yes yes
Media Images and videos ImageStimulusStart, ImageStimulusStop; VideoStimulusStart, VideoStimulusStop yes yes no no
Snapshot* Intervals mapped to a snapshot Snapshot[name] Interval Start, Snapshot[name] Interval End no no yes no

 

*Snapshot TOI’s are created automatically when eye tracking data is mapped onto that snapshot or stimuli. When you select a snapshot for analysis, your TOI will be composed of all the intervals defined by the two events ”Snapshot[name] Interval Start” and ”Snapshot[name] Interval End”. The ”Snapshot[name] Interval Start” event is created by the first gaze point/fixation mapped onto the snapshot. The ”Snapshot[name] Interval End” event is created by the last gaze point/fixation mapped on to the snapshot. If there is a gap of 5 s between two mapped gaze points, a ”Snapshot[name] Interval End” event is created for the last gaze point/fixation before the gap, and a ”Snapshot[name] Interval Start” event is created on the gaze point after the gap.

Custom-generated TOIs or simply, custom TOIs

Custom Times of Interest (custom TOI) are a new data analysis concept introduced in Pro Lab. The custom TOI is a flexible tool that allows the researcher or data analyst to precisely define and assign portions of their experiment to be analyzed. As described in the prior section on intervals, custom TOI are created by specifying the starting point and end point, either of which can be system-generated or user-generated events, the steps are simply:

  1. Create one or two user-generated events if not using any of the recording-generated events.
  2. Create the custom TOI in Pro Lab selecting one event as the start point and another as the endpoint.
  3. If necessary, select a backing visual (frame) from the Media Library on which to show gaze data visualizations and draw Areas of Interest.

The Media Library is the place where frames captured from the recording can be stored and called up for use as the backing visual for frame-based TOIs. The steps to create a backing visual from your recording is also simple:

  1. Find a frame in the video that you would like to use as the backing visual.
  2. Click the plus sign list box just above the timeline to the right of the current time indicator and choose “Save frame as media.”
  3. Name the file in the popup box that appears and press OK.

You are also able to create a TOI and assign a background image (frame) in one step. If you select the plus sign list box just below the window to the right and choose “Create TOI with frame.”

Watch a tutorial

Watch the video above to see how to set up TOIs, after that we’ll focus on an example of it's potential application.

Working with more complex TOIs

Adding offsets to custom TOI start and end events

Many eye-tracking studies do not use tasks that are fixed in time, like viewing an image for some amount of time. Rather, they have a more complex task that they can complete slowly or quickly, or it is part of a larger task where the demands are random and outside the direct control of the researcher. To be able to statistically compare experimental conditions, we cannot just calculate metrics such as total fixation duration for an object in such a task, because the total time the objects are in view may vary considerably, and an object in view for a long time will simply accumulate more fixations. For such studies, it would make more sense to decide on a fixed analysis window (a time of interest) and then project this around a certain event of interest. In Pro Lab, you can create these types of analysis intervals or adjust the existing intervals, by setting offsets to the start and stop events of your Times of Interest. 

To create TOI interval offsets follow these steps:

  1. Open a Recording tab.
  2. On the Time of Interest panel, click the "plus" icon to create a new custom TOI. Alternatively, if you want to apply an offset to an existing custom TOI click "Edit custom Time of Interest" (the pencil icon). 
  3. On the Create/Edit custom Time of Interest window select two events as a start and end point for your interval or intervals. 
  4. Toggle the Offset control, to show the seconds (s) and milliseconds (s) spin boxes. 
  5. Type your offset value in seconds (s) and/or milliseconds (s), in the spin boxes. If you use a negative value you will shift the start or end point of the interval to the left side of the start or end event on the timeline. A positive value shifts the start or end of an interval to the right side of the event. 
  6. Click Ok to save your custom TOI.

Application examples:

A researcher is interested in how long a shopper looks at a particular product until they ultimately put the product in the shopper cart. It can be problematic to count all fixations before this event. As the shopper approaches the aisle, he fixates the shelf with the product, but it is unclear whether the shopper is actually evaluating products at this time or is simply navigating towards the aisle. As this task lacks a distinct onset event but has a clear offset event (the final picking of a product), it makes sense to base the time of interest on the offset event. In this case, the researcher chooses to limit herself to the data between 10 seconds before the product choice and the actual product choice. When exporting the metrics, the researcher will have metrics based on 10-second intervals for all picked products, making it easy to fairly compare between different product categories, or whatever property is of interest.

 

TOI start point TOI end point
Offset:-10s Offset:0
Custom event: "Picked product type A" Custom event: "Picked product type A" 
Custom event: "Picked product type B" Custom event: "Picked product type B"

 

As another example, a researcher studies how a train traffic controllers react to unexpected events, such as accident messages, in a simulator. The researcher then manually codes these warning events in the recording, and then create a time of interest that use this event as an onset, and the same event plus 6 seconds as the offset. The researcher then exports metrics for the different areas of interest that cover the relevant information displays.

 

TOI start point TOI end point
Offset:0 Offset:6s
Custom event: "incoming warning message" Custom event: "incoming warning message"

 

As a final example, a designer in the automotive industry is testing various auditory alert signals, representing approaching cars from different directions. The areas of interest represent the two different side mirrors and the back mirror. For this study, the question of interest is how quickly the correct mirror is viewed, given a particular sound. Since drivers naturally look at the mirrors, even without any alert, it is important to have a control condition. The researcher then manually codes all audio alert events in the recording, and then create two times of interests, each spanning 2 seconds. Together, these TOIs cover the gaze behavior just before the alert, and just after the alert.

 

TOI "Control" start point TOI "Control" end point TOI "Test" start point TOI "Test" end point
Offset:0 Offset:2 Offset:-2 Offest:0
Custom event: "Alert Sound A" Custom event: "Alert Sound A" Custom event: "Alert Sound A" Custom event: "Alert Sound A"
Custom event: "Alert Sound B" Custom event: "Alert Sound B" Custom event: "Alert Sound B" Custom event: "Alert Sound B"

Creating a custom Time of Interest with multiple start and stop events

At the beginning of this learn article we have seen that we can use custom TOIs to group periods of the recording, that share the same start and end events into an analysis unit. However, in a lot of eye tracking experiments, it is common to encounter tasks and test designs that contain multiple periods in the recording (or stimuli), with distinct start and end events, that are part of the same process or that logically belong together. For example, you could make a TOI called “Baseline images” and another called “Experimental images”, and then associate the stimuli events with the respective TOIs. This will provide an informative TOI column in the metrics output, indicating what set the gaze on that stimulus belonged to. The TOI can thus be used as a way of keeping track of experimental conditions. To be able to group these periods (or stimuli) you can select multiple events that will count as the start or stop point for that custom Time of Interest (TOI).  

To create custom TOIs with multiple start and stop events follow these steps:

  1. Open a Recording tab.
  2. On the Times of Interest panel, click the "plus" icon to create a new custom TOI.
  3. Select two or more events as a start and end point for your TOI. If a TOI interval is not started, then any of the start events will cause it to begin. Once applied, any of the stop events will cause it to end.
  4. Click OK when you are done.

Application examples:

A researcher designs a study where a batch of 20 images is presented to the test participants. Although the content of the images is different, they are used to measure the same phenomenon, thus belong logically to the same analysis group. As a result, the researcher wants to assign all the periods in the recording when the test participants where exposed to those images into the same TOI. To achieve this, she creates a custom TOI where she multi-selects the 20 ImageStimulusStart events as the start point of the TOI and the 20 ImageStimulusStop events as the end point of the TOI. She also gives a name to the TOI that is representative of the phenomena or group.  The result will be a custom TOI with 20 intervals, that can be easily identified in the "Interval based (.tsv)" metrics export - the TOI column contains the group/phenomena name and Interval columns the image presentation order. 

 

TOI start point TOI end point interval
System-generated event: Image1StimulusStart  System-generated event: Image1StimulusEnd 1
System-generated event: Image2StimulusStart System-generated event: Image2StimulusEnd 2
System-generated event: Image3StimulusStart  System-generated event: Image3StimulusEnd 3
(...) (...) (...)
System-generated event: Image20StimulusStart System-generated event: Image20StimulusEnd 20

 

As another example, a researcher works with Tobii Glasses 2 to test consumer decision-making in the store. The different decisions are coded as events such as “picked up chosen product”, “picked up rejected product”, “checked shopping list”. Although these are initially analyzed as separate TOIs (using the event as start event, and then as end event the same event plus a temporal offset), the researcher would like to investigate if all decision-making follows a similar gaze behavior. To do this, she creates a “meta-TOI” that employs all events.

 

TOI start point TOI end point interval
Offsets:0 Offsets:6  
Custom event: “Picked up chosen product”  Custom event: “Picked up chosen product”  1
Custom event: “Picked up rejected product”  Custom event: “Picked up rejected product”  2
Custom event: “Checked shopping list”  Custom event: “Checked shopping list”  3

Conclusion

Times of Interest is a highly flexible data analytic tool in Pro Lab. Together with Areas of Interest, they provide useful and fine-grained capabilities for defining not only the spatial extent of your analyses (AOIs) but also the temporal span (TOIs). Used appropriately and with care, researchers can apply these tools to carry out powerful, sophisticated analyses of the most demanding stimulus presentations.

Related Articles