Structuring the stimulus presentation and dealing with order effects in Pro Lab

Tobii Pro Lab Design Screen based projects

Learn how to plan and structure your stimulus presentation order in Pro Lab

In a few cases, the structure of your experiment is very simple: just add the stimuli (the images and videos) you want to track the gaze on and run the experiment. However, in most experiments, it is not that simple. One reason for this is that we have order effects. This means that how we react to something is influenced by the order in which we perceive it.

For example, at the beginning of an experiment you don’t know what to expect, you perhaps don’t know what to look for to solve your task, or you have not even formed an idea of what the researcher is after yet. So how you view an image is different at the beginning of the experiment, compared to the end of the experiment. We could call this a training effect: you become better at solving the task with more training and experience of the task. Alternatively, you could also consider an effect of the opposite kind. At the beginning of the experiment, you are full of excitement and wonder at being part of your first psychological experiment. You are attentive and quick to respond. However, after 30 minutes of responding to close to a hundred trials of almost-similar stimuli, you find yourself low on energy and slow to respond. You are now the victim of a fatigue effect.

The consequence of these two order effects is that stimuli that are presented in the beginning is different from the material presented in the end, and this difference is not driven by the individual stimuli as such. You run the risk of falsely concluding that the first half of the stimuli was much easier than the latter half of the stimuli.

We want to prevent one or several stimulus items in carrying a disproportionate part of these order effects, as otherwise, it is unclear what viewing behavior is driven by the stimulus as such and what is driven by the inexperience or fatigue. Ideally, we want to distribute the learning and fatigue effects evenly across all stimulus items. One way to do this is to randomize the presentation order of the stimuli. If the experiment consists of a group of images, then we would simply shuffle the order, so for one participant an item could end up right at the beginning of the experiment, but for another, it could end up in the middle or at the end of the experiment. On average, after making many recordings with this experiment, the stimuli will end up roughly equally distributed in all positions. 

Shuffle/randomization

Figure 1. Shows a set of stimuli, here ordered 1-6, that is then shuffled in a random order for every new recording.

Structuring your experiment using shuffle, repetitions, sampling, and randomization

Depending on the experiment and the type of stimuli, different presentation orders are relevant. The simplest possible form would be to simply show all the stimuli in whatever order they have been put in the group, from the first to the last. Such a sequential order would be very appropriate if you have a group of images containing instructions for the participants. Obviously, they have a reading order and should not come randomly.

For other stimuli, like experimentally manipulated images, a random order makes the most sense, for the aforementioned reasons of order effects. In many experiments, however, the trial does not consist of just a group of images. You may have a fixation cross in the center of the screen to start off the participants in a search task from the same point. In this experiment, we do not want to just randomize the presentation order of everything, so we end up with several images or fixation crosses in a row. Instead, we want to have the fixation cross fixed at the beginning of every trial, and then just sample one image from a group of images, one at a time. 

Search experiment implementation

Figure 2. Shows a study design where a fixation cross is displayed at the beginning of the test and is then followed by a randomly sampled image from a pool of different stimuli images.

This sampling, however, can occur in different ways. The simplest to understand is to sequentially sample one item at a time. If you have a particular order you want to present the images, but every image should be preceded by some fixed element, then this choice makes sense. Then the fixed elements will occur, and then an image from the group will be sampled, until all of them have been sampled.

The most common choice, however, is random without replacement. This means you randomly sample one item from this group, and once it is consumed, it is not put back in the group. So, it will be impossible to pick the same item twice in a row (unless the list is repopulated with items again). A clear benefit of this approach is that it guarantees that the data set is balanced, in the sense that you show every item the same number of times.


The third way of sampling items is random with replacement. The difference, as revealed by the name, is that after sampling a particular element, it goes back into the group again. This means you can pick the same element again during the next draw.

If we have a group of stimuli, say three of them, then we want to iterate our grouping showing a fixation cross and a sample a new image. One way to do this is to wrap this group in another group with a repetition behavior and repeat it three times so all the stimuli are exhausted.

Search experiment implementation

Figure 3. When presenting the design shown in this figure to the experiment participant, they will see, as Trial 1, a fixation cross followed by a random visual search array. Then, as Trial 2, a fixation cross again, and one of the remaining arrays, and as Trial 3 a fixation cross followed by the last array stimulus.

In many experiments, showing something once does not give you enough data to uncover the effect of experimental manipulation. For an experiment where repeat viewing of the same material is not problematic, it would make sense to create several cycles of the same material. To add another cycle, and have two presentations of every stimulus, then one approach is to simply double the number of repetitions from 3 to 6. Note that for sampling randomly without replacement, the stimuli are not actually removed forever when they are sampled, they are simply temporarily excluded from selection. If the repetition continues with a fourth repetition, then all stimuli are back as available for selection as again. The second approach to produce two cycles is to wrap the group with three repetitions in another group with two repetitions. This would produce 2x3 = 6 stimulus presentations.

If the experiment simply consists of showing a random order of all the stimuli, then it may be an inelegant way to add a group with sample behavior and another group with repetition to be able to display all the material. The simpler way is simply to create a group with a shuffle behavior. This makes the overall build less complicated because there is one less group to keep track of.

However, it is often productive to consider several of these mechanisms together, to be able to build up a richer presentation flow in the experiment. For example, a trial can be seen as a group itself, containing a fixed part, and then a random part that samples from another group. This trial can then exist as different copies with small changes to the variants. Each of these trial groups can then be located inside a group which repeats, until all the material it samples is consumed. These groups are then contained in a single group which shuffles the presentation order of the repeated trial groups. This way it is possible to scale up an experiment from a pilot test to a full experiment very quickly. Such an example is illustrated below.

Search experiment implementation

Figure 4. Shows a more complex presentation order in a visual search experiment. This experiment starts randomly with a block of either X/O arrays or T/L arrays, and then performs three trials of these, with a fixation cross before each. Then it continues with the next block, with a fixation cross before each search array. 

Watch a tutorial

Watch the video above to see how to set up your stimulus presentation in Pro Lab using Group elements.

Playlist:

  • 0:00 Introduction to Group elements and the example experiment design
  • 2:00 Example experiment structure
  • 3:06 Creating a new project
  • 3:23 Adding images to the timeline
  • 4:00 Creating a group with action set to - show all, shuffle
  • 4:44 Creating a group with action set to - show all, sequential
  • 6:16 Creating a group with action set to - repeat
  • 7:14 Creating a group with action set to - sample, random without replacement
  • 8:04 Copy-pasting and editing a Group
  • 9:12 Introduction to Stimulus variables and how to create one
  • 10:14 Assigning a variable to a group
  • 10:33 Final result
  • 11:32 Conclusion

Using "Group" elements to implement the study structure in Pro Lab

Once you have designed your study structure, you are ready to implement it in Pro Lab. To start this process, go to the Design module of the software and start to organize your stimulus presentation elements on the timeline. The timeline is a linear representation of your presentation sequence. Besides being able to add stimuli elements directly to it, e.g. images or videos files, you can also add a "Group". A “Group” is a container, with its own timeline, to which you can add other presentation elements. You can assign different presentation rules or actions to each "Group". These allow you to "break" the linear or sequential presentation of the timeline and introduce presentation loops, sampling rules, and shuffling (randomization) of presentation elements. You can use "Groups" to apply the rules and actions discussed in the previous sections to the stimulus presentation.

Available "Group" actions

  • Show all: displays all elements in the groups...
    • Sequential: ...in sequential order
    • Shuffle: ...in random order without replacement
Action: Show all sequential
Action: Show all shuffle
  • Repeat: displays repeatedly "n" times and in sequence the content of the group.
Action: Repeat
  • Sample: samples "n" number of elements from the content of the group...
    • Sequential: ...in sequential order
    • Random without replacement: ...in random order without repeating the same elements
    • Random with replacement: ...in random order and may repeat elements
Action: Sample sequential
Action: Sample random without replacement
Action: Sample random with replacement

Try and implement an experimental design using groups

Let's see how you can implement the example in Figure 2 in Pro Lab. This time create an "Instruction" and "Thank you" element and add it to the experiment:

  1. Download the images below this section - "Visual search images"
  2. Start by creating a new "Screen Based" project and naming it “My first search study”.
  3. Click on Design to open the Design tab. On the main window, a timeline “Timeline 1” with a pre-defined “Eyetracker Calibration” element should appear.
  4. Create and add an "Instruction" and "Thank you" elements using an image editing software (e.g. Microsoft Paint). Save those images as .PNG files.
  5. Add the "Instruction" and "Thank you" elements to the timeline. 
  6. Click on the "Instruction" element and in the ”Properties” panel check “Keypress” and uncheck “Time” in the “Advanced on” options. This allows the participant to read the instruction at his or her own pace and then proceed to the next element by pressing any key on the keyboard.
  7. Click on the "Thank you" element and this time set a fixed display time - in the “Advanced on” option change the display time to 3 seconds.
  8. Next, Add a "Group" element to the timeline and place it in between the Instruction and "Thank you" elements.
  9. In the “Properties” panel change the “Name” of the group to “Search block”, keep the “Action” property set to “Show all” and the “Presentation order” to “Sequential”. The “Sequential” action ensures that the elements that will be added to the group are shown according to their order on the timeline.
  10. Double-click on the “Search block" element to open the group’s timeline.
  11. Add the "Fixation cross" element to this new timeline. 
  12. Click on the "Fixation cross" element and in the “Properties” panel, set “Advance on” to 2 seconds to display the image for 2 seconds.
  13. Add a new "Group" element to the "Search block" timeline, and place it after the "Fixation cross" element.
  14. Name this group “Search array sampler”, since it is used to sample one image from its content.
  15. In the properties pane, change the “Action” property to “Sample” set the “Presentation order” to “Random with replacement”. These settings make sure that the images are sampled at random and that none is shown more than once during the stimulus presentation.
  16. Add the images to the "Search array sampler" group... Double-click on the “Search array sampler” element to open the group’s timeline, then open windows file explorer, select the 3 search array stimuli images and drag to the “Search array sampler” timeline.
  17. While all images are still selected, in the “Properties” panel,  set “Advance on” to “Keypress” and uncheck “Time” in the “Advanced on” options. These properties will apply to all the images.

You are now ready to run your first test recording. Every time you run a recording the subject will see the following presentation structure: Instruction element > "Fixation cross" element > Randomly selected "Search array" element > "Thank you" element.

"Group" elements is a versatile feature that allows you to combine different presentation rules to implement the different levels of experiment design complexity in Tobii Pro Lab. 

Related Articles