English English

Scaled and distributed data collection for screen-based projects

Tobii Pro Lab Screen based projects

In this article you will learn how to conduct a distributed data collection with Pro Lab, for screen-based projects.

Distributed data collection enables you to both scale your study and save time. By distributed data collection, we mean, the ability to deploy multiple data collection sessions across different locations and eye tracking setups. For example, you can reduce the time needed to reach the sample size goal of your study by running simultaneous data collection sessions with two or more eye tracking setups, or, collaborate with other research teams to perform large- scale and/or geographically separated studies. In this article you will learn how to conduct a distributed data collection with Pro Lab.

Distributed data collection scenario

In order to see how we implement it in Pro Lab, let’s imagine the following scenario: As a lead researcher you are responsible for running a study where you need to collect data from two subject groups, located in two different geographical locations. In this scenario you can opt to travel to each location with your equipment and perform the data collection yourself, or, instead collaborate with local research groups, and outsource the data collection to their labs.


Let’s go with the second option, as it has the advantage of speeding up your data collection and cuts down on the travelling costs. From previous contacts and collaborations, you know that there are two research groups, one in each location, that have a similar Tobii Pro Eye tracker and a Pro Lab license and have been conducting similar studies, so you decide to contact them and propose to collaborate in this project. You agree the following:

  1. You will take the role of the lead researcher and be responsible for designing the study and analyzing the data.
  2. Each partner research group will recruit the participants and collect the necessary data in the target locations.
  3. You will use Pro Lab to present your stimuli, record and process the data.

The biggest challenge you have in this scenario, is that outsourcing your data collection requires providing multiple teams access to the experimental materials and procedures needed to complete the same study. This adds higher requirements on transparency and careful documentation of the data collection process. This includes, recruiting and debriefing participants, operating the setup, etc. The following workflow example illustrates how you can take advantage of different features in Pro Lab to help keep the process consistent and transparent across multiple teams.

Workflow example

Below you will find a possible workflow to keep data collection consistency in this scenario:
 

  1. Prepare your experiment design then brief and discuss it with your collaborators.
  2. Implement the experimental design in Tobii Pro Lab.
    1. Create a project and build the experiment trial structure. Tip: Keep your participant experimental design  in mind (this will later help you decide how to implement in the most efficient way your independent variables, read Between- and within-subject experimental designs in Pro Lab for examples of implementation of different designs, and Structuring the stimulus presentation and dealing with order effects in Pro Lab to learn how to build your experiment trial structure).
    2. Add your independent and extraneous variables to the project (e.g. participant variables, stimulusi variables , timeline names, etc.). Read What are experimental variables? for more information.
    3. Define your sample size (n)  and add (n) participants to your project. Tip: You can number and order your participants to reflect your sampling strategy and pre-assign values of the planned the participant variables values  to the participants. (read Independent Variables and Participant Groups to learn on how to add participant variables).
    4. Create AOIs, AOI tags and manual TOIs if your analysis requires it. You will use it in step 4 when you to test your implementation (read the following articles to learn how to: Digging Into Areas of Interest and Digging into intervals and Times of Interest). 
    5. Define and configure  your metrics export file, by selecting the relevant metrics (dependent variables) and data (Independent variables). You will use it in step 4 when you test your implementation (read the following articles to learn how to: Calculating and exporting eye tracking metrics in Tobii Pro Lab and Understanding Pro Lab's eye tracking metrics). 
  3. Write a data collection protocol. Include participant recruitment information, data quality checks and recording exclusion criteria.
  4. Test your implementation and data protocols. Perform a small pilot study  in your lab to see if the instructions and criteria are clear and that you can export the independent and dependent variables in the correct format.
  5. Once you are happy with your implementation and protocol, export two versions of the project (read the section below Pro Lab project import and export steps):
    1. Version A, containing half of the participants, or the participants you have assigned to location A.
    2. Version B, containing the other half, or the participants you have assigned to location B.
  6. Send the generated zip files to the respective research groups (i.e. according to location) along with the data collection protocols.
  7. Each research group will then open the project in Pro Lab (read the section below entitled Pro Lab project import and export steps to learn how to) and execute the data collection according to the instructions in the protocol. This includes recruiting the participants and documenting any deviations to the data collection process.
  8. Once the data set is complete each research group exports the data with the relevant recordings and sends back the generated zip file, together with any documentation generated during the sessions.
  9. Import the two files into the original Pro Lab project (read the section below Pro Lab project import and export steps), review the recordings and perform your metrics export for further analysis.

In this example, you used the the participant management features in pro lab to set your sample size, define the sampling order and standardize the participant information accross sites. Additionaly you used the export features to localise your participants (by selecting only to export part of them to each group). 

It is important to instruct the data collection teams do not change the stimulius and trial structure in the Design module of Pro Lab. Any changes won’t be able to later be re-imported into the original project and may create conflicts when aggregating the data.

  

Distributed data collection offers different advantages when it comes to saving time or scaling the data collection phase of a study. This in turn allows studies to achieve high statistical power by increasing the sample size and opens the opportunity for different research groups to share and replicate the same study and verify the generalizability of the tested effects.

This article introduces one possible scenario of distributed data collection and how to best implement it with Tobii Pro Lab. The scenario can be used as a start point to implement a distributed data collection procedure in your own study and tweeked to your own needs and type of collaboration.  

Related Articles