English English

Scaled and distributed data collection for screen-based projects

Tobii Pro Lab Screen based projects

In this article you will learn how to conduct a distributed data collection with Pro Lab, for screen-based projects.

Distributed data collection enables you to both scale your study and save time. By distributed data collection, we mean, the ability to deploy multiple data collection sessions across different locations and eye tracking setups. For example, you can reduce the time needed to reach the sample size goal of your study by running simultaneous data collection sessions with two or more eye tracking setups, or, collaborate with other research teams to perform large- scale and/or geographically separated studies. In this article you will learn how to conduct a distributed data collection with Pro Lab.

Distributed data collection scenario

In order to see how we implement it in Pro Lab, let’s imagine the following scenario: As a lead researcher you are responsible for running a study where you need to collect data from two subject groups, located in two different geographical locations. In this scenario you can opt to travel to each location with your equipment and perform the data collection yourself, or, instead collaborate with local research groups, and outsource the data collection to their labs.


Let’s go with the second option, as it has the advantage of speeding up your data collection and cuts down on the travelling costs. From previous contacts and collaborations, you know that there are two research groups, one in each location, that have a similar Tobii Pro Eye tracker and a Pro Lab license and have been conducting similar studies, so you decide to contact them and propose to collaborate in this project. You agree the following:

  1. You will take the role of the lead researcher and be responsible for designing the study and analyzing the data.
  2. Each partner research group will recruit the participants and collect the necessary data in the target locations.
  3. You will use Pro Lab to present your stimuli, record and process the data.

The biggest challenge you have in this scenario, is that outsourcing your data collection requires providing multiple teams access to the experimental materials and procedures needed to complete the same study. This adds higher requirements on transparency and careful documentation of the data collection process. This includes, recruiting and debriefing participants, operating the setup, etc. The following workflow example illustrates how you can take advantage of different features in Pro Lab to help keep the process consistent and transparent across multiple teams.

Workflow example

Below you will find a possible workflow to keep data collection consistency in this scenario:
 

  1. Prepare your experiment design then brief and discuss it with your collaborators.
  2. Implement the experimental design in Tobii Pro Lab.
    1. Create a project and build the experiment trial structure. Tip: Keep your participant experimental design  in mind (this will later help you decide how to implement in the most efficient way your independent variables, read Between- and within-subject experimental designs in Pro Lab for examples of implementation of different designs, and Structuring the stimulus presentation and dealing with order effects in Pro Lab to learn how to build your experiment trial structure).
    2. Add your independent and extraneous variables to the project (e.g. participant variables, stimulusi variables , timeline names, etc.). Read What are experimental variables? for more information.
    3. Define your sample size (n)  and add (n) participants to your project. Tip: You can number and order your participants to reflect your sampling strategy and pre-assign values of the planned the participant variables values  to the participants. (read Independent Variables and Participant Groups to learn on how to add participant variables).
    4. Create AOIs, AOI tags and manual TOIs if your analysis requires it. You will use it in step 4 when you to test your implementation (read the following articles to learn how to: Digging Into Areas of Interest and Digging into intervals and Times of Interest). 
    5. Define and configure  your metrics export file, by selecting the relevant metrics (dependent variables) and data (Independent variables). You will use it in step 4 when you test your implementation (read the following articles to learn how to: Calculating and exporting eye tracking metrics in Tobii Pro Lab and Understanding Pro Lab's eye tracking metrics). 
  3. Write a data collection protocol. Include participant recruitment information, data quality checks and recording exclusion criteria.
  4. Test your implementation and data protocols. Perform a small pilot study  in your lab to see if the instructions and criteria are clear and that you can export the independent and dependent variables in the correct format.
  5. Once you are happy with your implementation and protocol, export two versions of the project (read the section below Pro Lab project import and export steps):
    1. Version A, containing half of the participants, or the participants you have assigned to location A.
    2. Version B, containing the other half, or the participants you have assigned to location B.
  6. Send the generated zip files to the respective research groups (i.e. according to location) along with the data collection protocols.
  7. Each research group will then open the project in Pro Lab (read the section below entitled Pro Lab project import and export steps to learn how to) and execute the data collection according to the instructions in the protocol. This includes recruiting the participants and documenting any deviations to the data collection process.
  8. Once the data set is complete each research group exports the data with the relevant recordings and sends back the generated zip file, together with any documentation generated during the sessions.
  9. Import the two files into the original Pro Lab project (read the section below Pro Lab project import and export steps), review the recordings and perform your metrics export for further analysis.

In this example, you used the the participant management features in pro lab to set your sample size, define the sampling order and standardize the participant information accross sites. Additionaly you used the export features to localise your participants (by selecting only to export part of them to each group). 

It is important to instruct the data collection teams do not change the stimulius and trial structure in the Design module of Pro Lab. Any changes won’t be able to later be re-imported into the original project and may create conflicts when aggregating the data.

  

Exporting a project in Pro Lab 

  1. Open an existing Pro Lab project.
  2. On the Project overview click Export to open the export menu and select Project.
  3. On the Project export popup menu deselect the Recordings and Participants you want to exclude from the export.
  4. Click Browse and select a target folder where you want to save the file and name.
  5. Name your export file (you can choose a different name from the current project name) by entering typing in the File name text box.
  6. Click Export to export the project to the folder you selected.
 

Opening an exported project for the first time

  1. Unzip the project to a target folder.
  2. Open the un-zipped project in Pro Lab.
  3. Open the Project overview to check the content of the project – Recordings, Participants, Events and Variables.
 
Importing recordings and participants into the orginal Project
  1. Open the original project in Pro Lab project.
  2. On the Project overview click Import to open the import menu and select Project.
  3. On the Import Lab recordings popup window select the exported zipped project file with the data you wish to import.
  4. Next on the Import Lab recordings popup menu deselect the Recordings you want to exclude from the export.
  5. Click Export to export the project to the folder you selected.
  6. The new recordings will show up in the Recordings list in the Project overview.

! The imported project needs to have been generated from the original project.

 

Table 1. Project import conflicts - this table lists the import behavior of Pro Lab if changes are made to the participant variables and values during data collection. The general rule of thumb is that changes to variables with pre-assigned values, or directly to the pre-assigned values, are detected by Tobii Pro Lab and dealt as an information conflict between the original file and the data collection file. The import outcome will contain new participants containing the changes and keep the old participants with the original values. This will alert the test lead or the researcher responsible for the data analysis that changes were done to the original variables and values and a decision needs to be done. 
Original project     Data collection project  Original project
Participant Participant variable Value Change Import outcome
Participant01 Eye color Set to Green Changes value to Blue Conflict, Pro Lab creates a new participant entitled Participant01(2) with Eye color value set to Blue
Participant01 Eye color  Not specified Changes value to Blue No conflict, the value Blue is assigned to Participant01
Participant01 Eye color Set to Green or Not specified Adds a new participant variable No conflict, the new variable and respective value(s) are added to Participant01 in addition to the existing variables
Participant01 Eye color Set to Green Adds a new value Brown to the existing variable Eye color, and selects it Conflict, Pro Lab creates a new participant entitled Participant01(2) with the Eye color value set to Brown
Participant01 Eye color Not specified Adds a new value Brown to the existing variable Eye color, and selects it  No conflict, the value Brown is assigned to Participant01
Participant01 Eye color Set to Green Deletes the existing variable Eye color Conflict, Pro Lab creates a new participant entitled Participant01(2)
Participant01 Eye color Not specified Deletes the existing variable Eye color No conflict, however Pro Lab discards the change and keeps the variable Eye color for Participant01, with the value set to Not specified
Participant01 Eye color Set to Green Deletes the value Blue from the variable Eye color No conflict, the value Green remains assigned to Participant01
Participant01 Eye color Set to Green Deletes the value Green from the variable Eye Color Conflict, Pro Lab creates a new participant entitled Participant01(2)
Participant01 Eye color Not specified Deletes the value Blue or Green from the variable Eye color No conflict, the variable Eye color value remains Not specified for  Partcicipant01 
Participant01 Eye color Set to Green or Not specified Renames Variable Eye color to Iris color No conflict, a second variable named Iris color is added to Participant01
Participant01 Eye color Set to Green Renames the value Green to Grey in the variable Eye color Conflict, Pro Lab creates a new participant entitled Participant01(2) with the Eye color set to the value Grey
Participant01 Eye color Not specified Renames the value Green to Grey in the variable Eye color No conflict, the value Grey is added to the variable Eye color as a new value 
Pro Lab project export and import steps Show less

Distributed data collection offers different advantages when it comes to saving time or scaling the data collection phase of a study. This in turn allows studies to achieve high statistical power by increasing the sample size and opens the opportunity for different research groups to share and replicate the same study and verify the generalizability of the tested effects.

This article introduces one possible scenario of distributed data collection and how to best implement it with Tobii Pro Lab. The scenario can be used as a start point to implement a distributed data collection procedure in your own study and tweeked to your own needs and type of collaboration.  

Related Articles