Eye tracking Guerrilla Games’ Killzone 3

Valsplat, a Dutch user experience firm, used eye tracking to test Guerrilla Games' popular shooter game, Killzone 3. For high development-cost products such as video games, usability testing is a key component to ensure positive customer reactions and sales results.

Background to UX testing of game

Video gaming is a multi-billion dollar industry with steady sales growth worldwide. The release of a new game is the result of years of work and large investments. The developing company must make most of its sales within the first few weeks of a new release due to the competitive nature of the market.

Guerrilla Games is a Dutch video game developer known for its first-person shooter games Killzone and Killzone 2. The battleground environments in the Killzone games are vast and complex in structure, programming, and graphics. The work involved to create these virtual worlds for player interaction is substantial.

Guerrilla Games' Killzone 3, the biggest game ever made in the Netherlands (at the time of this study), is a 3-D first-person shooter game set on a futuristic planet. With a large development budget, quality assurance and gaming experience were crucial elements to the process. Guerilla Games collaborated with usability experts at Valsplat to create a series of gameplay testing sessions.

A Guerrilla Games‘ Killzone 3 screenshot.

As a game researcher, eye tracking players gives me very useful insights into player behavior, and I don’t have to make assumptions about where a player is actually looking.

Jeroen van der Heijden, Valsplat

Tools and methods

Guerrilla Games and the team at Valsplat developed a series of seven test sessions over a six-month period. In each session, six to eight players went through the entire nine levels of gameplay. Observers looked at biometric data, as well as questionnaires, to see participants' confusion, boredom, or frustration throughout the process.

Late in the development stages, Valsplat added eye tracking to perfect elements of the game. The usability tests in this round aimed to solve issues like encounters, boss fights, onscreen instructions, game objectives, and path finding.

Since Killzone 3 was designed to be played in widescreen format, a Tobii Pro T60 XL eye tracker was used in the study. The HDMI signal was split in two, with audio provided via headphones and the game displayed via DVI to maintain typical PlayStation 3 quality. This output was duplicated to enable real-time remote viewing. The research team used the Pro T60 XL to study both individual and cooperative gaming.

A usability test run in a group using Tobii Pro eye trackers.
Gaze plots overlaid on the Guerrilla Games‘ Killzone 3 video game.

Results

It became apparent to the Guerrilla Games developers that some of the eye tracking findings would not have been discovered using other usability testing methodologies. Here are some gameplay aspects that were highlighted during the Pro T60 XL tests with Valsplat:

Pathfinding: It can be difficult to understand how a player gets lost and why they might not see the intended path or the exit in a scene. Eye tracking revealed where the participant was actually looking. With this information, instead of highlighting the exit, developers might change the process by toning down objects stealing focus in the scene.

Visual storytelling: Plenty of effort is spent on elements outside of gameplay in order to make the player's world more immersive. For developers, this includes skybox work, background fights, or creating room themes. Eye tracking illustrated which parts the scene the player saw and, more importantly, did not see. Information like this can guide developers to improve placement and frequency when creating a virtual world.

Menu navigation, HUD, and tutorial texts: If a player is given a mission and fails to read the instructions, they may not understand the next steps in the game. This behavior can be detected early in the development process with eye tracking, allowing adjustments to be made in the timing of the text and the dialogue. HUD (heads-up display) is on-screen information shown as part of the user interface during gameplay. Eye tracking can also show what type of information is missed in this display or in the menu navigation.

Split-screen play: Using a two-player, single-screen interface, developers can study whether a user will look at their partner's side of the screen, as well as what they're looking at and why. The benefit to gameplay is uncertain, but this data can generate interesting options for future game development.

We are quickly realizing the potential of using eye tracking as an additional tool in game testing, giving us unique information about our users.

Seb Downie, Producer for Killzone 3, Guerrilla Games