A Scientist's Guide to Frame Dropping

In this short guide, we explain what frame drops are, why they are important, and how to avoid them in your studies.

Let’s start with a simple definition:

A Frame Drop (also known as a frame skip or FPS drop) occurs when an image is not drawn fast enough to be submitted to the graphics card on the next video frame. Instead of presenting the image, the content of the previous video frame is repeated. The new image is then presented in the following video frame.

Frame drops are not always noticeable to the viewer. The most obvious cases of frame drops are on dynamic displays, where a moving stimulus may appear to jitter rather than move smoothly.

Frame drops can also cause problems for frame-sequential stereoscopic displays, where images are presented to the left and right eyes on consecutive frames. If a frame is dropped, this can de-synchronize output to the left and right eyes and destroy the 3D effect.

Even when frame drops are not obvious to the viewer, they can have negative consequences for a study. For example, if you are presenting a stimulus for a fixed time, a frame drop can cause the stimulus to appear or disappear later than desired.

Why Do Frame Drops Happen?

Frame drops usually occur because your system cannot keep up with the speed of the requested video output. Key culprits include:

• Overloaded CPU (e.g., many background processes draining resources)

• Limited CPU resources (e.g., battery power, power-saving mode)

• Old, underpowered or overloaded GPU

• Requesting a “Wait” for a specific amount of time (thus blocking drawing processes)

• Checking for keypresses, mouse presses or other user input

• Complex graphics processing between frames (e.g., blending, rendering complex images)

Diagnosing Frame Drops in Your Experiment

Several psychology software tools, including Psychtoolbox, test for frame dropping when an on-screen window is first opened.

This is usually done by sending Flip commands to the graphics card and waiting for a timestamp; if the timestamp between flips is longer than the anticipated elapsed time of a single video frame, this is an indication the frame was dropped. If the system detects many frame drops within a short period, it may throw a warning or error message.

These preliminary tests are useful for diagnosing issues with your system. However, they won’t detect frame drops caused by stimulus generation within your study. For this, it is useful to keep track of the timestamps returned on each frame flip, and compare them to the expected duration.

What To Do if You Have Too Many Frame Drops

The team at PsychoPy has come up with a helpful list of suggestions for how to reduce frame dropping by your system.

Some of these methods are simple, such as turning off unnecessary processes on the PC, while others require some planning and code modifications.

Several of our software commands and modes are designed “defensively” to be robust against frame dropping. For example:

• Pixel Mode triggers will always execute on the video frame where the pixel is shown. If a frame is dropped, delaying pixel onset, the trigger will reflect the actual pixel onset.

• Similarly, commands executed with a register write on Pixel Sync will always occur on the frame the pixel sequence appears on. If this frame is delayed, the Pixel Sync commands will reflect the actual sequence onset.

• Stereo BlueLine Mode checks each video frame for a marker indicating which eye the image should be sent to, which ensures the correct polarization of the video frame regardless of when it appears. This ensures frame dropping cannot desynchronize the stereo output.