What is Pixel Identity Passthrough?

Pixel Identity Passthrough refers to the 1-to-1 mapping of user-assigned pixel values to the display.

Why does it matter?

VPixx devices rely on pixel identity passthrough for several functions. Specifically:

Pixel Mode uses pixel values to generate automated digital triggers
Pixel Sync listens for a specific sequence of pixel values, to trigger a hardware sync
Some of our high bit depth modes combine pixel values or assign new values to pixels using custom colour lookup tables
Stereo blue lines (PROPixx and VIEWPixx3D) rely on pixel identity passthrough in the bottom row of the image to assign video frames to the left or right eyes.

Often problems encountered with these specific features can be traced back to pixel value transformations made by experiment software or the graphics card. In this guide, we describe some of these transformations, and provide general advice on how to measure, disable and/or accommodate them when using VPixx device features.

Common sources of pixel value transformation

Pixel value transformations can occur with or without the user’s knowledge. These transformations may be global, i.e., applied to the entire image, or local, i.e., applied to specific locations or stimuli within an image.

Below are some examples of common pixel value transformations and where they are implemented.

Name	Implemented by	Effect
Blending	Experiment software	Part of the image rendering pipeline in software; mixes the colours of different image layers that spatially overlap to create a transparency effect. Many algorithms exist for blending. Commonly used for applying image masks and filters, or morphing between images. May be a global or local effect.
Anti-aliasing or interpolation	Experiment software	Modifies pixels along edges to smooth jagged and pixelated contours. Commonly used for text or high-contrast shapes and lines. A local effect which only affects edges.
Colour space transformations	Experiment software	Colour spaces may use normalized values (e.g., 0 - 1 or -1 to 1) rather than explicitly specifying precise n-bit values (e.g., 0-255 for 8 bits per colour). Transformations between colour spaces may introduce rounding errors that change the final display output. Global effect.
Colour corrections	Experiment software, graphics card	This method uses lookup tables to systematically transform colour values assigned by the user to some other value specified in by the table. Commonly used to systematically alter colour values according to some requirement (e.g., gamma correction). Global effect.
Image borders and overlays	Experiment software, other software	Some UI tools, recording software or antivirus software, can apply a thin border or overlay to your display. Within experiment software, this is typically an interactive UI border or taskbar which may or may not always be visible. A local effect that only changes the specific area where the border or task bar appear.
Dithering	Graphics card	A technique used by modern graphics cards that either temporally or spatially “jitters” pixel values to create the illusion of a high bit-depth display. We have an entire guide to how to detect and disable it here: Diagnosing and Disabling Dithering in the Graphics Pipeline

None of these transformations is bad, per se. Some of them are very useful for designing experiment stimuli. Measuring and understanding the transformations present in your system has three key benefits:

Identify undesirable transformations and disable them
Ensure desired transformations have the intended effect, and thus can be replicated
Understand how transformations affect pixel identity passthrough in order to continue using VPixx features (see our final section for more on this).

How to measure changes to pixel identity passthrough

On typical display hardware, pixel identity passthrough cannot be verified as it requires direct access to the video signal after it leaves the graphics processor.

As such, researchers often have to take it on faith that undesirable transformations have been turned off, and their applied transformations are working as expected. Often, the first sign that something is wrong happens when a special feature like Pixel Sync that relies on pixel identity passthrough mysteriously fails.

Fortunately, most VPixx devices have specialized tools that allow users to read the pixel values of the top row of the video signal directly. This is done using the GetVideoLine command. This command returns a 2D array with one column for each pixel, and one row for R, G and B values. GetVideoLine will fetch data from the current video frame.

Time (18).png — *GetVideoLine returns the R.G,B values of the pixels in the top row of the display, as measured in the video signal **after** leaving the graphics card.*

Below are some examples of calls to GetVideoLine in our MATLAB and Python APIs.

MATLAB Example

MATLAB

%Register update to collect most recent state from hardware, followed by vline
Datapixx('RegWrRd'); 
vline = Datapixx('GetVideoLine');

Python Example (Libdpx wrapper)

PY

#Register update to get most recent state from hardware, followed by a vline
DPxUpdateRegCache()
vline = DPxGetVidLine()

Python Example (Object oriented)

PY

#Register update to get most recent state from hardware, followed by a vline
vpx.updateRegisterCache()
vline = vpx.getVideoLine()

GetVideoLine can only read the top row of pixels on the display. If you suspect local pixel value transformations elsewhere on the screen, you may need to move the particular stimulus with the transformations such that it overlaps with this top row for testing purposes.

GetVideoLine is available with the following VPixx devices:

DATAPixx Series I/O hubs
VIEWPixx /3D and VIEWPixx 12-bit
PROPixx controller

MATLAB USERS: By default, PsychImaging windows that invoke the ‘UseDatapixx’ configuration intentionally blank the top row of pixels to reserve them for PsychImaging’s built-in pixel sync commands. You will need to disable this blanking to check vline and verify pixel identity passthrough. To do so, open PsychDatapixx.m and comment out the following lines during your test. We suggest re-enabling this feature when not testing vline.

How to deal with pixel value transformations

Once you have identified pixel transformations and their source, you have a few options:

If you don’t want the transformations

Some local transformations can be disabled on a case-by-case basis. For example, in PsychoPy you can set the line property ‘Interpolate’ to false for certain lines. See our Python Blue Line demo for a demonstration of this.

Global effects like dithering may be disabled via Psychtoolbox commands or your graphics card control panel. See this page for more details.

In some cases you can disable unwanted colour corrections by explicitly setting the graphics card lookup table to be linear (a gamma of 1). In MATLAB, Psychtoolbox permits this with the following command:

MATLAB

% Load a linear ramp into the graphics card's LUT, such that the graphics
% card does not further modify our pixel values.
Screen('LoadNormalizedGammaTable',window,linspace(0,1,256)'*ones(1,3));

Similarly, you can disable text antialiasing in MATLAB with:

CODE

Screen('Preference', 'TextAntiAliasing', 0);

If you do want the transformations:

The following advice pertains to users wishing to implement Pixel Mode, Pixel Sync or Stereo Blue Lines. In almost all cases, pixel value transformations will have unpredictable and undesirable effects on our high bit depth modes. We do not recommend keeping any transformations while using high bit depth modes.

If you want to keep your pixel value transformations while using VPixx features that require pixel identity passthrough, you have two options:

First, you can try to use pixels that are unlikely to be transformed. Maximum values, where one or more of the RGB channels are set to 255 (e.g., full red, cyan, magenta, white), are good candidates. This is a particularly useful strategy for creating Pixel Sync triggers or Stereo Blue lines.

If this is not possible-- for example, if you need to specify multiple colours in Pixel Mode, the next option is to reverse-engineer your pixel values so that the expected transformations produce the correct value for your desired output.

Consider, for example, a common global transformation: a simple gamma correction using a gamma of 2.2.

If this type of correction is applied to your pixel values, then it is presumed all of your pixels will undergo the following transformations:

Gamma corrections applied to R, G and B values of a pixel

Here’s a visualization of the transformation process:

*Gamma correction: A simple method of correcting for display gamma to linearize luminance output*

Thus, if we know the pixel value we need to produce on the display hardware, we can input an adjusted pixel value, such that the corrected output is equal to our desired pixel value. This output can then be verified using GetVideoLine.

To continue the following example, say we want to pass a red pixel with the value [128,0,0]. We therefore must solve for:

Solving for R, we get a rounded value of 55. Therefore to achieve the Display output [128, 0, 0] we must pass [55, 0, 0].

This general approach can be used to reverse-engineer pixel identities to achieve the desired output. It is always a good idea to verify this output via GetVideoLine, or in the case of Pixel Mode, by checking your trigger output.

Summary

Pixel identity passthrough refers to the 1-1 mapping of user-assigned pixel values, to display output. In practice, many pixel value transformations may be applied, with or without the user’s knowledge, such that display output is different than the input.

Some of VPixx special tools for our displays and synchronization, notably Pixel Sync, Pixel Mode, Stereo Blue Line and our high bit depth display modes, rely on pixel identity passthrough. Pixel value transformations can cause these features to malfunction. Thus, it is a good idea to identify, and, where possible, disable pixel value transformations by your software and graphics card. The command GetVideoLine allows the user to read the top line of pixels in the video signal provided to the display, making it easy to identify pixel value transformations.

In the event that a user cannot disable pixel value transformations, it may be possible to work around them by a) choosing maximum pixel values that are unlikely to be affected, or b) determining and passing the untransformed pixel value, such that the transformed result produces the desired effects. Note this strategy will only work for some features. For others, (specifically high bit depth modes) we strongly advise turning off all transformations and ensuring pixel identity passthrough is preserved.