M16: 16-bit greyscale and limited colour, full resolution

M16 allows the user to display in high-bit-depth grayscale, at any supported resolution (e.g., full HD 1920 x 1080) and 120 Hz refresh. It works by reassigning the 8 bits of the red and green colour channels of a given pixel as a single 16-bit greyscale output.

The remaining 8 bits of the blue colour channel may be optionally used to index a user-defined 256 x 3 CLUT containing 16-bit R, G and B pixel values. Images assigned to a row in this table will be drawn in the designated colour as an overlay on the main greyscale image.

To use M16 mode you will need to ensure your image drawing software is capable of specifying 16-bit values. In MATLAB, the PsychImaging tools available through Psychtoolbox include a function to instruct drawing commands to use 32-bit floating point values. Enabling this setting allows users to exceed the standard 8 bpc colour value assignment.

Examples

Drawing a grey rectangle in M16 mode (MATLAB/Psychtoolbox)

Note this example requires the use of Psychtoolbox for MATLAB. The first step is to enable 16-bit encoding and M16 using wrapper commands in PsychImaging. Then we can assign a single 16-bit grey value, expressed as a proportion of the maximum (e.g., from 0-1), to the drawing commands available with the Screen() function.

MATLAB

%Establish the correct colour and video settings
AssertOpenGL;
PsychImaging('PrepareConfiguration');
PsychImaging('AddTask', 'General', 'FloatingPoint32Bit');
PsychImaging('AddTask', 'General', 'EnableDataPixxM16OutputWithOverlay');

%If using the PROPixx - uncomment the following to change the sequencer 
%for linearized output (trades off luminance). Make sure to reset to 0 at
%the end of your script.
%Datapixx('SetPropixxDlpSequenceProgram', 6);
%Datapixx('RegWr');

%Open a full-screen window with a black background
screenNumber = max(Screen('Screens'));
[win, winRect] = PsychImaging('OpenWindow', screenNumber, [0,0,0]);

%Determine our target gray value as a proportion of the maximum possible value
max16BitGray = 65535;
myGrayLevel = 1026/max16BitGray;

%Draw our rectangle in the top left corner of the window and flip
Screen('FillRect', win, myGrayLevel, [0, 0, 400, 400]);
Screen('Flip', win);

Adding a 16-bit colour overlay

The following example builds off of example 1. Here we are adding code to generate some red text as an overlay.

To add colour as an overlay, we must first define a CLUT, and open a special overlay window using Psychtoolbox commands. Then we will draw our text to the overlay window with the appropriate index to our lookup table. Note that the lookup table is expecting RGB values with 16 bpc (i.e., 0-65535 levels, expressed as a proportion from 0-1).

On Windows, due to PTB’s scaling methods, row 1 is ignored. Row 2 is indexed as 1, Row 3 as 2, etc. The MATLAB example below demonstrates this.

MATLAB

%Generate our colour look-up table (CLUT) for our main display.
%CLUT values are 0-1; the output is 16 bit (0-65535). 

%!! ON WINDOWS!!
% PTB scales colour by 255/256, therefore CLUT counting starts at the second row.
% See example below.

%!! ON OS-X !!
%CLUT assignment seems to have issues with mapping onto CLUTs. Consider making multiple (~5) rows 
%The same colour and calling the middle row to ensure a buffer. 

%The clut is first populated with black and then the first two rows are changed to blue and red
myTable=repmat([0,0,0], [256,1]);
myTable(2,:) = [0,0,1]; %Row 1 in Windows
myTable(3,:) = [1,0,0]; %Row 2... etc
Datapixx('SetVideoClut', myTable);
Datapixx('RegWr'); %straight calls to our Datapixx API require an explicit register write or update

%Get our overlay window
overlay = PsychImaging('GetOverlayWindow', win);

%Turn off antialiasing, since it can create strange effects when using CLUTs
Screen('Preference', 'TextAntiAliasing', 0); 

%Let's put some blue text in the overlay. The last argument here is the index of our lookup table
DrawFormattedText(overlay, 'Here is some text', 'center', 40, 1);  
Screen('Flip', win);

Using a DATAPixx3 with a third-party display? You can still use the overlay feature. Simply follow the same procedure above; the DATAPixx3 will pass 8-bit instead of 16-bit values to the display.

Adding “hidden content” to the experimenter console

This example builds off the previous two examples.

In a typical VPixx setup, there is a second monitor connected to the Video Out 2 or Console Out port on the VPixx hardware. This second “console” monitor always shows a copy of the video to the stimulus display.

Because this view is a copy of the video signal, it does not put any additional pressure on the control PC’s graphics card. It is therefore ideal as an experimenter’s view during data collection. However, it cannot show unique material to the experimenter unless a special video mode is used.

For example, a researcher might want to keep track of participant performance on their console display, but hide this information from the participant. The PLDAPs Toolbox (pronounced “platypus”) by Eastman and Huk [1] uses this strategy to show real-time eye tracking data and trial metadata on the console during experiments.

To add unique content to the console, we must:

Define a colour which the hardware should ignore (a ‘transparency’)
Create a second 256 x 3 16-bit RGB CLUT for the console monitor
Populate the main display CLUT with transparencies in designated rows, where the corresponding row in the console CLUT contains a non-transparency colour

When drawing content to the overlay, we index a single row that invokes the stimulus display CLUT and the console display CLUT on respective monitors. In this case, the overlay text is ignored on the stimulus display but shown on the console.

MATLAB

%Let's set an arbitrary transparency colour of full green
myTransparency = [0,1,0];
Datapixx('SetVideoClutTransparencyColor', myTransparency);
Datapixx('EnableVideoClutTransparencyColorMode');
Datapixx('RegWr');

%Create a copy of the first CLUT and populate row three with our transparency (stimulus display) 
%and full blue (console) 
myConsoleTable = myTable;
myTable(3,:) = myTransparency; %Row 2 in Windows
myConsoleTable(3,:) = [0,0,1]; %Row 2 in Windows

Datapixx('SetVideoClut', [myTable; myConsoleTable]);
Datapixx('RegWr'); 

%Draw our text with index 3
DrawFormattedText(overlay, 'Trial 2 begins', 'center', 40, 2);  
Screen('Flip', win);

Transparency colours can also be used in the console lookup table if you want to do the reverse of this example and show something only on the stimulus display instead.

For more examples of M16 mode, please see our demo on Understanding CLUTs, M16 Mode, and Color Transparency.

Blending, interpolating and antialiasing can cause strange results when using CLUTs. It is best to avoid these in our high bit depth modes like M16. See What is Pixel Identity Passthrough? for more details.

References

[1] Eastman, KM. and Huk, AC. (2012) PLDAPS: a hardware architecture and software toolbox for neurophysiology requiring complex visual stimuli and online behavioural control. Front. Neuroinform. 6:1. doi: 10.3389/fninf.2012.00001