Global mouse control with Tobii EyeX/4c, with heavy data filters.
- Documentation and References
- Setting up the 4c
- Getting started
- Using Tobii SDK in your own projects
- Creating your own data streams from the Tobii SDK
- Smoothing Algorithm
- Libraries used in this Application
- Known Issues
- Operating System Support and Language Bindings
- Tobii License Information
Tobii Handbook (Helpful tutorials, slightly outdated)
Tobii API Documentation (Unfinished, slightly outdated)
- Position the eye tracker below your screen, angled slightly up.
- Plug the eye tracker into your computer's USB slot
- Download the Tobii Eye Tracking Core Software
- Open and follow the on-screen instructions to install.
- Click the Tobii Eye Tracking menu from the bottom right of the Window's taskbar
- Set-up a profile for each user (for most accurate results) and calibrate
To open or run this application locally, first clone/download/fork this repository.
Run the application from ./TobiiFormApp/Program.cs, the main class.
If you start a project from scratch, you will need to add the Tobii Core SDK into your project first. Here is a method to reference the SDK using NuGet Package Manager. The installation guides from Tobii's development website are outdated, so following this guide is recommended.
- Open Visual Studio
- Ensure that Nuget Package Manager is installed by clicking the Tools menu and looking for 'Nuget Package Manager'.
- If it is not there, go to 'Tools -> Get Tools and Features', search for NuGet Package Manager, and install.
- Expand the Tool menu and click the Options
- Expand the NuGet Package Manager from the left tree view
- Choose the Package Sources
- If nuget.org https://api.nuget.org/v3/index.json isn't listed as a package source:
- Type org in the name field
- Type https://api.nuget.org/v3/index.json repository URL in the source field
- Click on Update
- Restart Visual Studio
- Ensure that you have the nuget package source from the above step. Close NuGet Package Manager options if you have it open.
- Expand your project in the Solution Explorer
- Right click on the References
- Choose Manage Nuget Packages… from the menu
- Click Browse
- Choose the org package source to the right
- Search for Tobii
- Choose package Tobii.Interaction
- Click the latest stable version (I am using v0.73)
- Click Install
To create any data streams using the Tobii SDK, you first need to instantiate a host from the Tobii Interaction package.
using Tobii.Interaction;
...
private Host host = new Host();
The host creates a connection between the application and the Interaction engine, andis the main provider for the various eye tracking features. Intantiating a host provides one Tobii eye tracking context, and it is possible to create more than one host in a process.
To start a gaze point data stream, call Streams.CreateGazePointDataStream() on a host. Calling GazePoint(x,y,ts) on the data stream exposes functional access to the stream, where x is the gaze's X position relative to the top left corner of the screen, where y is the gaze's Y position relative to the top left corner of the screen, and where ts is the timestamp of when the data node was recorded.
GazePointDataStream gazePointDataStream = host.Streams.CreateGazePointDataStream();
gazePointDataStream.GazePoint((x, y, ts) => {
Console.WriteLine("Gaze Position and Timestamp: {0}\t X: {1} Y:{2}", ts, x, y));
});
Another option on how to implement the same functionality is by using the 'Next' event on the gaze point data stream instance
GazePointDataStream gazePointDataStream = host.Streams.CreateGazePointDataStream();
gazePointDataStream.Next += OnGazePointData;
private OnGazePointData(object sender, StreamData<GazePointData> streamData){
Console.WriteLine("Gaze Position and Timestamp: {0}\t X: {1} Y:{2}", streamData.Data.Timestamp, streamData.Data.X, streamData.Data.Y);
}
You can also choose to unfilter or lightly filter the gaze data in the creation of the data stream. There are currently only two options, unfiltered and lightly filtered. By default it is set at lightly filtered, but you can specify to unfiltered the stream by passing the enum value in the CreateGazePointDataStream() method params.
GazePointDataStream gazePointDataStream = host.Streams.CreateGazePointDataStream(GazePointDataMode.Unfiltered);
“Lightly filtered” is an adaptive filter which is weighted based on the age of the gaze data points GazePointData and the velocity of the eye movements. This filter is designed to remove noise and in the same time being responsive to quick eye movements. It is recommended to keep the filter on.
There are also three other types of data streams currently.
- Eye Position (physical location of the eyes)
- Fixation (stream of position and time of where the gaze gets fixated)
- Head Pose (phyiscal location of the head, only works on the 4c).
You can destory a host by calling the dispose method on a host instance.
host.Dispose();
This application contains basic live data smoothing techniques and algorithms to help smooth the noise of the eye tracking data stream.
If you are interested in possibling improving performance, Professor Manu Kumar from Stanford University has published a paper containing pseudocode for the implementation of a smoothing algorithm specifically for eye tracking.
It can be found here: http://hci.stanford.edu/cstr/reports/2007-03.pdf
- Window's scale setting must be set at 1:1 or 100%, otherwise gaze coordinates will be offset and not where the user is actually looking.
- A complete uninstall is required between each use with 2.11. There are no issues with the Tobii 4c
Operating Systems Supported: Windows7, Windows8, Windows10
Language Bindings: C#
This application and any applications developed during the hackathon event are under Tobii's Interactive Use license agreement. This means that the SDK can be used freely to develop games or other software where eye tracking data is used as a user input for interactive experiences.
However, this application and any applications developed during the hackathon event must not be used for analytic use. Tobii defines analytical use as the following.
Analytical Use is where eye tracking data is either (a) stored; or (b) transferred to another computing device or network; in both cases where the intent is to use or make it possible to use eye tracking data to analyze, record, visualize or interpret behavior or attention.