Tracking Devices

Authors: Gregory Baratoff and Scott Blanksteen

Tracking devices allow a virtual reality system to monitor the position and orientation of selected body parts of the user. Many interaction devices (see "Interaction Devices") incorporate a tracking device of some sort in order to measure the position and orientation of the body part they're attached to. In an HMD, the position and orientation of the head is measured. This information defines the user's viewpoint in the virtual world, and determines which part of it should be rendered to the visual display, as well as influencing the generation of acoustic stimuli. Attached to a glove, a tracking device measures the position and orientation of the hand. Based on this information, a hand can be rendered in the virtual world at the same position with respect to the user, providing feedback that is often necessary for dextrous manipulation.

Tracking devices, also called 6-degree-of-freedom (6-DOF) devices, work by measuring the position (x, y, and z coordinates), and the orientation (yaw, pitch, and roll) with respect to a reference point or state. In terms of hardware, the following three components are in general required : a source that generates a signal, a sensor that receives the signal, and a control box that processes the signal and communicates with the computer. Depending on the technology used, either the source or the sensor is attached to the body, with the other placed at a fixed spot in the environment, serving as a reference point.

The usefulness of tracking devices in virtual environments depends to a large degree on whether the computer can track the movements of the user fast enough to keep the virtual world synchronized with the user's actions. This ability is determined by the lag, or latency, of the signal, and the sensor's update rate. The signal lag is the delay between the change of the position and orientation of the target being tracked and the report of the change to the computer. Lags above 50 milliseconds are perceptible to the user and affect human performance. The update rate is the rate at which measurements are reported to the computer. Typical update rates are between 30 and 60 updates per second.

The precision with which actions can be executed in the virtual world depend on the resolution and accuracy of a tracking device used. Whereas the resolution is fixed for a given device, the accuracy usually decreases with the distance of the sensor from the source. The range of a tracking device is the maximum distance between sensor and source up to which the position and orientation can be measured with a specified accuracy.

Interference, or sensitivity to environmental factors, can limit the effectiveness of tracking devices. Depending on the technology used, they can be sensitive to large metal objects, radiation from display monitors, various sounds, and objects coming between source and sensor. During the design of the physical environment, these factors should be carefully considered, so that the user doesn't have to stay aware of the properties of the physical environment while engaged in a task in a virtual environment. This is especially important in fully immersive virtual environments where the outside view is completely blocked.

Most currently used tracking devices are active, in that the sensor, or sometimes the source, is attached to the target to be tracked. In passive tracking the target is monitored from a distance by one or several cameras. Although this approach is to be favored from the user's perspective, at this point in time this is not a technologically viable solution.

Current tracking devices are based on electromagnetic, acoustic, mechanical, or optical technology. A presentation of each of these approaches follows, together with a discussion of their advantages and disadvantages.

Mechanical tracking devices

These devices measure position and orientation by using a direct mechanical connection between a reference point and the target. Typically, a light-weight arm connects a control box to a headband, and encoders placed at the joints of the arm measure the change in position and orientation with respect to the reference point. The lag for mechanical trackers is very short (less than 5msec), their update rate is fairly high (300 updates per second), and they are accurate. Their main disadvantage is that the user's motion is constrained by the mechanical arm. An example of such a mechanical tracking device is the Boom developed by Fake Space Labs.

Inertial tracking devices represent a different mechanical approach, relying on the principle of conservation of angular momentum. These trackers use a couple of miniature gyroscopes to measure orientation changes. If full 6-DOF tracking ability is required, they must be supplemented by some position tracking device. A gyroscope consists of a rapidly spinning wheel suspended in a housing. The mechanical laws cause the wheel to resist any change in orientation. This resistance can be measured, and converted into the yaw, pitch, and roll values. Inertial tracking devices are fast and accurate, and since they don't have a separate source, their range is only limited by the length of the cable to the control box or computer. Their main disadvantage is the drift between actual and reported values that is accumulated over time, and can be as high as 10 degres per minute.

Optical tracking devices

Most optical tracking devices currently used in virtual environments are for tracking head position and orientation. Basically, they come in two variants. In the first one, one or several cameras are mounted on top of the HMD, and a set of infrared LEDs is placed above the head at fixed locations in the environment. In the alternative setup, the cameras are mounted on the ceiling, or a fixed frame, and a few LEDs are placed at fixed and known positions on the HMD. In both approaches, the projections of the LEDs on the cameras image planes contain enough information to uniquely identify the position and orientation of the head. Various photogrammetric methods exist for computing this transformation.

The optoelectronic ceiling tracker developed at the University of North Carolina is an example of a head tracker. The system consists three cameras mounted on the HMD, and 1000 infrared LEDs placed uniformly across the ceiling. The computer pulses the LEDs sequentially and processes the images to detect the flashes. Based on the locations of the flashes, the position and orientation of the head are calculated. The range of the optoelectronic ceiling tracker is limited only by the area of the ceiling covered by LEDs, and is thus easily expandable. Its update rate is between 50 an 80 Hz, and the lag varies between 20 and 80 ms. The resolution is 2 mm in position and 0.1 degrees in orientation. A problem with this tracker is that for some positions of the head very few or none of the LEDs are visible to the cameras, leading to 'blind spots' of the tracker.

The alternative design solution is exemplified by the Honeywell LED array system. Four infrared LEDs, arranged in a prescribed pattern on the HMD, are monitored by a camera mounted at a fixed position in the environment. As with the UNC system, the LEDs are pulsed one at a time, and the positions of the resulting flashes on the camera images together with the known relations between the LEDs are used to compute the position and orientation of the head. A variant of this system avoids the use of active elements, that is the LEDs, on the HMD. A reflecting pattern is placed on the HMD, and is illuminated by an external infrared source to make it visible to the camera.

Optical trackers in general have high update rates, and sufficiently short lags. However, they suffer from the line of sight problem, in that any obstacle between sensor and source seriously degrades the tracker's performance. Ambient light and infrared radiation also adversely affect optical tracker performance. As a result, the environment must be carefully designed to eliminate as much as possible these causes of uncertainty.

Electromagnetic Tracking Devices

Electromagnetic tracking devices function by measuring the strength of the magnetic fields generated by sending current through three small wire coils, oriented perpendicular to one another. These three coils are embedded in a small unit that is attached to whatever the system needs to track - typically, the user. The current has the effect of making each wire into an electromagnet while the current is flowing through it. By sequentially activating each of the wires, and measuring the magnetic fields generated on each of three other perpendicular wire coils, it is possible to determine the position and orientation of the sending unit.

These tracking units may experience interference operating in the vicinity of CRTs or other devices that produce magnetic fields, as well as metal objects, such as office furniture, that disrupt magnetic fields. Another disadvantage to these tracking devices is that the working volume tends to be rather small.

The most well-known producer of electromagnetic sensor technology is Polhemus. Their systems provide extremely low latency (on the order of 5 milliseconds) and have the ability to track multiple objects concurrently.

Acoustic Tracking Devices

Acoustic tracking devices use ultrasonic (high-frequency) sound waves for measuring the position and orientation of the target object. There are two ways of doing this: so-called time-of-flight tracking and phase-coherence tracking.

Time-of-flight tracking works by measuring the amount of time that it takes for sound emitted by transmitters on the target to reach sensors located at fixed positions in the environment. The transmitters emit sounds at known times, and only one is active at a time. By measuring when the sounds arrive at the various sensors, the system can determine the length of time it took for the sound to travel from the target to the sensors, and thereby calculate the distance from the target to each of the sensors. Since there will only be one point inside the volume delimited by the sensors that satisfies the equations for all three distances, the position of the target can be determined. In order to find position, only one of the transmitters is needed. Orientation is determined by the differences in location indicated by these calculations for each of the three sensors.

Time-of-flight trackers typically suffer from a low update rate, brought about by the low speed of sound in air. Of course, another problem is that the speed of sound in air is affected by such environmental factors as temperature, barometric pressure, and humidity.

Phase coherence tracking works by measuring the difference in phase between sound waves emitted by a transmitter on the target and those emitted by a transmitter at some reference point. The phase of a sound represents the position on the sound wave, and is measured in degrees: 360 degrees is equivalent to one wavelength difference. This is clear if one thinks of a sound that is a pure sine wave. The graph of the sine and cosine describes a circle as the angle progresses from 0 degrees to 360 degrees. After 360 degrees (one cycle, or wavelength), the graph returns to its starting point. As long as the distance traveled by the target is less than one wavelength between updates, the system can update the position of the target. By using multiple transmitters, as with time-of-flight tracking, orientation can also be determined.

Since they work by periodic updates of position, rather than by measuring absolute position at each time step, phase-coherence tracking devices are subject to error accumulation over time.


Through our investigation of tracking devices, we see that many different approaches have been tried, all of which have their own advantages and disadvantages. It is clear that the mouse and keyboard of virtual reality have yet to be discovered. All of the devices we have described are good for some environments and tasks, and fail on others, and, while we don't claim that the mouse and keyboard are perfect, they are certainly effective in a broad range of tasks, easy to use, not cumbersome, and inexpensive. Researchers are still actively seeking the tracking device with a large working volume, high accuracy and resolution, very short lag time and high update rate, that is convenient for the user. Until such devices are found, it will be hard to achieve the virtual reality goal of having the computer disappear.

[Table of Contents][Next Chapter]

Human Interface Technology Laboratory