Figure 10: Different Hand Gestures

For navigation the gestures and tilt sensor orientation information is used. The gestures are mapped to different speeds: slow, fast, back, and none. The tilt sensors are mapped to the rotation. See table below.

Table 3: Glove Data Mapping for Navigation

As one rotates the hand to the right, the view moves to the right, and similarly for the left. The figure below shows the mapping of hand orientation to view orientation.

Figure 11: Mapping Hand Rotation to Change in Orientation

When manipulating the properties of an object the information from the fingers were used essentially as sliders. Sliders as typically used on windows dialog boxes are bars that are moved along a straight line that allow the choice of values between a maximum and a minimum. A volume control for a sound player may use a slider bar, for example. The fingers are used in this same way. The flex of the finger has a maximum and a minimum flex value. This value is mapped to the property that it is controlling. See table below.

Table 4: Glove Data Mapping for Object Manipulation

Note: F1, F2, …, F5 correspond to the thumb, index, …, and pinky fingers respectively.

There are other ways to map this data. This seemed to be the most straightforward at the time of implementation. Another way to control object properties would be by using the different gestures. The gestures could map to controls such as "increase" and "decrease". This interaction technique would result in an entirely different means of interaction. With the device interface other glove interaction ideas can be quickly prototyped and tested.

The magnetic tracker used is the Polhemus Fastrak. The device driver for the Polhemus returns six bytes of information: three bytes for position and three bytes for orientation. The data from the device driver is packaged by the Polhemus interface into a position vector and an orientation vector.

The device mapping stage uses the information from the device interface to generate the following mappings:

Table 5: Magnetic Tracker Mapping for Navigation

As the table above illustrates, moving the sensor forward or backward controls navigational speed. At the moment the speed is a switch function. You are either going forward, or backward, or you are not moving. It would be easy enough to change this mapping to a proportional or exponential controller as a function of the Y coordinate. The navigation orientation is controlled by the yaw value of the orientation sensor. The figure below demonstrates how this mapping is computed.

Figure 12: Polhemus Mapping Hand Rotation to Change in Orientation

All object property manipulation is done using the position vector of the sensor. In the case of position, the object in the virtual world is mapped to move in correspondence with the movement of the tracker sensor. As the sensor is moved to the right the virtual object will also move to the right. In the case of scaling, only coordinate values in the positive quadrant are mapped. Since negative scales do not make sense (unless it is used to invert the shape), only positive scale values are accepted.

Table 6: Magnetic Tracker Mapping for Object Manipulation

In order to change the color of an object, the color cube interaction technique was used. With the color cube, the three color components (red, green, and blue) are mapped to each axis of the cube. In one corner lies the (0,0,0) coordinate that maps to black. At the other extremity is (1,1,1) which maps to white. If you translate along one of the axis, the corresponding color component will change. Mapping the tracker input to the color cube was easy as it is a direct mapping. In this case, as with object scaling, only values in the positive quadrant were accepted.

Speech input was emulated using a dialog box (see Figure 13). The mapping however does not change. In the case of a voice recognition system, you would program the system to accept a set of voice commands. The voice emulator program provides access to a set of simulated voice commands that are contained in a dialog box list. In both cases a command is a result of the stimulus. For this reason the device mapping to a command will remain constant, as it will only be the device interface that changes.

Figure 13: Snapshot of Voice Emulator Dialog Box

The commands in the dialog box list (see Figure 13) are used to map to the navigation and object manipulation. Table 7 shows that the mapping is direct in its meaning. The only mapping that is required is converting a command into a message for the client (Table 8).

Table 7: Speech Recognition Data Mapping for Navigation

Table 8: Speech Recognition Data Mapping for Object Manipulation

The keyboard can also be used as an input device, as it is the most widely used input device for computers. Although keyboards may not be ideal inputs for 3D interaction, it is a common interface and one that should be addressed. The keyboard input maps similarity as voice input. A set of key presses are interpreted and mapped to a set of commands.

The keys used for navigation are the same as are used in many popular 3D games (such as Quakeä ). The way in which it is mapped, however, is different. An increase speed command, does just that, increase the speed. In conventional interfaces in order to move forward, one must repeatedly press the up arrow. In order to reduce the overall bandwidth of the system only single speed values are sent to the client. The client will set the current speed to a value and continue moving at that speed until it receives a command that tells it to do otherwise. In order to stop, the user must slow down, or press the down arrow key. The same logic holds true for changes in orientation. Only start rotate and stop rotate commands are used, as opposed to continually streaming orientation change commands to the client.

Table 9: Keyboard Data Mapping for Navigation

For object manipulation most operations require six degrees of freedom or three degrees of freedom if rotation is not needed. X, Y, and Z position changes have been mapped to the right/left arrows, up/down arrows, and page up/down keys respectively. A similar mapping is done for color manipulation.

Table 10: Keyboard Data Mapping for Object Manipulation

What was been shown thus far is an architecture and toolkit that add interaction functionality to VRML-based worlds. In this regard, the client/server architecture can be used with VRML applications and a number of different input devices can be easily used, and tested for 3D interaction. The next issue I will discuss is system performance using the test configurations described above to enable the use of alternative input devices (e.g. keyboard, speech, position tracker, and glove) to navigate and manipulate objects within the world described in Figure 9.