Can Virtual Images Provide Ambulatory Cues?

We began by using physical cues, to provide a basis for comparing the effectiveness of virtual cues. As mentioned in Section 4.2, T.R.'s unmedicated gait consists of a 3--4 inch shuffle per step.

As T.R. had indicated, placing a series of cards on the ground about 26 inches apart allowed him to walk with a normal stride length and speed, although he had to attend to the cues to do so.

The next step was to provide ``real'' cues indirectly through the VV Sport, instead of having T.R. see the cue directly. We trained a video camera on the row of cards, and fed the output from the video camera into the VV Sport. T.R. walked forward parallel to the cards, but with the cards out of his visual field. The video camera moved forward parallel to him, providing an indirect visual cue.

This technique produced moderately good initiation steps, but not smooth sustained ambulation. T.R. believed that the technique could have worked in principle. There were problems related to keeping a good synchronization between the motion and angle of the camera and T.R.'s motion and viewing angle.

T.R. and the HITL researchers had independently assembled collections of taped images to provide a range of test cues to be played back in the VV Sport. In addition, the HITL researchers and Virtual Vision collaborators wrote simple graphics animations, which ran on an Amiga 3000 and an IBM PC, respectively. These animations were fed live into the VV Sport.

The main advantage of the live animations was the ability to re-program them as the study proceeded. A second advantage of the Amiga 3000 was that we were able to use the output of a video camera as the background for the animation. This allowed us to test the effectiveness of matching the animation background to the floor as seen through the unobstructed part of the visor, rather than providing a black background.

The test images included film of an escalator approaching or receding, filmed movement over a tiled floor pattern, as well as static or dynamic computer animations. These animations consisted of a series of moving or static bars, grids, or flat squares. Some of the squares lay horizontally (with or without a shadow below), others vertically. We tried images with and without perspective, although we did not make an effort to precisely align the animation perspective with the real perspective the subject was seeing.

By setting the animated objects to move down the screen at about the rate that the subject was walking forward, we could give the impression that the object was at a fixed location on the floor. The animations were set to start over when they finished, thus providing a continuous visual cue.

For animations which are either a grid or which are moving in a plane, one wants the plane to correspond to the floor when the subject views the animation through the VV Sport display. Due to the angle of the subject's gaze, this means that viewed on the computer terminal the animation must appear to be coming out of the screen. We experimented with angles ranging from 0 to 35 out of the screen.

The static images turned out to be essentially useless (including those produced by T.R. himself). As T.R. stepped forward, the image would appear to move forward with him, destroying the ``spatial illusion'' of an obstacle to be stepped over.

The film of the escalator approaching perhaps would have been effective, except that it was moving too slowly. This again removed the sense of an obstacle at a fixed location in space. Another possibility is that the escalator provided too much context which was clearly inconsistent with what T.R. was seeing in the real world (the sides of the escalator, etc.).

Although filmed as a control case for the approaching escalator, we had some hopes that the receding escalator would fire a sort of ``catching up'' response, but it had no effect.

We got our best results with animations involving simple yellow squares moving down the screen at approximately the walking rate. We tried to arrange things so that two images were in view, about two and three steps ahead of the subject. This produced a series of good initiation steps, but not sustained full-stride ambulation.

Our general impression from this portion of the study was that the single most important factor was maintaining the ``spatial illusion'' of an object in a fixed location (``space stabilized'') which needed to be stepped over.

A realistic image which did not provide spatial stabilization could not produce kinesia paradoxa. By contrast, yellow squares moving down the screen at approximately the walking rate did produce kinesia paradoxa.

Fortunately, it turned out that the space stabilization did not need to be exact, at least to achieve the level of results observed. The animations used were moving at a fixed rate in a fixed direction down the screen. Inevitable head movements and walking speed variations prevent such a technique from achieving precise space stabilization . Other than at the crude level described, we did not investigate how variations in the space stabilization precision affect the quality of the gait.

We created a number on animations on the Amiga 3000 which showed that T.R. was able to switch speeds by switching his attention between different stimuli moving concurrently at different speeds in the VV Sport display.

It seems likely that adding realism to a properly space stabilized image will make it more effective. However, the research described in this thesis did not pursue this factor.

Using the output from a video camera trained at the floor as the background for the animations appeared to be counter-productive. It reduced the contrast of the stimulus, when compared to using a simple black background.

This portion of the study was mainly a search for factors which would ``leap out'' as being critical. Space stabilization did so. The other factor which ``leaped out'' was field-of-view, to be discussed in the next section. Factors which did not appear to be as crucial (but which will need to be looked at carefully in future research) include perspective, shadows, color, vertical versus horizontal aspect ratio of stimuli, and flat versus 3-dimensional cues. We tried presenting the display beneath both the dominant and non-dominant eye, without observing a dramatic difference. Oddly enough, the angle of moving animations relative to the plane of the ground did not seem to be crucial either.