| Publications Page | HITL Home |

The Adaptive Effects of Virtual Interfaces: Vestibulo-Ocular Reflex and Simulator Sickness
by Mark Draper

Chapter 3
General Hypotheses

This chapter summarizes the two major hypotheses under investigation. Each hypothesis is described and defended using previous research results, supporting logic, and researcher assertions. Additional aspects and related issues are then considered. The intent of this chapter is to specifically describe the theoretical framework underlying the research performed. Additional details regarding virtual interfaces, virtual interface stimulus rearrangements, the VOR, VOR adaptation processes, and simulator sickness can be found in Chapter 2.

3.1 Hypothesis 1: Virtual interfaces May drive Vor adaptation

3.1.1 Description and Support

The first hypothesis states that stimulus rearrangements found in virtual interfaces can induce VOR recalibrations during typical, unrestricted, goal-driven interactions with the VE. This hypothesis is straightforward, as illustrated in Figure 14. It is an abbreviation of more detailed and complete neurophysiological models of VOR adaptation presented by Ito and Robinson (See Section 2.2). The novel aspect of this hypothesis lies within the shaded area.

Virtual interfaces are believed to induce visual-vestibular sensory rearrangements regarding self-motion (DiZio & Lackner 1992; Furness, 1994). These sensory rearrangements may result in the occurrence of retinal image slip during head movements (Peli, 1995). The function of the VOR is to help maintain a stable image on the retina during head movements through the initiation of involuntary compensatory eye movements (Robinson, 1981). Detected retinal slip during head movements may be interpreted as an error signal, activating VOR recalibration processes (Robinson, 1981). VOR gain and phase responses are then recalibrated to reduce existing retinal slip. Thus, sensory rearrangements arising during head movements are theorized to drive VOR adaptation processes. Given the importance of active head movements (Leigh, Dell’Osso, & Kosmorsky, 1993) and the potential for adaptation to occur after short exposure periods (Collewijn, et al., 1983), it is logical to hypothesize that VOR gain and/or phase adaptation can occur as a result of short-term, active head-coupled interaction with virtual interfaces (if the stimulus rearrangements are predictable and of sufficient magnitude).

Kramer, et al. (In Press) demonstrated that a VR HMD can be used to drive VOR adaptations both in gain and phase. These researchers provided a prerecorded virtual OKN stimulus (horizontally oscillating vertical stripes) through a HMD that was fixed to a rotating chair. The subject used a chin rest to steady the head. Through various visual and inertial manipulations, these researchers were able to achieve VOR adaptations similar to those attained using a physical OKN drum. This research was among the first to provide evidence that virtual images can drive VOR adaptations.

The virtual interface in Kramer’s et al. (In Press) work was designed exclusively to evoke VOR adaptation. The experiment was constrained by the following factors: 1) no active movements were performed by subjects, 2) system time delays were minimized, 3) a highly artificial visual scene was used, and 4) subjects had no meaningful task to perform. In contrast, this dissertation focused on the potential for VOR adaptation to occur in typical VEs with normally occurring stimulus rearrangements (e.g., time delays) while subjects made active, unrestricted head movements in the performance of a meaningful task. In other words, the question in this dissertation was whether virtual interfaces unintentionally induce VOR adaptation when these interfaces are used in an intended application.

VOR research has documented the occurrence of VOR adaptation through natural head movements (Gauthier & Robinson, 1975; Paige & Sargent, 1991). However, most of these studies were for moderately long durations and none used virtual interfaces. This research centered on short duration exposure (i.e., 30 minutes) to typically occurring stimulus rearrangements generated by virtual interfaces.

No other previous research investigating the relationship between virtual interfaces and VOR adaptation has been discovered. As a result, Dr. Viirre and I performed a small pilot study to explore this issue with a commercially available head-coupled VR system (this study is explained in more detail in Section 4.2.4). VOR and VVOR measures were recorded before, during, and after a 12 minute exposure to an interactive head-coupled virtual interface. The subject’s task was simply to play a virtual reality game which required repeated yaw head rotations. Eye and head position data were collected with a magnetic search coil system for maximum spatial and temporal resolution (Fuchs & Robinson, 1966; Robinson, 1963). Although only one subject was tested, the analysis indicated recalibration of the VOR to the virtual interface.

Research has shown that virtual interfaces can alter other oculomotor movements. Mon-Williams, Wann, and Rushton (1993) demonstrated that poor HMD optics, stereoscopic image quality, and poor interpupilary calibration resulted in a measurable change in visual function in 50% of subjects after only 10 minutes exposure to a virtual interface. These changes included decompensated heterophoria (i.e., deviation from perfect binocular positioning) and altered near point of convergence. A follow-on study (Rushton, et al., 1994) indicated that a higher quality HMD, better image quality, and bi-ocular viewing (instead of stereoscopic viewing) reduced but did not completely eliminate these visual changes.

Kotulak and Morse (1995) investigated oculomotor adaptations as a potential source of eyestrain in a monocular HMD used operationally by the US military. These researchers found that accommodation and vergence responses significantly differed between asymptomatic and susceptible pilots in three different circumstances that often occur in flight. Although the specific issues underlying their research differ from those of this dissertation, it is revealing that oculomotor adaptations occurred and were capable of differentiating symptomatic from asymptomatic subjects with regards to specific sickness symptoms (which supports the second hypothesis of this dissertation as well). As a final example, Kennedy and Murry (1993) found that increasing the update rate on a computer display altered saccadic eye movements in a reading task. They subsequently hypothesized that this alteration may be a source of visual fatigue and eyestrain while reading computer generated text.

Many researchers have asserted that virtual interfaces likely induce changes in the VOR and other oculomotor movements. Ebenholtz (1992) stated that "VR can drive adaptive changes in the VOR". Peli (1995) argued that the VOR may need to adapt to virtual environments (especially head-coupled systems with long latencies or low update rates) and lamented the lack of research into this issue. He also questioned which environment the VOR would tune to in the case of augmented (see-through) displays. Viirre (1996) not only believed that VR can adapt the VOR unintentionally but that it can be made to do so intentionally to facilitate clinical treatment of patients with vestibular disorders.

3.1.2 Type of Head Movements under Consideration

As stated earlier, the focus of this research was on VOR adaptation resulting from volitional head movements by the subject to complete a meaningful task. Both head translations and head rotations can trigger VOR adaptations in the presence of significant stimulus rearrangements (Viirre, et al., 1986). This is because both canal and otolith signals are involved in the vestibular signal that triggers compensatory oculomotor responses (Robinson, 1981; Tiliket, Shelhamer, Tan, & Zee, 1993). However, this research focused solely on head rotations (canal input) due to the relatively minor contributions of the otoliths to the VOR in the specific test conditions used in these experiments (i.e., horizontal rotational oscillations with no close target fixations and no off-axis rotations).

3.1.3 Stimulus Rearrangements under Investigation

This research concentrated on those stimulus rearrangements typically found in virtual interfaces that result in visual-vestibular sensory rearrangements during head rotations. Three commonly occurring stimulus rearrangements that meet this requirement are system time delays, image scale distortions, and limited DFOV.

3.1.3.1 System Time Delays

Normal physiologic latency between the beginning of a head rotation and beginning of a VOR compensatory response is approximately 4 to 11 ms (Sharpe & Johnston, 1993). This results in little to no VOR phase offset between eye and head motion over the range of natural head movements.

Any system time delay in a virtual interface that extends beyond this threshold. constitutes a stimulus rearrangement. This time delay introduces a phase lag between the head movement and the resulting optic flow (of the virtual image) in the opposite direction. This phase lag demand is hypothesized to stimulate VOR phase adaptation.

Figure 15 demonstrates this hypothesis. The four frames illustrate a top-down view of a subject wearing a HMD and viewing a virtual scene that is coupled to head movements. The subject is performing a VVOR. Frame 1 shows a stationary subject focusing on a spot in a center of the virtual image. Frames 2 and 3 indicate a head movement to the right. Signals from a head tracking system, represented by squiggly lines, report the change in head position to the computer that updates the visual scene. In Frame 2, the signal still has not resulted in an update in the visual scene (due to the existence of a system time delay). In Frame 3 the visual scene has been updated in response to earlier head movements but it lags behind the current movement of the head. In Frames 2 and 3, the dashed lines indicate gaze direction assuming zero time delay and the solid lines indicate actual gaze. In Frame 4, the head has stopped moving while the motion of the visual scene continues to completion.

Fixed system time delays produce a variable phase-lag demand on the VOR adaptation system with specific phase lag demand increasing with increasing head movement frequency. Figure 16 demonstrates how a 100 ms time delay results in different phase lags depending upon head movement frequency.

Some studies indicate that VOR adaptation is fairly specific to the particular frequency(s) used during exposure to the stimulus rearrangement (Khater, Baker, & Peterson, 1990; Lisberger, Miles, & Optican, 1983; Peng, Baker, & Peterson, 1994; Zee, 1996). However, other studies have not come to this conclusion (Demer, et al., 1989). Research has not been uncovered which describes the effects of a specific time delay (i.e., a variable phase lag demand) on VOR gain and phase adaptive response across several frequencies.

If VOR phase recalibration occurs in response to a fixed time delay, it could follow one of two courses. It might result in a variable phase adaptation of the VOR across all natural head movement frequencies, as shown in Figure 17. This is unlikely if one assumes a frequency-domain model of the VOR adaptation process. The VOR would seemingly require an extended period of time at each frequency to reliably establish the nature of the stimulus rearrangement, recalibrate the phase for that frequency, then store this value for extremely rapid retrieval when the same frequency head movements are made in the future at this time delay. Nonetheless, it is reasonable to speculate that various internal deficits to the vestibular system might result in a fixed time delay demand being placed upon the VOR adaptation system. For example, changes in transmission time would create a variable phase change demand across frequency. If the VOR was designed to recalibrate to these types of deficits, the phase response may appear similar to Figure 17.

A second scenario is the following. Head movements may be naturally constrained to a small range of frequencies when a person is confronted with a stimulus rearrangement such as a system time delay. Therefore, VOR adaptation may occur for this reduced frequency set only, thus making the phase adaptation ‘frequency-specific’ (Figure 18). This approach, involving fewer frequencies and a more stable/predictable stimulus rearrangement, would also seemingly result in faster adaptation. Research has provided indications that head movements are indeed constrained while using certain virtual interfaces (Pausch, Snoddy, Taylor, Watson, & Haseltine, 1996; So & Griffin, 1995).

Few studies have investigated VOR phase adaptation and even fewer are directly relevant to this dissertation. Kramer, Shelhamer, and Zee (1995) demonstrated that a pure phase change demand might modify the VOR in gain as well as phase. They found that a phase lead demand resulted in gain decreases but they did not find an effect of phase lag demand on gain. Powell, et al. (1996) also investigated phase adaptation and found a complex relationship between gain and phase adaptation magnitudes, but this research used a direction adaptation protocol, cats for subjects, and passive rotations.

Time delays may have other effects on the user, besides the potential for inducing VOR phase adaptation. Research has shown that time delays decrease performance of control tasks and can reduce satisfaction with the interface (Durlach & Mavor, 1995; So & Griffin, 1995).

3.1.3.2 Scale Changes

Virtual interfaces may often contain image scale distortions, whether intentional or accidental. Image scale factor distortions are easily induced by creating an inequality between GFOV and DFOV angles. These scale changes result in virtual magnification or minification of the virtual scene with a corresponding increase or decrease in optic flow velocity that corresponds to the magnification level. This change in optic flow rate generates a gain adaptation demand. Similar rearrangements using magnifying or minifying spectacles as stimuli have resulted in VOR gain adaptation, if the rearrangement was of large enough in magnitude and predictable (Collewijn, et al., 1983; Gauthier & Robinson, 1975; Robinson, 1981). Therefore, image scale distortions resulting from GFOV/DFOV inequalities in virtual interfaces are hypothesized to drive VOR gain adaptations.

Figure 19 demonstrates the effect of different image scales on the eye motion required to maintain fixation on a distant, stationary target (i.e., for a VVOR task). In each of the image scale conditions (shown as 200%: magnification, 100%: normal, and a 50%: minification), the subject’s head is shown moving to the right 10 degrees. The resulting optic flow varies with image scale factor. Therefore, the eye movement rotations required to maintain fixation on a visual target also varies with scale factor. Dashed lines indicate starting positions of head and eyes and the solid lines indicate final positions. These differences in eye movements for the same head movement illustrate the differing gain demands placed on the VOR system in the different image scale conditions.

Previous VOR adaptation research investigating short-duration exposures often restricted exposure interaction to passive head movements at a specific frequency of rotation. In contrast, this dissertation considered active head movements at unconstrained frequencies of motion. Active head movements are theorized to enhance VOR adaptation (due in part to the addition of efference copy signals) but unrestricted head movements across a range of frequencies in a limited exposure period may decrease overall adaptation if frequency specificity is correct. Therefore the results of these dissertation experiments are not easily predicted by the literature.

3.1.3.3 DFOV

DFOV may modulate the magnitude of any oculomotor adaptation that occurs in virtual interfaces. Although Lisberger, Miles, and Zee (1984) found that a full visual field is not required for adaptation to occur, their results suggest that full FOV motion did result in much greater adaptation taking place. This indicates that large FOV displays drive adaptation to a higher level than small FOVs. However, Shelhamer, Tiliket, Roberts, Kramer, and Zee (1994) found no difference in VOR gain adaptation whether the stimulus was full-field or a single LED, which presents a compelling argument against influence of DFOV on VOR adaptation. Thus more research is needed.

There are data suggesting that for maximum adaptation to occur, a small DFOV should not allow concurrent viewing of the normal, real-world environment. Research by Demer, et al. (1989) demonstrated that VOR gain adaptation was minimal when the periphery of the magnifying spectacles was not occluded. However, as DFOV increases and the peripheral stimulation decreases, VOR adaptation has been shown to reliably occur (Cannon, et al., 1985).

3.1.4 The Relative Movements of Head, Eye, and Gaze

This section characterizes the relative motions of head, eye, virtual target, and gaze (i.e., eye in space) as a result of added time delays and changes in image scale within a VE. For simplification, these equations assume that the eye perfectly tracks a distant, ‘fixed’ virtual target (V_target) within the VE while the head undergoes passive sinusoidal rotation at a single frequency (as in the case of VVOR testing). All defined motions (H, E, V_target and G) are velocities.

The Head motion is given by:

H = Acos(wt)

where A is the amplitude of head motion (1/2 peak-to-peak) and w is the oscillation frequency in radians. The VE is directly driven by head movements (i.e., it is head-coupled) such that it moves (with respect to the head) in equal and opposite direction to the head motion in order to maintain the apparent space-stability of virtual targets. If there is no image scale change and no system time delay, the motion of the virtual fixation target (V_target) with respect to the head can be described by:

V_target = -Acos(wt)

However, changes in image scale would change the amplitude of V_target motion in response to head movements, such that the equation would become:

V_target = Bcos(wt)

where ‘B’ is the new amplitude of V_target that is equal to the image magnification factor (S_image) multiplied by ‘-A’:

B = -A*S_image

In addition, an existing system time delay would act as a fixed phase lag component to the V_target in response to head movement, such that the equation for V_target becomes:

V_target = Bcos(wt + P)

where P is the added phase change in degrees.

So far, the motion of the head (H) and virtual target with respect to the head (V_target) have been described. Assuming that the eye perfectly tracks the V_target, the eye-in-head motion (E) necessarily equals V_target motion. Therefore, eye motion can also be described by:

E = Bcos(wt + P)

Thus, E should respond to changes in image scale by changing its gain response to amplitude B and it should respond to changes in system time delay through increases in phase lag to P.

For completeness, gaze motions under the different conditions will be described. The equation for gaze (G) is:

G = H + E

Assuming no changes in image scale and no system time delays, E perfectly opposes H and therefore G equals 0. This is an ideal but unreasonable goal of virtual interface design, given current technology limitations. In reality, both image scale factor distortions and system time delays often exist which cause sinusoidal movement of the V_target with regards to space. If only a system time delay exists, gaze becomes (after substitutions and reduction)

G = 2Acos(P/2)cos(wt - P/2)

As can be seen, a phase change in the V_target with regards to the head results in a net movement of G with a amplitude equal to 2Acos (P/2) and a defined phase relationship to head motion equal to (P/2). If both image scale distortions and time delays exist, the equation for gaze becomes more complex. After multiple substitutions and reductions, the general equation for G is:

G=SQRT(A² + 2ABcos(P) + B²)cos[wt + arctan((BsinP)/(A + Bcos(P)))]

In this case, the amplitude of gaze equals SQRT(A² + 2ABcos(P) + B²) and the phase lag is the result of arctan((BsinP)/(A+BcosP)). Therefore, it is obvious that the V_target and G moves with regards to space, as a function of image scale changes and existing system time delays.

3.1.5 Stimulus Predictability Concerns

A general requirement of adaptation is that stimulus rearrangements must be predictable (Welch, 1986; Welch & Cohen, 1991). Therefore, an issue of importance in this dissertation is the relative stability of the virtual interface stimulus rearrangements explored. Scale changes generated by GFOV/DFOV inequalities are thought to be nearly as stable as the wearing of magnification or minification spectacles, but system time delays can be more variable in nature.

As described earlier, time delays are generally a function of tracking hardware and computing power. These are relatively stable factors of a virtual interface. However, system time delay also is effected by complexity of the VE, computational load from parallel processes, and distance from transmitter for emitting tracking devices (if an emitting type of head tracker is a part of the interface).

It is preferable to first obtain the influences of stable time delays before attempting to understand the influences of additional variability. Therefore this research was specifically designed to generate relatively fixed time delay stimuli.

3.1.6 Summary

In summary, there is support from a pilot study, existing literature, and researcher assertions for the hypothesis that virtual interfaces can drive VOR recalibration. Time delays and image scale distortions occurring in virtual interfaces may generate sensory rearrangements which drive VOR gain and/or phase adaptation.

3.2 Hypothesis 2: Adaptability Governs Sickness Susceptibility

3.2.1 Logic and Support

This hypothesis assumes that Hypothesis 1 (that virtual interfaces can induce VOR adaptation) is confirmed. This hypothesis also assumes that the sensory rearrangement theory of simulator sickness is valid. Figure 20 provides an overview of the hypothesized relationship between VOR adaptation and simulator sickness.

 

Figure 20 is described as follows. Head-coupled virtual interfaces containing system time delays and/or image scale deviations from 1.0X magnification will present visual-vestibular stimulus rearrangements. The user’s sensory systems transduce these stimuli and generate internal sensory patterns. These sensory patterns are compared (via a ‘neural comparator’) to expected sensory patterns (held in a ‘neural store’) obtained from recent experience. The result of this comparison is the generation of a sensory rearrangement signal. If the magnitude of this signal is very small, the information flow terminates and no adaptation takes place. However, if the magnitude is significant and if there is a functioning vestibular apparatus, this sensory rearrangement signal will trigger VOR recalibration activities, a build-up of effect on simulator sickness, and other adaptation processes. These adaptation processes then update the ‘neural store’ which reduces the magnitude of the sensory rearrangement. This, in turn, reduces VOR recalibration requirements and the build-up of effect on simulator sickness.

Generalized adaptation processes are represented by gray boxes and double lines in Figure 20. The hypothesized individual ‘adaptability’ trait serves to govern the overall rate of adaptation. Reason and Brand (1975) theorized that this stable adaptability trait reflects the rate at which a person typically adjusts to conditions of sensory rearrangement (see Section 2.4.6). In sensory rearrangement theory terminology, adaptability is the time it takes for the ‘neural store’ of expected combinations of motion signals to be updated once a sensory rearrangement occurs. A person with high adaptability would rapidly adjust to sensory rearrangements and would therefore avoid motion (or simulator) sickness. A person who had low adaptability would be prone to more sickness symptoms due to the increased duration of the mismatch before the neural store was updated.

There is some experimental evidence for the existence of an individual adaptability trait. Studies have shown that those most prone to motion sickness tend to adapt more slowly to new combinations of motion (Reason & Graybiel, 1972). However, empirical successes have been modest because of the complexities of predicting a syndrome such as motion sickness. Still, some investigators consider differences in adaptability to be the single most important determinant of inter-subject difference in susceptibility to motion sickness (Griffin, 1990; Guedry, 1991; Kennedy, Dunlap & Fowlkes, 1990; Reason & Graybiel, 1972).

If an fixed adaptability trait exists within individuals that indicates relative adaptive ability to sensory rearrangements, than an objective measure of this adaptability trait would likely predict individual susceptibility to simulator sickness. VOR adaptation response is suggested as an objective measure of this adaptive ability to altered visual-vestibular motion cues. The remainder of this section defends the appropriateness of this metric by first describing the commonalties between VOR adaptation and simulator sickness and then discussing the rationale behind specifically investigating VOR adaptation time-course as the adaptability measure. Additional support for the hypothesized relationship between the VOR and simulator sickness can be found in Section 2.2.4 and Section 2.4.4.6.

Virtual interfaces often generate visual-vestibular sensory rearrangements during head movements, which are theorized as capable of driving VOR adaptation (Hypothesis 1). VOR adaptation is often marked by the occurrence of sickness symptomology that is similar to simulator sickness (Demer, et al., 1987; Demer, et al., 1989; Gauthier & Robinson, 1975; Gonshor & Melvill Jones, 1976b; Istl-Lenz, et al., 1985). According to the sensory rearrangement theory, visual-vestibular sensory rearrangements regarding motion and orientation are also strongly implicated as inducing simulator sickness (Reason & Brand, 1975). Given that the same catalyst is involved for both processes, it is reasonable to examine if a correlation exists between simulator sickness and VOR adaptation.

Additionally, a functioning vestibular apparatus is a fundamental requirement for both VOR adaptation and simulator sickness (Reason & Brand, 1975). The role of the vestibular organ in the VOR and VOR adaptation is obvious, but it also has an essential role in simulator sickness (and motion sickness) as well. No bilateral labyrinthine-defective subject (either through birth, disease, or surgical interference) has ever been made to experience any form of motion or simulator sickness. VOR adaptation is also impossible in bilateral labyrinthine-defectives. This further implies a relationship between simulator sickness and the VOR.

If VOR adaptation and simulator sickness are correlated, the association is not a perfect correspondence, nor is it a cause-and-effect relationship. VOR adaptation can occur without the onset of simulator sickness symptoms and simulator sickness can occur without concurrent VOR adaptation. Therefore, one must look deeper to uncover an objective measure of adaptability.

Prior to VOR adaptation, maximum retinal slip of the visual scene exists. Retinal slip during head movements can result in oscillopsia and it also generates a signal that the visual response is inappropriate for a given vestibular stimulus. Thus at the onset of VOR adaptation, the visual-vestibular sensory rearrangement is likely at its greatest magnitude (Guedry, 1991). This suggests that fast VOR adaptation would reduce the exposure time to the maximal stimulus mismatch while slow VOR adaptation would prolong it. As simulator sickness is often characterized by a build-up of response over time, longer adaptation periods allow more build-up of response to occur to the initial, maximally-provocative sensory rearrangement.

Both time-course of adaptation and level of adaptation achieved after a fixed exposure period may provide meaningful correlation with sickness likelihood. The relationship is hypothesized to be:

[sickness level] Û ¦ ( d (VOR)/d t , max(VOR))