Prediction of Simulator Sickness
in a Virtual Environment

[Previous Chapter][Table of Contents]


Discussion

Phase I

The primary focus of this research investigated the prediction of sickness. The goal was to address the following question: Can predictive techniques be used to model sickness on characteristics of an individual?

For sickness measured as a function of the Total Severity score (specifically, ln[TOTAL+1] - to be referred to as simply "sickness"), this research was successful in finding a model. For sickness measured as a function of the subscale scores, this study lacked enough data to employ linear regression techniques to find a model. Thus, for these measures, models were not found.

The model found for sickness was as follows:

LNTOTAL = 3.27 - 0.162 AGE + 0.0191 GENMRA + 0.00656 AGEMRA + 0.0277 AGEPRO - 0.0323 MRAPRO

Approximately 35% of the variance in sickness is explained by this model and the model fits the data well. Because of the numerous interaction terms in the model, however, it is somewhat difficult to interpret.

It should be kept in mind that this study only investigated individuals in the 19 to 46 year age range, with the majority of participants being in their 20's. Furthermore, the 46-year old was not included in the regression analysis. Thus, the age range used for the regression analysis was 19 to 32. Although the models developed in this research can be used to predict sickness for individuals outside of this age range, caution should be used when doing so. Before definitive statements are made about age, more research should be conducted with individuals representing the entire age range.

Several conclusions can be made based on the results of this regression analysis. The primary conclusion is that sickness can, in fact be modeled on characteristics of an individual. Furthermore, such a model can explain a reasonable proportion of the variance in sickness. It was noted in the Introduction that factors related to both the VR system and the task performed in the VE could also be associated with sickness. Thus, a model which included such factors, in addition to characteristics of the individual, would likely explain a larger proportion of the variance than does the model developed here. The focus of this study, however, was to model sickness on characteristics of the individual alone.

The model developed in this study portrays a complicated relationship between sickness and age, gender, mental rotation ability, and pre-exposure posturalstability. Based on the model, no clear simple relationship exists between sickness and any one of these four characteristics.

Gender does not appear to influence sickness directly but, rather, interacts with mental rotation ability in its effect on sickness as indicated by the presence of the interaction term in the model. The relationship suggested by this term is diagrammed in Figure 1.

Figure 1. The modeled relationship between sickness and mental rotation ability.

This relationship suggests that as mental rotation ability improves, predicted sickness increases for females but decreases for males. Why this relationship should exist is not clear from nor is it suggested by the literature. It should be noted that only the trends are predicted to differ, not the actual sickness scores between males and females. That sickness is not predicted to differ for the genders is indicated by the lack of a gender term by itself in the model. This is supported by the finding that the mean sickness scores did not differ significantly for the males and females in this study.

Age, mental rotation ability, and pre-exposure postural stability all interact with each other in their effect on predicted sickness. No clear relationship exists between sickness and any one of these four variables. Furthermore, of these three characteristics, only mental rotation ability appears to interact with gender.

The literature reviewed suggested that predicted sickness would follow an inverted "U" relationship with age: predicted sickness would be lower for younger and older individuals and higher for middle aged individuals. The data in this study did not appear to follow this pattern. It should be remembered, however, that these data had a very limited range of 19 to 32 years - ages during which, according to the literature reviewed, susceptibility to motion sickness should be slowly decreasing. Thus, the limited range of the data likely precluded finding a clear inverted "U" relationship.

The utility of the developed model lies in the fact that it could be developed - thus indicating that sickness can be successfully modeled on characteristics of an individual. Although it is not proposed that this model would hold for all individuals in all VR systems for all tasks performed, a certain degree of generalizability could be expected. As noted, the proportion of variance in sickness explained by the model could likely be increased by including characteristics of the system and task. This, however, would reduce the generalizability of such a developed model. Clearly there is a trade-off between generalizability and performance of any model.

Finally, as noted in the summary for the Review of the Literature, it appears that no model has been developed using linear regression techniques to predict sickness as measured by the SSQ on characteristics of the individual. Thus, there is no previous model to which the model developed in this study can be compared. The sole model found in the literature was that presented by Reschke (1990). Because that model was a logistic model, it is not directly comparable to a linear regression model.

Phase II

A second aim of this research was to provide insight into the effects of low-end VR exposure by addressing the question: Is VR exposure associated with decrements in postural stability? The data in this study failed to support the hypothesis that ataxic decrements occur in conjunction with low-end VR exposure. There are two possible explanations for this result.

First, mild ataxic decrements may actually occur in conjunction with low-end VR exposure, but there was insufficient power to detect them. This study had sufficient power to detect a moderate effect (by Cohen's, 1988, standards) but it did not have sufficient power to detect a small effect. Recall also that the measure actually used to quantify postural stability - the Prototype measure - is likely much less sensitive than the measure proposed - the y-velocity measure.

Alternatively, ataxic decrements may not be associated with low-end VR exposure or an effect may exist but be so small as to be of no practical significance. Postural tests are known to have a learning component (Hamilton, Kantor, & Magee, 1989) and, if the ataxic effect was small, it could be that learning masked that effect. It could also be that ataxic decrements occur only for those at the extremes of sickness.

The fact that ataxic decrements were not found in this study even though sickness occurred supports findings presented by Kennedy, Lanham, Drexler, and Lilienthal (1995). They found that with repeated exposure to a simulator, sickness decreases over time but ataxia increases. Thus, it could be that the participants in this study were at the beginning stage of these relationships. Although there were high levels of sickness, there was no significant ataxia.

Nevertheless, ataxia is a well-documented effect of simulator exposure (e.g., Kellogg & Gillingham, 1986; Kennedy, Fowlkes, et al., 1993). However, level of post-exposure ataxia is likely a function of several factors including the time of the exposure, the individual's level of VR experience, and the task performed in the VE.

Previous studies investigating ataxia have typically involved military pilots and flight simulators. Such individuals generally have much experience with the simulator and are subjected to long exposures, usually about 2 hours. As discussed above, Kennedy, Lanham, Drexler, et al. (1995) found that ataxia following simulator exposure increases as experience with the system increases. Furthermore, the intensity and duration of ataxia has been found to increase with longer simulator exposures (Fowlkes et al., 1987). Although VR experience was not specifically investigated herein, the participants were assumed to have no lingering adaptation from previous VR experiences. Furthermore, the exposure in this study was comparatively very short. Thus, if ataxia is a function of an individual's level of experience with the system (i.e., adaptation) as well as exposure time, then these participants would not likely reveal the effects.

Recent VR research, however, suggest that ataxia may indeed occur with exposure. Rolland, Biocca, Barlow, and Kancherla (1995) found degradation in hand-eye coordination and errors in pointing accuracy following the wearing of a see-through HMD. Although they are not the same as decrements in postural stability, these results demonstrate that negative aftereffects are indeed possible. There have been anecdotal observations of individuals demonstrating significant ataxic decrements following a 30-minute VR exposure (K.M. Stanney, personal communication, April 9, 1996). An important characteristic of that particular exposure may be that those individuals spent their time in the VE traversing a maze, a task which involved both forward and left/right represented movements. The task employed in this research - Ascent - on the other hand, involved represented movements primarily in the forward direction only. Thus, the kinematics of the task performed in the VE may have an important effect on the occurrence of ataxic decrements.

Taking all of this information into account, the final conclusion is that ataxic effects, if they occur in conjunction with low-end VR exposure, are likely to be small for short exposures with certain tasks. As with the developed model, this finding should not be expected to generalize perfectly to every VR system and every task performed. It is possible that some system configurations and some tasks performed in a VE might be more conducive to ataxic after-effects, although no research to date has addressed this issue. The finding of no ataxia almost surely does not generalize to longer exposures.

Additional Findings

Occurrence of Sickness

In this study, it was concluded that sickness did occur in conjunction with low-end VR exposure. Of the 40 research participants, only 6 (15%) reported no symptoms whatsoever following exposure. One severe case involving vomiting after only a short exposure. Furthermore, lingering or delayed effects were reported by 14 (35%) participants (not including the severe case which also included delayed effects).

The level of sickness found in this study was compared to sickness found in other studies involving simulator systems. The average Total Severity score found in this study is slightly greater than that reported for Navy flight simulators (Kennedy, Jones, Lilienthal, & Harm, 1993) but is about the same as that reported for most of the VR experiments at the U.S. Army Research Institute (Knerr, Goldberg, Lampton, Singer, & Witmer, 1996). Thus, it appears that the level of sickness found in this study is fairly typical of VR systems in general.

Because of the apparent representativeness of the sickness data in this study, the level of sickness reported herein could be used as a benchmark for future research with VR systems. Kennedy, Jones, et al. (1993) report that the 75th percentile may be a useful index. The 25th, 50th, and 75th percentiles for all of the sickness measures used in this study were reported in Table 1.

Gender Differences in Sickness

Another additional finding concerned differences in sickness between genders. This research found a gender difference in the same direction as that of other researchers (e.g., Kennedy, Lanham, Massey, Drexler, et al., 1995) but the differences obtained in this study were not found to be statistically significant. There did, however, appear to be greater variation in the scores for females.

Correlation Between IPD and Eyestrain

A negative correlation was observed between inter-pupillary distance and both the Oculomotor Discomfort subscale score and the rating of the "Eyestrain" symptom, although only the correlation between IPD and the symptom was significant. This suggests that, for the i*glasses!™, a smaller IPD is associated with a more severe rating for the symptom. This supports the finding of Regan and Price (1993) who suggested that diverging one's eyes (as would be required of individuals whose IPD is smaller than the system configuration) is likely to cause greater ocular stress than would converging. Furthermore, it appears that IPD might be related specifically to only the eyestrain symptom rather than the cluster of oculomotor discomfort symptoms. As discussed shortly, this finding has specific implications for the design of HMDs.

Correlation Between Sickness and Final Level Reached in Ascent

A significant negative linear correlation was obtained between sickness and final level reached in Ascent. This suggests that individuals who performed better at the game experienced less sickness. An alternative interpretation is that individuals who experienced more sickness performed worse at the game.

It should be noted that, except for the participant who experienced severe sickness, all participants were in the VE for the same amount of time. Furthermore, because Ascent is only available bundled with the i*glasses!™ and no participant volunteered having experience with that HMD, it is reasonable to assume that all participants were equally inexperienced with the actual game.

This finding has implications for the relationship between sickness and level of performance. Specifically, this study suggests that sickness is related to a decrease in performance. However, because level of experience with other games and computers in general was not controlled for in this study, a definitive relationship cannot be established based on the correlation alone.

Overall Conclusions from This Study

Considering all of the findings in this research, three points summarize the important results. First, sickness - as measured by the SSQ - can occur in conjunction with low-end VR exposure. Furthermore, this sickness can be severe and it can involve lingering or delayed effects. Second, a predictive model for sickness can be developed based on characteristics of the individual. Based on the model developed in this study, a complicated relationship exists between predicted sickness and gender, age, mental rotation ability, and pre-exposure postural stability. Sickness is not predicted to differ for the genders but, rather, gender interacts with mental rotation ability in its effect on sickness. Third, ataxic decrements in postural stability do not appear to be associated with short exposures to low-end VR with certain tasks.

Areas for Future Research

Use of Different Modeling Techniques

The analysis employed herein investigated the prediction of sickness using linear regression techniques and only first degree terms. Further investigation is needed to determine the exact nature of the relationship. Within the confines of linear regression, three additional techniques may be warranted. First, instead of using only first-order terms, model fitting could be attempted using higher order terms such as squares or cube functions. Typically, the use of such terms is suggested by either scatter plots of the variables or residual diagnostics. The data analyzed in this study did not suggest that such terms were needed. With larger samples and different variables, such terms may be appropriate.

A second technique would be to force regression through the origin. In other words, not include a y-intercept in the model. This technique is appropriate when it is theoretically sound to assume that the value of the dependent variable is zero when the values of all of the independent variables are zero. In a simple model of, say, sickness on age, such an assumption makes sense (i.e., an individual of 0 years experiences 0 sickness). In more complex models involving many variables, however, such an assumption may not be appropriate.

A third technique would be the use of generalized linear modeling methods (Weisberg, 1985). These methods are linear techniques in that the response depends upon the predictors only through a linear form. However, they extend linear regression techniques by means of two elements: a link function, which specifies the relationship between the linear predictor and the expected response, and an error function, which specifies the distribution of the errors.

Recall that models could not be found using linear regression techniques for the subscale data in this experiment due to the categorical nature of the responses. Logistic regression, a special case of generalized linear modeling, is appropriate for data in which the response consists of two categories. If sickness data were converted to categories of Sick and Not Sick, logistic regression would then be the appropriate method to use for developing a model. Recall that the one model found in the literature which had been developed to predict space motion sickness (Reschke, 1990) was developed using such techniques. The effective performance of that model suggests that generalized linear modeling techniques may be more appropriate than linear regression techniques for modeling sickness, especially as measured by the SSQ subscale scores.

Non-linear regression techniques could also be attempted (Weisberg, 1985). These techniques differ from linear techniques in that the response is modeled as a nonlinear function of the parameters. Such techniques are much more complex computationally and, furthermore, do not always yield solutions.

Effect of Experience and Longer Exposures

As was discussed previously, there is some literature to suggest that ataxia increases with increased exposure time. As was also discussed, there is some literature to suggest that sickness and ataxic effects of exposure change as an individual gains more experience with a simulator. This study investigated the effects of only a short VR exposure. Although an attempt was made to limit adaptation from previous VR experiences, this study did not investigate the effects of an individual's amount of VR experience or level of adaptation. Clearly, more research is needed to address these matters. Specifically, future research must determine the effects of length of VR exposure and an individual's level of VR experience and adaptation.

Effect of Different Tasks on Post-Exposure Ataxia

As was discussed, there have been anecdotal observations of ataxic decrements following exposure to a virtual maze. This research, on the other hand, employed a simple task involving relatively limited represented movements. Further research is needed to determine how characteristics (e.g., scene complexity, kinematics) of the task performed in a VE are related to the occurrence of post-exposure ataxia. It could be that VR applications involving limited represented movement - such as teleoperation or simple games - may pose limited risks of ataxia, whereas applications involving a high degree of represented movement - such as highly dynamic games - may pose greater risks of ataxia.

Relationship between IPD and Visual Symptomatology

An additional analysis conducted in this study investigated the relationship between inter-pupillary distance and visual symptomatology and found a significant negative linear correlation between IPD and the "Eyestrain" symptom on the SSQ. This suggests that individuals with smaller IPDs experienced more eyestrain. This finding has specific implications for users of HMDs with fixed IPD settings, which might be typical for low-end HMDs. This is clearly an area in which further research may have direct design implications for manufacturers of HMDs. Namely, such devices may need IPD adjustments (the i*glasses!™ lack any form of IPD adjustment) so that eyestrain can be avoided in susceptible users.

Performance Measurement

The finding of a negative correlation between sickness and level reached in Ascent suggests that sickness may be associated with decreased performance in a VE. This is an area which is in dire need of more research because of its direct implications to the use of VR systems for training. To date, it appears that the effect of sickness on performance in a VE, or other simulator, has not been adequately addressed.

Similarly, another important area in which more research is needed is in the assessment of performance changes which may occur following exposure to VEs. There are many warnings not to drive or engage in other dangerous activities after exposure and the military has several protocols which limit activities following simulator exposure, but very little has been done to validate such claims. Performance measures which relate to such tasks as driving could be used or other performance measures - such as change in dark focus, vision tests, and past pointing - are also possibilities which require further investigation. Furthermore, performance changes could be equated to drug or alcohol consumption to put such changes in easily understandable "units", as was done by Kennedy and Lilienthal (1995). A worse case outcome might be if subjects show no signs of sickness but do show performance decrements. The insidious implication of this would be that VR users need not be obviously impaired to actually be impaired (i.e., have degraded skills and functions).

It should be noted that proper measurement of performance requires a fair amount of time to establish learning curves and norms. Alternatively, tasks which are well-normed could be used so that performance could then be compared to those norms.

Implications for Manufacturers of Simulators and VR Devices

As this and several other studies (Lampton, Kolasinski, Knerr, Bliss, Bailey, & Witmer, 1994; Regan & Price, 1994) have shown, simulator sickness is clearly possible after exposure to VR systems. In some instances, such sickness can be quite severe. Potential users need to be informed of the existence of the phenomenon of simulator sickness, the symptoms which define the syndrome, and the possibility that they may experience it. Beyond this, specific users may need to be warned that they are at special risk. As the technology advances and VR technology gains widespread use, manufactures of such systems will need guidance in addressing these matters. Because of the criticality of this matter, it is important that definitive recommendations not be made on the results of just one study. The present study, however, does suggest some direction for future research.

Specifically, it was demonstrated that a model for the prediction of sickness can be developed based on characteristics of the individual. Future research, for example, could look at using only those characteristics - such as age and gender - that are readily obtainable. The results of this study, however, suggest that there may be no simple relationship between sickness and characteristics of the individual.

Ataxia poses a special concern for users of VR systems because ataxic decrements may linger for some time after exposure. Although this study did not find that ataxic decrements in postural stability were associated with short exposures to low-end VR, it may have revealed an important aspect of ataxic effects. Namely, keeping exposures short may be one way to prevent ataxia. Exactly how long "short" is, however, is not clear. Controlled, carefully designed studies are needed to address this issue before clear recommendations can be made.

Lingering and delayed aftereffects are another important hazard, as indicated by the results of this study. Although not all of the effects reported may have been attributable to the VR exposure, it is clear that this matter must be further investigated. Research is needed to determine the nature of the possible effects as well as how long such effects may last and how long after exposure they are likely to occur.

Overall, this research appears to have raised many more questions than it answered. But it has provided some direction for future research with respect to some potential problems associated with widespread low-end VR use. Virtual Reality technology has many positive aspects, but its negative aspects must be controlled as well. It is only by addressing all of these issues to the point of a thorough understanding that we can move past the problems and fully enjoy all of the benefits of VR technology.