![[Previous Page]](/icons/button-back.gif){kind=icon}
![[Table of Contents]](/icons/button-contents.gif){kind=icon}
![[Next Page]](/icons/button-next.gif){kind=icon}
As will be recalled, the hypothesis of this study was that VR would be useful as an educational tool due to its high degree of interactivity and immersion. While the students in the VR treatment did significantly improve their post test scores over their pre test scores, the main result of this study was that interactivity and not immersion is the important factor in learning about atomic and molecular structure. This conclusion was due to the result that students in the VR and Mac Interactive treatments scored well on both of the tests. In most comparisons, their scores were significantly better than the Mac Run, Video, and Control groups. However, the VR and Mac Interactive students were not significantly different from each other, which leads to the conclusion that interactivity is the important feature. For both the oral and written tests, Figure 22 illustrates the comparison among the different treatments in terms of the delta, which is the average improvement between the pre test and the post test. Lines are drawn connecting treatments which are not statistically different from each other. A vertical line is also drawn dividing treatments whose post test score was significantly higher than the pre test score.
Figure 22: Oral and Written Test Significance
The Mac Run and Video treatments, which should have had the same results for all of the tests, were not significantly different from each other for the oral test. However, they were significantly different from each other for the written test. Surprisingly, students in the Mac Run treatment scored as well on the written test as students in the VR and Mac Interactive groups. The Video treatment group did not significantly improve on the post written test as compared to the pre written test. This implies that immersion and interaction were not the only factors involved in influencing improvement on the written test. One possible factor is the original classroom instruction that the students received. The written test would be more susceptible to this factor, since that test was meant to mimic the classroom test. So, although there was no significant difference among the two different school populations in the pre test score, the Mac students might have been remembering something they had been taught while experiencing the Mac chemistry world, while the other students were really learning or not learning the subject during their experience of the chemistry world.
The spatial ability factor, as measured by the DAT score was not significant for the oral test and only marginally significant for the written test. No conclusions can be drawn from this data about how spatial and non-spatial learners create and manipulate their mental models.
For the long term study of the VR, Video, and Control treatment students, none of the groups retained any of the gains that they might have made in the post tests. However, while the VR students did not maintain their significant improvement on the oral test, they did not slip back to their original pre test level. The VR extended oral post test score is significantly higher than the VR oral pre test score.
The results of this study have not shown VR to be superior to other methods of instruction, namely an interactive Macintosh method. However, interactivity, an inherent part of VR, has been shown to be important. I had hoped that the immersion aspect of the VR treatment would make the chemistry concepts more concrete than the illustrations in the Mac Interactive treatment. To allow for the possibility that VR is significantly better than Mac Interactive despite the presented evidence, I need to examine the differences between the two treatments that might have skewed the study. I have identified five areas of difference, which are: training, world design, assessment, hardware resolution, and student population.
The most obvious difference between the VR and Interactive Mac treatments is that everyone in the VR treatment was experiencing that medium for the first time, while no one in the Mac treatment was new to the Macintosh computer. I did not train the VR participants to a level of expertise, so they were given the cognitive task of navigating in VR along with learning the chemistry lesson. Although I also did not train the Mac students in the use of the computer, they all used the tool easily. As I ran the experiment, it was clear to me that the VR students were struggling with maneuvering in the virtual world while trying to remember what needed to be done to build the virtual atom. The Mac students were able to concentrate exclusively on building the atom. Evidence of this lies in the difference of error rate for the two groups. I kept track of how many times a student made a mistake in setting the correct energy level and/or spin. Typically the error was that they forgot to change the setting as opposed to setting the indicator to the wrong level. The VR students made many more errors than the Mac students. Table 19 shows the error rates for the two groups. The VR group consistently made more errors than the Mac group. Only one third of the VR group made no errors while one half of the Mac group made no errors. The likely explanation is not that the VR group was less capable, since their pre test scores were not significantly different from the Mac group, but rather that the VR group had to concentrate on additional tasks. This drew attention away from the task of learning about chemistry. Further study of the use of VR in education should explore the importance of familiarity with VR itself.
Table 19: Error Rates
N Spin Error Energy Any Error
Error
0 1 2 0 1 0 1 2 3
Mac 14 79% 14% 7% 71% 29% 50% 43% 7% 0%
VR 33 58% 33% 9 58% 42% 33% 42% 21% 3%
Another area of difference between the VR and Mac groups also relates to the relative newness of VR. Designing a virtual world to take advantage of the immersion aspect of VR is non-trivial. In the same way that early filmmakers merely recorded stage plays without using the uniqueness of the new medium, my virtual chemistry world did not dramatically use the immersion capability. I believe that I failed to use the full potential of immersion in the world. If VR has the capability of providing an immersive experience, but immersion is not exploited, then VR theoretically has no benefit over the Interactive Mac. A future study could creatively use immersion in a virtual world in order to analyze VR. One chemistry world example is to somehow highlight and encourage viewing the atom and molecule from various perspectives including from inside the structure.
Assessment was another area that could have been improved in this study. The main problem was asking students to sketch a 3D object on a 2D piece of paper. The VR students saw the orbitals in 3D and were asked to draw them on a 2D piece of paper. The Mac students saw the orbitals on a 2D screen and were asked to draw them on a 2D piece of paper. Therefore, the Mac students were at a clear advantage since they did not have to translate the image they saw. Late in the study, I spoke with a VR student who felt that her understanding of the shape of orbitals had greatly increased, but who had done poorly on the test. I asked her about this disparity and she answered that she understood it in her mind, but could not draw something like that on a piece of paper. A future study could try different assessment techniques. One option would be to have the students sculpt the shape of the orbitals in some 3D medium such as clay, instead of drawing the shape. Another less intensive method of assessment would be to show several 3D models and have the students choose the correct model for a particular orbital.
The resolution difference between the Macintosh monitor and the VPL EyePhones might have influenced the results of this study. Students in the VR treatment wore low resolution EyePhones and saw "fuzzier" objects than those in the Macintosh group. This might have kept the students from feeling truly immersed in the virtual chemistry world, thereby reducing the VR experience to the low immersion treatment of the Macintosh.
Finally, another area of difference between the VR group and the Mac group was in student population. The two groups came from different schools and therefore had different chemistry classes and teachers. This was an unfortunate experimental design due to reasons outside of the researcher's control. Obviously, any future studies should strive to eliminate any unnecessary confounding factors like this. How much this difference affected the outcome of this study is unknown.
These differences between the VR and Mac treatments warrant strong consideration when drawing any conclusions from this study. As a final note, I watched many students go through the various technologies. The most enthused students I saw were the ones who experienced VR. Certainly VR was exciting and new for the students, but more importantly, I witnessed "a-ha" moments from many of them. One student scored zero on his pre-test. While he was in the VR world, he kept saying over and over, "oh, that's what the teacher meant." On the post-test, he scored a zero. I don't know what knowledge he gained, but I am convinced that with VR, he understood something he didn't know before.