The Effect of Student Construction of Virtual Environments on the Performance of High- and Low-Ability Students

Over the last three years, our laboratory has been engaged in a project that has brought demonstrations of immersive virtual reality (VR) to around 7000 children in Washington and Nebraska, has built and assessed a number of VEs designed to meet particular learning objectives, and has worked with students and their teachers to build their own VEs as part of regular curriculum units (Winn, 1995). This paper describes the third of these activities in which we studied the effects of having students, in grades 4 to 12, build their own immersive VEs.

For this paper, a virtual environment is understood to be an immersive, 3-dimensional environment created entirely from a database by a computer. The database consists of objects modeled by CAD software as 3-dimensional graphic objects. These objects are programmed to behave in certain ways as they interact with each other and with a "participant" who is visiting the VE. The participant wears a helmet in which the graphic objects are displayed stereoscopically in two eyepieces. The participant's head is tracked electromagnetically so that the computer can recompute and redisplay the objects from the participant's changing viewpoint in real time. The participant holds a wand, whose position is also tracked. The wand appears in the field of view as a hand or tool. By pointing with the wand and pressing buttons on it, the participant can move around inside the VE, pick up objects and interact with them in a number of ways. Stereophonic sound, which can provide realistic or arbitrary auditory cues, completes the picture.

Visiting VEs helps students learn content under some circumstances (Byrne, 1996; Dede, 1992, 1995; Rose, 1996; McLellan, 1996). There are three contributing factors. The first is immersion. Immersion in a VE makes it possible for students to experience what they are learning about in an entirely new way. VEs can simulate objects and actions that occur in the real world. But in particular, VEs can represent in directly visible and manipulable forms concepts and procedures that are intangible and invisible in the real world. What is more, students can interact with these objects and can actively experience phenomena in the virtual world in ways that are more natural than those normally employed when interacting with computers (Bricken, 1991) -- by moving and looking around, by pointing and gesturing, and by picking objects up to manipulate them or examine them. These activities enable students to experience phenomena through their own eyes, ears and hands rather than through the eyes of a teacher or textbook writer, providing what Clancey (1993) has called "first hand" experience of the world which, we believe, contributes a great deal to the sense of "presence" students can feel in a VE (Barfield & Hendrix, 1995; Hoffman, Hullfish & Houston, 1994; Zeltzer, 1992). For example, in Dede and Loftin's "Science Space", a student may experience first-hand what it is like to be a ball that reacts to forces acting on, and to collisions with another ball, (Dede at al., 1996; Loftin et al., 1993). The student can learn Newtonian mechanics by becoming and by observing a ball as it responds to student-induced changes in gravity, mass, velocity and elasticity. This is not possible in any other environment.

The second contributing factor to students learning in VEs is the interaction that VEs foster. Indeed, a study by Byrne (1996) suggested that interaction is a more important facilitator of learning than immersion for some kinds of task. Educational technologists have, of course, always understood that a student must interact with an environment for learning to occur (Anderson, Corbett, Koedinger & Pelletier, 1996; Psotka, 1995). However, the potential naturalness of interactions with objects in a VE makes interaction much easier and therefore more useful than in other types of environment.

Finally, most students find VEs entirely engaging (Bricken & Byrne, 1993; Taylor, 1997; Winn, 1995). Part of the reason for this is doubtless the novelty of VR and its association in children's minds with computer and video games. Another reason is the uniqueness of the experience and the empowerment it brings to young students who can control the computer to do their bidding in complex and sophisticated ways. We also believe, and set out to determine in this study, that it also enables some students to understand concepts and principles that have hitherto been opaque and baffling which is intrinsically motivating.

Having students construct their own VEs also enables them to learn content (Osberg, 1997). Building a VE requires students to construct knowledge of the domain of knowledge the VE embodies. In our work, students take responsibility for mastering content, deciding how that content is to be represented in the VE and how it is to behave. The VE is therefore a projection of students' understanding, or mental models, into an entire world of their own creation. We believe that arriving at the understanding necessary to build a VE enjoys all the advantages of allowing students to construct knowledge for themselves, under guidance, rather than have it fed to them (See Dede, 1995; Winn, 1993; Chapters in Duffy, Lowyck & Jonassen, 1993). We also believe that constructing a VE engages those cognitive and perceptual skills that are brought to bear when a student makes any physical construction (Harel & Papert, 1991). Turning a mental model into an artifact is, of course, design (Simon, 1981). We have had our students work with us as apprentice instructional designers. Having them make an environment to teach other children is a powerful learning tool. The responsibility they take on in so doing is very motivating.

From this, we conclude that VEs offer a very unique way for students to learn. They can gain unique perspectives and experience in a virtual place through the first hand experience VEs allow. When they build a VE, they construct their own mental models of content and then project these for all to see and share onto an entire world that they design and make. The technology is exciting to use and the projects are high motivating.

The main purpose of this study was to test the hypothesis that the unique experiences of building and visiting VEs would be more useful to some students than others. Learning in "traditional" classrooms often requires students to master abstract and esoteric symbol systems before they can understand content. Our expectation was that by allowing students, first, to make decisions about how their "world" was to appear and behave, then by putting into their hands the tools to actually build it and then having them visit their VE and perform tasks in it would be particularly helpful for students who do not do well with a more traditional, symbol-oriented pedagogy. We therefore examined the extent to which students' general ability, and secondary school students' spatial reasoning ability and spatial orientation ability predicted performance after learning by building and visiting VEs and after learning the same content in more traditional ways. Also, because earlier observations had suggested that gender might also interact with spatial ability to predict performance, we looked for gender differences in our performance data.

Subjects.

Three hundred and sixty-five students from grades 4 to 12 took part in the world-building project. However, only in a few of the secondary schools, where more than one class at the same grade level was studying the same material, was it possible to make comparisons between the students building VEs and students learning in a more traditional way. In some of these classes, students who did not build a world still got to visit one. Also, attrition was quite high in some schools because the final data collection took place close to the end of the school year and some students graduated. We worked with intact classes.

Procedure.

At the beginning of the year, teachers taking part in a larger project to bring VR to their classrooms were invited to submit short proposals for having their students build curriculum-related VEs. Fourteen proposals were received and accepted. Each school proposed to build a different world, though most were related to science curricula. The number of students participating in the construction of each VE, and the roles each played in the construction process, varied from school to school.

We took a four-step approach to constructing VEs: Planning, modeling, programming and experiencing. The entire process took from six to ten weeks with numerous visits by project staff to the students and their teachers, with, again, considerable variability form school to school. During the planning phase, students worked in groups to make decisions about how the VE should look and behave. They were given the task of constructing a VE in which other students could learn the content they were studying. This required them to find ways to show objects and to design metaphors for invisible objects and procedures. Modeling required the students both to learn the 3D CAD software we used for the project, running on Macintoshes in their classrooms, to design their objects on paper and then to draw them in three dimensions on the computer. Programming was conducted by laboratory staff. This involved assembling the objects into the VE, following the students' instructions, and imbuing the VE with the intended behaviors. For the experiencing phase, students visited the VEs they had created. They were given specific tasks to perform. After performing these tasks, which took from ten to fifteen minutes, students completed knowledge posttests and the general and spatial ability tests, and completed a questionnaire.

Instruments.

Students took teacher-constructed posttests over the content they had been studying. Because each group of students built a different VE, and each teacher therefore wrote a different posttest, the scores were first standardized to allow comparisons across VEs and the pooling of data across schools. Students also took a test of general ability, Raven's "Progressive Matrices" (Raven, 1958). This is a test that is not affected by students' mastery of language, and which has been normed for students we were working with. Scores were converted to age-corrected percentiles for analysis. Middle and High School students took an additional four spatial ability tests from the Ekstrom (1976) battery, card rotations, cube comparisons, paper folding and surface development. The sum of the first two of these gave a measure of spatial reasoning ability and the second two assessed spatial visualization. Students also completed a questionnaire. This consisted of 24 five-point scales that solicited student ratings in a number of areas including enjoyment, the sense of` "presence" in the VE (the extent to which students felt they were really in the VE and not in the classroom), and potential impediments to learning such as difficulty seeing and moving around in the VE and tendency to nausea. Students who did not build worlds and who were in a "traditional" class answered an eight-item subset of these questions that were concerned with the VR experience not with building a world.

Results.

Students were blocked on ability on the basis of their Raven's scores with students scoring in the middle third of the range of scores excluded from the analysis (final N=45). Posttest scores were submitted to a two-way ANOVA involving "World-building" and "Traditional" groups crossed with high and low ability. There was no main effect for group. However, the interaction of group with ability was significant, F(1,44)=2.91, p<.10. Low-ability students who did world-building (M=68.62%, SD=20.75) significantly outperformed those studying in the traditional way (M=42.55%, SD=26.28), F(1,44)=8.67, p<.01. For high-ability students, there was no difference in performance(MVR=60.16%, SD=18.75, MTraditional=60.89%, SD=19.36).

Students were also blocked on their spatial ability and spatial visualization scores. No significant main effects or interactions were found for either measure with content posttest performance as the dependent variable.

Contingency tables were built from the questionnaire rating scales for pairs of questions whose relationships were of interest. The (2 test showed that most of these associations were significant. Therefore, in order to arrive at a more parsimonious description of the data, two principle components factor analyses of the questionnaire were performed with varimax rotation. The first included all items on the questionnaire, that is those items answered by students who built worlds. This produced eight factors with Eigenvalues > 1.0, accounting for 69.2% of the variance. The second factor analysis included only those questionnaire items answered by all students, those who built worlds and those in the "traditional" group who visited a VE. This produced three factors with Eigenvalues > 1.0, accounting for 62.0% of the variance.

Students' factor scores were obtained for both factor analyses. Factor scores from the first factor analysis were then compared across students blocked, as before, on general ability (Raven scores), spatial reasoning (Ekstrom spatial scores) and spatial visualization (Ekstrom visualization scores). Four of the eight factors produced significant findings. Factor II, with an Eigenvalue of 2.25 accounting for 9.4% of the variance, had loadings on items assessing enjoyment and the sense of presence. Factor IV, with an Eigenvalue of 1.61 accounting for 6.7% of the variance, had loadings on items reporting the extent to which students made drawings and used 3D models before drawing objects on the computer. Factor V, with an Eigenvalue of 1.40 accounting for 5.8% of the variance, had loadings on items assessing students' degree of nausea and general malaise. Factor VII, with an Eigenvalue of 1.10 accounting for 4.6% of the variance, had loadings on items where students assessed the extent to which they collaborated with other students. T-tests using factor scores for factor II showed that high general ability students reported making paper drawings and 3D models of objects before modeling them on the computer more than low ability students, t(41) =1.82, p<.10. Factor scores for factor IV showed high spatial reasoning students enjoyed visiting their world and experienced higher levels of presence more than low spatial students, t(22)=2.47, p<.05. Factor scores for factor V showed that high spatial reasoning students were also likely to feel less nausea and less dizziness than low spatial students, t(22)=1.74, p<.10. Factor scores for factor VII showed that students with higher spatial visualization scores collaborated more with other students with low spatial visualization scores, t(22)=1.72, p<.10.

Factor scores from the second factor analysis were submitted to ANOVA with group (world building versus traditional treatments) as the independent variable and ability, spatial reasoning and spatial visualization as predictors. Two of the three factors produced significant results. Factor I, with an Eigenvalue of 2.51 accounting for 31.3% of the variance, had loadings on items to do with enjoyment and presence. Factor II, with an Eigenvalue of 1.40 accounting for 17.5% of the variance, had loadings on items that assessed students' ability to find and identify objects, to understand the task they were performing, and to navigate in the world. Analysis of factor scores for factor I showed that high spatial reasoning students experienced more enjoyment and presence than low spatial students, F(1,51)=5.66, p<.05. For factor II, high general ability students found it easier to find objects, perform the task and navigate than low ability students, F(1,92)=3.48, p<.10. Interestingly, no significant findings were obtained for group.

Because presence is fundamental to enjoyment and to performance in a virtual world, because of the role spatial reasoning plays in presence ratings, and because it appeared from other associations obtained from the rating scales that gender also affected presence ratings, further analysis was performed on a composite "presence" score obtained by adding the ratings on the two presence items on the questionnaire giving a maximum score of 10. Two-way ANOVA involving two levels of gender and two levels of spatial reasoning ability was performed on presence scores. For gender, F(1,53)=6.53, p<.05. For spatial ability, F(1,53)=5.58, p<.05. For the interaction of gender with spatial ability, F(1,53)=5.51, p<.05. Spatial reasoning ability did not affect presence ratings for boys, (MLow Spatial=7.47, sd=1.26, MHigh Spatial=7.47, sd=1.93). However, low spatial girls reported lower presence than high spatial girls, (MLow Spatial=5.14, sd=1.96, MHigh Spatial=7.38, sd=1.41, t(20)=2.82, p<.01).

As expected, the world-building activity improved the posttest performance of low ability students who built worlds when compared to those learning in a traditional manner. This suggests that the aggregate of innovative learning activities afforded by world building helped students understand the material who do not have high general ability as measured using a traditional test. The lack of difference between high ability students in both groups simply reconfirms that brighter students can learn from a variety of approaches and therefore, for them, the innovative nature of the world-building strategy had no effect. It remains to determine in a more controlled study which aspects of world building and of experiencing a VE made the greatest contribution to this gain - immersion, interaction, motivation, or the ability to learn concepts and principles directly without the need to master an abstruse and abstract symbol system first.

The lack of prediction of performance by spatial reasoning and spatial visualization ability was somewhat surprising. After all, a VE is a place where all learning occurs in 3-dimensional space. We suspect, with hindsight, that technical difficulties in navigating and operating in the VEs might have canceled out any advantages accruing from high spatial ability. The VE was just not conducive to using these abilities. Moreover, the finding that spatial ability predicts presence and enjoyment supports this interpretation since we know that presence and enjoyment predict performance and that navigating and operating difficulties reduce presence and enjoyment (Winn, 1995). This also suggests that high spatial ability, by heightening presence and enjoyment, might influence performance indirectly. However, this explanation awaits empirical verification. Also, students collaborated extensively designing and building their VE. This division of labor might have helped low spatial students compensate for their low spatial ability.

The first factor analysis showed that high general ability students used the recommended strategies of drawing and using three-dimensional models to help them visualize objects before modeling them on the computer. It seems that the more able students use appropriate strategies for performing particular tasks. Also, students with high spatial reasoning ability reported less physical malaise in the VE. This suggests that students who are good at manipulating objects are less likely to suffer the side effects that motion through space sometimes induces. The finding that students with high spatial visualization ability reported collaborating with other students more often than students low on this ability has no obvious interpretation. Perhaps other students turned to them for help when they needed to visualize objects in the VE from different points of view.

The factor scores from the second factor analysis suggest, in addition to the finding about enjoyment and presence, that more able students found it easier to perform tasks in the VE, including the ability to find objects and navigate in it, than less able students. One interpretation of this finding, that is corroborated by the result reported above, is that working in a VE requires the commitment of cognitive resources more available to high ability students. This would be the case if the interface is not sufficiently intuitive, requiring purposeful attention in order for the student to use it. Anecdotally, we can report that, in a grade 11 Chemistry study in which students spent close to an hour working in a VE while producing think-aloud data, the first fifteen minutes in almost every case were dominated by comments about the interface, and only later did subjects' thoughts turn to Chemistry. (Data from this project have not yet been completely analyzed.)

Girls with low spatial reasoning ability reported experiencing less presence than girls with high spatial reasoning ability. This difference was not found for boys, who reported higher levels of presence than girls. It is possible that boys have different ways from girls of becoming engaged in a VE. Maybe they have more exposure to computer games and have developed better skills for manipulating the interface than girls. Maybe they are more easily fooled into believing a VE is real than girls. This finding requires further study.

We consider this study to be exploratory. A good theoretical framework has not yet been constructed from which to construct good hypotheses about the questions that arose a priori and ex post facto in this study. Moreover, the setting of the study - intact classrooms in different schools working on different topics - was not conducive to producing anything but messy data. Nonetheless, what analysis these data permitted have confirmed the general findings that creating and visiting VEs helps less able students understand material. This may be because working with VEs allows these students to use learning strategies that are not called upon in traditional classrooms where emphasis is on learning more symbolically. Presence is clearly a key to learning in VEs and is related to spatial reasoning ability. And gender is a factor, whose precise role we still need to determine, but which should not be ignored. As the technology for building and learning in VEs advances both the quality of the VE and the ease of working in it, more carefully controlled studies will be possible and we will be in a position to conduct studies of precisely which features of VEs facilitate the learning of what kinds of content for which students.

Anderson, J.R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1996). Cognitive tutors: Lessons learned. Pittsburgh, PA: Carnegie Mello University, Unpublished manuscript.

Barfield, W. and Hendrix, C. (1995, March). Experiments investigating presence. Proceedings of the meeting of the Virtual Reality Annual International Symposium, Chapel Hill, NC.

Bricken, M. (1991). Virtual worlds: No interface to design. In M. Benedikt (Ed.), Cyberspace: First steps. Cambridge, MA: MIT Press.

Bricken, M., & Byrne, C.M. (1993). Summer students in virtual reality: A pilot study on educational applications of virtual reality technology. In A. Wexelblat (Ed.), Virtual reality applications and explorations. Cambridge, MA: Academic Press Professional.

Byrne, C.M. (1996). Water on tap: The Use of Virtual Reality as an Educational Tool. PhD. Dissertation, department of Industrial Engineering, University of Washington, Seattle, WA.

Clancey, W.J. (1993). Situated action: A neuropsychological interpretation: Response to Vera and Simon. Cognitive Science, 17, 87-116.

Dede, C. (1992). The future of multimedia: Bridging to virtual worlds. Educational Technology, 32, (5), 54-60.

Dede, C. (1995) The Evolution of Constructivist Learning Environments: Immersion in Distributed, Virtual Worlds. Educational Technology; 35 (5), 46-52.

Dede C.J., Salzman, M., & Loftin, R.B. (1996). The development of a virtual world for learning Newtonian mechanics. In Brusilovsky, P., Kommers, P., & Streitz, N. (Eds.), Multimedia, Hypermedia, and Virtual Reality. Models, Systems and Applications. Proceedings of the . First International Conference, MHVR`94. Berlin: Springer Verlag.

Duffy, T.M., Lowyck, J., & Jonassen, D.H. (1983). Designing environments for constructive learning. New York: Springer.

Ekstrom, R. (1976). Kit of Referenced Tests for Cognitive Factors. Princeton, NJ: Educational Testing Service.

Harel, I., & Papert, S. (1991) (Eds.). Constructionism. Norwood, NJ: Ablex.

Hoffman, H.G., Hullfish, K.C. and Houston, S. (1995). Virtual Reality Monitoring. Seattle, WA: Human Interface Technology Laboratory Technical Report, P-95-1.

Loftin, R.B., Engleberg, M., & Benedetti, R. (1993). Applying virtual reality in education: A prototypical virtual physics laboratory. Proceedings IEEE 1993 Symposium on Research Frontiers in Virtual Reality. Los Alamitos, CA: IEEE Computer Society Press.

McLellan, H. (1996). Virtual realities. In D. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology. New York: Macmillan, 457-490.

Osberg, K.M. (1997, March). The effect of having grade seven students construct virtual environments on their comprehension of science. Paper presented at the annual meeting of the American Educational Research Association, Chicago, Il.

Psotka, J. (1995). Immersive tutoring systems: Virtual reality and education and training. [Available as an HTML document from http://www.hitl.washington.edu].

Raven, J. (1958). Standard Progressive Matrices. San Antonio, TX: The Psychological Corporation.

Rose, H. (1996). Design and Construction of a Virtual Environment for Japanese Language Instruction. Seattle, WA: University of Washington, College of Education. Unpublished masters thesis.

Simon, H. (1981). The Sciences of the Artificial. Cambridge, MA: MIT Press.

Taylor, D.W. (1997, March). Student responses to their immersion in a virtual environment. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.

Winn, W.D. (1993). A conceptual basis for educational applications of virtual reality. Human Interface Technology Laboratory Technical Report. Seattle, WA: Human Interface Technology Laboratory. [Available as HTML document from http://www.hitl.washington.edu].

Winn, W.D. (1995). The Virtual Reality Roving Vehicle project. Technological Horizons in Education, 23, (5), 70-74.

Zeltzer, D. (1992). Autonomy, interaction and presence. Presence, 1, 127-132.

Footnote.

1. This project was funded with a grant from the US WEST Foundation to the Human Interface Technology Laboratory at the University of Washington. We gratefully acknowledge the support of US WEST while taking full responsibility ourselves for all opinions and any errors that occur in this paper.