Figure 1
- Concept Map Pre- and Post-test Built vs. Chosen Cycles by Group| Publications Page | HITL Home |
The Human Interface Technology Laboratory's (HIT Lab) Learning Center has been providing students with the opportunity to construct their own virtual environments since 1990, by working through special programs such as the Pacific Science Center's Technology Camp, and through other educational environments. Starting in 1995, the Learning Center created the Virtual Reality Roving Vehicle (VRRV)1 program, which allowed students to take a more active role in the entire virtual development process by taking the technology directly into the classroom.
The pilot for the VRRV project was Kellogg Middle School. We worked with 120 seventh grade students in the Kellogg Classroom of Tomorrow (KCOT) program (Kellogg Middle School, 1993; 1996), a constructivist-based extension of the Apple Classroom of Tomorrow (ACOT) project (Apple Computer, Inc., 1994; Dwyer, 1994; Yocam, Wilmore, & Dwyer, 1992; Baker, Herman & Gearhart, 1989). These students were studying wetlands ecology. The pilot program was designed to test our assumptions about the educational value of bringing world-building and experiencing technology directly to the students, especially with regard to students' comprehension and motivation by providing them direct access to the technology within their traditional learning environment.
The 'technology' provided to the students included a 3-D modeling package that could be used on their classroom MacIntosh computers, instruction in how to develop a virtual environment, and the Division ProVision100, a computer capable of presenting real-time 3-D stereoscopic images in an 'immersive' headset, and tracking both the head (helmet) and hand (wand) movements of the participant as he or she traverses the virtual environment.
Constructivism is a form of relativism that is primarily a subjective, rather than objective perspective. Under the Constructivist paradigm, there is latitude and acceptance for a variety of ways of knowing (Duffy & Jonassen, 1992; Lakoff, 1987; Bruner, 1973, 1990; Percy, 1992; Belenky et al., 1986; Pascuel-Leone, 1979, 1980).
As stated by Duffy & Jonassen (1992), on our human ability to frame our environment with meaning:
Constructivism provides an alternative epistemological base to the Objectivist tradition. Constructivism, like objectivism holds that there is a real world that we experience. However, the argument is that meaning is imposed on the world by us, rather than existing in the world independently of us. There are many ways to structure the world, and there are many meanings or perspectives for any event or concept. Thus there is not a correct meaning that we are striving for (p. 3).
One means of constructing knowledge is to create meaning by doing, and that the resulting knowledge 'creation' is unique unto the individual. Research has been conducted on the practical application of constructivist principles in the classroom (Brooks & Brooks, 1996, 1993; Wittrock, 1987, 1991; Wittrock & Alesandrini, 1990). It is clear from the results of these studies that it is possible to provide learning environments in which constructivist practices can be practically implemented.
Most information provided to students in the past has been pre-designed by such individuals as the teacher, the textbook author, or the multimedia developer. Though knowledge construction can and does occur with pre-designed information (Duffy, Lowyck & Jonassen, 1983, Duffy & Jonassen, 1992; Bruner, 1990), there is opportunity for fostering deeper understanding by bringing the student into the design process itself (Mones-Hattal & Mandes, 1995, Winn, 1995).
With advances in visual and interactive technologies such as virtual reality, the process of knowledge construction and meaning making from a visual and auditory perspective can be more fully explored. By creating their own environments, students can develop their own set of objects, relationships, and behaviors that are meaningful to them, and that can be shared and experienced through full-body interaction.
Creating support for knowledge construction within the student is a critical component to the success of developing self-motivated, intellectually stimulated learners (Wiske, 1994; Unger, 1994; Poplin, 1991; Duffy & Jonassen, 1992; Arnold, 1991). Virtual reality can make a unique contribution to knowledge construction because it is an environment in which students can imbed and extend their understanding in both a visual and an interactive fashion. In creating a virtual world, students can attribute meaning to objects, relationships and behaviors in a way that mirrors their personal understanding (Osberg, 1995).
The concept of learning through virtual reality has proven to be of positive value in some instances (Byrne, 1993, 1996; Osberg, 1995b; Dede, Salzman & Loftin, 1996; Rose, 1996; McLellan, 1996, Bricken & Byrne, 1992), especially for lower-functioning male students (Winn, 1997). However, much of the research conducted to date has been with pre-constructed virtual environments; environments created by designers and instructors, rather than by the students themselves.
Virtual reality provides students with an opportunity to interact directly with information embodied in a visual (Mones-Hattal & Mandes, 1995; Gigliotti, 1996), potentially 'intelligent' (Rose, 1996) form. Interaction is a critical component to students' knowledge construction, whether in a virtual or traditional educational environment (Byrne, 1996; Psotka, 1995). However, virtual reality provides more than just an opportunity for interaction; it engages the whole body in a manner that is valuable for developing somatic memory (Kraft & Sakofs, 1989; Samuels & Samuels, 1985; Dychtwald, 1977), and provides the participant with an opportunity to perceptually engage with the environment as if they were physically present in the computer-generated 'space' (Hoffman, Hullfish, & Houston, 1994; Zeltzer, 1992). The potential value of this dichotomy has been a point of discussion with virtual reality theorists (Hiem, 1994) and practitioners (Mones-Hattal & Mandes, 1995, Loftin & Kenney, 1995), alike.
Furthermore, especially when developing their own virtual environments, students feel a great deal of personal power over their learning process, and become engaged in facilitating their own and other's learning (Winn, 1995; Osberg, 1995b). The world-building process required four-steps: planning, design, programming and experiencing (Osberg, 1995a). By developing their own virtual learning environments, students have a great deal of latitude and control over their learning process by participating in the planning and design phases, rather than coming into the process only during the experiencing phase.
In this study, students had to master the content required to understand their wetland cycle, become 3-D modelers, understand the technology used to program and present their 3-D representations, and to consider how to engage the participant in 'experiencing' one of the cycles from start to finish. The combination of learning and directly applying what they had learned was very empowering for most of these students.
The main purpose of this study was to test the hypothesis that learning about a wetland cycle using constructivist principles (student-directed information retrieval and compilation, collaborative virtual environment design and construction) would yield greater comprehension of subject matter than learning about a wetlands cycle through traditional means (teacher-directed classroom lecture, single textbook readings, worksheet completion). Our second hypothesis was that the traditional classroom approach would be more educationally efficacious than a no instruction control.
Subjects.
Subjects in this study were 117 middle school (Grade 7 and 8; ages 12-14) students attending Kellogg Middle School in north Seattle. The subjects were randomly assigned to four groups; Carbon (n = 30), Energy (n = 27), Nitrogen (n = 30), and Water (n = 30), the names of which represent the wetland cycle these students constructed .
These students were part of a 4-classroom experimental program focusing on the implementation of constructivist learning, integrated multiple-content area curriculum blocks, and the use of technology as an integral part of the learning process. The subjects were either individually (when in their own classroom) or team-taught (when two classrooms are combined) by two teachers per classroom.
For the purposes of this study, all four classroom teachers were an integral part of the process, as was an additional 'traditional' science teacher, who came out of retirement to participate out of personal interest.
Design
Students in each of the four groups:
The treatments, based on instructional strategies, were:
We were able to obtain data sets for 88 out of 117 students for statistical analysis. The other 29 were too incomplete to be of value. Of these 88 students, 47 were boys and 41 girls. Not all students had complete data sets (quantitative, concept map and interview data), but all 88 students had complete scores for at least two of the three measures.
The subject area studied was wetlands ecology. Within this subject area, four cycles were studied: carbon, energy, nitrogen and water.
The design was developed to study the response of the subjects to one independent variable; pedagogy, with three levels; the constructivist approach, the traditional approach, and no instruction. Each student studied three of the four cycles; one using constructivist learning strategies, and two using traditional strategies. The control treatment was provided by one of the KCOT teachers, in which the students studied other non-related subject matter. The constructivist learning paradigm was paired with the virtual environment creation process as part of the learning paradigm. Students were self-directed with regard to their information research regarding their wetland cycle. The two traditional cycles were studied using textbooks and worksheets in a teacher-directed classroom environment. The last cycle was not studied directly, and served as a means to test our assessment instruments.
Procedures
The study lasted two weeks, 4 days a week on site at Kellogg Middle School. There were two 1.5 hour blocks per day of teaching time in the KCOT classrooms; from 9:00 - 11:30, and from 12:00 - 1:30. During each of these blocks, each of the four classrooms held class simultaneously; one in virtual environment development and constructivist learning, one on traditional science, and two on other subjects currently being studied.
Each group had a total time-on-task for constructivist learning and world building of 3 hours per week; 6 hours total. The same held true for each of the other cycles, except of course for the no instruction control. All testing was conducted either before the project began, or after completion of the project. This meant that almost all classroom time could be spent on-task.
CONSTRUCTIVIST INSTRUCTIONAL PROGRAM
During the first 1.5 hour constructivist block, and half of the second 1.5 hour block, Carbon Group subjects had the opportunity to look over materials related to general wetlands ecology, and the carbon cycle specifically. These materials were selected by the students from library guides. They could also view Internet-accessible information on wetlands ecology and their particular cycle, review CD-ROM and video-disk materials about the process, and develop an understanding of the concepts based on their experiences with these materials. There was no direct instruction in the constructivist classroom.
The four steps or phases of world building are Planning, Building, Programming and Experiencing. By providing a progressive structure to the process, we were able to create an environment in which students had ample opportunity to think deeply about their constructions, and to enhance their visio-spatial skills through drawings and models created both by hand and with the assistance of the computer.
During the second half of the second 1.5 hour block, subjects met in groups of 10, to begin the design phase of world development. These groups were led by a Human Interface Technology Laboratory (HIT Lab) Virtual Reality Roving Vehicle (VRRV) representative. In these groups, roles were assigned, to make certain that all aspects of the design/build process would be covered. After roles were assigned, subjects discussed their view of what should be contained in a virtual wetland environment designed to portray the cycle that they were studying. Each group of 10 students designed their environment from the ground up. The design process included:
Sketching and modeling of objects was highly encouraged. Sufficient, although limited time to talk and develop ideas within their groups was provided, to discuss visual representations, metaphorical concept representation, stage setting and design.
During the third and fourth 1.5 hour blocks, each subject took part in the building phase of world development by creating their set of assigned objects on their classroom computers, either alone or in groups, using Swivel 3-D and a MacIntosh computer as their modeling tools. Students used their paper-and-pencil sketches of their objects to help them construct the 3-D version of the object on the computer. By drawing their objects from a top, side and cross section view, students were able to 'map' their drawings on to the Swivel 3-D software interface used to model the objects in 3-D. We found this type of pre-construction analysis helpful for simple objects, and critical for complex, multi-component objects.
Another component of the design process was to teach students fundamental concepts of composition and functionality. Composition of the world was discussed in terms of both aesthetic and functional appeal (Gigliotti, 1996). We used the metaphor of dressing a three-dimensional stage when talking to students about this issue (Laurel, 1991). Considerations included where the participant would enter, what they would have in their near and far environment, whether a clear path had been established, or whether the participant was intended to explore at will, and so on. We also presented composition with respect to the learning goals of the environment.
VRRV staff constructed the final virtual environment from the students' objects, drawings, and behavior matrices, on the Division ProVision 100, a proprietary system including both the programming/rendering language and virtual reality hardware.
During the last two days of the two-week program, students took part in the experiential portion of the 4-step process by experiencing both their own virtual environment, and an environment designed by another student group. The equipment used for the experiential portion of the project was the Division ProVision 100, a proprietary immersive system that includes a substantially enhanced 486 processor, wand, headset, and tracking system for both the participant's head and hand.
This process took place in a portable close to the students' regular classrooms. We had two machines set up within the portable, one interview station, and a large conference table at which students could fill out their surveys. There were generally 5 to 10 students present at any given moment during this stage, as we needed to have students both 'under the helmet' and 'on deck', to get everyone through two environments in the time frame allowed. We also videotaped students experiencing their 'created' environment.
TRADITIONAL INSTRUCTIONAL PROGRAM
During the first 1.5 hour traditional block, subjects were guided by the traditional science teacher in reading wetland, water, and nitrogen-specific portions of the textbook entitled Life Science: The Challenge of Discovery. Handouts provided the students with page numbers tied to the cycle being studied, a key word list, and a set of study questions to be answered in class during the discussion session. Students were told that they could seek out additional sources of information should they so desire. It should be noted that most students chose to stay in their classroom and work with the existing text.
During the second, third and fourth 1.5 hour blocks, students were given flowcharts and worksheets to fill out, with specific page numbers relating to the text. Both the flowcharts and worksheets had 'clues', in that part of the cycle being studied was already drawn or described. Students were expected to 'fill in the blanks'.
At the end of the two-week project, some of the students (n = 41) got to experience a virtual world that they had studied using traditional learning strategies. The remainder of the students (n = 25) experienced a virtual environment in which they had had no instruction.
NO INSTRUCTION CONTROL
The No Instruction Control provided no direct instruction regarding a particular wetland cycle; in fact, the teacher who had the control group taught a completely unrelated subject during her time with the students. However, there was a great deal of overlap between the cycles studied, and students already knew the water cycle in particular from earlier studies. Some also knew the energy cycle to a degree. Though no direct instruction was provided, it can be assumed that some of the information gleaned during general wetlands study, or study of the other cycles may have provided useful information about the no-instruction cycle to the students.
Instruments.
All subjects were pre- and post-tested by having them draw concept maps (pictorial/verbal representations of the individual's view of that particular cycle) of both their constructed virtual wetland cycle, and a cycle of their choice, and by an objective multiple-choice instrument that addressed all four cycles that had been designed by the KCOT teachers in conjunction with the visiting 'traditional' science teacher.
Students were also interviewed after experiencing both their own virtual learning environment, and the virtual learning environment created by another group of students. After they had experienced both environments, the students completed a survey about their reactions to both the design and experience process.
We were able to conduct statistical analysis on all of the measures collected: Objective pre- and post-tests scores, concept map pre- and post-test scores for both the world that they built, and a chosen world that they wished to represent, Interview data for all students who experienced VR, and Survey information.
Regarding the objective tests, an ANOVA with test occurrences as a within-subjects factor of pre- and post-test scores revealed a significant improvement in scores overall, F(1, 79) = 97.58, p < .001. There was no significant main effect based on the world that the students built, F(3, 79) = 1.78, p > .05, and no interaction effects.
Based on these findings, we were unable to ascertain that instructional paradigm alone had an effect on the children's understanding of particular cycles. However, treatment effects were found for concept map data.
The concept maps provided the richest data from all of the measures taken. Rated using holistic scoring techniques, each cycle described was carefully examined to determine if the student had provided information in a manner fitting the criteria by raters blind to treatment. Scores for concept maps ranged from 0 - 4. Interrator reliability for concept map analysis was significant (Z = -3.5279; p < .001).
Sixty-seven subjects drew four maps: two 'built' (before and after the world they built and experienced (T1)), and two 'chosen', i.e. a self-selected representation of one of the three other cycles being studied (before and after the treatment had been administered (either T2 or T3)). Not surprisingly, many of the children selected water as the cycle to represent for their 'chosen' drawing; the cycle best known to them prior to their experience in this project. Mean scores by group by pre- and post-test for both built and chosen maps is presented in Table 1, below.
Mean Scores
Built vs. Chosen Pre-test Post-test
by Group Built Chosen Built Chosen
Carbon (n = 14) 1.57(1.45) .93(1.14) 2.57(1.40) 2.64(1.22)
Energy (n = 17) 1.00(0.50) 2.53(1.46) 2.71(0.92) 2.71(1.40)
Nitrogen (n = 14) .93(0.47) 3.00(1.53) 2.86(0.95) 3.71(1.50)
Water (n = 23) 2.26(1.18) 1.48(1.08) 3.30(0.70) 2.22(1.28)
Grand Means (n = 68) 1.53(1.14) 1.93(1.48) 2.91(1.00) 2.60(1.35)
Note: Standard deviations in parentheses.
Concept maps were analyzed by ANOVA in two ways. The first analysis consisted of two within-subjects factors, pre- and post-test measures, and built vs. chosen measures. Group was used as a between-subjects factor. A second analysis was conducted on instructional treatment (Constructivist, Traditional, and No Instruction), as a within-subjects factor for Constructivist vs. Traditional, and Constructivist vs. No Instruction, and as a between-subjects factor for Traditional vs. No Instruction.
The first analysis, a within-subjects ANOVA comparing pre- and post-test scores, by built vs. chosen world yielded no main effect based on the world that the children drew, F(3, 63) = .71, p > .05. However, there was a significant pre-post effect, F(1, 63) = 71.75, (p < .001).
This pre-post effect is consistent for all concept map measures. It is clear that the students' cognitive gains in procedural and relational knowledge improved. What is also clear is that the wetlands project resulted in significant comprehension and understanding of the subject matter, regardless of group.
Further concept map analysis indicates no significant main built effect, F(1, 63) = 2.04, p > .05, a significant interaction effect between built vs. chosen and group, F(3, 63) = 15.56, p < .001, a significant interaction F(1, 63) = 10.47, (p < .005) between pre-post and built-chosen, and a significant interaction effect F(3, 63) = 6.10, (p < .01) between pre-post, built-chosen, and group.
These findings indicate that not only were the pre- and post-test scores significantly different, but the scores varied based on whether the children had built the world the represented in their drawings, or whether it was the world that they had chosen to represent. Furthermore, the additional interaction between group, pre-post and built-chosen indicates that the group the children were in also had a significant effect on this interaction.
As can be seen in Figure 1, below, the Carbon group experienced the most significant gains in both built and chosen worlds. In all other groups, pre-post differences were much more substantial for the world that they built, rather than the world that they chose to represent. It is interesting to note that the Carbon group contained the three intellectually challenged children in the KCOT program. In previous (Osberg, 1993b) and subsequent (Winn, 1995; 1997) research, it has been found that virtual environment design tends to help learning impaired children even more substantially than non-learning impaired students. These findings could be indicative of this trend.
Figure 1
- Concept Map Pre- and Post-test Built vs. Chosen Cycles by GroupWhat can be seen in comparing group information is that there are gains for all groups in all instances, and for three of the four groups, gains are much less substantive for the chosen concept map representations. This is what we had anticipated based on the fact that we perceived that most subjects, given their choice, would draw a cycle known to them, namely water. However, for the Carbon group, the gains for chosen world representations were even more substantive than for their built environment. This inconsistency can be attributed, we believe, to the less robust knowledge base of some of the Carbon group members.
Built world concept map gains were consistent for all groups, resulting in at least a +1.0 rise in scores on a 4-point scale.
Another interesting way to look at this same data set is to alter the graphics to show the built vs. chosen scores for each group. An illustration of this analysis is presented in Figure 2, below.
Figure 2
- Concept Map Pre- and Post-test Scores by Group by Built vs.
Chosen CycleWhat can be seen from this graph is that for both built and chosen concept map representations for all groups, the post-test scores are consistently higher than pre-test scores, and that the Water group did the best on the pre-test, again because this cycle was already known.
Treatment comparisons yielded the most interesting results of all of the analyses conducted. In comparing concept map scores based on treatment (Constructivist, Traditional and No Instruction), we conducted a within-subjects ANOVA to compare Constructivist vs. Traditional scores, and Constructivist vs. No Instruction scores, and a between-subjects ANOVA comparing Traditional vs. No Instruction scores.
To illustrate the comparisons more fully, a means table is presented below which will be referenced throughout this section. In Table 2, means for each treatment group for both built and chosen cycle representations, and pre- and post-test scores are presented.
Mean Scores
for Pre- and Post-tests Treatment
by Treatment Constructivist Traditional No Instruction
Constructivist vs. Traditional (n = 43)
Pre-test 1.67(1.27) 1.74(1.56)
Post-test 2.79(1.04) 2.67(1.34)
Constructivist vs. No Instruction (n = 24)
Pre-test 1.28(0.84) 2.25(1.29)
Post-test 3.12(0.93) 2.46(1.38)
Traditional (n = 43) vs. No Instruction (n = 24)
Pre-test 1.74(1.56) 2.25(1.29)
Post-test 2.67(1.34) 2.46(1.38)
Note: Standard deviations in parentheses.
Constructivist vs. Traditional Treatment Analysis
For the Constructivist vs. Traditional comparison (n = 43), using a within-subjects ANOVA, we found a significant F(1, 42) = 58.23, (p < .001) pre-post effect, but no significant effect for the cycle illustrated under either instructional paradigm F(1, 42) = .01, (p > .05), and no interaction effects, F(1, 42) = .41, (p > .05).
This finding indicates that instructional paradigm did not significantly affect the children's ability to represent a wetlands cycle; that both the Constructivist and Traditional learning paradigms resulted in significant gains. Though the means in each case varied, as can be seen in Table 2, above, they did not differ significantly, which refutes my original hypothesis that the Constructivist learning paradigm would provide more substantive results than Traditional education, at least using the assessment criterion that we established.
However, by comparing the actual drawings, there are other conclusions that can be drawn. The concept maps illustrated under both the Constructivist and the Traditional treatment are visually richer, more complex, and more accurate after the virtual world building experience than those created during the pre-test. This comparison is consistent across groups. Both are technically correct, yet there is clear value in the creative representation of information, especially in the transformation or translation of information from one symbol system to another.
This transformation process is one means to assess the development of students visual literacy (Mones-Hattal & Mandes, 1995; Kirby, Moore & Schofield, 1988), which is defined by Farmer (1987) as 'the abilities to read and interpret visually and to express oneself honestly and accurately by translating visual symbols into verbal language and vice versa'.
In this study, almost source information regarding wetlands ecology was primarily text based, with the addition of limited videodisk, CD-ROM or Internet-based images and sounds. The translation process was embodied in the subjects' ability to transform this text-based source information into an interactive, visual representation by creating a virtual environment. The comparison between the pre and post-test representations is clear; the effects of translating that information into a visual representation provides an additional element to the students knowledge base above and beyond 'technical' correctness.
This richness could be described, in Gibsonian (1986) terms, as arising from the 'cognitive process that includes mental rehearsal, introspection, and visualization, and distinguishes itself as a thought process different from verbal thinking whereby each exposure to the visual image permits the observer to become a keener interpreter of the visual display, i.e. to see more and more element within the display over time'. In the case of these two concept maps, what you are seeing as the result of the constructivist learning experience, reified in world design, is that keener perspective; the ability to utilize the visual component in addition to the textual description to clarify and elucidate the carbon cycle.
This relates to Morris & Hampson's (1983) image taxonomy in that what this student has chosen to represent has become 'real' for her; when she thinks about these cycles, it is in this form. It also relates to Mones-Hattal & Mandes (1995) perception that virtual reality involves visual thinking, which deeply affects our perceptions, and our memory.
Constructivist vs. No Instruction Treatment Analysis
For the Constructivist vs. No Instruction treatment comparison (n = 24), we found significant pre-post effect, F(1, 23) = 18.40, (p < .001), no significant built-chosen treatment effect, F(1, 23) = .45, (p > .05) and a significant pre-post, built-chosen treatment interaction, F(1, 23) = 18.25, (p < .001).
In conducting paired t-tests to analyze mean scores for pre-post/built-chosen worlds, we found the pre- and post-test means for the Constructivist treatment were significant (p < .001), but that the No Instruction treatment means did not vary significantly.
This finding indicates that concept maps improved significantly for worlds that the children built themselves, but not for worlds that were drawn by choice in which children had received no instruction. This supports a portion of my original hypothesis that constructivist learning is certainly more valuable than no instruction whatsoever. Even so, the children's representations were much more pictorial after the world building experience, as was true for the comparison between Constructivist and Traditional treatments.
Traditional vs. No Instruction Treatment Analysis
For the Traditional vs. No Instruction comparison (n = 67), we performed both a within-subjects analysis for pre-post measures, and a between-subjects analysis comparing the individuals in the Traditional vs. the individuals in the No Instruction treatment group. In doing so, we found no significant score differences between the two treatments, Traditional vs. No Instruction, F(1, 65) = .06, p > .05, a significant pre-post effect, F(1, 65) = 5.35, p < .05 and a significant pre-post, treatment interaction effect, F(1, 65) = 4.00, p = .05.
In conducting paired t-tests to analyze mean scores for pre-post/treatment interaction, we found the difference between Traditional pre- and post-scores was significant (p < .001), but that the No Instruction pre- and post-test scores did not vary significantly.
This implies that though Traditional education provided the students opportunity to improve their knowledge on that particular wetland cycle, the No Instruction treatment does not does not. The interaction effect must be attributed to the differences in instructional treatment.
Summary of Treatment Analyses
Based on the findings above, both the Constructivist and Traditional educational approach were both educationally valuable in that pre- and post-test scores for both treatments improved significantly. Further analysis indicated that the Constructivist approach is more educationally valuable than No Instruction, but that the comparison between the Traditional and No Instruction treatments did not yield significant differences between the two treatments. However, pre- and post-test scores were significantly different for the Traditional approach, indicating educational improvement, but not for the No Instruction treatment.
These analyses were based on subjects' built vs. chosen cycle representations. Given that the students chose to represent was most often water, regardless of whether the treatment for that cycle was Traditional or No Instruction, it is not surprising to see no significant treatment difference between Traditional and No Instruction.
What was surprising was to see no significant treatment difference between Constructivist and Traditional treatments. In the end, it becomes a matter of interpretation, and of desired outcomes. If we choose to foster and value creativity and alternative forms of knowledge representation in our educational settings, such as was illustrated in the concept map comparisons above, then the Constructivist approach is one way to facilitate such knowledge acquisition and application. If instead we choose to focus on the 'technically correct' version of knowledge recall and application, without giving additional thought to the inventiveness or cognitive value in translating that information, both the Constructivist and Traditional approaches are equally educationally valuable.
The interviews were conducted just after the students had completed their virtual experiences, both within the world that they built, and the world that they visited that had been constructed by another student group. Interviewers asked students to recall the cycle just experienced, as represented in the virtual environment.
Many of the students, in addition to using words to describe their experiences, moved their bodies in the same way that they had while in the virtual environment. This indicates a somatic 'memory' that is not described in the text-based data, but is well worth mentioning (Kraft & Sakofs, 1989).
Interviews were rated on a scale of 0-4 using a similar set of criteria as used to rate the concept maps. The information gathered during these interviews was intended to test whether the students remembered the steps to each respective cycle, as well as the key components required to complete each cycle. The evaluator also kept track of how often the student needed to be prompted, whether steps were remembered in order or whether the remembrances were somewhat scattered, and other comments that the students had about their experiences. The interviews were also video-taped for review purposes.
The interview data, analyzed using within-subjects ANOVA, indicate a significant main effect F(3, 65) = 2.79, (p < .05) based on the world that the subjects built. This indicates that children's ability to accurately describe their experiences, regardless of whether the world was the one that they themselves built, or was built by another student group depended at least partially on the group in which they were in.
There was also a significant F(1, 65) = 14.68, (p < .001) built-experienced main effect, and a significant interaction effect, F(3, 65) = 4.37, (p < .01) between built vs. experienced and group.
In analyzing the mean scores for each group, the differences based on group illustrated in the table below appear to be dictated by low Carbon Interview scores for their experienced (not built) world. The three learning impaired students in the KCOT program had all been placed in the Carbon group; the group that experienced Nitrogen world; the most difficult cycle to understand. All of the other groups experienced the three easier cycle worlds, carbon, energy and water, which is at least a part of the reason that they were able to accurately recount and describe their experienced cycles.
Mean Scores
Group Built Experienced Exp. World
Total Sample (n = 69) 4.06(1.07) 3.57(1.31)
Carbon (n = 18 ) 3.94(1.16) 2.78(1.48) Nitrogen
Energy (n = 19) 4.26(1.28) 4.37(.895) Water/Nitro
Nitrogen (n = 19) 4.00(.882) 3.53(1.31) Energy/Carbon
Water (n = 13) 4.00(.913) 3.54(1.05) Energy
Note: Standard deviations in parentheses.
In fact, the group with the highest mean Interview scores for both built and experienced environments was the Energy group, who had contact with what the teachers felt were the two easiest cycles, energy and water. The Nitrogen and Water groups were both very close in scores for both built and experienced environments.
Interview and Concept Map Comparisons
To compare Interview and concept map data, we wanted to review which educational paradigm had been employed regarding the worlds experienced by the students. We present the following table describing the built and experienced worlds, coupled with the educational paradigm or treatment for each in Table 4, below.
Built Worlds Experienced Worlds
Group (Constructivist) Traditional No Instruction
Carbon (n = 18 ) Nitrogen (n = 18)
Energy (n = 19) Nitrogen (n = 4) Water (n = 15)
Nitrogen (n = 19) Carbon (n = 11) Energy (n = 8)
Water (n = 13) Energy (n = 13)
Total (n = 69) Total (n = 33) Total (n = 36)
When integrating the information from Table 3 and Table 4, we can derive Table 5, below. In both the carbon and nitrogen cycles, children were required to describe chemical processes, unlike the water and energy cycles which were more natural-object rather than process oriented. In other words, the carbon and nitrogen cycles were more abstract than water or energy, which may have been a source of difficulty in verbally describing the cycle experienced than in describing the two 'easier' subjects. However, the scores for the visual representations provided in the concept maps yielded no significant difference between Constructivist and Traditional treatments. This supports my position that abstract information is easier to describe using visual tools.
Mean Scores
Group Built Experienced Exp. World
Total Sample (n = 69) 4.06(1.07) 3.57(1.31)
Carbon (n = 18 ) 3.94(1.16) 2.78(1.48) Nitrogen=T
Energy (n = 19) 4.26(1.28) 4.37(.895) Water=N/Nitro=T
Nitrogen (n = 19) 4.00(.882) 3.53(1.31) Energy=N/Carbon=T
Water (n = 13) 4.00(.913) 3.54(1.05) Energy=N
Note: Standard deviations in parentheses. Treatment in bold
script; T =
Traditional, N = No Instruction.
The Survey was analyzed using frequency distributions. Questions were asked about both the process of developing the virtual environment, and about the experience of being in the virtual space. We were trying to ascertain which portion(s) of the project subjects deemed to be most valuable or enjoyable, and whether they would consider undertaking such a project in their educational environments in the future.
Results indicate that the students very much enjoyed the "Virtual Wetlands", as we billed the project. They liked both building and visiting their environments, and wanted to incorporate the use of virtual technology into the curriculum for students who would be studying this subject in the future. Almost all of the students wanted to experience virtual reality again.
Using the combination of a 7-point Likert scale and two essay questions at the end of the survey, we found that often students tended to polarize towards either the top or bottom of the scale, with very few questions having a 'normal' distribution. Trends indicated by the fourteen questions on the survey are described below by the kind of issue addressed by the question; Process, Learning and Teaching, Task Understanding and Difficulty, Perceived Value and Overall Enjoyment, Physical Discomfort, and Presence. An Essay Question analysis follows. For a more complete analysis of the Enjoyment Survey Data, see Osberg (1997).
Process
Process questions dealt with students' perceptions of whether they worked primarily alone, or whether they worked with their partners, their group, or both. Most students indicated that they worked most closely with their partner, followed by there group. This is an important finding, as the project had been designed as a collaborative learning opportunity. We were happy to see that students recognized this.
Learning and Teaching
The third and fourth questions on the survey had to do with learning and teaching. We were seeking understanding about what the students thought the end goal of the project was; to learn themselves, or to develop a system whereby others could learn what they learned, which implies that they have first learned the material themselves. We also wanted to know whether they perceived that the locus of control for learning was within themselves, or outside their control.
What we found was that students clearly understood the goal of the project; to teach other middle school students wetlands ecology. However, they were divided on the question of control. Most students ended up right in the middle of the bell curve, between "we taught the teachers" and "the teachers taught us". This also makes complete sense when you take into account that 1/2 of their 'teaching' experience during this project was provided by the 'traditional' science teacher. The question was couched around the virtual world building component of the project, but we think many students looked at the question from the perspective of the project overall.
Task Understanding and Difficulty
The questions regarding task understanding and ability were designed to develop an understanding about how much the students understood what they were to accomplish while in the virtual environment, as well as how well they were able to accomplish it. Being in a virtual space, especially for a novice, can be quite challenging. In the wetlands environments, students could rise and sink to any elevation they desired by 'flying vertically'. This is not always the case; movement can be constrained to a certain plane or level, but in our four environments, students could fly to any altitude they chose. Therefore, their normal mode of locomotion (walking) was temporarily supplanted by the sensation of 'flying'. This can be very disconcerting for some individuals, especially for adults. Children seem to be more adaptable to alternative forms of perceptual 'movement'.
Answers on the two questions regarding task understanding and difficulty of completing the task indicated that students understood what was expected of them in terms of performance within the virtual environment, but that some students had difficulty with navigation and manipulation of objects. This last point could have been potentially mitigated through additional acclimatization of students in a virtual space, but unfortunately our project time constraints did not allow for such training. Subsequent VRRV students had the opportunity to experience a virtual space prior to their construction process, and this aided both their understanding of a three dimensional environment as well as their ability to both navigate within the space, and manipulate objects as well.
Perceived Value and Overall Enjoyment
Two questions had to do with the students' perceived value of virtual reality as a learning tool, and one with the overall enjoyment factor of the project as a whole.
In asking the students' how they best thought their cycle could be conveyed to next year's 7th graders, whether through the means generally available to them in the constructivist classroom, or whether to incorporate virtual reality into the learning experience, most of the respondents answered that they would rather incorporate a virtual reality component into the learning process, rather than learn the material through traditional means.
When asked if they would like to visit a virtual environment again, almost all (92%) answered in the high end of the scale, indicating strong interest in such an endeavor.
Regarding the design/build process, most students indicated that they had enjoyed the process very much. In fact, in discussions with students during and after the project, almost everyone wanted to use the technology and the process for their very next project in class, as well as for a special district-wide science competition. To facilitate their development projects, we left the modeling software on the computers at Kellogg. Unfortunately, we were not in a position to provide the display technology as well, an issue addressed further in the discussion section.
It was very satisfying to see such a positive response with regard to both the design/build and experiential portions of the project. Clearly, this was a project that was perceived by the students to be of value, and one that they enjoyed as well. As an individual who subscribes to the notion that learning can indeed be fun, we were very pleased with these results.
Physical Discomfort
We asked two questions about potentially negative physical feelings that the students might have experienced while in the virtual space; one on nausea and one on dizziness. It is a well known fact that for a small percentage of the population, being in a virtual environment can cause vertigo, headaches, and nausea. (Prothero, et al., 1995). We were interested to see how many individuals experienced these negative physical feelings in this particular project.
The first question dealing with these issues asked the students how "sick to their stomach" they felt inside the virtual environment. Most (80%) of the respondents answered that they did not feel at all sick to their stomach. However, the remaining 20% of students experienced at least some sensations of nausea.
When asked how dizzy they felt in the virtual space, 74% of the respondents answered that they experienced little or no dizziness. 22% of the respondents answered that they had experienced at least some dizziness. These figures are consistent with the sensations of nausea described above. Though we were very careful to keep the students in a small area, where movement was minimized, it is clear that the technology does not provide a physically comfortable experience for, in this case, nearly a quarter of the subjects. Nonetheless, 92% of the respondents wanted to go back into a virtual environment, regardless of physical discomfort.
Presence
'Presence' is the perceptual and somatic sense of being in a particular place (Hoffman et al., 1996, Prothero, et al., 1995; Prothero & Hoffman, 1995; Barfield & Weghorst, 1993). Generating a sense of presence is one of the key features of virtual reality, and is facilitated primarily through encompassing the visual field in a relatively 'natural' manner, to preclude alternative perceptual input from confounding or confusing the experiencer.
The virtual environment, at least at this stage in the development of the field, is still 'cartoony'; most of the objects are not exact replicates of what we might expect. Organic material is particularly difficult to replicate, and the wetlands environment is rife with it. Yet, these environments were compelling enough to engage the students in such a manner as to temporarily 'suspend their disbelief', even in the noisy portable in which we conducted the experiential portion of the study.
Interestingly enough, it was in the 'presence' questions that we derived our most 'normal' distributions. All of the hype aside, subjects were able to articulate whether they really felt as if they were in their wetland environment or not. As all three of the questions' distributions are so evenly matched, we feel that this data in particular has value. The other issue is whether a sense of presence is required in an environment such as this. It can be argued both ways; on one hand that a sense of presence means that the individuals perceptions are more 'engaged', leading to a deeper experience. On the other hand, even if an individual doesn't perceive him or herself to be in a separate reality, it may make it easier to transfer what has been learned back out into what we normally consider 'reality' (Hoffman, Hullfish & Houston, 1995).
We found that most of the subjects did not feel that their virtual environment constituted a separate reality, or that they forgot the real world while they were in the virtual environment. However, most students did indeed perceive that they were in the virtual wetland; that they were in a "different place". This indicates a sense of presence of being in the wetland environment, which is different than asking if the wetland was real or if the real world 'went away', as discussed above. This could relate to Zeltzer's (Presence, 1992) distinction between interaction with an interface, which he calls 'presence' and interaction with content, which he calls 'logical interaction'.
What this means is that even though subjects may not have actually perceived the virtual environment to be real, they still got a sense of being in the virtual wetland. This could be because they had built one of the environments and were looking for the aspects of the environment that they had designed. It could also be due to familiarity with expected objects in the virtual wetland. All subjects in a particular group knew what was going to be 'in' their space, and so might have been more inclined to see the space as a wetland, even an imagined wetland as it was.
Nonetheless, this lack of a sense of 'reality' did not impede their learning or enjoyment. Later data from the VRRV project, however, indicates that enjoyment is often strongly related to a sense of presence. (Winn, 1995).
The last analysis performed was with regard to the value of world building versus the experiential portion of the process. Because we had measures for 21 students who went through the entire world-building experience but did not don the helmet and experience their creation, we were able to compare these students' test scores to those students who both took part in the world-building exercise, and got to experience two worlds as well.
By conducting a between-subjects ANOVA on the objective test scores, we found no significance between world-building and experiencing F(1, 81) = 1.54, p > .05, a significant pre-post effect, F(1, 81) = 78.14, p < .001, and no interaction effects.
In performing the same analysis on the concept map data, we found slight significance between students who got to experience virtual reality versus those who did not F(1, 65) = 3.22, p < .10, a significant pre-post effect F(1, 65) = 54.08, p < .001, and no interaction effects.
In a further within-subjects ANOVA comparing concept map data for built vs. chosen representations coupled with the virtual reality experience variable, we found no significant difference for main effect F(1, 65) = .13, p > .05, and no interaction effects F(1, 65) = .06, p > .05.
Based on these results, it appears that the primary cause for the substantial pre-post improvement in scores can be directly attributed to educational treatment, as opposed to experiencing the virtual environment. However, it was the experiential portion of the project that was highly motivating for most of the children. The reward of seeing what they had worked so hard to create was an end-goal that was clearly defined and achievable based on the students own hard work. The students who didn't get to experience their environments were very disappointed. Therefore, even though the experiential portion of the project did not affect the students cognitive measures, it was a critical affective component to the project.
Results from the quantitative measure indicated significant improvement between pre- and post-test measures for all groups, but it was unclear from these results which treatment(s) might have been more effective. Treatment analysis was performed on the concept map data.
Regarding concept map analysis, we found improvement between pre- and post-test measures, regardless of group. We also found significant differences between scores on the world that children built, versus the world that they chose to represent, and interactions between all of the above.
Based on our initial hypotheses, our findings yielded some unexpected results, in that the Constructivist approach, including world-building, did not prove to be significantly more educationally valuable than the Traditional approach. However, as discussed above, some of the measures, most notably the concept maps, could have been analyzed with a different set of criterion, which may have yielded much different results. We have seen that the students' representations became much more graphic and process oriented after their virtual environment creation process, indicating that world building had an effect on their understanding of both the Constructivist environment which they embodied as a virtual space, the Traditional environments that they studied, and even the cycles in which no instruction was provided.
In support of our initial hypotheses, we found the Constructivist approach was more educationally valuable than the No Instruction control, though results were not significant when comparing the Traditional and No Instruction results. Pre-test scores indicate that students had prior knowledge regarding at least one, if not two of the wetland cycles studied, and that there was also a lot of content cross-over between the wetland cycles selected for comparison. However, since this wetland ecology module required these specific cycles to be studied, some of this overlap was necessary.
The group that had the most significant effect on scores overall was the Carbon group. This was the one group in which the three learning-impaired students had been place. As a group, they experienced substantial gains in both their Constructivist and Traditional cycles. These results dovetail with findings discovered in subsequent analyses; that VR helps lower-functioning students more substantially than higher-functioning students (Winn, 1997). What we have not discovered is what components of the virtual experience are particularly valuable to this population.
Furthermore, we feel that another reason that there were no significant differences between the Constructivist and Traditional treatments is that our student population were already experienced in the Constructivist learning paradigm; that there was nothing new, aside from world-building, presented to them during the wetlands ecology module. Their day-to-day activities in the classroom were rich with collaborative, inquiry-based learning opportunities. Most of the students were already able researchers, who knew how to formulate an information search, and to collate and present information. It is my assumption that the anomaly for these students with regard to this project was to have had a period of 'traditional' science each day for the duration of the project. As constructivists, they took the information provided and put it into their own schemas just as they did in the constructivist classroom.
One positive finding that the quantitative data doesn't describe is how very much the students' language and presentation techniques changed and grew over the course of the project. Students began to speak in terms of their 'perspective', and 'rotating their view'. They seemed more willing to consider part-to-whole relationships in their other classes; a trend which was noticed by all four KCOT teachers. These fundamental language shifts became an inherent part of students vocabulary, and were applied across domains, from personal relationships, to the study of history and mathematics. This is an undocumented finding that warrants additional research.
The concept maps provided by the children show a clear movement towards the incorporation of visual metaphors in their post-tests, which can be attributed directly to their virtual environment construction process, as this was the only component of the project that included visual representations, yet it affected the manner in which students chose to represent information regardless of what instructional paradigm had provided the initial content.
We found that asking students to mentally and physically take objects apart and reconstruct them in drawings from multiple perspectives was infinitely valuable to their understanding of the relationship between 2-D and 3-D representations. We were surprised at how their world view changed based on this activity-- this is one of the most powerful aspects of virtual world design that we have encountered (Winn, 1995, 1997; Osberg, 1995b; 1997).
By visualizing objects from both the real and virtual environment in their mind's eye, and on paper, students begin to realize that part-to-whole relationship and its relevance to modeling, and on a grander scale as well (Samuels & Samuels, 1975, Richardson, 1980; Richardson, 1969, 1994; Kaufman, 1984). We were very cognizant of how students' language began to change; how they spoke of whole-to-part relationships rather than part-to-whole as a reflection of their 'new' perspective on life. We were also clear that there is an amazing amount of power and latitude when it comes to describing and developing a 'world'; the sense of responsibility for developing an environment, especially a learning environment granted students an autonomous authority over their learning process that most had not experienced prior to this project.
We didn't get nearly as deeply involved in the semiotic side of the project as we would have liked. Further research into the nature of the knowledge construction process, as described by Cunningham (1992). It would be especially valuable to relate that process to the creation of virtual objects, how they are developed, and how they come to have meaning, both as symbols and as directly accessible objects. Furthermore, research into developing meaningful virtual tools would be useful (Rose, 1996), as well as designing and testing new navigational paradigms. All of these opportunities involve the use of signs and metaphors to make meaning in a virtual space.
From a Constructivist perspective, additional research could be conducted on the general value of constructivist learning; learning for depth vs. depth (Brooks & Brooks, 1993), how and when to incorporate visual and verbal representations (Mones-Hattal & Mandes, 1996), whole body learning and experiential education (Kraft & Sakofs, 1989), meaningful self-directed learning (Poplin, 1991), and examining one's reasoning for developing certain knowledge constructions (Minstrell, 1989; 1992; Minstrell, Stimpson & Hunt, 1992).
It would be especially interesting to see how virtual reality can continue to be used to assist in the learning process. We feel we have just scratched the surface of this topic; we still do not have a strong understanding of when to use the world-building process and the technology; for what content areas, for what age groups, and under what circumstances.
The metaphysics of virtual reality are a fascinating subject unto itself (Heim, 1993; Osberg, 1993a, 1994a; Gigliotti, 1996). Certainly, as our philosophical basis for understanding our environment changes, so too will the nature of our perceptions. We find this particularly fascinating when one considers what might be learned from 'alternate realities' in which our perceptions, indeed our belief systems may be engaged in ways we can only contemplate at this time.
Regarding the incorporation of ethics into the creation and distribution of virtual environments, and the sociological effects of both the Internet and virtual reality (Prothero, 1996), we have much opportunity to begin to understand how this technology both draws people together (Reingold, 1993; McLuhan, 1964), but also separates them (Bowers, 1988, 1992).
We don't necessarily as of yet have a firm handle on what makes a virtual environment educationally useful and enjoyable, so from a perceptual perspective, we would like to see additional research conducted within the virtual environment itself, testing navigational paradigms, effective use of color and texture, spatial manipulations of scale and orientation, and the more prevalent and effective use of audial and haptic feedback.
On the practical side, the access and administrative aspects alone require in-depth analysis if the world-building process and virtual reality technology are to ever become an integral part of the learning process.
There is also the issue of multi-participant environments, which has not been well discussed. There is a great deal of opportunity in this area, since we do not, at this time, have a strong understanding of what the 'loading factor' of any given environment might be.
In conclusion, we feel the world-building process, coupled with the opportunity to experience one's virtual learning environment is a powerful, motivating way in which to learn, at least about wetlands ecology.
By incorporating the student's creativity and design skills, metacognitive skills, freedom to make their own design and navigational decisions, and their whole body into the learning process, we are providing students with a very wide avenue of opportunity for cognitive, somatic and affective growth and experience.
To be sure, there are still gaping holes in how the technology can best be employed, how to provide access to a wide number of students, how to develop an effective ethical system, and how to avoid the pitfalls of trying to promote the inappropriate use of any technology.
The field is rife with opportunity, both for research and development. Our best opportunity is to incorporate good human-computer interface design processes; to test and re-test our assumptions and how they affect our developments, and to encourage and support student involvement from idea generation to usability assessment.
The best designers that we work with have two things in common: they are all about 13 years of age, and they are absolutely fearless. By the time students get to high school, the cognitive atrification process has already begun, and their interests are often focused elsewhere. Younger adolescents seem much more willing to try out a variety of ideas and perspectives.
To this end, we propose the development of virtual educational programs that allow students, particularly middle school students, the opportunity to participate in and contribute to the development of a virtual educational network. As one female student from this project said: "I don't think we are ready for this technology; our teachers don't know enough about it yet." Shank (1992) would agree. But the students do.
Several students mentioned the difference between verbal presentation, and creating cause and effect relationships themselves. One male student said "I didn't understand the nitrogen cycle when the teacher explained it. I do now!!". A female student said "It's harder to learn this stuff out of a book; here I could go with it as it was happening. I was in control!" Another male student said "I knew absolutely nothing about this prior to building my world in VR. I used it to learn. I learned a process. It was fun, so I'll remember it more.", and another said "I understand it better now that I've experienced it."
All of the students wanted more of everything; more time,
more realism, more water, more mud, more animals and plants, more
behaviors, more sound, more applications, more environments.
And we intend to try and give it to them.
Apple Computer, Inc. (1994). The Imperative to Change Our Schools. Cupertino, CA: Apple Professional Development Center.
Arnold, J. (1991). Towards a middle level curriculum rich in meaning. Middle School Journal, November 1991, 8-12.
Baker, E.L., Herman, J.L., Gearhart, M. (1989). The ACOT Report Card: Effects on Complex Performance and Attitude. Presented at the annual meeting of the AERA, San Francisco, 1989.
Barfield, W., Weghorst, S. (1993). The sense of presence within virtual environments: A conceptual framework. In Salvendy, G., & Smith, M.J. (Eds.) Human-computer interaction: Software & Hardware Interfaces.
Belenky, M.F., Clinchy, B.M., Goldbreger, N.R., Tarule, J.M. (1986). Women's Ways of Knowing. Basic Books.
Bowers, C.A. (1988). The Cultural Dimensions of Educational Computing: Understanding the Non-neutrality of Technology. NYU: Teachers College Press.
Bowers, C.S. (1992). Ideology, educational computing and the moral poverty of the Information Age. In Against the Grain: Critical Essays on Education, Modernity, and the Recovery of the Ecological Imperative. New York: Teachers College Press.
Bricken, M. and Byrne, C. (1992). Summer Students in Virtual Reality: A Pilot Study on Educational Applications of Virtual Reality Technology. Seattle, WA: Human Interface Technology Laboratory at the University of Washington, Technical Publications R-92-1.
Brooks, J.G., Brooks, M.G., (1993). In Search of Understanding: The Case for the Constructivist Classroom, Alexandria, VA: ASCD.
Brooks, M.G., Brooks, J.G. (1996). Creating the Constructivist Classroom. Course materials from an ASCD workshop by the same title, New Orleans, LA, March 14-15, 1996.
Bruner, J.S. (1966). Towards a Theory of Instruction. New York: W.W. Norton & Company, Inc. and Harvard University Press.
Bruner, J.S. (1971). The Relevance of Education, Cambridge: Harvard University Press.
Bruner, J. S. (1973). Beyond the Information Given; Studies in the Psychology of Knowing. New York: W.W. Norton & Company.
Bruner, J.S. (1990). Acts of Meaning, Cambridge: Harvard University Press.
Byrne, C.M. (1993). Virtual Reality and Education, Seattle, WA: Human Interface Technology Laboratory at the University of Washington, Technical Report R-93-6.
Byrne, C. M. (1996). Water on Tap: The Use of Virtual Reality as an Educational Tool. Unpublished Ph.D. Dissertation, University of Washington, College of Engineering.
Cunningham, D. (1992). Beyond educational psychology: steps towards an educational semiotic. Educational Psychology Review, 4(2), 165-194.
Dede, C. (1990). The evolution of distance learning: Technology mediated interactive learning. Journal of Research on Computing in Education, 22(3), 247-264.
Dede, C. (1992). The future of multimedia: Bridging to virtual worlds, Educational Technology, 32(5), 54-60.
Dede, C. (1994). Evolving from multimedia to virtual reality. Educational Multimedia and Hypermedia, 1994. Proceedings of ED-MEDIA 93: World Conference on Educational Multimedia and Hypermedia. Association for the Advancement of Computing in Education.
Dede, C., Salzman, M., Loftin, R.B. (1996). The development of a virtual world for learning Newtonian mechanics. Originally published in the Proceedings of the Multimedia, Hypermedia, and Virtual Reality conference, MHVR '94. Berlin: Springer-Verlag.
Duffy, T. & Jonassen, R. (Eds.) (1992), Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Lawrence Erlbaum.
Dwyer, D. (1994). Apple classrooms of tomorrow: what we've learned. Educational Leadership, April 1994, 4-10.
Dychtwald, K. (1977). Bodymind. NY: Putnam's Sons.
Farmer, S. (1987). Visual literacy and the clinical supervisor. Clinical Supervisor 5, 41-71.
Gibson, J.J. (1986). The Ecological Approach to Visual Perception. Lawrence Erlbaum, Hillsdale, NJ.
Gigliotti, C. (1996). Aesthetics of a Virtual World, Leonardo.
Hampson. P.J., Morris, P.E. (1990). Imagery, consciousness, and cogntive control: The BOSS system revisited. In P.J. Hampson, D.F. Marks, J.T.E. Richardson (Eds.) Imagery: Current Developments, 1-38. NY: Routledge.
Heim, M. (1993). The Metaphysics of Virtual Reality. New York: Oxford University Press.
Hoffman, H.G., Hullfish, K. and Houston, S.J. (1995). Virtual Reality Monitoring. VRAIS '95 March 11-14, 1995. Chapel Hill, NC. Los Alamitos, CA: IEEE Computer Society Press.
Hoffman, H., Prothero, J., Wells, M. and Groen, J. (1996). Virtual Chess: Meaning Enhances the Sense of Presence in Virtual Environments. HIT Lab Technical Report P-96-3.
Kaufmann, G. (1984). Mental imagery and problem solving. In A. Sheikh (Ed.) International Review of Mental Imagery, volume 1. NY: Human Sciences Press.
Kellogg Middle School KCOT Students. (1993). The Seattlelite: A Youth's Guide to the North Seattle/Shoreline Area. Seattle, WA: Kellogg Middle School.
Kellogg Middle School. (1996). Program Options Brochure. Seattle, WA: Kellogg Middle School.
Kirby, J. R., Moore, P. J. & Schofield, N. J. (1988). Verbal and visual learning styles. Contemporary Educational Psychology, 13, 169-184.
Kraft, R. & Sakofs, M. (Eds.) (1989). The Theory of Experiential Education. Boulder, CO: Association for Experiential Education.
Lakoff, G. (1987). Women, Fire, and Dangerous Things. Chicago: University of Chicago Press.
Laurel, B. (1991). Computers as Theatre. Reading, MA: Addison-Wesley.
Loftin, R.B., Engelberg, M., & Benedetti, R. (1993). Applying virtual reality in education: A prototypical virtual physics laboratory. IEEE (0-8186-4910-0).
Loftin, R.B., Kenney, P.J. (1995). Training the Hubble space telescope flight team. IEEE Computer Graphics and Applications, 15(5), 31-38.
McLellan, H. (1996). Virtual realities. In D. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology. New York: Macmillan, 457-490.
McLuhan, M. (1964). Understanding Media: the Extension of Man. NY: McGraw Hill.
Merickel, M. (1992). The Creative Technologies Project: A study of the relationship between virtual reality (perceived realism) and the ability of children to create, manipulate, and utilize mental images for spatially related problem solving. (Eric Document ED 352 942).
Minstrell, J. (1989). Teaching science for understanding. In Resnick, L. & Klopfer, L. (Eds.) Toward the Thinking Curriculum: Current Cognitive Research. Alexandria, VA: Association for Supervision and Curriculum Development.
Minstrell, J. (1992). Facets of student knowledge and relevant instruction. In Duit, R., Goldberg, F., & Niedderer, H. (Eds.) Research in Physics Learning: Theoretical Issues and Empirical Studies. Kiel, Germany: University of Kiel.
Minstrell, J. Stimpson, V. & Hunt, E. (1992). Instructional design tools to assist teachers in addressing students' understanding and reasoning. Paper presented at the annual meeting of the AERA, San Francisco, April, 1992.
Mones-Hattal, B., Mandes, E. (1995). Enhancing visual thinking and learning with computer graphics and virtual environment design, Computers & Graphics, 19(6), 889-894.
Morris, P.E., and Hampson, P.J. (1983). Imagery and Conciousness. Academic Press: NY.
Osberg, K.M. (1993a). Virtual Reality and Education: A Look at Both Sides of the Sword. HIT Lab Technical Report R-93-7. Seattle: Human Interface Technologies Laboratory.
Osberg, K.M. (1993b). Virtual Reality and Spatial Cognition Enhancement: A Pilot Study. Seattle, WA: Human Interface Technology Laboratory at the University of Washington.
Osberg, K.M. (1994a). Rethinking Educational Technology: A Postmodern View. Human Interface Technology Lab Technical Report R-94-4.
Osberg, K.M. (1994b). Distance Learning and Virtual Reality: A Collaborative Learning Opportunity, Proceedings of the Computer Managed Learning conference, December, 1994, Melbourne, Australia.
Osberg, K.M. (1995a). The Teacher's Guide to Developing Virtual Environments. Human Interface Technology Laboratory Special Publication, VRRV Project Support.
Osberg, K.M. (1995b). The VRRV Report. VR in the Schools: University of North Carolina at Chapel Hill College of Education.
Osberg, K.M. (1996). But What's Behind Door Number 4? Virtual Reality and Ethics: A Discussion. Proceedings of the First International Conference on Virtual Reality and Ethics, University of Michigan, Ann Arbor, MI, October 4 - 6, 1996.
Osberg, K.M. (1997). Constructivism in Practice: Meaning-Making in the Virtual Environment. University of Washington, College of Education, Doctoral Disseration.
Pascual-Leone, J. (1979). Intelligence and experience: A Neo-Piagetian approach. Instructional Science, 8, 301-67.
Pascual-Leone, J. (1980). Constructive problems for constructive theories: the current relevance of Piaget's work and a critique of information processing simulation psychology. In R.H. Kluwe and H. Spada (eds.) Developmental Models of Thinking (263-96). NY: Academic Press.
Percy, W. (1954). The Message in the Bottle. New York: The Noonday Press.
Poplin, M. (1991). Constructing Meaning: Visions and Views. Schenectedy, NY: New York State English Council.
Prothero, J., Parker, D., Furness, T. and Wells, M. (1995). Towards a Robust, Quantitative Measure for Presence. HIT Lab Technical Report P-95-8.
Prothero, J.D. and Hoffman, H.G. (1995). Widening the Field-of-View Increases the Sense of Presence in Immersive Virtual Environments. HIT Lab Technical Report R-95-5.
Psotka, J. (1995). Immersive tutoring systems: Virtual reality and education and training. [Available as an HTML document from http://www.hitl.washington.edu].
Pylyshyn, Z. (1973). What the mind's eye tells the mind's brain. Psychological Bulletin, 80 (1), 1-24.
Pylyshyn, Z. (1981). The imagery debate: analogue media versus tacit knowledge. Psychological Review, 88 (1), 16-45.
Pylyshyn, Z., and Demopoulus, W. (1986). Meaning and Cognitive Structure. Ablex: NJ.
Rheingold, H. (1995). Virtual Communities. New York: Simon & Schuster.
Richardson, A. (1969). Mental Imagery. NY: Springer Publishing.
Richardson, A. (1994). Individual Differences in Imaging: Their Measurement, Origins, and Consequences. NY: Springer Publishing.
Richardson, J.T.E. (1980). Mental Imagery and Human Memory. London: McMillan.
Rose, H.A. (1996). Design and Construction of a Virtual Environment for Japanese Language Instruction, University of Washington, Seattle, WA; Howard A. Rose.
Samuels, M., & Samuels, N. (1975). Seeing With the Mind's Eye. New York: Random House.
Shank, G. (1992). Educational semiotic: threat or menace?. Educational Psychology Review, 4 (2), 195-221.
Unger, C. (1994). What teaching for understanding looks like. Educational Leadership, February, 1994, 8-10.
Winn, W. (1992). The assumptions of constructivism and instructional design. In Duffy & Jonassen (Eds.), Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Lawrence Erlbaum.
Winn, W. (1993). A Conceptual Basis for Educational Applications of Virtual Reality. HIT Lab Technical Report R-93-9. Seattle: Human Interface Technologies Laboratory.
Winn, W. (1994). It's virtually educational, Information Week, March 28, 1994, p36.
Winn, W. (1995). Virtual Reality Roving Van Project. T.H.E. Journal, 5(12).
Winn, W. (1997). Global Warming: It's a Gas! A Collaborative Virtual Environment for Studying the Effects of Global Warming. To be presented at the 1997 AERA Conference, March 24-28, Chicago, IL.
Winn, W., Bricken, W. (1992). Designing virtual worlds for use in mathematics education: the example of experiential algebra, Educational Technology, 32(12), 12-19.
Winn, W., Hoffman, H., Osberg, K. (1995). Semiotics and the Design of Objects, Actions, and Interactions in Virtual Environments. Presented at the Symposium "Semiotics and cognition: Issues in the symbolic design of learning environments", AERA, San Francisco, CA, April 1995.
Wiske, M.S. (1994). How teaching for understanding changes the rules in the classroom. Educational Leadership, February, 1994, 19-21.
Wittrock, M.C. (1987). Teaching and student thinking, The Journal of Teacher Education, November-December 1987, 30-33.
Wittrock, M.C. (1991). Generative teaching of comprehension, The Elementary School Journal, 92(2), 169-184.
Wittrock, M.C., Alesandrini, K. (1990). Generation of summaries and analogies and analytic and holistic abilities. American Educational Research Journal, 27(3), 489-502.
Yocam, K., Wilmore, F., Dwyer, D. (1992). Situated Teacher Development: ACOT's Two-Year Pilot Project. Paper presented at the annual meeting of the Society for Technology and Teacher Education, Houston, TX, 1992.
Zeltzer. 1992. Presence.
Footnote.
1. This project was funded with a grant from the US WEST Foundation to the Human Interface Technology Laboratory at the University of Washington. We gratefully acknowledge the support of US WEST while taking full responsibility ourselves for all opinions and any errors that occur in this paper.