![[Previous Chapter]](/icons/button-back.gif){kind=icon}
![[Table of Contents]](/icons/button-contents.gif){kind=icon}
![[Next Chapter]](/icons/button-next.gif){kind=icon}
Statistical analysis was conducted on all of the measures collected: objective pre- and post-tests scores, concept map pre- and post-test scores for both the world that they built and a chosen world that they wished to represent, interview data for all students who experienced VR and survey information. ANOVA tables relating to these results can be found in Appendix F: ANOVA Tables.
Regarding the objective tests, an ANOVA with test occurrences as a within-subjects factor of pre- (M = 8.87) and post-test (M = 12.35) scores revealed a significant improvement in scores overall, F(1, 79) = 97.58, p < .001. There was no significant main effect based on the world (T1) that the students built, F(3, 79) = 1.78, p > .05, and no interaction effects. These results are illustrated in Table 18.
It was unclear whether instructional paradigm alone had an effect on the children’s understanding of particular cycles based on these findings. However, treatment effects were found for Concept Map data.
The Concept Maps provided the richest data from all of the measures taken. Rated using holistic scoring techniques, found in Appendix C, Concept Map Scoring Criteria, each cycle described was carefully examined to determine if the student had provided information in a manner fitting the criteria. Raters were blind to treatment. Scores for concept maps ranged from 0 - 4. Interrator reliability for concept map analysis, using the Wilcoxon Matched Pairs test was significant (Z = -3.5279; p < .001).
Sixty-seven subjects drew four maps: two "built" (before and after the world they built and experienced (T1)), and two "chosen", i.e. a self-selected representation of one of the three other cycles being studied (before and after the treatment had been administered (either T2 or T3)). Not surprisingly, many of the children selected water as the cycle to represent for their "chosen" drawing; the cycle best known to them prior to their experience in this project. Mean scores by group by pre- and post-test for both built and chosen maps is presented in Table 11, below.
Mean Scores
Built vs. Chosen Pre-test Post-test
by Group Built Chosen Built Chosen
Carbon (n = 14) 1.57(1.45) .93(1.14) 2.57(1.40) 2.64(1.22)
Energy (n = 17) 1.00(0.50) 2.53(1.46) 2.71(0.92) 2.71(1.40)
Nitrogen (n = 14) .93(0.47) 3.00(1.53) 2.86(0.95) 3.71(1.50)
Water (n = 23) 2.26(1.18) 1.48(1.08) 3.30(0.70) 2.22(1.28)
Grand Means (n = 68) 1.53(1.14) 1.93(1.48) 2.91(1.00) 2.60(1.35)
Note: Standard deviations in parentheses.
Table 11 - Concept Map Means for Pre- and Post-tests, and Built vs. Chosen Environments
Concept Maps were analyzed by ANOVA in two ways. The first analysis consisted of two within-subjects factors, pre- and post-test measures, and built vs. chosen measures. Group was used as a between-subjects factor. A second analysis was conducted on instructional treatment (Constructivist, Traditional, and No Instruction), as a within-subjects factor for Constructivist vs. Traditional, and Constructivist vs. No Instruction, and as a between-subjects factor for Traditional vs. No Instruction.
The first analysis, a within-subjects ANOVA comparing pre- and post-test scores, by built vs. chosen world yielded no main effect based on the world that the children drew, F(3, 63) = 1.02, p > .05. These results are illustrated in Table 19. However, there was a significant pre-post effect, F(1, 63) = 71.75, (p < .001) and no interaction effects F(3, 63) = .71, (p > .05). These results are illustrated in Table 20.
This pre-post effect is consistent for all concept map measures. It is clear that the students’ cognitive gains in procedural and relational knowledge improved. What is also clear is that the wetlands project resulted in significant comprehension and understanding of the subject matter, regardless of group.
Further concept map analysis indicates no significant main built effect, F(1, 63) = 2.04, p > .05, and a significant interaction effect between built vs. chosen and group, F(3, 63) = 15.56, p < .001. These results are presented in Table 21. Further analysis yeilded a significant interaction F(1, 63) = 10.47, (p < .005) between pre-post and built-chosen, and a significant interaction effect F(3, 63) = 6.10, (p < .01) between pre-post, built-chosen and group. These results are presented in Table 22.
These findings indicate that not only were the pre- and post-test scores significantly different, but the scores varied based on whether the children had built the world the represented in their drawings, or whether it was the world that they had chosen to represent. Furthermore, the additional interaction between group, pre-post and built-chosen indicates that the group the children were in also had a significant effect on this interaction.
As can be seen in Figure 4, below, the Carbon group experienced the most significant gains in both built and chosen worlds. In all other groups, pre-post differences were much more substantial for the world that they built, rather than the world that they chose to represent. It is interesting to note that the Carbon group contained the three intellectually challenged children in the KCOT program. In previous (Osberg, 1993b) and subsequent (Winn, 1997) research, it has been found that virtual environment design tends to help learning impaired children even more substantially than non-learning impaired students. These findings could be indicative of this trend.
Figure 4 - Concept Map Pre- and Post-test Built vs. Chosen cycles by Group
What can be seen in comparing group information is that there are gains for all groups in all instances, and for three of the four groups, gains are much less substantive for the chosen concept map representations. This is what had been anticipated based on the assumption that most subjects, given their choice, would draw a cycle known to them, namely water. However, for the Carbon group, the gains for chosen world representations were even more substantive than for their built environment. This inconsistency may be attributable to the less robust knowledge base of some of the Carbon group members.
Built world concept map gains were consistent for all groups, resulting in at least a +1.0 rise in scores on a 4-point scale.
Another interesting way to look at this same data set is to alter the graphics to show the built vs. chosen scores for each group. An illustration of this analysis is presented in Figure 5, below.
Figure 5 - Concept Map Pre- and Post-test scores by Group by Built vs. Chosen cycle
What can be seen from this graph is that for both built and chosen concept map representations for all groups, the post-test scores are consistently higher than pre-test scores, and that the Water group did the best on the pre-test, again because this cycle was already known.
Treatment comparisons yielded the most interesting results of all of the analyses conducted. In comparing concept map scores based on treatment (Constructivist, Traditional and No Instruction), within-subjects ANOVAs were conducted to compare Constructivist vs. Traditional scores, and Constructivist vs. No Instruction scores, and a between-subjects ANOVA comparing Traditional vs. No Instruction scores.
To illustrate the comparisons more fully, a means table is presented below which will be referenced throughout this section. In Table 12, means for each treatment group for both built and chosen cycle representations, and pre- and post-test scores are presented.
Mean Scores
for Pre- and Post-tests Treatment
by Treatment Constructivist Traditional No Instruction
Constructivist vs. Traditional (n = 43)
Pre-test 1.67(1.27) 1.74(1.56)
Post-test 2.79(1.04) 2.67(1.34)
Constructivist vs. No Instruction (n = 24)
Pre-test 1.28(0.84) 2.25(1.29)
Post-test 3.12(0.93) 2.46(1.38)
Traditional (n = 43) vs. No Instruction (n = 24)
Pre-test 1.74(1.56) 2.25(1.29)
Post-test 2.67(1.34) 2.46(1.38)
Note: Standard deviations in parentheses.
Table 12 - Concept Map Means for Pre- and Post-tests by Treatment
4.1.2.1. Constructivist vs. Traditional Treatment Analysis
For the Constructivist vs. Traditional comparison (n = 43), using a within-subjects ANOVA, a significant F(1, 42) = 58.23, (p < .001) pre-post effect was found. These results are presented in Table 23. Further analysis yeilded no significant effect for the cycle illustrated under either instructional paradigm F(1, 42) = .01, (p > .05), as shown in Table 24, and no interaction effects, F(1, 42) = .41, (p > .05), as shown in Table 25.
This finding indicates that instructional paradigm did not significantly affect the children’s ability to represent a wetlands cycle. Both the Constructivist and Traditional learning paradigms resulted in significant gains. Though the means in each case varied, as can be seen in Table 10, above, they did not differ significantly, which refutes my original hypothesis that the Constructivist learning paradigm would provide more substantive results than Traditional education, at least using the assessment criterion that we established.
However, using the actual drawings in (Ann Graham, Carbon) Figures 6 and 7 as a comparison, there are other conclusions that can be drawn. In this example, the concept maps illustrated under both the Constructivist and the Traditional treatment are visually richer, more complex and to a degree more accurate after the virtual world building experience than that created during the pre-test. This comparison is consistent in reviewing the subjects’ concept maps, regardless of group or treatment. Both are technically correct, yet there is additional value in the creative representation of information, especially in the transformation or translation of information from one symbol system to another. This transformation process is one means to assess the development of students visual literacy (Mones-Hattal & Mandes, 1995; Kirby, Moore & Schofield, 1988), which is defined by Farmer (1987) as "the abilities to read and interpret visually and to express oneself honestly and accurately by translating visual symbols into verbal language and vice versa".
Pre-test Representation
Post-test Representation
Figure 6 - Constructivist Treatment Pre- and Post-test Concept Map comparison
Pre-test Representation
Post-test Representation
Figure 7 - Traditional Treatment Pre- and Post-test Concept Map comparison
In this study, almost all source information regarding wetlands ecology was primarily text based, with the addition of limited videodisk, CD-ROM or Internet-based images and sounds. The translation process was embodied in the subjects’ ability to transform this text-based source information into an interactive, visual representation by creating a virtual environment. The comparison between the pre and post-test representations is clear; the effects of translating that information into a visual representation provides an additional element to the students knowledge base above and beyond technical accuracy.
This richness could be described, in Gibsonian (1986) terms, as arising from the "cognitive process that includes mental rehearsal, introspection, and visualization, and distinguishes itself as a thought process different from verbal thinking whereby each exposure to the visual image permits the observer to become a keener interpreter of the visual display, i.e. to see more and more element within the display over time".
This expanded perspective may be the result of the constructivist learning experience reified in world design. Students used the signs they created to experiment with interactions, leading to deeper understanding of the meaning behind the signs. Cunningham’s (1992) abductive reasoning model was used during the design process, but was particularly applicable during the experiential portion of the project. Children, through the development of hypotheses and the process of experimentation developed an understanding of the salient characteristics and components used in each cycle. This visual, experiential understanding allowed students to utilize the visual component in addition to the textual description to clarify and elucidate the carbon cycle in their concept map post-tests. This process was consistent for all groups.
These post-test concept map representations relate also to Morris & Hampson’s (1983) image taxonomy in that what this student has chosen to represent has become "real" for her, even though the representations may not be directly analogous to the natural world. In other words, when she thinks about these cycles, it is in this form. It also relates to Mones-Hattal & Mandes (1996) perception that virtual reality involves visual thinking, which deeply affects our perceptions, and our memory.
4.1.2.2. Constructivist vs. No Instruction Treatment Analysis
For the Constructivist vs. No Instruction treatment comparison, a significant pre-post effect was found, F(1, 23) = 18.40, (p < .001). These results are presented in Table 26. Further analysis yeilded no significant built-chosen treatment effect, F(1, 23) = .45, (p > .05), as presented in Table 27, and a significant pre-post, built-chosen treatment interaction, F(1, 23) = 18.25, (p < .001), as presented in Table 28.
In conducting paired t-tests to analyze mean scores for pre-post/built-chosen worlds, the pre- and post-test means for the Constructivist treatment were significant (p < .001), but the No Instruction treatment means did not vary significantly.
This finding indicates that concept maps improved significantly for worlds that the children built themselves, but not for worlds that were drawn by choice in which children had received no instruction. This supports a portion of my original hypothesis that constructivist learning is certainly more valuable than no instruction whatsoever. Even so, the children’s representations were much more pictorial after the world building experience, as was true for the comparison between Constructivist and Traditional treatments (Isaac Ralston, Nitrogen), as illustrated in Figures 8 and 9, below.
Pre-test Representation
Post-test Representation
Figure 8 - Constructivist Treatment Pre- and Post-test Concept Map comparison
Pre-test Representation
Post-test Representation
Figure 9 - No Instruction Pre- and Post-test Concept Map comparison
4.1.2.3. Traditional vs. No Instruction Treatment Analysis
For the Traditional vs. No Instruction comparison, both a within-subjects analysis for pre-post measures, and a between-subjects analysis comparing the individuals in the Traditional vs. the individuals in the No Instruction treatment group were conducted. In doing so, no significant score differences were found between the two treatments, Traditional vs. No Instruction, F(1, 65) = .06, p > .05, as presented in Table 30. Further analysis yeilded a significant pre-post effect, F(1, 65) = 5.35, p < .05 and a significant pre-post, treatment interaction effect, F(1, 65) = 4.00, p = .05, as presented in Table 31.
In conducting paired t-tests to analyze mean scores for pre-post/treatment interaction, the difference between Traditional pre- and post-scores was significant (p < .001), but the No Instruction pre- and post-test scores did not vary significantly.
This implies that though Traditional education provided the students opportunity to improve their knowledge on that particular wetland cycle, the No Instruction treatment does not does not. The interaction effect must be attributed to the differences in instructional treatment.
Based on the findings above, both the Constructivist and Traditional educational approach were both educationally valuable in that pre- and post-test scores for both treatments improved significantly. Further analysis indicated that the Constructivist approach is more educationally valuable than No Instruction, but that the comparison between the Traditional and No Instruction treatments did not yield significant differences between the two treatments. However, pre- and post-test scores were significantly different for the Traditional approach, indicating educational improvement, but not for the No Instruction treatment.
These analyses were based on subjects’ built vs. chosen cycle representations. Given that the students chose to represent was most often water, regardless of whether the treatment for that cycle was Traditional or No Instruction, it is not surprising to see no significant treatment difference between Traditional and No Instruction.
What was surprising was to see no significant treatment difference between Constructivist and Traditional treatments. In the end, it becomes a matter of interpretation, and of desired outcomes. If we choose to foster and value creativity and alternative forms of knowledge representation in our educational settings, such as was illustrated in the concept map comparison in section 4.1.2.1 and 4.1.2.2, then the Constructivist approach is one way to facilitate such knowledge acquisition and application. If instead we choose to focus on the ‘technically correct’ version of knowledge recall and application, without giving additional thought to the inventiveness or cognitive value in translating that information, both the Constructivist and Traditional approaches are equally educationally valuable.
The interviews were conducted just after the students had completed their virtual experiences, both within the world that they built, and the world that they visited that had been constructed by another student group. Interviewers asked students to recall the cycle just experienced, as represented in the virtual environment.
Many of the students, in addition to using words to describe their experiences, moved their bodies in the same way that they had while in the virtual environment. This indicates a somatic memory that is not described in the text-based data, but is well worth mentioning (Kraft & Sakofs, 1989).
Interviews were rated on a scale of 0-5 using a similar set of criteria as used to rate the concept maps. The information gathered during these interviews was intended to test whether the students remembered the steps to each respective cycle, as well as the key components required to complete each cycle. The evaluator also kept track of how often the student needed to be prompted, whether steps were remembered in order or whether the remembrances were somewhat scattered, and other comments that the students had about their experiences. The interviews were also video-taped for review purposes. Rating criteria are described in Table 13, below.
CARBON CYCLE
ENERGY CYCLE
NITROGEN CYCLE
WATER CYCLE
The interview data, analyzed using within-subjects ANOVA, indicate a significant main effect F(3, 65) = 2.79, (p < .05) based on the world that the subjects built, as presented in Table 33. This indicates that children’s ability to accurately describe their experiences, regardless of whether the world was the one that they themselves built, or was built by another student group depended at least partially on the group in which they were in.
There was also a significant F(1, 65) = 14.68, (p < .001) built-experienced main effect, and a significant interaction effect, F(3, 65) = 4.37, (p < .01) between built vs. experienced and group, as presented in Table 34.
In analyzing the mean scores for each group, the differences based on group illustrated in Table 14, below, appear to be dictated by low Carbon Interview scores for their experienced (not built) world. The three learning impaired students in the KCOT program had all been placed in the Carbon group, the group that experienced Nitrogen world (the most difficult cycle to understand). All of the other groups experienced the three easier worlds, carbon, energy and water, which is at least a part of the reason that they were able to accurately recount and describe their experienced cycles.
Mean Scores
Group Built Experienced Exp. World
Total Sample (n = 69) 4.06(1.07) 3.57(1.31)
Carbon (n = 18 ) 3.94(1.16) 2.78(1.48) Nitrogen(T;18)
Energy (n = 19) 4.26(1.28) 4.37(.895) Water(N;4)/Nitro(T;15)
Nitrogen (n = 19) 4.00(.882) 3.53(1.31) Energy(N;8)/Carbon(T;11)
Water (n = 13) 4.00(.913) 3.54(1.05) Energy(N;13)
Note: Standard deviations in parentheses. Treatments: T=Traditional, N+ No
Instruction. Number of subjects per treatment follows semicolon.
In fact, the group with the highest mean Interview scores for both built and experienced environments was the Energy group, who had contact with what the teachers felt were the two easiest cycles, energy and water. The Nitrogen and Water groups were both very close in scores for both built and experienced environments.
The survey was analyzed using frequency distributions. The survey itself can be found in Appendix D.
Questions were asked about both the process of developing the virtual environment, and about the experience of being in the virtual space to ascertain which portion(s) of the project subjects deemed to be most valuable or enjoyable, and whether they would consider undertaking such a project in their educational environments in the future.
Results indicate that the students very much enjoyed the "Virtual Wetlands", as we billed the project. They liked both building and visiting their environments, and wanted to incorporate the use of virtual technology into the curriculum for students who would be studying this subject in the future. Almost all of the students wanted to experience virtual reality again.
Using the combination of a 7-point Likert scale and two essay questions at the end of the survey, we found that often students tended to polarize towards either the top or bottom of the scale, with very few questions having a normal distribution. The fourteen questions on the survey are described below by the kind of issue addressed by the question; Process, Learning and Teaching, Task Understanding and Difficulty, Perceived Value and Overall Enjoyment, Physical Discomfort, and Presence. An Essay Question analysis follows.
4.1.4.1. Process Analysis
Regarding the building process, we first asked how each individual had faired with their partner, and their group. Frequency distributions for each of these questions is provided in Figure 10, below.
Figure 10 - Frequency data regarding pair and group participation
When asked whether they had worked together as partners during the project, 87% of the respondents answered from 5 - 7, indicating that they and their partner(s) worked together, rather than working individually. When asked if they interacted with others in their group, 72% of the respondents answered from 5 - 7, suggesting that they interacted often or frequently with their other group members. This indicates that the study facilitated collaborative learning, both in pairs and in the larger group context. This is one of the foundations of the constructivist paradigm; to mentor and support each others’ learning processes.
The third and fourth questions on the survey had to do with learning and teaching to discover what students thought the end goal of the project was; to learn themselves, or to develop a system whereby others could learn what they learned, which implies that they have first learned the material themselves.
Figure 11 - Frequency distributions regarding learning and teaching perceptions
The students were asked what the purpose of the project was; to teach themselves, or to teach other students. Figure 11, above, illustrates that 63.6% of the respondents answered between 5 - 7, indicating that the project was designed to provide them an opportunity to develop and environment to be used to teach other students. This was the way that the design process was introduced to the students, though it implies that they too are learning the material prior to sharing it with others. 10% of the students responded with a 1, indicating that the project was designed solely to teach the wetland cycles to themselves. Slightly over 20% of the students responded with a 4, indicating that both cases were true.
Figure 12 - Frequency distribution regarding teaching locus of control
Figure 12, above, illustrates the students’ response to their perceptions on who was doing the teaching; whether they themselves were in a position to ‘teach the teachers’, or whether the teachers continued to ‘teach them’. This question was poorly worded, especially since the bulk of the learning that took place in the context of world-building was under the Constructivist treatment, where all the learning in the classroom was self-directed. Nonetheless, a substantial (54%) portion of the students answered either 5, 6, or 7, at the top end of the scale indicating that the ‘teachers continued to teach us’. This response is a bit baffling. Perhaps the students’ assumed that the Traditional treatment was representative of teacher presence and control, which did continue throughout the course of the project, or that they thought of HITL representatives as ‘teachers’.
The questions regarding task understanding and ability to complete the task were designed to develop an understanding about how much the students understood what they were to accomplish while in the virtual environment, as well as how well they were able to accomplish it. Being in a virtual space, especially for a novice, can be quite challenging. In the wetlands environments, students could rise and sink to any elevation they desired by ‘flying vertically’. This is not always the case; movement can be constrained to a certain plane or level, but in our four environments, students could fly to any altitude they chose. Therefore, their normal mode of locomotion (walking) was temporarily supplanted by the sensation of ‘flying’. This can be very disconcerting for some individuals, especially for adults. Children seem to be more adaptable to alternative forms of perceptual ‘movement’.
Figure 13 - Frequency distribution regarding task understanding
The frequency distribution for the first question, regarding understanding of the task (which was to interactively complete the cycle portrayed in the environment), is illustrated in Figure 13, above. 86.5% of the students answered at the high (5 -7) end of the scale, indicating that they understood the task at hand. The remaining 13.5%, who answered between 1 - 4, were not as well informed about the task going into the virtual environment.
Figure 14 - Frequency distribution regarding task difficulty
When asked about the difficulty of completing the task, as illustrated in Figure 14, above, 73.7% of the students answered between 1 - 3, indicating relatively little difficulty in completing the task. However, the remaining 26.3% answered between 4 - 7, indicating moderate to severe difficulty in completing the task.
What these figures indicate is the need for greater opportunity to acclimatize to the virtual environment. Had there been more time, a ‘training round’ in another virtual space would have been advantageous, which was a practice we incorporated later into the project. Kellogg Middle School, as the pilot program, suffered in this regard. Children were perhaps not as well prepared for the navigational and interactive components of a virtual space as they could have been with a practice round under their belts.
Two questions had to do with the students’ perceived value of virtual reality as a learning tool, and one with the overall enjoyment factor of the project as a whole.
Figure 15 - Frequency distribution regarding next year's students wetlands curriculum
In asking the students’ how they best thought their cycle could be conveyed to next year’s 7th graders, whether through the means generally available to them in the constructivist classroom, or whether to incorporate virtual reality into the learning experience, 92% of the respondents answered between 5 - 7, indicating they would rather incorporate a virtual reality component into the learning process. These data are illustrated in Figure 15, above.
Figure 16 - Frequency distribution regarding desire for another virtual reality experience
When asked if they would like to visit a virtual environment again, as illustrated in Figure 16, above, 92% again answered 5, 6 or a very resounding 7, indicating strong interest in such an endeavor.
Figure 17 - Frequency distribution regarding enjoyment of the design/build process
In Figure 17, above, the frequency distribution for the question regarding enjoyment in designing and building the virtual environment is illustrated. 87.8% of the respondents answered between 5 - 7, indicating that they enjoyed the process, 53% of which enjoyed the process very much. Those that answered between 1 - 3 comprised 4% of the respondents. The remaining 8.2% answered 4, indicating a neutral position on their enjoyment level.
In discussions with students during and after the project, almost everyone wanted to use the technology and the process for their very next project in class, as well as for a special district-wide science competition. In fact, we left the modeling software on the computers at Kellogg, and at all other environments in which we engaged in a world-building process for this very purpose. Unfortunately, we were not in a position to provide the display technology as well, an issue addressed further in the discussion section.
It was very satisfying to see such positive values with regard to both the design/build and experiential portions of the project. Clearly, this was a project that was perceived by the students to be of value, and one that they enjoyed as well.
Two questions were asked about potentially negative physical feelings that the students might have experienced while in the virtual space; one on nausea and one on dizziness. It is a well known fact that for a small percentage of the population, being in a virtual environment can cause vertigo, headaches, and nausea. (Prothero, et al., 1995). It was interesting to see how many individuals experienced these negative physical feelings in this particular project.
The first question dealing with these issues asked the students how "sick to their stomach" they felt inside the virtual environment. As illustrated in Figure 18, above, 80% of the respondents answered between 1 - 3, indicating that they did not feel at all sick to their stomach. At the high end of the spectrum (5 - 7), 15% indicated that they did indeed feel nauseous during their virtual experience. 5% of the respondents answered a 4, which might mean they experienced some nausea.
When asked how dizzy they felt in the virtual space, as illustrated in Figure 19, above, 74% of the respondents answered on the low (1 - 3) end of the scale, indicating no or very little dizziness. 22% of the respondents answered on the high end of the scale (5 - 7). We were very careful to keep the students in a small area, where movement was minimized. Yet, it is clear that the technology does not provide a physically comfortable experience for, in this case, nearly a quarter of the subjects. However, 92% of the respondents wanted to go back into a virtual environment, regardless of physical discomfort.
"Presence" is the perceptual and somatic sense of being in a particular place (Hoffman et al., 1996, Prothero, et al., 1995; Prothero & Hoffman, 1995). Generating a sense of presence is one of the key features of virtual reality, and is facilitated primarily through encompassing the visual field in a relatively natural manner, to preclude alternative perceptual input from confounding or confusing the experiencer.
The Division headset is a full helmet, part of the ‘immersive system’ gear complement provided by the manufacturer. It weighs 7 pounds, and has a subtended view of about 150 degrees. Research has indicated that the minimum field of view requirements needed to generate a sense of presence is approximately 110 degrees. (Prothero, et al., 1995; Furness, 1986, 1989) In addition to field-of-view issues, the frame rate of the display is also a key component in generating a ‘natural’ feel to the display. When the frame rate gets much below 30 frames per second, the virtual motion perceived in the headset can get very jerky and unnatural. Furthermore, screen flicker is perceptually distracting, contributing to a lack of a sense of presence.
The virtual environment, at least at this stage in the development of the field, is still ‘cartoony’; most of the objects are not exact replicates of what we might expect. Organic material is particularly difficult to replicate, and the wetlands environment is rife with it. Yet, these environments were compelling enough to engage the students in such a manner as to temporarily ‘suspend their disbelief’, even in the noisy portable in which we conducted the experiential portion of the study.
Interestingly enough, it was in the ‘presence’ questions that we derived our most ‘normal’ distributions. All of the hype aside, subjects were able to articulate whether they really felt as if they were in their wetland environment or not. As all three of the questions’ distributions are so evenly matched, this data in particular has value. The other issue is whether a sense of presence is required in an environment such as this. It can be argued both ways; on one hand that a sense of presence means that the individuals perceptions are more ‘engaged’, leading to a deeper experience. On the other hand, even if an individual doesn’t perceive him or herself to be in a separate reality, it may make it easier to transfer what has been learned back out into what we normally consider "reality" (Hoffman, Hullfish & Houston, 1995).
Figure 20 - Frequency distribution regarding a sense of presence (I)
In Figure 20, above, subjects were asked if their wetland environment ‘became a reality’ for them; and if ‘they forgot the real world’. Just over 50% of the respondents answered between 1 - 3, indicating that the virtual environment did not become a reality for them, or that they did not forget the real world. Another 14% of the respondents answered 4, indicating a neutral view on whether the environment was perceptually ‘real’ or not. The remaining 36% felt that, at least at some level, the virtual space was an alternate ‘reality’. Of that 36%, 13.1% answered 7 - almost all the time, whereas 17.2% answered 1 - not at all.
The graph is more heavily weighted toward the low end, indicating that most of the subjects did not feel that their virtual environment constituted a separate reality, or that they forgot the real world while they were in the virtual environment. In truth, these two issues should have been separated into two questions. However, as has been seen in other sections of this survey analysis, this lack of a sense of presence did not impede their learning or enjoyment. Later data from the VRRV project, however, indicates that enjoyment is often strongly related to a sense of presence. (Winn, 1995).
Figure 21 - Frequency distribution regarding sense of presence (II)
In asking the subjects whether they felt as if they were in the wetland, we see an interesting shift towards the high end of the scale. As illustrated in Figure 21, above, 48% of the respondents answered 5 - 7, indicating a relatively strong sense of being in the wetland environment, which is different than asking if the wetland was real or if the real world ‘went away’, as discussed above. This could relate to Zeltzer’s (Presence, 1992) distinction between interaction with an interface, which he calls ‘presence’ and interaction with content, which he calls ‘logical interaction’. 26% answered with a 4, indicating that they perhaps had a moderate sense of being in the wetland environment. At the low end of the scale (1 - 3), another 26% of the subjects answered that they had a very weak sense of being in the wetland.
What this means is that even though subjects may not have actually perceived the virtual environment to be real, they still got a sense of being in the virtual wetland. This could be because they had built one of the environments and were looking for the aspects of the environment that they had designed. It could also be due to familiarity with expected objects in the virtual wetland. All subjects in a particular group knew what was going to be ‘in’ their space, and so might have been more inclined to see the space as a wetland, even an imagined wetland as it was.
The frequency distribution for the last presence question, on whether subjects felt they were in a ‘different place’ (neither real nor unreal) is illustrated in Figure 22, below.
This graph is very similar to the graph illustrated in Figure 19. The question was asked to assess a general sense of presence. 51% of the respondents answered 1 - 3, indicating that they perceived they were indeed in a ‘different place’. 21% of the respondents answered 4, indicating they neither agreed nor disagreed that they were in a ‘different place’. At the high end of the scale (5 - 7), 28% of the respondents disagreed with the statement, indicating that they did not feel like they were in a ‘different place’.
This data support the findings reported for question 8, regarding the sense of being in the wetland. Instead of a sense of whether the environment was real or not real, or whether the ‘real’ world was no longer perceived, it seems clear that some of the subjects felt present in the wetlands environment, and that they felt that environment was ‘different’ from the real world.
There were two essay questions on the Satisfaction Survey:
We received responses from 97 of the participants, most of whom answered both questions. Responses were analyzed according to the following categories, listed in Table 15, below:
LIKED BEST LIKED LEAST
1 - Building 1 - Difficulty Building
2 - Experiencing 2 - Difficulty Experiencing/Simsickness
3 - Building and Experiencing 3 - Difficulty Building and Experiencing
4 - Final Product/Sense of Completion 4 - Didn't get to program final product
5 - Overall Process 5 - Didn't like overall Process
6 - Focus on Hardware Aspect 6 - Not Enough Time!
7 - Educational Focus 7 - Educational/Research focus
8 - Negative Comments 8 - Positive Comments
9 - Group Issues
10 - Misc. Responses
Table 15 - Survey Essay Response Categories
Frequencies from the first question, regarding what subjects liked BEST about the virtual reality project are illustrated in Figure 23, below.
Figure 23 - What students liked best about their virtual reality experience
As can be seen from the graph above, the bulk (n = 34) of the positive responses were regarding the experience of visiting the virtual wetlands. This was to be expected, since the reward for learning about the wetland environments was to experience the students’ creations.
Other children preferred either the building process (n = 17), or enjoyed both building and visiting their environment (n = 18). Nine of the subjects said that they liked the final product best; the sense of accomplishment of finally getting to view the product which they had been working on so diligently. The overall process was listed as the best component by six of the subjects, whereas nine subjects focused their attention on the hardware itself, mainly the use of the helmet.
Two students indicated that learning about wetlands was for them the best aspect of the project, and two students did not like anything about the project whatsoever. It is interesting to note that in both these cases, the students’ partner was not present for the bulk of the project. In the second essay question, regarding what they liked the least, both students mentioned the lack of a partner as being highly detrimental to their enjoyment of the project overall.
It is clear that most of the positive responses (n = 69) had to do with activities directly associated with either the development process or the experiential component of the project, or both. It is unfortunate that more students didn’t have an appreciation for the educational portion of the project (n = 2).
Figure 24 - What students liked least about their virtual reality experience
This point is reinforced by the data from the second essay question, illustrated in Figure 24, above, in that the bulk (n = 20) of the negative responses were about the students’ dislike of the research or bookwork component of the project. However, it is clear from the test scores that indeed, the children learned, perhaps in spite of themselves!
Negative aspects of the project that were mentioned by subjects include building the objects and world (n = 13), not enough time (n = 9), experiencing VR (n = 8, including 4 specific references to simsickness), a negative group or partner experience (n = 8), and fifteen miscellaneous comments, ranging from "Having to wait to see the finished product", to "The talking of the teachers".
Many children (n = 21) answered that there was absolutely nothing that they would have changed about the project, making comments such as "I don’t think there was anything wrong with it; I had lots of fun!", and "Nothing. There was nothing in my opinion that I didn’t like."
It is a continued affirmation of the value of the project that all of the subjects, except two, had positive feelings to report as illustrated in Figure 23. Out of the 97 responses on the second essay question, illustrated in Figure 24, 21 of the subjects wouldn’t have changed anything, leaving 76 respondents who had, for the most part, a single negative comment to make.
It appears that the virtual environment design process, and experiencing the subjects virtual creations was viewed as the most valuable aspect of the project. It also appears that 20% of the students who responded did not like the research or bookwork component of the project. These students did not make the connection that the research component was just as important as learning how to create objects in 3-D, or how to define and refine the interface for learning.
4.1.5. World Building vs. Experiencing: A Comparison
A post-hoc analysis was performed was with regard to the value of world building versus the experiential portion of the process. Because measures were collected for 21 students who went through the entire world-building experience but did not don the helmet and experience their creation, comparisons could be made between these students’ test scores and those students who both took part in the world-building exercise, and got to experience two worlds as well.
By conducting a between-subjects ANOVA on the objective test scores, no significance between world-building and experiencing was found F(1, 81) = 1.54, p > .05, as presented in Table 38. Further analysis yeilded a significant pre-post effect, F(1, 81) = 78.14, p < .001, and no interaction effects, F(1, 81) = .35, p > .05, as presented in Table 39.
In performing the same analysis on the concept map data, slight significance was found between students who got to experience virtual reality versus those who did not F(1, 65) = 3.22, p < .10, as presented in Table 35. Additional analysis yeilded a significant pre-post effect F(1, 65) = 54.08, p < .001, and no interaction effects F(1, 65) = .13, p > .05, as presented in Table 36.
In a further within-subjects ANOVA comparing concept map data for built vs. chosen representations coupled with the virtual reality experience variable, no significant difference for main effect was found F(1, 65) = .13, p > .05, and no interaction effects F(1, 65) = .06, p > .05, as presented in Table 37.
Based on these results, it appears that the primary cause for the substantial pre-post improvement in scores can be directly attributed to educational treatment as opposed to experiencing the virtual environment. However, it was the experiential portion of the project that was highly motivating for most of the children. The reward of seeing what they had worked so hard to create was an end-goal that was clearly defined and achievable based on the students’ own hard work. The students who didn’t get to experience their environments were very disappointed. Therefore, even though the experiential portion of the project did not affect the students cognitive measures it was a critical affective component to the project.