The Impact of Three-Dimensional Immersive Virtual Environments on Modern Pedagogy

INTRODUCTION

The purpose of this paper is to get some ideas onto the table from which to begin our deliberations about using virtual reality and other technologies to teach about global climate change. This paper is not intended to propose an agenda. It does not pretend to be exhaustive or exclusive. Nor is it intended as a first draft of any final report we might write. I hope that there will be some discussion, clarification, rebuttal or rejection of the points below by email before we meet.

In this paper I raise issues that I believe to be relevant to research concerning the potential of virtual reality to teach complex material. These include: Attributes of Virtual Reality (VR) and Virtual Environments (VEs), why we are focusing on Global Change as our subject matter, approaches to learning in VEs, student characteristics, and the risks of advocacy for VR. I hope that these thoughts will inspire debate and argument about the relative merits of VR for helping students learn science.

The goal of our workshop is to identify issues and research questions concerning the "added value", if any, of using three-dimensional immersive VEs to help students understand complex topics in science. "Virtual reality" means different things to different people. At one extreme are fully immersive systems where participants experience presence in a virtual environment that a high-end computer presents to them through stereoscopic eyepieces and stereophonic earphones enclosed in a helmet. Interaction with the environment is through quasi-natural gestures and actions, sometimes speech recognition, and sometimes through the use of virtual tools. At the other extreme are non-immersive systems, that run on a desktop computer, and that present two-dimensional environments with which students interact using a keyboard and mouse.

Research projects have provided data that suggest students can learn from both immersive and desktop VR and from environments created and presented using technologies that lie between these two extremes. Whether students can learn in virtual environments is therefore not the issue. What is at issue is whether we can use immersive VR and related "high-end" technologies to help students learn complex subject matter more easily, more successfully and possibly earlier than they can from other technologies and their attendant pedagogies. Also important is the issue of whether VR can help students retain what they learn longer and whether VEs can compensate for some students' lack of skill with the symbol systems used in more traditional classrooms. If students’ understanding or performance are no better or only marginally improved after working with high-end VR compared to when they have worked with a desktop program, then it is difficult to justify the cost of using high-end systems for education.

Many matters arise from the general objective of determining the added value of immersive environments for learning. Some of these lead naturally and directly to research questions and testable hypotheses. Others suggest cautions and constraints that impinge on a research agenda. Others simply need to be kept in mind. The following serve as places to start our deliberations.

VR shares many characteristics with other technologies. Indeed, some claim that VR is no more than a fairly straightforward extension of multimedia systems (Dede, 1992). If VR is to improve student comprehension and performance relative to other technologies, it will do so because of its unique characteristics not through the characteristics it shares with other technologies. (Ignoring this premise has led to problems with a lot of earlier research in educational technology. See Clark [1983, 1985]). Part of our task is therefore to identify the unique attributes of VR that might improve understanding and performance. These attributes can then be manipulated as independent variables in experimental studies of VR.

Zeltzer (1992) has proposed a framework for thinking about the characteristics of VR. He proposes that computer-based environments can be characterized along three dimensions which he calls autonomy, presence and interaction.

Autonomy reflects the extent to which the environments function on their own, without (and sometimes in spite of) user input. Systems with low autonomy, like many tutorials and drill and practice programs, sit doing nothing until the student enters an answer to a question or clicks on a navigation icon. Highly autonomous environments, on the other hand, follow their own goals, evolve and develop whether the user does anything or not. Real-time simulations and many games fall into this category. Some military training programs are populated with intelligent agents whose goal is to defeat the trainees and whose strategies will succeed unless trainees take precautions and countermeasures. In autonomous environments, the student and the program may be collaborators or adversaries.

Presence is the sense that the participant is indeed in the place represented as a VE and not in the laboratory with a helmet on. Defining, engendering and measuring presence is the focus of several important research projects (Barfield et al., 1995; Hoffman et al., 1995). Evidence exists that a high level of presence is associated with enjoyment, ability to navigate and perform tasks, and even some learning (Winn, 1995). By and large, it is easier to experience presence in immersive VEs than in desktop programs, although some people claim that reading a good novel can create a sense of presence just as effectively as an immersive VE. With computer-supported environments, the sense of presence increases as the interface becomes more intuitive, more transparent (Bricken, 1991) and the VE becomes more like the real world. This suggests that the place of an environment on the presence dimension depends in large part on interface issues and technical issues such as polygon count, field of view, tracker accuracy and speed and the rate at which the views of the VE are rendered by the computer.

While presence clearly has a lot to do with how participants may interact with the environment, Zeltzer uses "interaction" to mean the extent to which the interaction between participant and environment logically follows the laws that govern the environment. For example, in Dede, Salzman & Loftin’s "Newton World" (1996), which simulates many principles and concepts of Newtonian mechanics, the way an object moves changes according to the prescriptions of Newton’s laws of motion as the student increases or decreases gravity. In Byrne’s "Atom World" (1996), an atom’s behavior and structure change appropriately as the student adds electrons to an orbital. The assumption here is, that for this kind of subject matter, learning occurs as students observe the consequences of their actions in a VE and those consequences had better be consistent and accurate if the students’ understanding is to be correct. The place of an environment on the interaction dimension depends on the fidelity and accuracy of the way it shows reactions to student actions.

Different technologies are characterized by different degrees of autonomy, presence and interaction. Broadcast television, for example, is high on autonomy, low on interaction and variable on presence depending largely on a program’s genre. Computer tutorials are highly interactive, but low on presence and autonomy. "Shoot-em-up" computer games are high on interaction and presence, but low on autonomy. Zeltzer claims that only VR is high on all three. This suggests that, for him, VR is only quantitatively different from its components. If this is the case, then research into the effectiveness of VR for learning will show that simply using more program features that increase autonomy, presence and interaction will be sufficient to bring about significant improvements.

Others argue for qualitative differences between immersive VR and non-immersive systems. Many people will tell you that visiting a VE is unlike anything they have experienced before. These claims and experiences have some support from cognitive science and philosophy. For example, Clancey (1993) has used the term "first-hand" to describe experiences with environments that are unmediated and personal. Traditionally, children learn about the world in two rather different ways. One way is simply to interact with the world, often informally, by experimenting with objects in the immediate environment. The other way is through formal schooling.

Not only is schooling largely conducted through relatively abstract symbol systems, but what students interact with is presented from the point of view of the person using the symbols to describe it. In other words, students see the world through the eyes of the textbook writer or teacher rather than through their own. This is not necessarily bad. Words, pictures and other symbolic forms can be powerfully instructive, can persuade and can arouse emotions. But on other occasions, the case can be made for restoring to children, in virtual environments, the first-hand experiences they enjoy when interacting directly with real environments. Many of the strategies employed in situated learning and constructivist learning attempt to do this, suggesting that VR may well go hand-in-hand with these approaches to learning (Dede, 1995). Research that found evidence for the greater success of these approaches to learning in immersive rather than non-immersive environments would support the argument that VEs are qualitatively different from other kinds of learning environments.

There are, of course, many other ways to describe and categorize the attributes of VR. However, manipulating autonomy, presence and interaction in experimental studies of learning outcomes might be a useful place to start a research agenda. Determining the appropriateness of constructivist and traditional learning strategies for implementation in VEs is also important to consider.

There are several reasons to focus on global change as the subject domain within which to discuss and study the relative merits of VR. First, it is topical. The greenhouse effect, changes in climate, floods and famines that might be attributed to global changes in the environment are in the scientific and popular press and in the public eye. They are issues that many children are already aware of. Global change is also an international matter. Technology now makes it possible to create virtual communities that can discuss and study issues of worldwide import. Sharing ideas in such communities that can include students from many different countries and cultures, each bringing a different perspective to the discussion, is as enriching as it is informative.

Global change is also complex. It transcends and unites many different disciplines, from the basic sciences to the applied, from chemistry and biology to environmental science and sociology. The complexity of global change has two advantages. First, it makes it difficult to teach in traditional ways without oversimplification. It is to a large extent what Spiro et al. (1992) has called "an ill-structured domain". Traditional approaches to teaching and learning have not been all that successful at teaching ill-structured content. For this reason, there is a great deal of room for improvement in helping students understand global change. If VR or another technology can make a difference, that difference will be readily apparent. Second, the difficulty of teaching about global change means that using it as a test of the effectiveness of VR will provide a serious challenge for the technology to overcome. It will be a rigorous and telling test.

Next, global change is a phenomenon that scientists use computer technology to study. Data about global change are frequently easier to understand when displayed as two- and three-dimensional graphics. Hypotheses about the mechanisms that drive global environmental phenomena can be tested by computer simulations. Data visualization and computer simulation are fundamental to the creation of virtual environments. The marriage between scientific research into global change and VR is therefore a natural one. And VR can bring the databases, visualization techniques and simulations, developed by environmental scientists, to students to help them learn about global change without oversimplifying the material.

Largely because of the complexity of the subject matter, children's understanding of the concepts and principles of global change is rife with misconceptions. A number of these are well documented (Boyes, 1993; Francis, Boyes, Qualter, & Stanisstreet, 1993). Whenever I ask a class of upper elementary or middle school students what they know about the greenhouse effect, someone will inevitably say it is caused by the depletion of ozone in the atmosphere. What is more, students’ misconceptions about global change and science generally are very resistant to change even in the face of evidence scientists find convincing (Brewer & Chinn, 1991; Chinn & Brewer, 1993). A lot of attention in science education is given to helping students gather data and test hypotheses that should correct their misconceptions. One way of assessing the usefulness of VEs in helping students understand science is therefore to examine the extent to which misconceptions are reduced after experimenting in a VE. Put another way, is "evidence" about global change gathered by students in a VE more convincing than evidence gathered by them in other ways?

Of course there is always a danger that VEs will create their own misconceptions about global change, even as they remove others. This danger arises in part from the often metaphorical manner in which objects and events are presented in VEs. An advantage of VEs is that they can present to students, in ways that afford direct inspection and interaction, phenomena that cannot be directly inspected and interacted with in the real world. Since these phenomena have no directly perceptible properties, creators of VEs must invent them. And therein lies the problem; these representations are often relatively arbitrary.

In a HIT Lab created VE that represents aspects of global change, students control the quantities and rates of pollution by turning wheels. Yet there is nothing intrinsically "wheel-like" about air pollution. In another of our VEs, built for younger children, students control processes that operate in wetlands ecology, such as the Nitrogen cycle, by interacting with virtual animals and plants and making them interact with each other. While the visible consequences of these actions are "accurate", the animals and plants are really placeholders for biochemical processes not for real objects. If objects such as ducks and pond weed predominate in students’ mental models of the Nitrogen cycle at the expense of processes such as Nitrogen fixation, then one might legitimately conclude that the VE had given rise to a misconception.

There are, therefore, several reasons for using global change as the subject matter within which to assess the relative effectiveness of virtual environments. Some of these are technical, as in the availability of expertise, data sets and simulations of global change itself. Others relate to the rigor of research -– subject matter this complex will put VR to a telling test. Yet others are concerned with students’ sense of the importance of the topic and with their misconceptions about it. And VEs do allow experimenting, observation and data gathering to occur in ways that are different both from the laboratory (for example, you can speed up or slow down time in a VE and achieve massive changes in scale) and from researching extant documents.

Educators have long abandoned the idea that learning occurs as students are "filled up" with knowledge provided by the teacher and by teaching materials. Instead, just about all subscribe to the idea that students have to be active participants in learning if it is to occur. The idea that knowledge is generated from within students rather than being absorbed from outside is not new. But recently theories about how students construct knowledge from information have become more pervasive. The emphases and flavors of these theories are quite variable. But they are usually classified, rather loosely, under the heading "Constructivism".

Constructivist theories of learning make two assumptions that are particularly relevant to our deliberations. The first is that students construct their own understanding of what they are studying. They achieve this by interacting with their learning environments, using the knowledge and skills that they already have, to experiment and make sense of new experiences. These activities are often iterative. Students return to the same material again and again, each time bringing a new perspective to it by virtue of the intervening activities. While, in theory, student actions are not constrained beyond what is reasonable for the subject matter, they are typically guided either directly by a teacher or indirectly through the nature of the learning environment itself. The knowledge that is constructed is personal, often idiosyncratic. In science classrooms, constructivism is usually implemented in research projects where students construct hypotheses and gather data to test them.

The second characteristic of constructivist theories of learning is that knowledge construction is collaborative. Meaning is constructed socially, and where there is disagreement about what something means, negotiation occurs (Vygotsky, 1978). In this way, a balance is maintained between the idiosyncrasy of a student's knowledge and the meaning ascribed to objects and phenomena by the community and culture to which the student belongs. In classrooms, this aspect of constructivism appears as group projects and other forms of collaboration among students.

The ways in which theories of knowledge construction are applied and articulated with technology, including VR, are germane to our deliberations. Over the last decade or so, educational technology has evolved to the point where students can work with computer-generated learning environments (desktop, multimedia, immersive) that bring experiences into the classroom and home that have hitherto been unavailable in those venues. The focus of the educational technologist has shifted from developing systems that can substitute for teachers to those that can do things that teachers cannot do easily. As I mentioned earlier, we are now looking at the unique properties of technologies to do new things to help students learn (Kozma, 1991, 1994). We are using technology to create and provide rich and complex environments in which students can construct their own understanding of content in the same way that they do in other environments. (See chapters in Duffy & Jonassen [1992] and Duffy, Lowyck and Jonassen [1993] for many examples.)

The case can be made that VEs are good places for students to construct knowledge (Dede, 1995; McLellan, 1996). Students learn by interacting and experimenting, often iteratively, with virtual objects and phenomena as they do with real phenomena or phenomena represented in other ways. However, simply turning students loose in VEs with the task of constructing understanding is not likely to succeed. Strategies for providing necessary guidance, feedback to actions and collaboration, that are relatively easy to implement in a classroom, are not so straightforward in VEs. Nor do we yet know much about the best ways to implement them virtually.

The issue of guidance is particularly problematic. The easiest strategy is to have a teacher, or another student, watch what the student is doing in the VE on a TV monitor and talk the student through the difficulties that occur while performing tasks. This "low tech." approach is easy and cheap to implement. However, since the guide is outside the VE, whatever the guide says is an intrusion into the VE and reduces presence. A second option is to place an agent in the VE, who can even be invisible to the student, who is capable of monitoring the student's actions and, using synthesized or digitized speech, provides guidance verbally. The advantage is that presence is not affected nearly as much as in the former case. However, the complexity of the VE and the program that drives it is increased significantly. Finally, the guide might appear as an "avatar" of a second participant in the VE. This enables conversation and discussion to occur between the student and the guide within the VE, again without loss of presence but at a highest cost in complexity and programming.

The question of how a VE provides feedback to students is directly concerned with Zeltzer's (1992) notion of "interaction" that we looked at earlier. In VEs that simulate real objects and events, this requires no more than making the virtual world respond in the same way that the real world would to student actions. If air pollution increases, then smog should appear and air temperature increase when measured with a virtual thermometer. If sea temperature increases, polar ice caps should be seen to melt. In less realistic simulations, however, the provision of feedback is less straightforward.

Imagine a VE that obeys the laws of Algebra rather than of the Newtonian universe (see Winn & Bricken, 1992). In an Algebra world, an object representing a term in an algebraic expression must "change its sign" as the student carries it from one side of an equals sign to the other. Imagine that changing the sign is achieved by turning the object upside down. Now, one of three things may happen to provide feedback if the student forgets to invert the object. It may stay where the student leaves it. In this case, the system allows the student to make mistakes, on the assumption that the student will eventually find them, and gives no visible feedback. On the other hand, the object may float back to where it was before the student moved it. Now the student knows there is an error and must figure out what it is. Finally, the object might invert itself and settle into place where the student put it. In this case, the student learns that there was an error and the error was to forget to change the sign. These variations in feedback are not by any means unique to VEs. But they serve to illustrate the kinds of things we need to study when we investigate the effectiveness of VEs for learning.

By default, VEs are lonely places. To add more than one visitor to a VE requires extra work. Providing for collaboration to foster knowledge construction in VEs must be considered at two levels. (I am not considering the obvious strategy of have students visit a VE singly and then discuss their findings with others afterwards, outside the VE.) Technically, it is possible (though often costly in terms of machine performance) to put more than one student into a VE. Moreover, these people need not be in the same location. Students can work together, sharing tasks and the tools to accomplish them, and can talk to each other, either directly or over the phone if they are physically distant. At a second level, there is a whole host of issues about which we know very little but which are of importance to collaborative knowledge construction. Among these is the protocol to follow when collaborating with someone you have never met physically (assuming the students are from different schools or countries). Will students be shy working with people they have never met, or will the opportunity to assume a different persona embolden them? Given the opportunity to select or create their avatar -- the form in which they will appear to the other student in the VE -- what will they choose and is their choice important? How will they divide the labor? Will it be necessary to assign specific tasks to each before they enter the VE?

These issues, of course, are not specific to learning about global change. However, they are important for us to consider. It is likely that implementations of VR to teach global change, and other complex topics, will be based on some flavor of constructivism. If that is the case, then we need to assess different strategies for giving guidance to students while they are inside VEs, for providing them with feedback on their actions and for allowing them to collaborate with other students in the same VE.

Learning about global change in VEs may not be equally effective for all students. We have data that suggest that building and visiting VEs is more effective for less able students than for more able ones and that, not surprisingly, spatial reasoning ability predicts a student's presence, enjoyment and, to some extent, performance in a VE (Winn et al., 1997). The issue that underlies these and other observations is, at least in part, variability in students' skills with the various symbol systems used, traditionally, in school, in computer-supported learning environments and in VEs.

Salomon (1979) alerted us almost two decades ago to the way in which the symbol systems used by different media and cognition interact. He suggested that mere exposure to a medium could develop skills in using the symbol system the medium employed. In one study, Salomon (1979) reported that the extensive exposure of American children to television enabled them to develop skills in visual processing, while Israeli children, with less exposure to television, did not develop these skills. More generally, findings such as these led Salomon (1988) to conclude that the symbol systems used by computers can be internalized and used by students as tools to reason about problems. More recently, Salomon and his associates (Salomon et al., 1991) describe an application of these principles in which a computer program that helps students compose written documents also increases the amount of meta-cognitive awareness about writing (student awareness of the writing process itself). And any one of us who has watched youngsters playing computer games will express admiration, maybe grudging, for the skills they have developed to outwit the witch or to shoot down the enemy spaceship.

At first glance, all this somewhat removed from learning about global change in VEs. However, if we accept that the symbol systems used by a particular medium or technology can be co-opted by students as tools to help them think about problems and to construct knowledge, then we must seriously consider how the symbol systems used in immersive VEs and other learning environments help students construct their understanding of global change. At the very least, the variety of modalities (sight, sound and possibly touch) and symbolic forms (words, pictures, graphics) offered students who visit VEs is likely to reach more students than just teacher presentation or text. This is the reason we offer for our finding that working with VR helps less able students than more able ones (Winn et al., 1997).

This observation suggests a "shotgun" approach to designing VEs for learning about global change -- the more variety the better. We need to develop more precise learning prescriptions than that. While the extent to which immersive learning environments can cater to individual differences among students remains largely unknown, it is nonetheless useful to speculate about which student characteristics might be best served by constructivist strategies supported by VEs. Some students may learn best when simply told things. Others may learn best by discovering the rules that govern the behavior of objects in a VE. Yet others may prefer to take a more systematic approach to constructing knowledge from a VE, by applying the scientific method that requires them to construct, test and verify hypotheses. All of these possibilities require empirical verification.

One of the best predictors of student learning is what students already know about a subject before they begin to study some new aspect of it (Tobias, 1989). Two issues follow from this observation. The first is whether constructivist strategies, embodied in interactive VEs, assume that students have already mastered the basics of a content domain. It seems prudent not to place novices in a VE and expect them to arrive at some rational understanding of the content that is free of major misconceptions. Spiro et al. (1992) has suggested that some constructivist strategies are best applied to the acquisition of advanced rather than basic knowledge. A task for research into the effectiveness of VEs that help students learn about global change is therefore to determine what students already need to know about the content before they can be expected to learn from working in a VE.

Thinking optimistically, we might anticipate using VEs to introduce global change at an earlier point in the curriculum than it, or its component sciences, are currently taught. Conservatively, this notion is constrained by theories of cognitive development. Certain intellectual activities are simply impossible for students to perform at certain stages in their cognitive development. Conversely, if we assume that the science curriculum is itself somewhat conservative, we can expect that some students may be capable of understanding some topics much earlier than they are typically taught. If this is the case, we need to determine if VEs, or other technology-supported environments and learning strategies, can make it realistic to introduce complex science topics earlier in the curriculum.

In this last, brief, section, I draw attention to the fact that VR is "over-hyped" in the popular press and the popular imagination. What is more, those of us who study the way in which youngsters act in and learn from VEs are easily impressed by the enthusiasm with which students take to VR and by their adeptness with the technologies used for building and presenting VEs. It is hard to resist the conclusion that VR is all that it takes to help less able and less motivated students to become actively and enthusiastically engaged in learning complex material. As scientists, we must guard against such advocacy. It is our job to gather evidence about the relative effectiveness of VEs and to present that evidence, objectively, to the scientific community and to the community of practitioners who teach science to students.