INSTITUTE FOR DEFENSE ANALYSES IDA
Educational Uses of
Virtual Reality Technology
Christine Youngblut
January 1998
Approved for public release, distribution unlimited.
IDA Document D-2128
Log: H 98-000105 ©1998 Institute for Defense Analyses
1
1
Page 2
3
ã1998 Institute for Defense Analyses, 1801 N. Beauregard Street, Alexandria, Vir-ginia 22311-1772 ° (703) 845-2000.
Permission is granted to any individual or institution to use, copy, or distribute this
document in its paper or digital form so long as it is not sold for profit or used for
commercial advantage, and that it is reproduced whole and unaltered, credit to the
source is given, and this copyright notice is retained. This document may not be
posted on any web, ftp, or similar site without the permission of the Institute for
Defense Analyses.
This work was conducted under DASW01-94-C-0054, DARPA Assignment A-189,
for the Defense Advanced Research Projects Agency. The publication of this IDA
document does not indicate endorsement by the Department of Defense or any other
Government agency, nor should the contents be construed as reflecting the official
position of any Government agency.
2
2
Page 3
4
iii
©1998 Institute for Defense Analyses
PREFACE
This work was sponsored by the Defense Advanced Research Projects Agency. The
paper was reviewed by Dr. Dexter Fletcher and Dr. Steven Welke of IDA. The author expresses
her sincere gratitude to these reviewers for their most helpful review comments.
In particular, the author is indebted to all those researchers and teachers who took time
from their busy schedules to discuss their work and provide information for this paper.
3
3
Page 4
5
iv
©1998 Institute for Defense Analyses
4
4
Page 5
6
v
©1998 Institute for Defense Analyses
TABLE OF CONTENTS
Executive Summary.....................................................................................................
ES-1
1. Introduction ..................................................................................................................
1
1.1 Background...........................................................................................................
1
1.2 Approach............................................................................................................... 3
1.3 Scope..................................................................................................................... 4
1.4 Organization of Paper........................................................................................... 5
2. Introductory Programs .................................................................................................
7
2.1 Outreach Programs...............................................................................................
8
2.2 Web-Based Programs........................................................................................... 9
2.3 Teacher Education Programs................................................................................ 9
2.4 Collaborative Programs...................................................................................... 10
3. Current And Planned Uses of VR Technology ..........................................................
13
3.1 Student Use of Pre-Developed Virtual Worlds..................................................
16
3.1.1 Type of Use .............................................................................................. 16
3.1.2 Educational Subjects Supported .............................................................. 29
3.1.3 Pedagogical Support ................................................................................ 30
3.1.4 Support for Students with Special Needs ................................................. 36
3.1.5 Hardware and Software Issues ................................................................. 37
3.1.6 Extending Beyond Education .................................................................. 42
3.2 Student Development of Virtual Worlds............................................................ 43
3.2.1 Type of Use .............................................................................................. 50
3.2.2 Educational Topics Supported ................................................................. 50
3.2.3 Integration into the Curriculum ................................................................ 51
3.2.4 Support for Students with Special Needs ................................................. 54
3.2.5 Hardware and Software Issues ................................................................. 55
3.3 Multiuser, Distributed Worlds .......................................................................... 57
3.3.1 Type of Use .............................................................................................. 61
3.3.2 Educational Subjects Supported .............................................................. 61
3.3.3 Pedagogical Support ................................................................................ 61
3.3.4 Hardware and Software Issues ................................................................. 62
4. Evaluations of VR Usage ...........................................................................................
65
4.1 Evaluations of Student
Use of Pre-Developed Virtual Worlds.......................... 65
4.1.1 Evaluation of Effectiveness ..................................................................... 72
5
5
Page 6
7
vi
©1998 Institute for Defense Analyses
4.1.1.1 General Educational Effectiveness ............................................. 72
4.1.1.2 Effectiveness for Learning Disabled Students ........................... 76
4.1.1.3 Comparative Educational Effectiveness...................................... 77
4.1.1.4 Impact of Immersion ................................................................... 78
4.1.2 Evaluation of Usability ............................................................................ 81
4.1.2.1 General Usability......................................................................... 81
4.1.2.2 Usability for Physically and Learning-Disabled Students........... 82
4.1.2.3 Sense of Presence, Ease of Navigation, and Enjoyment ............. 83
4.2 Evaluations of Student-Development of Virtual Worlds ................................... 84
4.2.1 General Educational Effectiveness .......................................................... 87
4.2.2 Effectiveness for Learning-Disabled Students ......................................... 88
4.2.3 Comparative Educational Effectiveness .................................................. 88
4.2.4 Ability to Work Creatively and Enjoyably ............................................... 90
4.2.5 Student Characteristics that Impact Learning ........................................... 91
4.3 Evaluations of Multiuser, Distributed Virtual Worlds ....................................... 92
5. Conclusions ................................................................................................................
95
List of References
........................................................................................................ 107
List of Acronyms .........................................................................................................
111
Appendix A.
Locations of Participating Research Groups and Schools ..................... 113
6
6
Page 7
8
vii
©1998 Institute for Defense Analyses
LIST OF FIGURES
Figure 1. Constructivist Learning Environments............................................................. 2
Figure 2. MaxwellWorld................................................................................................ 14
Figure 3. Vari House...................................................................................................... 15
Figure 4. Example Vicher List of Things to See and Do, and Study Questions............ 33
Figure 5. Examples of Student Questions Used with MaxwellWorld........................... 34
Figure 6. Unit on Antarctica.......................................................................................... 44
Figure 7. Wetlands Ecology........................................................................................... 49
Figure 8. Student Tasking for Development of a Virtual Stage.................................... 53
Figure 9. NICE............................................................................................................... 58
Figure 10. Virtual Physics............................................................................................... 59
7
7
Page 8
9
viii
©1998 Institute for Defense Analyses
8
8
Page 9
10
ix
©1998 Institute for Defense Analyses
LIST OF TABLES
Table 1. Programs Introducing the Educational Potential of VR Technology................ 7
Table 2. Characteristics and Usage of Pre-Developed VR Applications....................... 17
Table 3. Classification of Applications.......................................................................... 16
Table 4. Educational Subjects........................................................................................ 29
Table 5. State and Other Learning Objectives............................................................... 29
Table 6. No Pedagogical Support.................................................................................. 30
Table 7. Type of Pedagogical Support........................................................................... 31
Table 8. Special Needs Applications............................................................................. 36
Table 9. Hardware Support for Pre-Developed Applications........................................ 39
Table 10. VR Development Software for Pre-Developed Applications.......................... 41
Table 11. Characteristics and Usage of Student Development of Virtual Worlds.......... 45
Table 12. Classification of Applications.......................................................................... 50
Table 13. Educational Subjects........................................................................................ 51
Table 14. Hardware Support for Student-Developed Worlds.......................................... 55
Table 15. VR Development Software for Student-Developed Worlds............................ 56
Table 16. Characteristics and Usage of Multiuser, Distributed VR Applications........... 60
Table 17. Classification of Applications.......................................................................... 61
Table 18. Educational Subjects........................................................................................ 61
Table 19. Type of Pedagogical Support........................................................................... 61
Table 20. Hardware Support for Multiuser, Distributed Worlds..................................... 63
Table 21. Completed Evaluations on the Use of Pre-Developed Virtual Worlds........... 66
Table 22. Types of Completed Evaluations..................................................................... 72
Table 23. Impact of Findings on Refinement of NewtonWorld...................................... 74
Table 24. Completed Evaluations Using Student Development of Virtual Worlds........ 85
Table 25. Types of Completed Effectiveness Evaluations.............................................. 87
Table 26. Student Opinions About VR Technology During a Summer Camp (1991).... 91
Table 27. Completed Evaluations Using Distributed Virtual Worlds............................. 93
9
9
Page 10
11
x
©1998 Institute for Defense Analyses
10
10
Page 11
12
ES-1
©1998 Institute for Defense Analyses
EXECUTIVE SUMMARY
Background and Purpose
Educating current and future generations of American children to live in an information
society is a critical issue. It is compounded by the recognized need to provide life-long
education for all citizens and to support a flexible workforce. Virtual reality (VR) technology
has been widely proposed as a major technological advance that can offer significant support
for such education. There are several ways in which VR technology is expected to facilitate
learning. One of its unique capabilities is the ability to allow students to visualize abstract
concepts, to observe events at atomic or planetary scales, and to visit environments and interact
with events that distance, time, or safety factors make unavailable. The types of activities
supported by this capability facilitate current educational thinking that students are better able
to master, retain, and generalize new knowledge when they are actively involved in
constructing that knowledge in a learning-by-doing situation.
The potential of VR technology for supporting education is widely recognized. Several
programs designed to introduce large numbers of students and teachers to the technology have
been established, a number of academic institutions have developed research programs to
investigate key issues, and some public schools are evaluating the technology. It has already
seen practical use in an estimated twenty or more public schools and colleges, and many more
have been involved in evaluation or research efforts.
This paper reviews current efforts that are developing, evaluating, or using VR technology
in education. It builds a picture of the states of the art and practice, and reviews some of the
critical questions that are being addressed. While the coverage of efforts is not intended to be
comprehensive, it does provide a representative sampling of recent and current activities.
Educational uses of the technology are broadly distinguished as those where students interact
with pre-developed VR applications and those where students develop their own virtual worlds
in the course of researching, understanding, and demonstrating their grasp of some subject
matter.
Pre-Developed Educational VR Applications
Over forty efforts in the category of pre-developed applications are reported in this paper.
In the majority of these efforts, a single student interacts with the virtual world; although this
student may be collaborating with others in his physical classroom, there is no collaboration in
11
11
Page 12
13
ES-2
©1998 Institute for Defense Analyses
the virtual world. Only three of the efforts currently support multiple users, and they provide
only very limited types of interaction between users.
Efforts using pre-development applications are almost equally split between those
conducted primarily as research efforts and those where VR applications were developed for
practical use (although several of the applications developed as part of research efforts are also
expected to see eventual practical use). The first practical use of an educational VR application
that has been identified occurred in 1993, and some twenty applications are expected to have
seen practical use by the end of 1997.
Nearly three-quarters of the applications are immersive, using either a head-mounted
display (HMD) or cave display to visually immerse a user in the virtual world. While a few
researchers have started to look at the use of spatialized sound and haptic interfaces, these are
not in practical use as yet.
Discounting applications intended for use by all age groups, the predeveloped applications
are nearly equally split between those designed for elementary and middle school students, for
high school students, and for college students (undergraduate and graduate). They cover a
broad range of subjects with, again, a fairly equal split between the arts/ humanities and
sciences. A few are designed to meet specific country or state learning objectives. The majority
support constructivist learning using an experiential or guided-inquiry paradigm. Several
applications are being developed to meet the special needs of students with learning
disabilities, autism, or certain physical disabilities.
Student Development of Virtual Worlds
More than twenty efforts are reported in the category of student development of virtual
worlds. Here, nearly two-thirds of the efforts have been conducted as a practical part of the
curriculum and the remainder, while also conducted in classrooms, are primarily regarded as
research activities. The first identified practical effort was undertaken in 1993, and students
will have been involved in at least twelve different virtual world building efforts by the end of
1997. Although a few of these efforts have been, or are intended to be, repeated on a regular
basis, most have been one-time events.
The efforts are equally split in their use of either a desktop or immersive VR system,
although the practical uses have largely entailed the development of desktop virtual worlds,
whereas the research activities have focused on student development of immersive worlds.
Most of the worlds have been developed by students working in groups. While these students
have needed little technical support for desktop world development activities, students have
been actively supported by researchers in the development of immersive worlds.
For this category, the majority of efforts are being conducted at the middle school level,
with more efforts at this educational level than at elementary school, high school, and college
12
12
Page 13
14
ES-3
©1998 Institute for Defense Analyses
levels combined. While these efforts address a smaller number of subjects than the pre-devel-opment
applications, the range is still extensive.
Evaluations
In total, thirty-five evaluations on the identified efforts have been completed. An additional
twenty evaluations are currently underway or already planned. Just as the majority of
educational uses of VR technology have involved pre-developed applications, so have the
majority of evaluations to date been performed on this category of applications. All efforts in
the categories of pre-developed applications and student world development that are classified
as research-oriented have been the subject of at least one evaluation. But while over half of the
pre-developed applications in practical use have been evaluated, only two of the eleven
practical-use efforts where students have developed their own virtual worlds have seen a
similar type of evaluation. While these figures seem low, it must be remembered that all current
educational uses of VR technology are, at least to some extent, exploratory, and even when no
explicit evaluations have been performed, the researchers and teachers are forming their own
opinions of the value of the technology.
Given this level of activity, what has been learned about the effectiveness and usability of
the technology? Although the current data is insufficient for any substantive conclusions to be
drawn, some initial findings can be posited:
° Use of both pre-developed VR applications and student development of virtual worlds
can be educationally effective. All studies that have investigated whether students using
VR technology could meet stated learning objectives found that some level of learning
does occur. However, the extent of such learning has varied. The few formal studies
(two for pre-developed applications and three for student-development efforts) that
have examined whether VR technology provides a more effective learning tool than
traditional classroom practices have shown that students using VR technology
performed at least equivalently and usually better than those using other forms of
instruction. Other studies have shown that while immersive applications have been
more effective than non-immersive applications, the key factor seems to be the
interactivity of these applications rather than the fact of immersion.
° Students enjoy working with virtual worlds and this experience can be highly
motivating. Reports of student enjoyment are common and several researchers and
teachers report striking improvements in student motivation. Surprisingly, students are
very tolerant of the low resolution and cumbersomeness of current HMDs. The main
problem seems to lie in navigating around the worlds. Occurrences of simulator
sickness symptoms are rare, and symptoms that do occur take the form of disorientation
and ocular discomfort, and not nausea.
13
13
Page 14
15
ES-4
©1998 Institute for Defense Analyses
° Teachers report their role in the classroom changing. Instead of being a teacher with all
the answers, teacher have found themselves acting as facilitators who support students
in their discovery of worlds and in their building ideas based on information gained
from those worlds. The use of pre-developed applications, however, can pose a problem
for lesson administration and monitoring students' progress.
° In practical terms, desktop VR is more suitable for widespread use than immersive VR
technology. Considering both the hardware and software requirements, desktop VR is
quite a mature technology. It is affordable in that a basic level of technology can be
achieved on most existing personal computers at either no cost or some minimal
software cost. The expected availability of increasing numbers of virtual worlds over
the Web is likely to promote its use. While immersive VR is being used in several
practical applications, this part of the technology is less mature, with shortcomings in
such areas as displays, system lag, and common interaction metaphors. Immersive VR
is also expensive, with a single hardware/ software platform (including HMD and other
specialized interface devices) starting at around $15,000. An unstable marketplace is
likely to slow the widespread use of immersive VR technology.
At this time, there is no data to support findings on the effectiveness of the technology for
collaborative learning, or the cost-effectiveness of VR-based education.
Concluding Remarks
It is important to note that current uses of the technology tend to be isolated examples of
what proponents of the technology can achieve. Moreover, almost exclusively, the studies have
concerned one-time use of virtual worlds and there is no information on how students respond
to the technology over the long term. Additional work is required to substantiate or refine
current findings and to answer specific questions, such as which characteristics of the
technology support particular types of learning and how use of the technology should be
integrated with other educational activities.
Existing data does suggest that this technology offers significant, positive support for
education. It indicates sufficient potential value to justify continuing research and development
activities and increasing practical evaluations of technology uses. Such work needs to occur
hand-in-hand with research into constructivist and collaborative learning. Thought must also
be given to how to train teachers in the use of the technology.
14
14
Page 15
16
1
©1998 Institute for Defense Analyses
1. Introduction
The purpose of this paper is to report on educational uses of virtual reality (VR) technology.
By presenting examples of both systems that are in practical use and those that are still the
subject of research and development, it provides a sense of the current state of the practice and
the state of the art. The paper also looks at researchers' and teachers' evaluations of VR
educational applications to see what has been learned, how critical questions are being
answered, and whether the technology is starting to live up to its promise.
1.1 Background
Many researchers and educational practitioners believe that VR technology offers strong
benefits that can support education. For some, VR's ability to facilitate constructivist learning
activities is the key issue. Others focus on the potential to provide alternative forms of learning
that can support different types of learners, such as visually oriented learners. Still others see
the ability for learners, and educators, to collaborate in a virtual class that transcends
geographical boundaries as the major benefit.
In traditional instructional environments, students are expected to learn by assimilation, for
example, by listening to an instructor lecture about a subject. Current educational thinking is
that students are better able to master, retain, and generalize new knowledge when they are
actively involved in constructing that knowledge in a learning-by-doing situation. This is a
philosophy of pedagogy called constructivism and its supporters vary, ranging from those who
see it as a useful complement to teaching-by-telling to those who argue that the whole
curriculum should be reinvented by students through gently guided discovery learning [Dede
1997a].
As noted by Jonassen [1994], the major distinction between traditional instructional design
and constructivism is that the former focuses on designing instruction that has predictable
outcomes and intervenes during instruction to map a predetermined conception of reality onto
the student's knowledge, while the latter focuses on instruction that fosters the learning process
instead of controlling it. Jonassen also points out that constructivists focus on learning
environments rather than instructional sequences, recommending features such as those
identified in Figure 1.
The support VR technology provides for constructivist learning is discussed in some detail
by Winn [1993]. Winn suggests that immersive VR technology allows three kinds of
15
15
Page 16
17
2
©1998 Institute for Defense Analyses
knowledge-building experiences that are not available in the real world, but that are important
for learning. These pertain to size, transduction, and reification. VR technology allows radical
changes in the relative sizes of the student and virtual objects. Using Winn's examples, at one
extreme, a student could enter an atom and examine and adjust electrons in their orbitals, thus
altering the atom's valence and its ability to combine to form molecules; or at the other
extreme, a student could acquire a sense of the relative sizes and distances in the solar system
by flying between planets. Transduction refers to the use of interface devices to present
information that is not readily available to human senses. For example, variations in the
intensity of some sound could be used to portray levels of radiation, or color could be used to
show the movement of oxygen through an environment. Together, transduction and the ability
to manipulate size support reification, which is the process of creating perceptible
representations of objects and events that have no physical form, such as mathematical
equations.
VR worlds can also be used to
circumvent the physical, safety, and cost
constraints that limit schools in the types
of environments they can provide for
learning-by-doing. For example, it would
be impractical to allow students studying
chemical engineering to further their
understanding of the underlying processes
by conducting experiments with the
equipment at an operating chemical
production plant. This type of activity can
be performed in a virtual world. As this example hints, VR learning environments can also
support the notion of situated learning where students learn while in the actual context where
that learning is to be applied.
Within the constructivist philosophy, various actual pedagogical approaches can be taken.
The most popular pedagogical approach is one of guided-inquiry where, by performing tasks
such as experiments, students are guided to uncover critical concepts for themselves. An
experiential approach is the second most common approach. While all virtual worlds allow a
user to experience a virtual situation, the term "experiential" is used in this paper to refer to
more than simple walkthroughs of a virtual world. Instead, educational VR applications
described as experiential require some further interaction on behalf of the student.
VR technology also can provide a different framework for education, one that is
independent of a physical classroom and the restrictions levied by the availability of
educational resources at any one physical location. In this context, the term "virtual classroom"
is used to imply more than the use of telecommunications technologies to provide an electronic
° Provide multiple representations of reality, thereby avoid-ing oversimplification of instruction and representing the
natural complexity of the real world;
° Focus on knowledge construction, not reproduction;
° Present authentic tasks (contextualizing rather than abstracting instruction);
° Foster reflective practice;
° Enable context-and content-dependent knowledge con-struction;
and
° Support collaborative construction of knowledge through
social negotiation, not competition among learners for rec-ognition.
(Based on Jonassen [1994].)
Figure 1. Constructivist Learning Environments
16
16
Page 17
18
3
©1998 Institute for Defense Analyses
simulation of a conventional classroom, although this can have merit of its own (see [Turoff
1995]). Instead, as best discussed by Tiffin and Rajasingham 1 [1995], the concept of a virtual
classroom embodies a new paradigm of learner-centered education suitable for lifelong
learning. In this paradigm, students of all ages participate in educational activities from their
home, place of work, or some type of "school house." Students shape their own curriculum to
meet their personal needs, joining classes appropriate to their learning objectives that are given
in venues suited to the topics of interest. Classes are not limited to the availability of
appropriate teachers in the region but can meet at a time convenient to all participants,
independent of the geographical locations of those participants. Access to libraries,
laboratories, and other educational resources is not limited to certain hours but available round
the clock.
This utopian view has some very real education control and social issues to contend with,
particularly for young students, and would require major rethinking and restructuring of the
educational system. Such changes are is unlikely to occur any time soon, if indeed this is the
right educational approach to take. Research on virtual classrooms is starting at three or four
sites internationally but due to the preliminary nature of this work, these efforts are not
discussed in this paper.
1.2 Approach
Educational uses of VR technology were identified by several means. Some uses were
already known to the author, others were identified by colleagues, or found mentioned in the
open literature or Web sites. As particular efforts were identified, the educators or researchers
involved were contacted for more information about their work. 2 In addition to collecting
detailed, factual data about each usage, information was sought that would allow investigating
critical issues about using VR technology for education.
To help clarify these issues and provide some type of yardstick by which progress can be
measured, several high-level questions were formed. Some of the most obvious questions
relate to effectiveness of VR-based education, namely:
° Does learning in virtual worlds provide something valuable that is not otherwise
available?
° How does the effectiveness of student use of pre-developed virtual worlds compare
with traditional instructional practices?
1 These authors prefer the term "virtual learning space" to avoid the suggestion that a virtual class is held in an
electronic version of a conventional classroom. The more common term "virtual classroom" is used here. 2
Exceptions are the teacher at H. B. Sugg Elementary School who conducted a study with student development
of virtual pyramids, and the researcher at Oregon State University who looked at the use of VR for promoting
awareness of spatial relations.
17
17
Page 18
19
4
©1998 Institute for Defense Analyses
° How does the effectiveness of student development of virtual worlds compare with
other instructional practices?
° How does the effectiveness of student use of pre-developed virtual worlds compare
with that of student development of virtual worlds?
° How does the effectiveness of immersive and non-immersive virtual worlds
compare?
° How well does VR technology support collaborative learning between students? Is
this collaboration educationally effective?
° Is VR-supported learning cost effective?
A closely related issue concerns where VR technology should and, equally important,
should not be used. Questions here address both educational content and student characteris-tics:
° For what type of educational objectives or material is VR technology best suited?
Where is it not suited?
° Are there specific student characteristics that indicate whether VR-based education
is appropriate? Does the technology benefit only certain categories of students?
Potential student and teacher acceptance of VR learning environments will depend on
many factors, including ease of interface use, and ease of integration into the classroom.
° Do students find VR interfaces easy to work with?
° Does the effective use of VR technology change the teacher's role in the classroom?
° What are student and teacher reactions to the use of this technology?
Practical questions that need to be considered are:
° Are the hardware platforms and minimum set of interface devices required
affordable to most schools?
° Are the needed software development tools commonly available?
° Is the technology currently mature enough for practical use?
These questions will be revisited at the end of this paper. While the information presented
in the paper is insufficient to provide definitive answers, it is does provide some useful
indicators of current trends and problem areas.
1.3 Scope
In this paper, the term "virtual reality" is used broadly to cover both immersive and non-immersive
VR. The sense that a user is actually present in a virtual environment is termed
"presence" and is an artifact of being visually immersed in the computer-generated virtual
world. Presence is often held to be the discriminating feature of VR applications. This view
would exclude from consideration non-immersive VR applications, that is, those that rely on a
18
18
Page 19
20
5
©1998 Institute for Defense Analyses
traditional desktop monitor or single projection screen for the display. At this stage in the
development of the technology this restriction may be a mistake, at least for the educational
uses discussed here. Although the cost of the special systems needed for immersive VR is
coming down, these systems are still beyond the scope of most school budgets. Also, there are
unresolved questions relating to health and safety issues that arise in the use of immersive
systems [Wilson 1996]. Moreover, there is no overwhelmingly conclusive evidence that
immersive systems are more effective in educational applications than their non-immersive
counterparts. Omitting non-immersive applications would mean ignoring many promising
efforts that have valuable information to offer. Accordingly, this paper addressed educational
uses of both immersive and non-immersive VR.
The scope of the study reported here is limited to educational uses of VR technology and
excludes training applications. In other words, the focus was restricted to those applications
intended to impart knowledge; not considered were applications designed to provide for the
development and practice of work-related skills. The study was further limited to consideration
of only graphical VR applications, ignoring their textual counterparts referred to as Multi-User
Domains or Dungeons (MUDs) or object-oriented MUDs (MOOs). Also, it is not concerned
with those instances where VR technology itself is being taught, but rather where VR
technology is used as the learning medium. Within this context, the paper provides information
on VR applications designed to teach particular topics, and those cases where students are
themselves developing virtual worlds. As such, it covers kindergarten through grade 12 (K-12),
college, and other educational venues.
1.4 Organization of Paper
Section 2 provides an overview of efforts designed to provide educators and students with
a basic appreciation of the potential of VR technology. In Section 3, specific educational uses
of VR are summarized and discussed. While the coverage of existing and planned work is by
no means comprehensive, a representative set of uses of the technology is covered. Section 4
focuses on evaluations that are underway or have been completed. These evaluations include
experiments comparing the educational effectiveness of VR applications with traditional
learning practices. Section 5 reviews the questions just raised with respect to what has been
learned and provides some remarks on the status of VR in education and critical research needs.
The names and locations of those researchers and teachers who participated in this study
are given in Appendix A. The paper concludes with a list of references and a list of acronyms
used.
19
19
Page 20
21
6
©1998 Institute for Defense Analyses
20
20
Page 21
22
7
©1998 Institute for Defense Analyses
2. Introductory Programs
The section provides an overview of some programs that are intended to provide a basic
appreciation of educational applications of VR technology. These programs are listed in Table
1, along with identification of the group that provides the program and the funding
organization. As can be seen in this table, various schools, colleges, state organizations, and
government agencies are all playing a role in promoting the use of VR technology in education.
The goals of the programs range from providing students with the opportunity to visit virtual
worlds, to placing VR software in the hands of teachers who will use it to meet their specific
teaching objectives in the classroom.
Table 1. Programs Introducing the Educational Potential of VR Technology
Program Name Provided By Participants Date
Outreach
Virtual Reality Roving Vehicle (VRRV)/ Washington University of Washington, Human Interface Technology Laboratory (HITL), Seattle, WA Teachers and students
grades 4-12
1994-1997
VRRV/ Nebraska, Phase I and II Education Service Unit #3 of Nebraska, Educa-tional Development Center, Omaha, NE Teachers and students 1995 onward
Mobile Aeronautics Education Laboratory (MAEL) NASA/ Lewis Research Center, Cleveland, OH Students grades 9-12 1997 onward
Web — University of Washington, HITL Teachers 1996 onward
Teacher
Education
VRRV/ Nebraska, Phase III Education Service Unit #3 of Nebraska, Educa-tional Development Center Teachers 1997 onward
Educators' VR Series Haywood Community College, Regional High Tech Center, Waynesville, NC Teachers 1994 onward
Virtual Reality in the Schools East Carolina University, Virtual Reality and Education Laboratory (VREL), Greenville, NC Teachers 1995 onward
Virtual Education -Science and Math of Texas (VESAMOTEX) Slaton Independent School District, Wilson, TX Teachers 1995-1997
VR Concentration, M. A. in Edu-cation East Carolina University, VREL Teachers 1996 onward
Collaborative VR in Education University of Illinois, National Center for Super-computing Applications (NCSA), Champaign, IL Teachers 1996-1997 Virtual Reality in the Schools East Carolina University, VREL Teachers 1995 onward
21
21
Page 22
23
8
©1998 Institute for Defense Analyses
2.1 Outreach Programs
Looking first at the outreach programs that have taken VR systems to elementary, middle,
and high schools, there are three programs to consider:
° Virtual Reality Roving Vehicles (VRRV)/ Washington,
° VRRV/ Nebraska, Phases I and II, and
° Mobile Aeronautics Education Laboratory (MAEL).
The VRRV programs have had the largest impact to date, using vans to bring VR to over
7,000 students and educators. VRRV/ Washington was the first outreach program to start. It
was conducted in two phases. The initial phase, called the Hors d'Oeuvre, provided
participants at various schools with presentations on VR technology and the opportunity to
visit virtual worlds. Researchers visited schools for one day or more to present and discuss
VR, and then provide a demonstration of commercially developed virtual worlds. The second
phase, called the Entrée, provided a more in-depth introduction to the technology by
supporting students. In this case, researchers spent several days with teachers and students
who, over a period of several weeks, developed their own virtual worlds on topics such as
Wetlands Ecology and Global Warming (these worlds are discussed in Section 3.2).
The first phase of VRRV/ Nebraska was modeled after VRRV/ Washington's Hors
d'Oeuvre. The second phase differed from VRRV/ Washington, however, by supporting
teachers in a one-week intensive effort where they were able to use the University of
Washington Human Interface Technology Laboratory (HITL)-developed Atom World to teach
atomic and molecular structures or, alternatively, choose their own approach for using VR
technology to support their curriculum. VRRV/ Washington and the outreach parts of VRRV/
Nebraska have been sponsored by the US West Foundation. They have now come to an end,
although analysis of collected data continues.
The third outreach program, MAEL, is just getting underway. Unlike the VRRV programs,
VR is only one of the educational tools demonstrated by MAEL. Using a 16-wheeler truck for
transportation, this program uses ten different types of workstations to teach students about
aeronautics, mathematics, and science. The immersive VR workstation provides a virtual
biplane that student use to explore aeronautical concepts. MAEL is intended to be used in two
ways: as an exhibit at air shows and other public aeronautic events, and as an educational tool
designed to visit schools and provide students with access to a predefined (or custom)
curriculum. MAEL began its first trip in January 1997, visiting schools near the National
Aeronautics and Space Administration (NASA) Ames Research Center and the NASA Dryden
Flight Research Center. It is expected to spend six months a year on the road for the next several
years. The feasibility of setting up permanent versions of the MAEL components at various
schools, colleges, and science centers is being investigated.
22
22
Page 23
24
9
©1998 Institute for Defense Analyses
In addition to exposing students and teachers to the potential of VR technology, these three
programs also have a research element. One of the goals of the Hors d'Oeuvre phase in the
VRRV programs was to determine the limitations and potential uses of VR technology for
education. With respect to the Entrée phase, the researchers assessed whether having students
build their own VR worlds helped them understand the concepts and principles they were
learning as part of the regular curriculum. The results from these evaluations are reported in,
respectively, Section 4.1 and Section 4.2. The research agenda of MAEL with respect to the
VR part of its program includes two fronts. The first of these is looking at students' reactions
to a virtual flight experience. The second is more opened ended and based on the use of video
cameras to record students' interactions with the VR workstation. The recordings will be used
to help refine the curriculum and educational programs and also to provide data for outside
researchers studying how students learn using advanced technology teaching methods. It is too
early for this program to have any results to report.
2.2 Web-Based Programs
Another type of program is illustrative of the great variety of roles that the Web offers.
Based on the understanding of educational VR applications they gained through the VRRV
program, the HITL has increased the VR resources available to teachers by providing a Web
site intended to make teachers aware of the use of VR technology in education. Currently, this
site provides some introductory information about VR and brief guidance on world building
using the Global Change application (see Section 3.1) as a model. A Virtual Reality Modeling
Language (VRML) version of this world is available for downloading. This effort has been
sponsored by the U. S. Department of Education, Funds for Innovation, Teacher/ Pathfinder
program. Should additional funding become available, the HITL might provide additional
resources in the form of on-line teacher support.
2.3 Teacher Education Programs
There are five programs that provide some type of education for teachers regarding the use
of VR technology. Together, these programs have introduced over 100 teachers from public
school systems, colleges, and universities to VR technology. The programs are:
° VRRV/ Nebraska, Phase III,
° Educators' VR Series,
° Virtual Reality in the Schools,
° Virtual Education -Science and Math of Texas (VESAMOTEX), and
° VR Concentration, M. A. in Education.
The third phase of VRRV/ Nebraska is just starting and its goal is to prepare and support
teachers in the use of VR for constructivist learning activities based on the desktop computing
23
23
Page 24
25
10
©1998 Institute for Defense Analyses
facilities schools already have available. To this end, Education Service Unit #3 of Nebraska
offers one-day workshops for educators and is expected, in some cases, to work directly with
teachers on special projects.
The first offering of the Educators' VR Series program was a one-day workshop given with
support from Autodesk Applied Software. More recently, a one-week workshop was provided
at the request of the North Carolina Center for the Advancement of Teaching. In the summer
of 1997, Haywood Community College plans to offer its first VR Summer Institute for
Educators.
The Virtual Reality in the Schools program offers an annual multi-day workshop for
teachers who are interested in learning about VR technology. This program provides further
support for teachers through its quarterly journal publication entitled VR in the Schools. This
journal provides updates on the Virtual Reality and Schools project, as well as reports on
relevant studies and current practical uses of the technology, reviews of VR-related software
and hardware, and general articles of interest. VREL also provides a number of pamphlets
discussing VR and education that provide high-level guidance for using VR in the classroom.
The VESAMOTEX program has two objectives: to promote the use of VR in education and
to bring VR into practical use to support science education at Slaton High School. In support
of the first of these objectives, several presentations have been given on the topics of VR and
the VRML to groups such as the Texas Association of Physics Teachers and the Texas
Computer Educators Association. Future plans include a discussion in one of the electronic
meeting rooms on the Web, where researchers working with VR will be invited to answer
questions posed by educators. (The efforts where VR technology has been used in the
classroom at Slaton High School are discussed in Section 3.2.) VESAMOTEX is supported by
the Texas 1995 Christa McAuliffe Fellowship.
The remaining type of program, and seemingly the only one of its kind at the moment, is
VREL's VR Concentration in its Master of Arts in Education (Instructional Technology
Specialist-Computer) degree course. Four different courses are offered, with the goal of
educating future teachers in the use of VR technology. These courses cover topics ranging from
evaluating VR hardware devices and software development tools, to considering ethical
implications in educational uses of the technology. One course, EDTC 6242, for example,
requires students to work with a local school to develop a VR application that meets specific
curriculum objectives, to develop supporting instructional materials for teachers, and then
conduct classroom evaluations of the effectiveness of the application they have developed.
2.4 Collaborative Programs
The VR in Education program is part of the National Science Foundation's (NSF) Resource
for Science Education program at the National Center for Supercomputing Applications
24
24
Page 25
26
11
©1998 Institute for Defense Analyses
(NCSA) at the University of Illinois. The purpose of the program is to engage practicing
teachers in the VR research community and to improve the understanding of educational issues
relating to VR in the classroom. The overall approach of the program is to:
° Partner with a nationally selected, diverse group of educators in a year-long VR and
education program,
° Work with educators to help design and test VR systems and applications for use in
classrooms at a variety of levels,
° Encourage educators to interact and form collaborations with NCSA staff and scientists
on projects applicable to their classrooms,
° Support educators for peer training, sharing of materials, and grant writing, and
° Pay attention to teacher feedback.
Nine teachers from across the country have made a one-year agreement to collaborate with
NCSA staff in integrating VR into classroom curricula. These selected participants teach
students ranging from kindergarten up through junior college. A series of workshops has
provided the teachers with an introduction to various VR hardware and VRML, together with
presentations and demonstrations of uses of the technology. During the course of these
activities, the teachers and NCSA researchers have collaborated in rating some existing VR
applications according to National Science Education Standards, and the group has worked on
creating a vision for the use of VR technology in the nation's schools. Additionally, all of the
teachers have been using the technology in their classrooms. For example, at Urbana High
School, Chicago, IL VR technology is being used in a physics class to help students understand
equipotential surfaces. (Details on the ongoing activities and results of the program were
unavailable prior to release of a forthcoming NSF/ North Central Regional Educational
Laboratory (NCREL) report.)
The Virtual Reality in the Schools program also provides direct support for the introduction
of VR technology into the classroom. Here, Virtual Reality and Education Laboratory (VREL)
staff work with individual teachers who have volunteered to join the project. These teachers are
given a copy of either a PC or Mac version of VR development software (Virtus WalkThrough)
and provided with hands-on training in its use. VREL staff members then work with the teacher
to help identify where VR can best be used in a particular area of the curriculum taught by the
teacher. Usually, this collaboration involves selecting specific items from the North Carolina
Competency-Based Curriculum Objectives and discussing how VR might be used as a means
of student attainment or as a measure of attainment. Once the teacher has designed the lessons
that will use VR as a teaching tool, VREL staff continue to be available for consulting and visits
to the school. They also help with evaluating success in meeting the specified curriculum
objectives. Up to the fall of 1996, seventeen teachers were signed up on this program.
25
25
Page 26
27
12
©1998 Institute for Defense Analyses
26
26
Page 27
28
13
©1998 Institute for Defense Analyses
3. Current and Planned Uses of VR Technology
The majority of educational uses of VR technology has involved student use of pre-developed
VR applications where students individually visit a virtual world to learn some basic
concepts or, for example, to develop an understanding of different periods in history.
Alternatively, students may be required to develop their own virtual world to guide the
research, understanding, and demonstration of their grasp of scientific or non-scientific
material. Uses of VR in these two categories are discussed separately in Section 3.1 and Section
3.2. Multiuser, distributed VR applications are discussed in Section 3.3.
It is interesting to briefly look at the proportions of uses in each of the three categories of
pre-developed, student-developed, and multiuser VR. The ratio is, roughly, 13: 7: 1, with 40 pre-developed
applications, 21 student-development efforts, and 3 multiuser applications. The
predominance of uses of pre-developed VR applications should not be surprising since these
applications provide a good first step for teachers and students in building their understanding
of the technology, and a more controllable venue for investigating basic questions pertaining
to educational uses of VR technology. Indeed, over half of the efforts in this category are
primarily research oriented. Given this fact, and considering the mastery of the technology
needed, the relatively large proportion of efforts involving student-development of virtual
worlds does, initially, seem somewhat surprising. In fact, the high number of efforts in this
category is attributable to the work of just two organizations that, between them, account for
nearly two-thirds of the cases.
The low number of uses of multiuser, distributed VR reflects the maturity of VR technology
itself: the integration of VR, networking, and telecommunication technologies is still in the
initial stages of research. Although only three of the efforts discussed here currently fall into
this category, several of the developers of pre-developed VR applications have stated their
intention to develop multiuser, distributed versions of their current applications at some point
in the future.
27
27
Page 28
29
14
©1998 Institute for Defense Analyses
Figure 2. MaxwellWorld
immediately accessible. The virtual hand is used to point to menu items, and a button on a Polhemus 3Ball device is used to make a selection. The hand also allows a student to place positive and negative charges of
various relative magnitudes by, for example, attaching a test charge to the tip of the virtual hand. Once a
Project Goal: Examine whether VR's sensorial immersion can help students remediate deeply rooted misconceptions
and construct accurate mental models of abstract science concepts.
Focus of Current and Planned Work: -Extend explorations on how multisensory immersion in-fluences
learning. -Investigate impact of frames of reference on learning.
-Experiment with collaborative learning among geograph-ically remote users inhabiting a shared virtual context.
MaxwellWorld Description: Students build 3D electric
fields and explore forces and energy by directly manipulat-ing
multiple 3D representa-tions (test charges, field lines,
equipotential surfaces, and flux surfaces). They can see,
hear, and feel the distribution of forces and energy through-out
the space. The field space in the virtual world occupies a
1-meter cube with Cartesian axes display for easy refer-ence.
Interaction is achieved via menus and a virtual hand.
Menus are attached to a user's wrist so they can be removed
from the visual field but are
charge configuration has been placed, the force on a positive
test can be attached to the vir-tual hand so that students can
see the line( s) of force extend-ing through that point; or the
tip of the virtual hand can be-come a "potential" meter for
exploration of the distribution of potential in the world.
Gaussian surfaces can also be placed using the virtual hand.
Electric field lines, potentials, surfaces of equipotential, and
lines of electric flux through surfaces can be instantiated
using the menus, and then ob-served and controlled interac-tively.
28
28
Page 29
30
15
©1998 Institute for Defense Analyses
Figure 3. Vari House
Project Goal: To integrate archaeological data with advanced computer graphics techniques to support educa-tion, data analysis, and the preservation of cultural heritage information.
Focus of Current Work: Field testing.
Vari House Description: Two linked virtual worlds show the Vari site as excavated and the Vari house as reconstructed by archaeologists. Links are also provided to supplemental information about Greece and the
Vari region. The reconstructed house shows both the exterior and interior of the building and a floor plan is included. Students are guided in the exploration of the worlds by answering questions that help develop crit-ical
thinking skills.
29
29
Page 30
31
16
©1998 Institute for Defense Analyses
3.1 Student Use of Pre-Developed Virtual Worlds
In an effort to start by demonstrating the range of applications in this category, Figure 2 and
Figure 3 and provide descriptive overviews of two example applications. The selection of these
particular applications over any others is not intended to imply "best-in-class," though both are
examples of excellent work. The first application, MaxwellWorld, provides an example of an
application developed as a research tool that runs on high-end machines and provides a fully
immersive and multisensory interface. The second application, Vari House, is an example of a
professionally-developed desktop product that was designed for practical use and is supported
by extensive teaching materials.
Though providing insight into the types of virtual worlds being developed, these two exam-ples
are a small subset of current applications. Table 2 3 provides summary details for all the
pre-developed VR applications that are considered here. These applications vary greatly in the
educational topics supported, their pedagogical approaches, the support they provide for stu-dents
with special needs, and the type of hardware and software platforms required. These top-ics
are all discussed later. First, however, the different ways in which pre-developed
applications are being used is discussed. .
3.1.1 Type of Use
Pre-developed VR applications can be
broadly distinguished as those developed for
practical classroom and exhibition use, and
those intended primarily for use as research
vehicles. The applications are nearly equally
split between these two groups, as shown in
Table 3. Some applications in research
category may eventually end up in practical
use, but this result is not seen as the major
purpose for their development.
For those applications reported here, the
first practical classroom use occurred in 1993.
The two applications that came into use that
year were both designed for learning-disabled
students. Two years later, 1995 saw four more
applications come into practical use and, since
then there has been a small but steady increase in practical usage. By the end of 1997, a total
3 In this and all subsequent tables, italics are used to denote planned activities.
Table 3. Classification of Applications
Practical Classroom and Exhibition Use Virtual Egyptian Temple Gebel Barkal: Temple B700
Virtual Bicycle Vari House Virtual Pompeii Science Education World
Classical Buildings a Energy Conservation O'Connor World Vicher (I and II)
Updike World Safety World Nursing World Map Interpretation World
Milling Machine 3D Map World VESL Makaton World
Virtual Biplane Life Skills World
Research Vehicles Cell Biology Greek Villa
Virtual Gorilla Exhibit NewtonWorld CDS MaxwellWorld
Room World PaulingWorld Great Pyramid a
a. Part of Virtus Corporation Archeological Gallery.
AVATAR House 3D Letter World Phase World
Street World Atom World Object World Global Change
Crossing Streets Zengo Sayu LAKE VRRV Hors d'Oeuvre
Spatial Relations World
30
30
Page 31
32
17
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-Developed
VR
Applications
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Display
Usage
User
Organization
Date
of
Use
C a r n e g i e M e l l o n U n i v e r s i t y , S I M L A B , P i t t s b u r g , P A
Virtual Egyptian Templ e
Walkthrough
of
an
Egyptian
temple
modeled
on
surviving
remains
of
2
ancient
temples.
User
can
walk
through
the
temple
compound
where
wall
paintings
come
alive
and
a
priest
avatar
serves
as
a
guide.
Supported
by
25
hours
of
reference
material
on
CD-ROM.
Understanding
of
Greek
culture.
Middle
-
High School
De sktop
Practical
use
Commercially
available,
1998
(productized
by
Mind
Experience
Technologies,
Campbell,
CA)
Virtual Bicycle
Using
a
bicycle
mounted
on
a
motion
platform,
user
controls
speed
of
bicycling
through
conditions
such
as
varying
road
surface.
The
user
encounters
several
accident
scenarios.
A
skill
monitor
scores
performance
and,
for
low
skill
levels,
an
agent
offers
information
and
guidance.
Train
and
rehabilitate young bicycle users.
5-15 years
Pro j ect i o n sc reen s (3 )
Practical
use
Schools
throughout USA
2000 onward
Virtual Pompeii
Walkthrough
of
a
replication
of
the
Theatre
Complex
of
Pompeii
prior
to
the
eruption
of
Mt.
Vesuvius
(A.
D.
79),
showing
historical
details
taken
from
literary
sources
such
as
Pliny
(the
elder)
and
historical
diggings.
The
world
includes
the
Large
Theater,
the
Temple
of
Hercules,
and
the
Triangular
Forum.
A
re-enactment
of
a
typical
theatrical
performance
is
provided.
Provide
insight
into
Pompeiian
life
and
culture.
All
ages
A l l f o r m s o f d i s p l a y
Exhibition
SIGGRAPH
'95
Fall
1995
Exhibition
DeYoung
Museum
(San
Francisco,
CA)
Fall
1995
Exhibition
Smithsonian
Castle
(Washington,
DC)
Summer 1997
Co rresp o n d e n c e S c h ool ( N Z )
Classical Buildings
Walkthrough
of
a
depiction
of
ancient
Greek
and
Roman
buildings,
and
other
large-scale
artifacts
such
as
roads
and
aqueducts.
Course:
Classical Studies GL 200.
High school, college
Desk t o p
Practical (optional
part
of
course)
Correspondence School, Wellington, NZ
Mid-1997 onward
31
31
Page 32
33
18
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Display
Usage
User
Organization
Date
of
Use
E R G E n g i n e e r i n g , I n c .
Cell Biology
Room
where
a
sick
child
asks
for
cells
(neuron,
muscle
cell,
intestinal
cell)
so
that
he
can
think,
move,
and
eat.
The
user
builds
these
cells
from
different
types
of
organells.
He
can
use
help
books
associated
with
each
self-explaining
object
or
proceed
by
experimentation.
Additional
explanations
are
given
as
correct/
incorrect
organells
are
added
to
a
cell.
Animations
show
completed
cell
function.
"Hands-on" experience
with
the
principles
of
human
cell
biology.
All
ages
HM D (stereo )
Evaluation
of
impact
of immersion
and
interactivity
The
Computer Museum, Boston, MA
March-June 1995
Exhibition
The
Computer Museum, Boston, MA
July
1995
Exhibition
TEPIA
Center,
Tokyo,
Japan
October-November 1995
Geor gi a I n st it u t e o f T e c hno l o g y , Grap hi cs V i sua l i z a t i o n &
Usab i l ity (GVU) Cen t er ,
Virtual Gorilla Exhibit
User
explores
a
depiction
of
the
Gorilla
Exhibit
at
Zoo
Atlanta,
including
4
viewable
gorilla
habitats
and
the
Gorillas
on
the
Cameroon
Interpretive
Center.
By
assuming
the
role
of
a
member
of
a
virtual
gorilla
family
(silverback,
adult
male
and
female,
juvenile,
or
infant),
the
user
can
test
behaviors
and
elicit
responses
reflecting
his
position
in
the
family
hierarchy.
(Extensions
will
address
animal
husbandry
and
conservation.)
Instruction
in
(1)
gorilla
behaviors
and
social
interactions
in
a
family
group;
and
(2)
zoo
habitat
design
and
layout
issues.
K-12
HM D (m on o)
Formative evaluation (conducted
at
Zoo
Atlanta)
Midway
and
Slaton
Elementary Schools, Westminster School, Trickum Middle School, Fayetteville High School
Spring 1996
Exhibition
SIGGRAPH
'96,
New
Orleans,
GA
August 1996
Exhibition
Zoo
Atlanta,
Georgia
Fall
1997 onward
Conceptual Design Space (CDS)
CDS
allows
students
to
create,
walkthrough,
and
modify
architectural
designs
while
getting
immediate
feedback
on
the
impact
of
changes
to
the
architectural
space.
These
designs
can
be
imported
from
a
CAD
system
or
developed
from
inside
the
VE.
Design
tools
support,
for
example,
creating
simple
shapes;
modifying
and
grouping
them;
and
applying
simple
geometric
transformations.
A
transformation
widget
supports
large-scale
object
manipulation
while
detailed
manipulations
use
menus.
ARCH
4613:
Advanced
Design
Studio
VI.
College students, architects
HM D (mon o)
Comparative evaluation of immersive design and user interface tools
Georgia
Institute
of
Technology
Spring 1995
Practical
use
Georgia
Institute
of
Technology
Winter/ spring 1995
Lord,
Aeck,
and
Sargent,
Inc.,
Atlanta,
Georgia
Winter/ spring 1995
32
32
Page 33
34
19
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Disp lay
Usage
User
Organization
Date
of
Use
Hay w oo d C o m m un i t y C o l l e ge
O'Conner World
Walkthrough
of
scene
from
O'Connell's
A
Good
Man
is
Hard
to
Find
to
identify
objects
or
conditions
that
do
not
belong,
are
important
to
the
scene,
or
missing.
ENG
254:
Major
American
Writers,
English
277:
Exploring
Literature
Through
Virtual
Reality.
College students
HM D (stereo )
Practical
use
Haywood Community
College
Fall 1995 onward
Updike
World
Walkthrough
of
scene
from
Updike's
A& P
to
identify
objects
or
conditions
that
do
not
belong,
are
important
to
the
scene,
or
missing.
English
277:
Exploring
Literature
Through
Virtual
Reality.
College students
Pro j ect i o n s c reen w/ pa ssive g l a sses
Practical
use
Haywood Community
College
Spring 1996 onward
Nursing World
Walkthrough
of
"typical"
residential
house
with
furnishings.
The
student
is
tasked
to
identify
furnishings
and
other
components
of
the
home
environment
that
could
be
obstacles
to
patient
rehabilitation.
For
use
in
several
courses
in
the
Nursing Curriculum.
College students
HM D (ste reo)
Practical
use
Haywood Community
College
Spring 1997 onward
Milling Machine
Simulation
of
the
actions
of
milling
machine
cutting
parts
from
metal
stock.
The
simulation
is
guided
by
a
Computer-Numerical
Control
(CNC)
student
program.
The
student
may
change
viewpoints
during
the
process
and
view
the
final
product.
MEC
207:
Introduction to CNC Machines.
College students
P r oj ec t i o n scre en w/ p a ssiv e glasse s
Practical
us e
Haywood Community
College
Fall 1997 onward
In t e rface T ech no l ogi e s C o r p o r a t i o n
Virtual
Envi-
ronment Science Laboratory (VESL)
Representation
of
the
solar
system
where
students
can
visit
any
planet,
and
the
sun,
manipulating
variables
such
as
mass
and
velocity.
Game-
like,
experiential
paradigm
that
teaches
critical
concepts.
As
a
user
learns
about
features
in
the
world,
he
gains
virtual
tools
(e.
g.,
that
shrink
or
expand
space,
and
slow
down
or
speed
up
time)
that
permit
further
explorations.
Games
also
allow
the
user
to
conduct
experiments
where
observation
of
results
facilitates
learning
of
linear
and
orbital
mechanics.
AAAS's
Benchmarks
for
Science
Literacy,
The
Physical
Setting:
The
Universe,
The
Structure
of
Matter,
Energy
Transformations,
Motion;
NRC,
National
Science
Education
Standards,
Content
Standards
9-12:
Science
as
Inquiry,
Physical
Science,
Earth
and
Space
Science.
Grad es 9-1 2 , i n cl ud i n g st u d en t s w i t h p h y s i c a l d i s a b i l i t i e s
HM D (s t e reo)
Subjective usability evaluation
Students,
teachers,
assistive
device
specialists,
human
factors
engineers, occupational therapists
1994-1996
Subjective effectiveness evaluation
Students
and
teachers
from
schools
in
Santa
Cruz
County,
CA
1994-1996
Effectiveness evaluation
Students
from
schools
in
Santa
Cruz
County,
CA
1994-1996
Practical
use
Several
schools
in
CA,
1997
Commercially
available,
1997
33
33
Page 34
35
20
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Disp l a y
Usage
User
Organization
Date
of
Use
J a m e s C o o k U n i v e r s i t y , S c ho ol of E duc at i on ( N Z )
Room World
Scene
of
a
square
room
without
doors
or
windows,
containing
27
color
objects
such
as
dining
and
lounge
furniture.
N/ A
Grade
6
Desk t o p
Evaluation
of
impact
of immersion
on
recall
Townsville
primary
schools,
Australia
Summer 1995
Great Pyramid
Depiction
of
the
Great
Pyramid,
the
inside
of
which
is
hollow
showing
various
passageways
and
chambers.
Appreciation
of
pyramid
structure.
Grade
7
Des k top
Subjective effectiveness evaluation
Townsville
primary
schools,
Australia
Summer 1995
3D
Letter World
Colored
letters
appearing
in
a
giant
alphabet
ring
suspended
in
the
air
against
a
background
of
blue
sky.
Letter
recognition.
Grade
2
Deskto p
Effectiveness evaluation (unable to be completed)
Townsville
primary
schools,
Australia
Late
1995
L earn i ng S i t e s, In c.
Gebel Barkal: Templ
e
B700, Nubia
Walkthrough
of
recreation
of
the
ancient
Nubian
temple
B700
(ca.
650-640
BCE)
and
its
environs.
Objects
are
linked
to
text,
image,
and
narrative
databases
containing
interpretive
and
diactic
information,
19th
century
drawings,
and
excavation
photos.
Understanding
of
Gebel
Barkal
site.
Graduate students, archeolo-gists
HM D (stereo )
Practical
use
VR
Lab,
Ministry
of
Education,
Republic
of
Egypt
Spring 1996
Exhibition
Museum
of
Fine
Arts,
Boston,
MA
Spring 1996
Practical
use
Commercially
available
since
1996
Vari
House, Greece
Walkthrough
of
linked
virtual
worlds
depicting
an
excavated
and
reconstructed
Hellenistic
farmhouse
(ca.
325-275
BCE)
in
Attica
and
its
environs.
Hyperlinked
files
contain
text
and
images
about
the
site,
and
problem-solving
tasks
for
teaching
critical
thinking,
planning,
and
specific
competency
objectives.
North
Carolina Standard Course
of
Study
Competency Goals and Objectives for World History, Culture, and Geography; Society for American Archaeology guidelines for teaching archaeology in public schools.
Grades 9-12
Desk top
Field
testing
10
schools
in
the
U. S.
and
Canada
Spring 1997 ongoing
Practical
use
Limited
prototype
available
on
Web
since
December
1996
Commercially
available
by
fall
1997
NAS A/ L e wis Re search Cent er
Virtual Biplane
Biplane
with
joystick
control
and
functional
instruments
(airspeed,
altimeter,
compass,
fuel
remaining).
Participants
can
look
forward
to
see
terrain
ahead,
look
out
of
the
sides
of
the
plane
to
see
terrain
below,
or
back
to
see
terrain
travelled
over.
Appreciation
of
flight.
Grades 9-12
HM D (stere o)
Practical
Various
Spring 1997 onward
34
34
Page 35
36
21
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Display
Usage
User
Organization
Date
of
Use
No rth Caro lin a S t ate Un i v . Comp . S c i e n ce & Com p . E n g.
De pts., Un iv . of North Caro lina M e d i c a l S c h o o l
Street World
Simple
street
scene
with
buildings
and
a
sidewalk,
one
car
and
one
stop
sign.
N/
A
Autistic children, ages 6-12
HM D (stere o)
Evaluation
of
whether
VR
can
benefit autistic children
Chapel
Hill
and
Wake
County
public
schools
Fall
1994
Object World
Simple
worlds
including
classroom
objects
and
features
(such
as
color).
Object
identification.
Autistic children, ages 6-12
HM D (stereo )
Comparative evaluation of VR and conventional teaching methods
Chapel
Hill
and
Wake
County
public
schools
June
1997 ongoing
O r e g o n S t a t e U n i v e r s i t y , S c h o o l o f E d u c a t i o n
Science Education World
Travelling
in
a
virtual
"pod,"
students
examine
plant
(the
Folanum
tuberosum)
structure,
anatomy,
and
physiology.
Two
levels
of
detail
are
shown,
including
mechanisms
used
to
transport
fluids
from
roots
to
leaves.
Speech
and
textual
information
are
available
on
request.
Conceptual understanding
of
nature
and experimental
method.
High school
HM D (stereo )
Pilot
study
Eugene
4-J
school
district
Spring 1997 ongoing
Spatial Relations Wor l ds
Series
of
3
worlds:
an
office-like
room
where
furniture
can
be
moved,
a
racquetball
court
where
objects
can
be
moved,
and
a
outdoor
environment
where
target
objects
are
to
be
distinguished
between
their
transformed
or
mirror
model.
Spatial
problem solving abilities.
Ages 8-11
H M D (stereo )
Evaluation
of
impact
of immersion
Elementary
summer
school
program
in
Novato,
CA
1992
S h e f f i e l d H a l l a m Un ivers i ty (UK)
Greek
Villa
Depiction
of
an
ancient
Greek
domestic
residence
with
8
rooms,
2
floors,
a
courtyard,
and
many
objects.
Sound
effects
contribute
to
the
atmosphere,
windows
and
doors
open
and
close,
and
some
animation
is
used.
National
Curriculum Key Stage 2 History Unit, "Ancient Greece."
Ages 8-10
Des k top
Effectiveness evaluation
St.
Patrick's Catholic School, Sheffield, UK
Spring 1996
Effectiveness evaluation
Firs
Hill
Junior
School,
Sheffield, UK
TBD
Practical
use
Commercially
available
TBD
35
35
Page 36
37
22
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/ Learning Objectives Supported
Intended Audience
Display
Usage
User
Organization
Date
of
Use
U n i v e r s i t y o f H o u s t o n , G e o r g e M a s o n U n i v e r s i t y , &
NAS A Jo hn son S p a ce Cen t e r
NewtonWorld
Open
corridor
activity
area
where
2
balls
of
various
masses
move
and
rebound,
and
moveable
cameras
record
events.
Signs
indicate
the
presence
of
absence
of
gravity
and
friction,
columns
and
floor
markings
support
judgements
of
distance
and
speed,
potential
energy
is
portrayed
by
tactile
and
visual
cues,
and
velocity
through
auditory
and
visual
cues.
Parameters
such
as
gravity
can
be
changed
via
a
control
panel.
The
world
supports
guided
inquiry
into
the
kinematics
and
dynamics
of
1D
motion,
with
scaffolding
to
advance
the
user
from
basic
to
advanced
activities.
A
user
is
tasked
to
predict
forthcoming
events,
experience
them,
and
explain
the
experience.
Exploration
of
Newton's
Laws
of
Motions
as
well
as
conservation
of
both
kinetic energy and linear momentum.
Grades 5-10
HM D (stere o)
Formative usability evaluation
High
school
in
Houston,
TX
Summer 1994
Subjective effectiveness evaluation
American
Assoc.
of
Physics
Teachers
'94
Summer
Meeting
Summer 1994
Evaluation
of
multisensory interface
Clear
Creek,
Clear
Lake
High
Schools,
Houston,
TX
December 1994 -May 1995
Evaluation
of
content
and
lesson structure
Deer
Park Elementary
School,
Rocky
Run Intermediate
School,
VA
Spring 1997 ongoing
Evaluation
of
age
group,
ego
vs.
exocentric viewpoints, multisensory interface
Elementary
and
Junior
High
Schools
in
Fairfax,
VA
Fall
1997
Field
testing
TBD
1998
MaxwellWorld
Boxed
area
where
a
user
positions
a
charge
configuration
and
uses
a
positive
test
charge,
electric
potential
meter,
or
electric
charge
line
to
query
the
potential
at
selected
points,
and
drops
test
charges
into
the
world.
Resultant
forces,
electric
field
lines,
potentials,
equipotential
surfaces,
and
lines
of
electric
flux
are
displayed.
Flux
of
an
electric
field
through
Gaussian
surfaces
can
be
visually
measured.
Exploration
of
electrostatics, leading up to
the
concepts
of
electric
field
(force),
electric
potential
(energy),
superposition,
and
Gauss's
Law.
Grades 5-10
H M D (stereo )
Formative usability, learnability, effectiveness evaluation
Robinson
High
School,
Fairfax,
VA
and
University
of
Maryland
Summer 1995
Comparative effectiveness evaluation
Robinson
High
School,
Fairfax,
VA
Spring 1996
Evaluation
of
effect
of
frame
of
reference
George
Mason
University
Fall
1997
Field
testing
TBD
Late
1998
36
36
Page 37
38
23
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/ Learning Objectives
Supported
Intended Audience
D i s p l a y
Usage
User Organization
Date
of
Use
Un i v ersity o f Ho usto n, Geor g e M a son Univ ersity , &
NAS A JS C (con tinue d)
PaulingWorld
Flythrough
of
a
molecular
structure
represented
in
various
forms.
(Being
extended
to
include
display
of
equipotential
surfaces
and
ability
to
interactively
explore
effects
of
atom
removal
and
substitution
through
direct
links
to
molecular
modeling
applications.)
Learn
about
probability
density,
wave
functions, nuclear charge,
and
atomic
orbital
shapes
for
single
atoms,
bonding
of
2
atoms,
differences
between
bonded
structures, determinants of bonding angles and length.
High
school
HM D (stereo )
Formative evaluations
TBD
Mid-1998
Univ ersit y of I l l i n o i s , N C S A ,
and T R I
Crossing Streets
Set
of
3
worlds
each
showing
a
different
street
intersection
(one
is
modeled
on
an
actual
street
near
2
schools.
Each
world
includes
traffic
patterns
with
moving
cars.
Students
are
tasked
to
cross
the
street
safely
in
a
range
of
situations.
Transportation-
related
skills.
K-12
CA VE
Evaluation
of
learning transfer
Various
public
schools
in
Urbana
and
Champaign
Fall
1996 ongoing
U n i v e r s i t y o f I o a n n i n a , D e p t . o f
P r i m a r y E d u c a t i o n (Greec e)
virtuaL Approach
to
the
Kernel
of Eutrophication
(LAKE)
Series
of
linked
virtual
worlds
demonstrating
the
process
of
lake
eutrophication.
The
user
can
manipulate
the
system
behavior
to
identify
key
factors
and
their
relationships.
Fifteen
predefined
starting
viewpoints
are
provided.
Basic
concepts
in
eutrophication.
College (education) students
Des k top
Evaluation
of
usability
and
comparison
of
navigation devices
University
of
Ioannina
1995-1996
Un i v ers i t y o f M i ch i g an , Dept . of C h e m i cal
E n g i n e e r i n g
Vicher (I and II)
In
each
of
two
Vicher
worlds,
students
are
guided
by
a
set
of
questions
and
a
list
of
things
to
see
and
do
to
learn
about
industrial
responses
to
catalyst
decay
and
non-
isothermal
effects
in
reactor
design.
Each
Welcome
Center
provides
educational
input
from
books
and
TVs.
Students
can
visit
6
different
reactor
rooms,
defining
operating
conditions
and
"operating"
all
the
CHE
344:
Reaction
Engineering
and
Design.
College students
HM D (stereo)
Formative evaluation
(I)
University
of
Michigan
Early 1995
Practical
University
of
Michigan
Spring 1995 onward
Formative evaluation
(II)
University
of
Michigan
Spring 1997 ongoing
37
37
Page 38
39
24
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
D i s p l a y
Usage
User
Organization
Date
of
Use
U n i v e r s i t y o f M i c h i g a n , D e p t . o f C h e m i c a l E n g i n e e r i n g
(co n ti n u ed )
Vicher (I and II) (continued)
equipment
to
observe
effects
of,
say,
changing
feed
rates
on
reaction
conditions.
Three
microscopic
exploration
areas
are
also
available.
Off-
site
beta
testing
University
of
North
Carolina,
Michigan
Tech,
Georgia
Tech
July 1996 ongoing
Publicly
available
(distributed
by
CACHE
Corporation,
Austin,
TX)
Fall 1997
Safety
World
Students
navigate
through
a
pilot
plant
(for
production
of
polyether
polyols
from
materials
such
as
ethylene
oxide)
to
learn
how
to
perform
safety
and
hazard
evaluations.
Safety
features
include
a
sprinkler
system,
pressure
relief
systems,
blowout
panels,
and
emergency
showers.
Plant
environs
include
a
hospital
and
a
river
providing
city
drinking
water.
Objects
provide
hypertext
information
and
photos
of
the
actual
plant.
Material
safety
data
sheets
are
also
available.
CHE
486:
Chemical
Process
Simulation and Design I.
College students
HM D (stereo )
Formative evaluation
(I)
University
of
Michigan
Fall 1995
Practical
University
of
Michigan
Fall 1995 onward
Formative evaluation (II)
University
of
Michigan
Fall 1996 ongoing
Publicly
available
(distributed
by
CACHE
Corporation,
Austin,
TX))
Fall 1997
Unive r sit y of M i sso uri, Dept . of Geo g r a p h y
Map Interpretation World
Flythrough
of
a
central
Missouri
landscape
based
on
geological
survey
data.
Vertical
scale
adjusts
by
factors
of
5
and
10
to
support
understanding
of
the
effects
of
vertical
exaggeration.
(Plan
to
address
addition
map
interpretation
issues).
Geography
137:
The
Language
of
Maps.
College students
Glass e s ( p a s s i v e , s h u t t e r )
Practical
University
of
Missouri
Fall 1996 onward
3D
Map
World
GIS-based
VE
that
shows
various
maps,
in
layers,
of
a
3D
terrain
map
that
can
be
used
to
investigate
environmental
problems.
Geography
137:
The
Language
of
Maps.
College students
Glass e s ( p a s s i v e , s h u t t e r )
Practical
University
of
Missouri
Spring 1998 onward
38
38
Page 39
40
25
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
D i s p l a y
Usage
User
Organization
Date
of
Use
U n i v e r s i t y o f N o t t i n g h a m , Dep t . Ma nu f act ur i n g En gi n eer i n g & O p er at i o ns Ma nag e me nt ,
VIRAR T Gro u p (UK)
Makaton
World
A
set
of
6
worlds,
each
of
which
depicts
a
"warehouse"
of
3D
examples
of
a
Makaton
symbol.
When
a
user
enters
the
warehouse,
an
animated
sequence
is
triggered
to
show
the
dynamic
hand
sign
for
the
symbol.
The
user
can
view
objects
from
all
angles
and
interact
with
them
to
see
their
function.
After
a
limited
number
of
new
symbols
have
been
encountered,
the
user's
ability
to
correctly
identify
their
meaning
is
tested
in
a
"reward
warehouse."
Teaches
Makaton symbols and associated sign language.
Severely learning disabled children
De sktop
Pilot
study
Shepherd
School,
Nottingham,
UK
Fall 1992
Effectiveness and usability evaluation
Shepherd
School,
Nottingham,
UK
Summer 1994
Practical
use
Shepherd
School,
Nottingham,
UK
Summer 1993 onward
Commercially
available
from
ROMPA,
Chesterfield,
UK
since
summer
1995
Life
Skills Wor l ds
Several
worlds
representing
a
virtual
city,
house,
supermarket,
skiing,
kitchen,
high
street,
town,
and
bowling
green.
Some
teach
life
skills;
in
the
virtual
supermarket,
for
example,
the
user
pushes
around
a
trolley,
selects
goods,
and
takes
them
to
the
checkout.
Others
provide
experiences;
in
the
city,
for
example,
the
user
can
experience
driving
around
encountering
traffic
lights,
road
works,
pedestrian
crossing,
one-ways
traffic
systems,
and
other
cars.
Supports
development of self-directed activity.
Severely learning disabled children
Desk t o p
Evaluation
of
skill
transfer, promotion of self-directed activity (Virtual Supermarket)
Shepherd
School,
Nottingham,
UK
Early 1996
Comparative evaluation of effectiveness (Virtual House)
Shepherd
School,
Nottingham,
UK
Fall 1996 ongoing
Practical
use
Shepherd
School,
Nottingham,
UK
Summer 1993 onward
Commercially
available
from
ROMPA,
Chesterfield,
UK
since
summer
1995
Evaluation
of
effectiveness
Shepherd
School,
Nottingham,
UK
Spring 1997 ongoing
39
39
Page 40
41
26
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/ Learning Objectives
Supported
Intended Audience
D i s p l a y
Usage
User
Organization
Date
of
Use
U n i v e r s i t y o f N o t t i n g h a m , VIRAR T Gro u p
(co n ti n u e d )
AVATAR House
Exploration
of
a
house
(kitchen,
bathroom,
living
room)
containing
recognizable
objects
that
emit
sounds
when
activated.
Rooms
are
designed
to
focus
the
user
onto
certain
objects
and
activities
to
provide
a
means
of
practising
skills
and
to
link
with
real-
world
activities.
Supports
development of concentration skills, improved attention span, self-confidence.
Autistic children
D e s k t o p
Evaluation
of
effectiveness
Shepherd
School,
Nottingham,
UK
1996 ongoing
U n i v e r s i t y o f P o rtsmo u th (UK)
Energy Conservation
The
virtual
world
provides
students
with
a
subsection
of
a
city,
composed
of
different
types
of
buildings.
Some
buildings
have
associated
energy
ratings
and
students
must
perform
energy
analyses
to
provide
energy
ratings
for
other
buildings.
Students
can
look
at
energy
data
to
evaluate
the
cost
of
improvements
under
budget/
time
restrictions.
National
Curriculum
in
Physics,
Key
Stages
3
and
4.
Ages 13-16
De sktop
Practical
St.
Luke's
School,
Portsmouth,
UK
Fall 1997
Un i v ersity of W a sh i n gto n , HIT L
Phase
World
Presentation
of
3D
graphs
(surfaces)
showing
relationships
among
volume,
pressure,
and
temperature
when
changes
in
state
occur.
The
user
can
fly
over
surface
and
at
interesting
points
zoom
in
to
observe
what
happens
at
the
molecular
level.
Understanding,
at
the
molecular
level,
of
what
happens
when
matter
changes
from
solid
to
liquid
to
gas,
and
relationships among pressure, temperature, volume.
Grade
11
HM D (st e reo)
Comparative effectiveness evaluation
Kennedy
High
School,
Seattle,
WA
January 1995 (analysis ongoing)
Atom
World
Open
area
with
sources
for
subatomic
elements,
metered
scale
for
changing
electron
charges,
atom
assembly
area,
and
notice
board
identifying
atomic
element
to
be
constructed.
Electron
shells
are
depicted
as
spheres
that
change
color
when
an
element
is
completed.
Review
of
basic
atomic
and
molecular
structures.
Grade
11
HM D (stereo )
Evaluation
of
impact
of immersion
and
interactivity
Garfield
High
School,
Seattle,
WA
1994-1995
VRRV
use
VRRV/ Nebraska participants
1996-1997
40
40
Page 41
42
27
©1998 Institute for Defense Analyses
Table
2.
Characteristics
and
Usage
of
Pre-
Developed
VR
Applications
(continued)
Developer
Name
of Application
Description
Courses/
Learning
Objectives
Supported
Intended Audience
Disp l a y
Usage
User
Organization
Date
of
Use
U n i v e r s i t y o f W a s h i n g t o n , H I T L (co n t i n u ed )
Zengo
Sayu
Japanese-style
tatami
room
containing
a
table,
chair,
a
number
of
spheres,
and
a
box.
Provides
a
whole
language
approach
to
teaching
some
Japanese
nouns,
verbs,
and
prepositions.
Teaching
Japanese.
College students
H M D (stereo )
Comparative effectiveness evaluation
University
of
Washington
January-March 1996
Practical
use
Windows
NT
version
commercially
available,
1997
(productized
by
FirstHand,
Inc.)
Global Change
Depiction
of
the
Seattle
landscape
and
Puget
Sound
from
space,
an
aerial
view,
and
a
ground
level
view.
Inquiry-
based
scenario
where
user
assumes
role
of
a
alien
visiting
a
world
with
problems.
By
setting
levels
of
industry,
cars,
and
forestation,
and
taking
measurements
as
moving
forward
or
back
in
time,
the
user
can
see
the
effect
of
these
factors
on
the
environment.
Controls
consist
of
wheels
to
set
the
level
of
the
factors
that
impact
global
change,
and
a
time
dial
to
set
the
year.
A
gauge
shows
current
temperature.
Understanding
the
basic
relationships among the causes and effects of global change.
Grades 7-10
H M D (stereo )
Effectiveness evaluation
VRRV
Entrée
participants
Fall
1996 (analysis ongoing)
Practical
Available
on
Internet
as
part
of
the
Teacher/
Pathfinder program
1996 onward
Proof
of
concept
for
multiuser, distributed
use
Childrens'
Hospital
and
local
school
January 1997 ongoing
VRRV
Hors
d'Oeuvre
A
selection
of
commercially
available
or
researcher-developed
virtual
worlds
that
demonstrate
VR
technology
and
applications.
N/ A
Grades 4-12
HM D (stereo )
Evaluation
of
enjoyment, ease of navigation
Various
school
in
Washington
and
Nebraska
1994-1997
41
41
Page 42
43
28
©1998 Institute for Defense Analyses
of 17 applications are expected to become part of regular classroom activities. In the case of
exhibitions, two applications were made available to the public for short periods in 1995, each
at two different venues. Depending on completion of construction at Zoo Atlanta in Georgia,
and on the necessary funding for equipment, the Virtual Gorilla Exhibit is expected to become
a permanent exhibit at Zoo Atlanta starting in fall 1997.
Most practical classroom applications have been integrated into specific courses or
curricula. While some have been developed to meet a teacher's needs for a particular class and
are not expected to be used elsewhere, others are intended for widespread use. Different
mechanisms are being used to make such applications publicly available. Practical use
applications that are already marketed as commercial products are VESL, Gebel Barkal:
Temple B700, Vari House, Makaton World, and Life Skills World. Learning Sites, Inc., has
taken a useful step in its marketing of Vari House by making a prototype of the application, and
sample supporting documents, available for free downloading on its Web site. This marketing
step gives potential buyers an opportunity to experience (part of) the Vari House world before
deciding whether to purchase a license that entitles them to complete copies of all materials.
The University of Michigan is making its Vicher and Safety worlds available at a minimum
cost that covers only the price of the materials used and associated shipping costs. Two
applications, Greek Villa and Zengo Sayu, that were initially developed as research vehicles
also are expected to become commercially available. The three ScienceSpace worlds
(NewtonWorld, MaxwellWorld, and PaulingWorld) are also expected to see practical use,
although how these worlds will become publicly available is not known at present.
42
42
Page 43
44
29
©1998 Institute for Defense Analyses
3.1.2 Educational Subjects Supported
Among them, the current set of
pre-developed educational VR
applications provide support for
students from elementary school to
graduate school. Discounting those
few applications that are intended for
use by all age groups, applications
are fairly equally split between those
designed for elementary and middle
school levels, those for high school
students, and those for college
students (undergraduate and
graduate).
The range of educational subjects
covered is quite broad, showing a
fairly equal split between the arts and
sciences. However, as can be seen in Table 4, there is a predominant subject in each of these
fields. For the arts, over one third of the applications address ancient civilizations, looking at
either just ancient structures, or also considering cultural concerns. In the case of the sciences,
the most popular subject is physics, followed by environmental sciences. The applications
designed for science learning show the greatest diversity of virtual worlds and, usually, the
most complex worlds. The virtual worlds used in the arts area tend to consist of simple
buildings and objects. The science-related applications show the most evidence of explicit
pedagogical support.
To support achieving their educational
goals with respect to specific subject
areas, a few of the applications were
developed to meet specific state
curriculum objectives or the requirements
of certain organizations, as indicated in
Table 5.
Reflecting the extent to which a pedagogy
is embodied in the virtual world, these
applications also differ widely in the
extent of teacher support that is provided.
Most notable are Vari House, which is accompanied by an 80-page Teacher Guidebook, 25-
page Student Workbook, background information sheets, and a special electronic mail list for
Table 4. Educational Subjects
Aeronautics Virtual Biplane Animal behaviors Virtual Gorilla Exhibit
Architectural design CDS Ancient Structures Great Pyramid, Classical Buildings
Ancient Structures/ Cultures Virtual Egypt, Virtual Pompeii, Gebel Barkal: Temple B700, Vari House,
Greek Villa Biology Cell Biology
Chemistry PaulingWorld Chemical Engineering Vicher (I and II), Safety World
Environmental Science Global Change, Eutrophication, Energy Conservation
Geography Map Interpretation World Industrial Arts Milling Machine
Language Makaton World, Zengo Sayu Letter Recognition 3D Letter World
Literature O'Conner World, Updike World Nature, Experiential Method Science Education World
Object Identification Object World Physics VESL, NewtonWorld, MaxwellWorld,
Phase World, Atom World Real Life Skills Virtual Bicycle, Crossing Streets, Street
World, Life Skills Worlds, AVATAR House
Rehabilitation Nursing World Spatial Relations Special Relationships World
Table 5. State and Other Learning Objectives
VESL -AAAS, Benchmarks for Science Literacy, The Physical Setting: The Universe, The Structure
of Matter, Energy Transformations, Motion. -NRC, National Science Education Standards,
Content Standards 9-12: Science, Inquiry, Physical Science, Earth and Space Science.
Vari House -North Carolina Standard Course of Study Competency Goals and Objectives for World
History, Culture, and Geography. -Society for American Archeology guidelines
for teaching archeology in public schools.
Greek Villa -(UK) National Curriculum Key Stage 2 History Unit, "Ancient Greece."
Energy Conservation -(UK) National Curriculum in Physics, Key Stages 3 and 4
43
43
Page 44
45
30
©1998 Institute for Defense Analyses
users; VESL, which is supported by a 3-day teacher workshop with accompanying manuals and
curricula material; and Virtual Biplane which, as part of the MAEL program, is supported by
pre-and post-visit activities for both teachers and students at the schools visited.
The majority of applications intended for practical classroom use are, of course, accessed
in the classroom. The exception is the Classical Buildings application used by the
Correspondence School, a distance learning institution in New Zealand. The learning institute
is incorporating existing virtual worlds of Greek and Roman buildings into its Classical Studies
course by mailing computer discs with these worlds, and the Virtus Player freeware needed to
run them, to students. The students can then walkthrough the worlds on their own PCs or Macs
to gain information appropriate to specific portions in the course.
Overall, the majority of research has been conducted with students using equipment
routinely available in the classroom. Many of the HITL's research efforts, however, have
involved the researchers taking special equipment to schools for, usually, short periods of time.
Alternatively, the evaluations of the ScienceSpace worlds (that is, NewtonWorld,
MaxwellWorld, and Pauling World) are being conducted by bringing students to the equipment
at the University of Houston and George Mason University.
3.1.3 Pedagogical Support
Just over a third of the applications rely on minimal interaction and simple walkthroughs
of a virtual world to support their educational objectives. These applications are identified in
Table 6. This approach can be very effective despite its lack of any specific embedded
pedagogy.
For example, the University of Michigan, Department of
Chemical Engineering's Safety World allows students to navigate
through a recreation of an actual pilot plant for the production of
polyether polyols in order to learn about analyzing plant safety.
Students can see safety features such as pressure relief systems
and emergency showers, and also consider the impact of possible
plant failures on the local environment. The walkthrough is
supported by links to hypertext information, photographs, and
material safety data sheets. As another example, Haywood
Community College's Milling Machine allows students to study
the effect of the programs they develop for driving a milling
machine by walking through a simulation of the procedure while
changing viewpoints. In one case, the purpose of the VRRV Hors d'Oeuvre effort was not to
teach any particular classroom subject but to provide participants with an awareness of VR
technology and its possible applications; here no pedagogy was needed. Some of the
Table 6. No Pedagogical Support
Virtual Egyptian Temple Virtual Pompeii
Classical Buildings CDS
O'Conner World Updike World
Milling Machine Nursing World
Room World Great Pyramid
3D Letter World Gebel Barkal: Temple B700
Science Education World Spatial Relations Worlds
Safety World VRRV (Hors d'Oeuvre)
44
44
Page 45
46
31
©1998 Institute for Defense Analyses
applications in this group, such as the Science Education World, are likely to add explicit
pedagogical support in the future.
Those applications that explicitly embody
some form of pedagogy all support
constructivist learning, using an experiential
or guided-inquiry paradigm. The particular
approach used in each case is identified in
Table 7. The Virtual Gorilla Exhibit takes an
experiential approach. By allowing
participants to assume the role (silverback,
adult male and female, juvenile, infant) of
different members in a virtual gorilla family
and interact with the virtual gorillas, this
exhibit teaches about gorilla social hierarchies
and behaviors. Another application following
an experiential paradigm is the Virtual
Bicycle. Here participants use a specially-modified
bicycle to cycle round a route where
various hazardous conditions occur. The rider must deal with these conditions, which are based
on statistical analysis of bicycle accidents, and his performance is guided and critiqued by a
virtual mentor. MAEL's Virtual Biplane is intended to provide students with a basic
appreciation of flight. This application differs from the others in that it is intended to support
additional educational activities. For example, the curricula at different workstations, including
an aircraft design workstation, miniature wind tunnel, amateur radio station, remote sensing
workstation, and flight simulator, are ultimately expected to interconnect to create the
experience of preparing for and performing a cross-country flight. The MAEL curricula as a
whole are being jointly developed by NASA/ Lewis Research Center and Cuyahoga
Community College. MAEL will be used to support the Science, Engineering, Math, and
Aerospace Academy (SEMAA) K-12 education program developed by these two groups. The
Crossing Streets application is based on previous work at the University of Illinois on general-case
instruction and behavior self-management models. Here, students are presented with three
different types of street intersections to learn to cross the street carefully under a variety of
traffic patterns.
The other experiential worlds are specifically intended for use by learning-disabled or
autistic students, and are discussed in the following section. By allowing students to experience
various activities in a virtual world, these worlds are intended to will help these students learn
basic skills that will help in their daily lives.
Table 7. Type of Pedagogical Support
Virtual Bicycle Experiential Cell Biology Guided-inquiry
Virtual Gorilla Exhibit Experiential VESL Guided-inquiry
Vari House Guided-inquiry Virtual Biplane Experiential
Street World Experiential Object World Experiential
Greek Villa Learning talk, guided-inquiry Crossing Streets Experiential
LAKE Guided-inquiry NewtonWorld Guided-inquiry, scaffolding
MaxwellWorld Guided-inquiry Pauling World Guided-inquiry
Vicher (I and II) Guided-inquiry Map Interpretation World Guided-inquiry
Makaton World Experiential Life Skills World Experiential
AVATAR House Experiential Energy Conservation Guided-Inquiry
Phase World Guided-inquiry Atom World Guided-inquiry
Zengo Sayu Whole language learning Global Change Guided-inquiry
45
45
Page 46
47
32
©1998 Institute for Defense Analyses
For the guided-inquiry applications, the pedagogical support is provided in different ways.
In several cases, the pedagogy is not embedded into the virtual world but provided by means
of associated textual materials. Vari House is one such application. This set of linked virtual
worlds provides students with walkthroughs of the archaeological excavation site in Vari,
Greece, and a reconstruction based on the building remains that were found. The desktop
format allows textual materials to be presented alongside the each virtual world, and these
materials provide a demonstration of how archeologists determined, for example, the
occupation of the building's occupants. Students are guided in developing their own critical
thinking skills by answering questions such as: What factors might tell us about the date of the
building? What do you think the circular stones with the central depressions were used for?
For Map Interpretation World, instructors plan to post questions that will guide student use
of the world as part of a Web-based course syllabus. As the course progresses, students can
return their responses to these questions electronically, including links to the virtual world in
their discussions as appropriate.
The Vicher worlds also rely on non-embedded pedagogy, guiding students' learning about
reaction engineering and design by means of a one-page list of things to do and see in the virtual
worlds and a short list of questions to answer. The example list and set of questions shown in
Figure 4 serves to indicate how well this approach can guide student interactions with virtual
worlds. Actual development of the Vicher worlds was guided by consideration of Bloom's
Taxonomy of Educational Objectives and Felder and Silverman's classification of learning
styles [Bell and Fogler, 1995]. Currently, the Vicher worlds target levels 2, 4, and 6
(comprehension, analysis, and evaluation) of Bloom's taxonomy and the application is
intended to support "active," "visual," "inductive", and "global" learners. The Vicher worlds
are more interactive than those previously discussed. They allow students, for example, to
define the operating conditions for several different types of chemical reactors so that they can
observe the effects of changes, looking for pertinent relationships.
The MaxwellWorld, LAKE, Global Change, and Phase World applications are similar in
the way they support guided-inquiry learning. Each allows students to change world
parameters and observe the effects. Using MaxwellWorld as an example, student tasking
typically follows the form of the teacher first describing the activity that is to be performed;
then the student predicts what is going to happen, completes the activity and observes what
actually happens; and finally the student describes what he sees and compares it to what was
predicted. Figure 5 provides an example scenario using this approach. NewtonWorld is also
similar to these applications: it allows students to change physical laws, such as the coefficient
of friction, and observe the effects on a pair of colliding balls in order to learn about the
kinematics and dynamics of one-dimensional motion. NewtonWorld, however, provides
additional pedagogical support by introducing scaffolding that advances students from basic to
46
46
Page 47
48
33
©1998 Institute for Defense Analyses
advanced activities. PaulingWorld currently is undergoing redesign but is expected to follow
this overall approach.
As with the Vicher worlds, the developers of the ScienceSpace worlds and LAKE, and the
HITL researchers have provided information on the pedagogical underpinnings of their
applications. The ScienceSpace researchers stress four concerns [Dede et al., 1997b]:
° Students must focus on or be engaged in an experience in order for learning to
occur,
° Meaningful representations are necessary to communicate information,
Figure 4. Example Vicher List of Things to See and Do, and Study Questions
List of Things to See and Do in Vicher I
Things to see and do in the Welcome Center: The main thing to do here is to practice using the controls
and get comfortable with the experience. Can you crawl under the tables? Ride the escalator? Watch some
TV to learn the use of the mouse and get a preview of other rooms. Get some help on something in the room.
When you are ready you can walk through any of the doorways to teleport to other rooms. But don't walk
though the exit until you are ready to quit the program!
Things to see and do in the Transport Reactor Room: The television (to your right) will explain the
equipment and how it works. Use the control panel to turn the equipment transparent. Observe the coking/
decoking process. How does this change when you change the flowrates? Watch the cutaway view of the
tracer coke and decoke. "Activate" any pellet for a closer view.
Things to see and do in the Time-Temperature Room: Watch TV. Turn on the reactor power to start an
experiment. The clock and calendar mark time. Observe how conversion declines with declining activity at
constant temperature. How long until shutdown? Activate the "HEAT FX" button to control temperature for
constant conversion. Then push power and note the time until shutdown. Try some different target conver-sions
to observe the tradeoffs.
Things to see and do in the Microscopic Areas: Outside the pellet, observe external diffusion. Fly inside to
observe reactions taking place inside the pores. The red hexane modules are the reactants -follow one until
it reacts. The orange intermediates will also react. If you have difficulties, watch the targets -they are rigged
for easy observation. (Note: The targets are NOT the only active sites -they are just easier to watch.) You can
fly through the targets for a closer look. Then activate or fly through the pictures to get back out.
Study Questions for Vicher I
1. In the transport reactor room, how do the flowrates of hexane and oxygen affect the coking/ decoking
process? Can you identify trade-offs of high versus low flowrates?
2. In the time-temperature room, flowrates are constant and conversion is controlled by temperature. Does the
catalyst activity decline faster or slower when higher (desired) conversions are chosen? Why? If the tem-perature
is controlled to yield a constant conversion, how does this affect catalyst degradation rates? State
at least two trade-offs between reactor performance and shutdown frequency.
3. Inside the catalyst pore, does a reaction take place every time a reactant hits the pore walls? Why or why
not? Describe the steps involved when a reaction does take place.
47
47
Page 48
49
34
©1998 Institute for Defense Analyses
° Multiple mappings of information can enhance learning, and
° Learning-by-doing and reflective inquiry can both induce learning.
In particular, to support mastery of complex scientific concepts, the ScienceSpace worlds are
designed to provide learners with experiential metaphors and analogies that aid in
understanding abstractions that are remote or contradictory to everyday experience. The
developers of the virtuaL Approach to the Kernel of Eutrophication (LAKE) base their world
design on general aspects of sensory ergonomics. The LAKE application consists of a series of
linked worlds that students can explore to experience successive stages in the development of
plant nutrient materials and organisms in lakes.
At the HITL, the researchers believe that immersion is the key issue and that the
psychological processes that become active in immersive VR are very similar to those that
operate when people construct knowledge through interaction with objects and events in the
real world [Winn 1993]. Their applications, therefore, are designed to embody psychological
theories pertaining to first-person experiences, non-symbolic interactions, and learning by
constructing knowledge.
In a setting of the solar system, VESL also allows students to manipulate variables such as
mass, velocity, and time with the objective of learning about linear and orbital mechanics.
VESL differs from the previous set of applications, however, in providing a virtual helper that
assists the students with exploring the virtual solar system. Context-sensitive help provides
additional support, addressing such issues as how to operate the controls, what to do next, and
how to interpret the results of experiments from a physicist's point of view.
Atom World and Cell Biology differ from the preceding applications in that participants
learn by constructing objects. In Atom World, students review basic atomic and molecular
Figure 5. Examples of Student Questions Used with MaxwellWorld
These questions were posed as part of a lesson that teaches about the concept of superposition, that is, that
each of the charges in the space influences the strength and direction of the electric field (force) at every
point in the space.
The student is looking at a dipole, and has just learned about superposition. Now he is asked to use that
concept to predict what will happen at a specific point in the field, and the field as a whole, when he deletes
a source charge:
Question 7. Predict. Now let's talk about that same trace. Use your finger to show what will happen
to that trace when you remove the positive charges. Explain why. What will the field be
like in general?
Question 8. Observe and Compare. Let's test your hypothesis. Point to the positive charge and dou-ble
click to delete it. Is this what you predicted? Based on what you just observed, de-scribe
specifically what the force meter on a test charge trace reflects in relation to a set
of charges.
48
48
Page 49
50
35
©1998 Institute for Defense Analyses
structures by constructing specified structures. These structures are built by selecting items
from different sources of subatomic elements and changing electron charges as necessary. Cell
Biology uses a game-like approach in guiding participants to learn basic principles of human
cell biology. A participant is provided with bowls of different types of organelles that must be
combined to form the different types of cell that a sick child needs. Energy Conservation takes
a different approach to knowledge construction by allowing students to investigate how the
energy saving properties of various building materials impact home energy costs. Students gain
practical insights in how to plan for energy-related home improvements by learning how to
balance the cost of particular improvements with expected savings over time under various
budgetary and time constraints.
Guided-inquiry is supported in the Greek Villa application by asking students to take on the
role of time-travellers and to see aspects of the Greek Villa as evidence from which deductions
about the Greeks can be made. Additionally, this work is investigating the use of "exploratory
talk" in virtual worlds. The concept behind exploratory talk is that a certain type of student-student
discourse can contribute to learning more than other patterns of interaction [Grove
1995]. Consequently, this work has attempted to foster students' exploratory talk and is
considering whether, and the extent to which, students engagement in this type of discourse can
be used as a measure of VR technology's educational value.
Finally, Zengo Sayu adopts a whole language approach for second language learning or,
more specifically, a combination of Asher's Total Physical Response (TPR) strategy and
Terrell's Natural Approach [Rose and Billinghurst, 1996]. As Rose reports, TPR is a direct
assimilation method where the meaning of the target language is conveyed through physical
demonstration and does not use any form of translation into the first language. The Natural
Approach is a modification of Asher's strategy that supports the concept of a "silent period"
for language absorption, incremental knowledge acquisition, concrete associations
development, and the use of speech technique to draw attention to critical aspects of the target
language. Zengo Sayu supports this combined approach by the use of speech and gesture
recognition and digitized speech output in the context of a Japanese-style room where objects
can be manipulated.
49
49
Page 50
51
36
©1998 Institute for Defense Analyses
3.1.4 Support for Students with Special Needs
A few groups have been looking at the potential of VR
technology to support students with learning disabilities.
The applications that have been developed in this area are
identified in Table 8. The leader in this area is the
University of Nottingham, Virtual Reality Applications
Research Team (VIRART) Group, which has worked in
collaboration with the University of Nottingham Medical
School, Department of Learning Disabilities. Here, as part
of their Learning in Virtual Reality (LIVE) program,
researchers have been working closely with staff from a local school for the learning disabled.
With the overall goal of developing a methodology for the use of VR technology in special
needs teaching, they have proposed a five-step approach that, as stated by Brown et al. [1995],
seeks to:
° Embed the development of virtual learning environments in contemporary
educational theory,
° Empower users and their care-givers to participate successfully in shaping and
defining the educational and rehabilitative applications developed,
° Design and execute a continual program of testing and use these results to refine the
virtual learning environments,
° Consider the ethical issues surrounding the involvement of people with disabilities
in research and development, and
° Develop a curriculum for use of these environments in special classrooms today.
The VIRART team has already used this approach in developing a number of VR
applications for severely learning-disabled students. Makaton World supports learning the
Makaton Symbol Vocabulary, a sign language used in the United Kingdom by people with
learning disabilities. The application consists of a number of separate virtual worlds, each
designed to teach a particular Makaton symbol and associated hand sign. Currently, 50
Makaton symbols are supported, drawn from the first four levels of the Makaton vocabulary.
The researchers hope to continue adding new symbols until the whole 350 symbol vocabulary
is included. Another series of virtual worlds, the Life Skills Worlds, provides students with an
opportunity to learn practical skills (e. g., shopping in a supermarket) or to experience events
that would otherwise be inaccessible to them (e. g., driving a car). Life Skills World is currently
being extended to include cafe, post office, recreation center, bank, and health center virtual
worlds.
Researchers at James Cook University, School of Education, developed a simple world
consisting of a ring of alphabetic characters suspended in empty space. The purpose of this 3D
Letter World was to determine whether exposure to the virtual world would help young
Table 8. Special Needs Applications Support for Learning Disabilities
Makaton World Life Skills Worlds
3D Letter World
Support for Autistic Students Street World
Object World AVATAR House
Support for Physical Disabilities VESL
Science Education World
50
50
Page 51
52
37
©1998 Institute for Defense Analyses
children who had exceptional difficulty in letter recognition. Unfortunately, because of the
difficulties the children experience in mastering basic navigation skills, this effort could not be
completed [Ainge 1996c]. The researchers attribute the students' difficulties to the lack of
adequate hand-eye coordination and spatial awareness. This conclusion suggests that further
work is needed to determine the necessary basic skills for exploration of virtual worlds and to
assess whether special preparatory training could assist learning disabled students in gaining
such skills.
This VIRART researchers are also working on providing support for autistic students. In
this case, the AVATAR House application allows a student to explore a virtual house where
rooms are designed to focus attention on recognizable objects and activities so that the students
can learn practical skills. Researchers at North Carolina State University, working with the
Treatment and Education of Autistic and Other Communications Handicapped Children
(TEACHC) program at the University of North Carolina Medical School, have also focused on
the use of VR to help autistic children. In their first effort, these researchers used Street World
to investigate the usability of the technology for this type of user. Based on the success of this
evaluation, the researchers are redesigning the VR application to teach identification of basic
classroom objects. The new application is called Object World.
Two groups have been working to develop educational VR systems for students with
physical disabilities. Following an analysis of the special needs of persons with spinal cord
injuries, Interface Technology Corporation developed VESL to provide a virtual physics
laboratory that could be used by students with such disabilities, as well as non-disabled
students [Nemire 1994]. After performing a study looking at the usability of spatial tracking
technology for students with cerebral palsy [Nemire 1995a] and developing special prediction
software that would help these students to select targets in a virtual world, Interface Technology
Corporation redesigned VESL to additionally support this class of students. Both these design
and development efforts included usability evaluations that considered feedback from not only
students and teachers, but also assistive device specialists, occupational therapists, and human
factors engineers.
The Oregon Research Institute also has taken care to ensure that its Science Education
World accommodates the needs of many physically challenged students. One of the
mechanisms they employ is a touchscreen. Future versions of the application are expected to
include speech recognition so that students can take notes during their investigations.
3.1.5 Hardware and Software Issues
For each developer of educational VR applications, Table 9 summarizes key aspects of the
hardware used. The type of hardware platform and peripherals required by an application can
have a significant impact on its development and operational costs. Though the high graphics
processing capabilities of a powerful Silicon Graphics, Inc. (SGI) machine are desirable for
51
51
Page 52
53
38
©1998 Institute for Defense Analyses
creating highly-detailed worlds with fast frame rates, the high costs of such machines can make
them impractical vehicles for elementary, middle school, high school and, in some cases, college
education. Recently, SGI, Hewlett-Packard (HP), and Intergraph have all introduced new lines
of graphics workstations that provide good graphics performance at substantially lower costs.
At the other end of the spectrum, Pentium PCs and the development of powerful graphic
accelerators are increasing the power of low-end machines. Consequently, while high-end
graphic workhorses have been used as platforms for some of those applications developed as
research tools, and as an initial development and evaluation platform for other applications, they
are becoming less widely used. The consortium developing ScienceSpace, for example, is
porting its applications to PCs, and the University of Washington HITL is considering a move
to HP workstations. On the other hand, the University of Michigan, Department of Chemical
Engineering, is porting in the reverse direction, from PCs to SGI mainframes to get the
processing power needed for its complex Vicher and Safety worlds. The range of SGI machines
in use includes the Onyx, Indigo, Crimson/ Reality Engine, and Maximum Impact platforms.
The most popular platform for educational VR applications is a PC. The majority of
developers are using Pentium PCs with graphic accelerators, although ERG Engineering, Inc.
and the Correspondence School have both used 386-or 486-level PCs. Other types of hardware
platforms, such as the HP J210 PA-RISC, Intergraph GLZ5, and Division PV 100 workstations,
have seen only limited use.
The choice of visual display represents another important decision, one that affects both cost
and the degree of immersion experienced by participants. Usually, the ideal situation is full
immersive viewing of a computer-generated virtual world, and with current technology this calls
for an HMD or CAVE display. HMDs are devices where two miniature display screens (one for
each eye) are positioned in front of the user's eyes and viewing through optical lenses that serve
to magnify the images. The user's sight is restricted to what can be viewed on the virtual scene
projected by the optical system and, hence, the user is visually immersed in the virtual world
that is presented. Stereoscopic viewing is achieved by presenting slightly different images on
the display screens. A tracking device is usually attached to the HMD so that the virtual scene
is updated appropriately as the user turns his head. A CAVE display takes a very different
approach. Here the user can move freely within a small "room" constructed of up to six rear
projection screens and some type of special glasses are used to provide stereoscopic viewing of
the virtual world in which the user is visually immersed. The majority of developers are using
relative inexpensive HMDs to provide stereoscopic viewing. While the capabilities of different
HMDs vary considerably, current low-cost products (less than $1,000) typically deliver a
resolution of around 300 x 400 pixels, and a horizontal field of view of around 30°. Some
developers have chosen to use HMDs but provide only monoscopic viewing, thus reducing
some of the need for heavy graphics processing.
52
52
Page 53
54
39
©1998 Institute for Defense Analyses
There are other factors that can influence display choice. If, for example, it is desirable for
several students to watch the interaction of one of their colleagues with an application, then the
most economical option may be to use a projection screen and provide inexpensive light
polarizing glasses so that the audience can watch a 3D display of what the immersed student
sees using an HMD. This type of combination viewing is used by students at Haywood
Community College. A more expensive alternative, used by the NCSA for Crossing Streets, is
to provide for simultaneous immersion of several participants in a CAVE.
Table 9. Hardware Support for Pre-Developed Applications
Developer
Platform Display Special I/ O
SGI Division
Workstation PC Macintosh Intergraph Workstation HP Workstation Monitor Monitor w/ Glasses HMD Projection Screen CAVE w/ Glasses CyberScope Speech I/ O Haptic Spatialized Sound Hand Device Motion Platform
Carnegie Mellon University, SIMLAB 3 3 3 3 3 3 3 3
Correspondence School 3 3 3
ERG Engineering, Inc. 3 3 3 a
a. Digitized speech output only
3 3
Georgia Institute of Technology, GVU
Center 3 33 3
Haywood Community College 3 3 3 3 3 3
Interface Technologies Corporation 3 3 3 3 3
James Cook University 3 3
Learning S ites, I nc. 3 3 3 3 3
NASA/ Lewis Research Center 3 3 3
North Carolina State University 3 3
Oregon Research Institute 3 3
Oregon State University 3 3 3
Sheffield Hallam University 3 3
University of Houston, George Mason University, & NASA JSC 3 33 a 3 3 3
University of Illinois, NCSA 3 3 3
University of Ioannina 3 3 3
University of Michigan 3 3 3 3 3 3 3
University of Missouri 3 3 3 3
Univ. of Nottingham, VIRART Group 3 3 3
University of Portsmouth 3 3
University of Washington, HITL 3 3 3 3 3
53
53
Page 54
55
40
©1998 Institute for Defense Analyses
In some cases, an application is designed to work with alternative display devices. The
widest selection is supported by the University of Michigan's Vicher and Safety World
applications. Here, the Visual I/ O HMD, Crystal Eyes shutter glasses, and Simsalabim's
CyberScope are all supported. (The CyberScope is an optical hood that attaches to a monitor
to provide a stereoscopic display.)
Overall, two-thirds of the developers (nearly three-quarters of the applications) provide
immersive viewing. For non-immersive, stereoscopic viewing, the use of shutter glasses with
a monitor seems to be preferred to a projection screen with passive glasses. Less than one third
of the developers rely entirely on the use of a standard desktop monitor for world viewing.
As shown in Table 9, other special input/ output devices are being used in addition to the
display devices. Often, it is the same small group of developers who provide these types of
support. The HITL's Zengo Sayu and Interface Technology Corporation's VESL both use
speech recognition and digital speech output, and digitized speech output is also provided in
ERG Engineering Inc. 's Cell Biology world and the PaulingWorld from the ScienceSpace
series of worlds. All but the first of these developers have also used spatialized sound in some
of their applications. The SIMLAB at Carnegie Mellon University also uses spatialized sound.
Haptic feedback has been employed in NewtonWorld and MaxwellWorld, in evaluating the
impact of multi-sensory feedback on learning effectiveness. The haptic feedback was provided
by two different haptic vests, both of which operate by converting sound waves to vibrations.
While some immersive and desktop applications still rely on a traditional mouse as the
primary input device, special devices such as joysticks, wands, and six degrees-of-freedom
mice are being used with some immersive applications. The most common of these special
devices is a joystick. In particular, data gloves are not widely used, largely because of problems
in resolution for interpreting gestures. The Street World and Object World designed for use by
autistic children use no hand-based input; instead, participants navigate through these worlds
via head tracking on the HMDs and walking in a small area. As previously noted, Interface
Technology Corporation and the Oregon Research Institute also make special accommodations
for physically disabled students: VESL includes a head wand and a spatial tracking device that
attaches to the user's hand, and Science Education World uses a touchscreen instead of a
mouse.
The Virtual Bicycle and Map Interpretation World provide the first examples of the use of
motion platforms in educational VR applications. The Virtual Bicycle uses a specially modified
bicycle that is mounted on a motion platform to represent the different types of surface that the
participant must traverse. For the Map Interpretation World, researchers are considering the use
of a motion platform to provide a sense of following geographical contour lines.
Developers' use of particular VR development packages is shown in Table 10. These
development systems vary greatly in the tools they provide. The most extensive being Sense8's
54
54
Page 55
56
41
©1998 Institute for Defense Analyses
WorldToolKit, a package that is available for platforms ranging from SGI machines to PCs.
Packages such as Superscape's VRT, Virtus' Virtus Walkthrough, and VREAM's Virtual
Reality Development System (VRDS) are less sophisticated and used primarily on PC
platforms. REND 386 and BRender are early packages that are available as freeware on the
Internet. DVise is the development package that comes with Division's VR workstations. Some
developers have chosen to create their own development packages or use available modeling
tools. In one case, at the University of Missouri, Department of Geography, a virtual world was
constructed using the Multigen database tool and Perfly for the creation of the walkthrough.
Other VR software development systems or 3D graphics packages in use include Alice,
Renderware, CyberSpace, Quicktime VR, Design it3D, 3D Design Center, Alias Modeling
Table 10. VR Development Software for Pre-Developed Applications
Developer
DVise Superscape
VRT
WorldToolKit Virtus WalkThrough/
Pro
VRDS BRender VRML Custom Other Carnegie Mellon University, SIMLAB 3
Correspondence School 3 3
ERG Engineering, Inc. 3
Georgia Institute of Technology, GVU Center 3
Haywood Community College 3 3
Interface Technologies Corporation 3
James Cook University 3 3
Learning Sites, Inc. 3 3
NASA/ Lewis Research Center
North Carolina State University 3 3
Oregon Research Institute 3
Oregon State University 3
Sheffield Hallam University 3
Univ. of Houston, George Mason Univ., & NASA JSC 3
University of Illinois, NCSA 3
University of Ioannina 3
University of Michigan 3
University of Missouri 3
University of Nottingham, VIRART Group 3
University of Portsmouth
University of Washington, HITL 3
55
55
Page 56
57
42
©1998 Institute for Defense Analyses
Software, AutoCAD, Modelgen, NCad, Ogre, and Hyperstudio. A simulation software
development package called VEGA, from Paradigm Simulations, Inc. has also been used. As
can be seen in the table, there are no strong favorites in this area.
One reason why such as wide range of development tools is being used is that no single tool
currently supports the range of functionality needed in the development of diverse virtual
worlds. Until recently, the incompatibility between these tools has sometimes presented
problems for application developers. The continuing development of VRML may, in time,
resolve such difficulties. VRML is not a development package that provides developers with a
range of tools, but a programming language for the development of virtual worlds that can be
viewed using various Web browsers. The main benefit of VRML is that of standardization of
virtual world data over the Web. Some development packages (such as VRT, Sense8's
WorldToolKit and WorldUp, and VREAM's forthcoming VRCreator) already provide VRML
compatibility. The most recent version of VRML, VRML 2.0, still has many limitations, such
as a lack of specifying how objects can interact with a multiuser technology. Even so, VRML
2.0 is in the process of becoming an International Organization for Standardization/
International Electrotechnical Committee standard (ISO/ IEC 14772), and VRML browsers are
expected to become widely available on the Web in the near future.
While only one of the developers (Learning Sites, Inc.) of educational VR applications
discussed here is currently using VRML, in January 1997 SGI posted information about the top
ten VRML educational worlds (reportedly judged by leading figures in the VRML community)
on their Web site. Vari House was selected as one of these award-winning educational
applications. The other nine applications are not discussed in this paper because they are
essentially VRML demonstration worlds not intended for immediate practical educational
research or use. More evidence of the interest in VRML for educational applications is
provided by Educational Service Unit #3 of Nebraska (see Section 2). Here, Service Unit staff
is currently searching for teachers who are willing to use VRML on the Internet to support a
collaborative project.
3.1.6 Extending Beyond Education
Several of the predeveloped VR applications also can be used for non-educational
purposes. For example, in their role as exhibition pieces, Cell Biology and Virtual Gorilla
Exhibit serve to both entertain and educate. In addition, some of the applications are intended
for use by both students and professionals engaged in specific work-related activities. The
Conceptual Design Space (CDS) was one such application, providing architects with tools to
aid their visualization of architectural spaces. Gebel Barkal: Temple B700 is intended to
support archeologists through the provision of links to nested datasets that include relevant
archaeological data.
56
56
Page 57
58
43
©1998 Institute for Defense Analyses
The work of Learning Sites, Inc., in general, deserves special mention for its breadth of
vision. The goal of this group is to create a globally integrated and interactive system of linked
virtual worlds that can be used for teaching, research, archaeological fieldwork, museum
exhibitions, and even tourism. Accordingly, the existing worlds are all based on detailed
archeological evidence and provide links to resources such as excavation notes, site drawings,
and related historical, geographical, and cultural material. The set of Learning Sites worlds is
expected to grow, and some of the existing worlds will be expanded. Gebel Barkal: Temple
B700, for example, will continue to expand as more temples are added. To this end, Learning
Sites, Inc. is currently in discussion with several archeologists about publishing their
excavation results using interactive VRML worlds.
3.2 Student Development of Virtual Worlds
As before, in an effort to start by trying to give some idea of the types of virtual worlds that
have been developed by students in the course of learning about particular topics, the Figure 6
and Figure 7 provide descriptive overviews of two of these worlds. A summary of the major
characteristics and usage of the efforts considered is given in Table 11. Some of the topics of
discussion in the previous section, such as embedded pedagogy and commercial availability,
are not applicable for student development of virtual worlds. Consequently, the structure of this
section differs slightly from the last, dropping some topics of discussion while adding those
more pertinent to student world building.
Before continuing, it should be noted that the work of West Denton High School,
Newcastle, England, is not included in the following discussions because of a lack of
information. This work was conducted in the early 1990s, but the teachers involved with the
work have left the school, which is no longer using VR technology. This work deserves some
mention, however, because it was Europe's first school-based VR project. One of the ways in
which VR technology was used was to support learning about workplace safety by having
students design and build virtual worlds of factories, keeping health and safety rules in mind.
Then, using a trackball, they could explore the virtual world, driving virtual lathes and forklift
trucks. Some of the VR work was submitted in partial fulfillment of the UK A-Level
examination in computing and the Business Technician Education Council (BTEC) diploma in
computer studies. It is unfortunate that the overall findings of this work have not been
disseminated.
Another effort that is not discussed at this time because actual development has not yet
started is one by Educational Service Unit #3 in Nebraska. This group is the Internet Service
provider for regional schools and regards VR and Web technologies as mutually compatible.
One of its goals is for students to develop a variety of skills that will allow them to build worlds
collaboratively over the Web. To achieve this goal, staff are developing a curriculum for use by
area teachers that will help them integrate 3D computing using a low-cost VR development
57
57
Page 58
59
44
©1998 Institute for Defense Analyses
Figure 6. Unit on Antarctica
Background on the Icebound Project: Don and Margie MacIntyre wintered over at Cape Denison in Ant-arctica in 1995. They lived in a 6' x 8' room, which they called the Gadget Hut, that they had designed and
tested. Don and Margie communicated with thousands of students across New Zealand, sharing their day-to-day trials and tribulations. (For example, at one time, Don and Margie almost died of carbon monoxide poi-soning
when a vent hole froze over.) Their way of life became a catalyst for many studies of Antarctica. A class of students at Evans Bay Elementary School, Wellington, NZ, participated in the project, exchanging
faxes and email with the MacIntyres and joining in an audio conference.
Work at Evans Bay Elementary School: In an ef-fort to teach communication and critical thinking
skills, students were assigned many research activ-ities that required use of VCRs, electronic bulletin
boards, electronic mail, CD, and VR technology.
VR Activity: Four students were tasked to design a
permanent Antarctica base large enough for two
people. The educational objective was to learn and
apply critical thinking skills in using knowledge
gained from research. The design had to show evi-dence
that all important aspects had been consid-ered.
For example, because of a lack of water for
fire fighting, the base was divided into separate sec-tions
with long connecting tunnels that were col-lapsible
to prevent any fires from spreading. The
students also included a refrigerator that would be
used to keep things warm, instead of cold.
Researcher's Comments: "The students did use critical thinking and applied research -but of course they
could have done this using paper -or have made a [physical] model. What the VR software did was allow
their ideas to be changed rapidly, explored and developed quickly -redesigning was a breeze. They could con-ceptualize
as a group -taking turns at driving the mouse on the PC. It also allowed presentation of a 3D walk-through
so that ideas could be recorded as a 'walkthrough movie' in software. Ideas were seen by the whole
group -in 3D. Overall, they used the software very successfully to collaboratively conceptualize, test and
present their ideas" [Carey 1997].
58
58
Page 59
60
45
©1998 Institute for Defense Analyses
Table
11.
Characteristics
and
Usage
of
Student
Development
of
Virtual
Worlds
Organi-zation
Class/ Course
Tasking
Learning
Objectives Supported
Intended Audience
Display
Usage
Students' Organization
Date
of
Use
E a s t C a r o l i n a S t a t e U n i v e r s i t y
EDTC
6242
Develop
a
VR
application
that
meets
specific
curriculum
objectives,
supporting
instructional
materials,
and
demonstration
scenario.
Conduct
empirical
evaluation
of
the
application's
educational
effectiveness.
Build
skills
in
applying
VR
technology
to
education.
College students
Desk top , HM D (stereo ) , g l as ses
Practical
(as
independent study course)
East Carolina State University
Spring 1996 onward
Ev a n s B a y I n te r m e d ia te S c h o o l
Class
work
on communications
technologies,
unit
on
Antarctica
Using
information
learned
through
communicating
with
a
couple
wintering
in
Antarctica
in
a
hut,
cooperatively
design
a
permanent
Antarctica
habitat.
Develop
critical
thinking
skills.
Ages
10-13
De skto p
Practical
Evans
Bay Elementary
School
Late 1995
Class
work
on communications
technologies,
unit
on
sports
Following
local
debate
on
the
advisability
of
building
a
new
sports
stadium,
develop
stadium
designs
and
walkthroughs
of
these
designs.
Support
development
of
3D
thinking
skills
and
problem
solving
design
issues
within
set
parameters.
Ages
10-13
De sktop
Practical
Evans
Bay Elementary
School
Mid-1995
Language
program
Work
cooperatively
to
create
worlds
and
stories
that
develop
within
them.
Video,
audio,
or
text
at
hot
spots
carry
the
narrative.
Develop
oral
language
part
of
English curriculum: story
telling,
group
discussion,
group
problem
solving;
and
from
the
visual
language
part:
present
ideas
in
a
visual
way.
Ages
10-13
Deskto p
Practical
Evans
Bay Elementary
School
Mid-1996 onward
Virtual
Museum
Working
in
pairs,
decide
on
a
research
topic,
make
a
mind
map
of
their
existing
knowledge,
list
research
questions
and
conduct
research.
Present
the
result
as
a
series
of
related
museum
displays,
using
supporting
graphics,
text,
movies,
and
sound.
Develop
skills
in
organizing
research
results.
Ages
10-13
De skto p
Practical (looked at comparative educational effectiveness)
Evans
Bay Elementary
School
Mid-
late 1996
Virtual
Stage
Working
to
scale,
design
stage
sets
for
witches'
scene
in
Macbeth
using
standard
sizes
of
large
wooden
cubes,
and
avatars
to
represent
actors'
positions.
Develop
mathematical concepts of scale and skills in group design.
Ages
10-13
Desk t o p
Practical
Evans
Bay Elementary
School
Mid-1996
59
59
Page 60
61
46
©1998 Institute for Defense Analyses
Table
11.
Characteristics
and
Usage
of
Student
Development
of
Virtual
Worlds
(continued)
Organi-zation
Class/ Course
Tasking
Learning
Objectives Supported
Intended Audience
Displa y
Usage
Students' Organization
Date
of
Use
Ev a n s Ba y I n t e r m e d i a t e
S c h ool (co n ti n u ed )
Virtual
Cemetery
Working
in
groups,
based
on
research
about
(in) famous
people
in
history,
build
a
cemetery
that
shows
a
classification
of
the
people,
with
mausoleums
with
epitaphs
and
movies
of
student
oral
presentations
about
the
people.
Develop
skills
in
organizing
research
results.
Ages
10-13
De skto p
Practical
Evans
Bay Elementary
School
Late 1996
Hayw ood Co mmu ni t y
C o l l e g e
English
277:
Exploring Literature
Through
Virtual
Reality
Use
VR
as
a
literary
analysis
tool
by
developing
a
series
of
simple
worlds
that
portray
scenes
from
a
reading
assignment,
and
a
final
world
that
addresses
teacher-
supplied
questions.
To
explore
how
writers
use
words,
images,
symbols,
and
settings
to
create
a
mood,
develop
characters,
or
dramatize
a
story
theme.
College students
H M D (s t e r e o ), p r o j e c t i o n scree n w/ p a ssiv e glasse s
Practical
Haywood Community College
Spring 1996 onward
H .B . S u g g E l e m e n t a r y
S c ho ol
Virtual
Pyramid
Working
in
groups,
build
a
pyramid,
move
objects
in
and
out,
and
view
the
pyramid
from
different
perspectives.
North
Carolina
Standard
Course
of
Study
Objectives
2.
1,
2.2,
and
2.3.
Grade
5
Des k top
Evaluation
of
effectiveness
H.
B.
Sugg Elementary
School
Early 1995
James Coo k Univ . e r s i t y , S c ho ol of E d u cat i o n
Mathematics
(3D
Shapes)
Working
in
pairs,
create
a
world
and
populate
it
with
a
specified
set
of
3D
shapes,
then
interact
with
it.
To
develop
skills
in
visualizing
and
recognizing
3D
shapes
form
various
viewpoints.
Grades 6-7
Deskto p
Comparative educational effectiveness evaluation
Townsville primary schools, Australia
May-July 1995
Historical
Home
Working
as
a
group,
select
a
period
in
history,
research
domestic
life,
and
built
a
virtual
home
that
illustrates
this
life
style.
To
develop
skills
in
researching
and
understanding
of
domestic
life
in
historical
cultures.
Grade
7
D e s k t o p
Comparative educational effectiveness evaluation
Townsville primary schools, Australia
Late 1997
K e l l y W a l s h H i g h S c h o o l
Math
Worlds
Develop
worlds
that
support
learning
in
pre-calculus
and
geometry
including,
for
example,
worlds
for
plotting
coordinates
in
3-D
and
functions
revolving
around
the
x
-axis.
Various
mathematical topics in pre-calculus
and
geometry.
Grades 10-12
HM D (stereo ) , shu t te r g l asses
Practical
use
Kelly
Walsh High School, Casper, WY
Fall 1993 onward
60
60
Page 61
62
47
©1998 Institute for Defense Analyses
Table
11.
Characteristics
and
Usage
of
Student
Development
of
Virtual
Worlds
(continued)
Organi-zation
Class/ Course
Tasking
Learning
Objectives Supported
Intended Audience
D i s p l a y
Usage
Students' Organization
Date
of
Use
K e l l y W a l s h Hi g h S c ho ol
( c o n t i n u e d )
Computer Programming
class
Research
an
educational
area
where
VR
seems
applicable
(not
math
or
programming
topics
and
something
that
cannot
be
done
by
hand)
and
create
an
educational
virtual
world
for
the
selected
topic.
Various
topics.
Grades 10-12
HM D (stereo), glasse s
Practical
Kelly
Walsh High School, Casper, WY
Fall 1993 onward
Comparative effectiveness
for
correcting science misconceptions
2
high
and
3
elementary schools in Natrona School District #1, WY
1993
-
1994
S l at on I n dep e n d en t S c h o o l D i s t r i c t
Class
on
Atomic
and
Molecular Structure
Working
in
groups,
research
a
particular
atomic
or
molecular
model
and
build
a
virtual
world
that
showing
understanding
of
the
atom
structures
involved
(numbers
of
protons
and
neutrons
in
the
nucleus,
and
electron
spacing).
To
develop
an understanding
of
an
atom
and
its
parts.
Ages
15-16
S h u tter g l asse s, p r o j e c t i o n s c r e e n
Practical
Slaton
High
School, Slaton,
TX
January 1996 onward
Class
on
Energy
Conservation
Working
in
groups,
select
a
(U.
S.
or
world)
area
you
would
like
to
live
in,
and
research
climatic
data
and
other
information
about
heating,
cooling,
house
structures,
etc.
Then
prepare
house
blueprints
and
create
virtual
worlds
that
provide
3D
renderings
of
the
blueprints.
To
provide
an understanding
of
costs
involved
in
home
maintenance
and
learn
to
develop
energy
conservation
techniques
to
reduce
the
cost
of
living.
Ages
15-16
S h u t ter g l a sses, p r oj ect i o n screen
Practical
Slaton
High
School, Slaton,
TX
January 1996 onward
U n i v e r s i t y o f W a s h i n g t o n , H I T L
Pacific
Science Centre, Summer Camp '91
Working
in
teams,
decide
on
a
world
to
build,
plan
the
work,
create
3D
objects
and
define
interactions
(HITL
staff
use
these
to
actually
build
the
virtual
worlds),
then
view
these
world.
Develop
an
understanding of VR technology.
Ages
10-15
HM D
Determine
if
students
could
work
creatively and enjoyably with VR
Pacific Science Center, Creative Technology Camp
Summer 1991
Pacific
Science Centre, Summer Camp '92
Working
in
teams,
take
an
abstract
concept
and
incorporate
it
into
a
virtual
world
with
an
emotional
theme.
Develop
an
understanding of VR technology.
Ages
10-15
HM D (stereo )
Assess
impact
of
gender,
race,
and
scholarship on ability to work creatively and enjoyably
Pacific Science Center, Creative Technology Camp
Summer 1992
61
61
Page 62
63
48
©1998 Institute for Defense Analyses
Table
11.
Characteristics
and
Usage
of
Student
Development
of
Virtual
Worlds
(continued)
Organi-zation
Class/ Course
Tasking
Learning
Objectives Supported
Intended Audience
Display
Usage
Students' Organization
Date
of
Use
U n i v e r s i t y o f W a s h i n g t o n , H I T L (c o n t i n u e d )
Course
on
HIV/
AIDS
prevention
As
a
group,
build
a
world
that
teaches
about
the
dangers
of
AIDS
and
the
precautions
that
can
be
used
to
protect
against
it.
(With
HITL
staff
support.)
HIV/ AIDS
awareness.
High school
HM D (stere o)
Assess effectiveness
for
"at-risk" students
and
VR's
role
in
a
curriculum
Southwest Youth and Family Services School, Seattle, WA
Spring 1993
Special
project
(Puzzle
World)
Build
3D
puzzle
pieces
that
fit
together
on
an
individual
and
group
level.
Development
of
3D
spatialization
skills.
Neu r ol og i cal l y i m p a i r e d s t u d e n t s ,
a g es 1 1 -1 4
HM D (s t e reo)
Evaluation
of
VE
building
for
cognitive development and spatial processing enhancement
Children's Institute for Learning Differences, Bellevue, MA
1993
Wetlands
Ecology
Study
a
wetland
life
cycle:
either
water,
carbon,
energy,
or
nitrogen.
Working
in
groups,
develop
a
virtual
world
that
demonstrates
understanding
of
the
wetlands
cycle
studied,
using
behavioral
models
to
represent
key
concepts.
Then
experience
that
virtual
world
and
one
demonstrating
some
other
wetlands
cycle.
Understanding
of
wetlands
ecology.
Grade
7
HM D (stereo )
Comparative effectiveness
of
building
and
visiting
worlds,
and
traditional instruction
Kellogg Middle School, Shoreline School District,
WA
Fall 1994
VRRV demonstration
VRRV participants
1995
-
1997
VRRV
Entrée
Students
follow
4-step
world-
building
process:
planning,
modeling,
programming,
and
experiencing
a
VE.
(With
HITL
staff
support.)
Experiencing
VR
technology
in
the
context
of
a
specific
curriculum.
Middle
and
high
school
HM D (s t e reo)
Evaluation
of
VR
limitations/ potentials, and whether VE building helps learning
14
high schools
and
middle schools
in
WA
1994
-
1997
A
a
group,
extend
a
basic
Tree
World
with
animals
that
live
in
the
tree
or
around
it,
and
the
things
these
animals
need
to
live.
Experiencing
VR
technology.
Grade
4-6
H M D (stereo )
Experimental
Elementary schools in WA
1994
VRRV
use
VRRV
use
1994-1997
62
62
Page 63
64
49
©1998 Institute for Defense Analyses
Figure 7. Wetlands Ecology
Developed Worlds: -Carbon Cycle. Demonstrated CO
2 formation,
O 2 formation, and decomposition. Objects in-cluded plants that used CO
2 and produced O 2 ,
and animals that consumed O 2 and produced CO
2 . Carbon was released into the cycle
through the decomposition of flesh or feces.
-Energy Cycle. Demonstrated the food chain and how energy transfers from one organism to another, includ-ing decomposition and its contribution to plant growth and regeneration. Objects included blue-green algae,
fish, dragon flies, birds, and a duck, turtle, fox, and alligator.
-Nitrogen Cycle. Demonstrated nitrogen fixing, movement of nitrogen through the food chain, denitrofication, decomposition (release of fixed and free nitrogen into the air and soil). Objects included free nitrogen, a light-ening
storm, rain transferring fixed nitrogen into the ground for absorption by plants, nitrogen fixing bacteria, plants with fixed nitrogen, denitrofying bacteria, and a duck, fox, dead ducks, and feces.
-Water Cycle. Demonstrated cloud formation (condensation), rainfall (precipitation), groundwater accumula-tion, and water vapor (evaporation). Objects included energy from the sun, water vapor, clouds, rainfall, and
a lake representing groundwater accumulation.
Project Goal: To test the hypothesis that learning about a wetland cycle using constructivist principles paired with the use of VR technology would yield greater comprehension of subject matter than learning about a wet-lands
cycle though traditional means.
Pedagogies Compared:
-Constructivist. Two 1.5-hour sessions were spent by students working individually, or with partners, study-ing general wetlands ecology information and information on an assigned cycle. They selected materials
from a library guide, the Internet, CD-ROMs, and video-disks to develop their own understanding of the un-derlying concepts. No direct instruction was provided. Groups of students then spent two more sessions plan-ning
a virtual world for the assigned cycle and creating objects and behaviors for the world. The plans and world components were integrated into a virtual world by HITL researchers. Students experienced the world
for their assigned cycle and a world developed by other students for another cycle.
-Traditional. For the first session, students were guided by a teacher in reading appropriate sections of a text-book. Handouts with page numbers tied to the assigned cycle, a keyword list, and a set of study questions to
be answered in discussion periods were provided. In the remaining three sessions, students completed flow-charts and worksheets, with specific pages numbers relating to the text. Then some of the students experi-enced
the virtual world for the cycle studied.
-No instruction. Students were given instruction on a unrelated subject.
63
63
Page 64
65
50
©1998 Institute for Defense Analyses
system into their classroom curricula. A Level 1, seven-to eight-day middle school industrial
tech module has been developed that provides students with a chance to learn the x, y, and z
coordinate system, navigation, and world design skills. Level 2 modules will focus on problem
solving skills. An example of a type of problem that might be used in a Level 2 module would
be the situation that faced the Apollo 13 ground crew when they had to fix a problem with CO 2
buildup using only the resources available in the spacecraft.
3.2.1 Type of Use
Unlike the pre-developed applications
which were roughly equally split between
practical use and research vehicles, only
about one-third of these efforts are
regarded as primarily research oriented.
As is shown in Table 12, the majority of
the student development of virtual worlds
has been conducted as a practical part of a
curriculum.
The first student development of
virtual worlds as part of classroom activities began in 1993 at Kelly Walsh High School. By the
end in 1996, twelve such efforts had been conducted. This fact does not imply, however, any
widespread nature of these efforts, because two thirds of the efforts were performed at Evans
Bay Elementary School or by the University of Washington's HITL. Indeed, the extent of VR
use at Evans Bay Elementary School is remarkable since the teacher leading the VR activities
is working independently and his equipment is all self-funded. The efforts at Evans Bay
Elementary School are all primarily practical in nature, whereas the HITL efforts are all
research oriented.
Not all of the practical efforts have been one-time events. The math and computer
programming classes at Kelly Walsh High School, East Carolina University's EDTC 6242, and
Haywood Community College's Exploring English Literature course are ongoing programs.
Also, Evans Bay Intermediate School expects to continue using its VR-based Language
Program.
3.2.2 Educational Topics Supported
There have been fewer instances where students developed virtual worlds than pre-developed
applications, and these virtual worlds do not cover the same breadth of educational
subjects. Even so, the range of topics covered is impressive and demonstrates the flexibility of
VR technology, as shown in Table 13. In this category of student-development of virtual
worlds, only the work at H. B. Sugg Elementary School has been reported as supporting specific
Table 12. Classification of Applications
Practical Use EDTC 6242 Exploring Literature Class
Unit on Antarctica Math Class Unit on Sports (Stadium) Computer Programing Class
Language Program Atomic/ Molecular Structure Class Virtual Museum Energy Conservation Class
Virtual Stage VRRV/ Washington Entrée Virtual Cemetery
Research Vehicles Virtual Pyramid Summer Camp '92
Mathematics (3D Shapes) HIV/ AIDS Prevention Course Historical Home Special project (3D puzzle)
Summer Camp '91 Wetlands Ecology
64
64
Page 65
66
51
©1998 Institute for Defense Analyses
state curriculum objectives. In this single case, topics in the North Carolina Standard Course of
Study Objectives 2.1, 2.2, and 2.3 were addressed.
The scope of student world development efforts varies greatly in terms of the need to
research underlying concepts, design and build a world, and support useful viewing of that
world. The amount of teacher or researcher support that students have needed in their world-building
activities has depended on the complexity of the case in hand and any prior experience
the students have. At one end of the spectrum, students at Evans Bay Elementary School now
work in groups without support to develop desktop virtual worlds. Projects such as the HITL's
Wetlands Ecology represent the other extreme, where students with only a minimal exposure
to VR technology (through the VRRV Hors d'Oeuvre program) were provided with extensive
support in the design and construction of immersive worlds.
Most of the worlds have been
developed by groups of students, rather
than individual students working
independently. Although many have
been simple walkthrough worlds
without much interaction, several
worlds have supported a variety of user
activities. The actual world
development activities themselves
have taken from a few days to a few
weeks of elapsed time. Except for the
work of the HITL, who transported
needed equipment between schools, all
these efforts employed equipment routinely available in the classroom.
3.2.3 Integration into the Curriculum
Teachers have had a variety of educational goals in encouraging students to develop their
own virtual worlds. In the work discussed here, Kelly Walsh High School has been the first
school where students developed virtual worlds as part of their regular classroom activities. In
math classes, VR technology is used to support pre-calculus and geometry instruction. One
activity requires students to build worlds where they can learn to plot 3D coordinates, and
another requires them to create geometric shapes to learn about angles and distances. More
complex activities are also undertaken, for example, where students are required to create a
world that models volumes of revolution. One of the goals of computer programming classes
is to learn about VR technology itself, and for their final project, students are tasked to research
an area in education where they feel VR technology would help students to learn the content.
They then create an educational world around this idea, producing virtual applications that are
Table 13. Educational Subjects
Ancient Structures Virtual Pyramid Communication Technologies Unit on Antarctica,
Unit on Sports (Stadium) Education EDTC 6242,
Computer Programming Class Environmental Science Wetlands Ecology,
Energy Conservation Class History Historical Home,
Virtual Cemetery Language Language Program
Literature Exploring Literature Mathematics Math Worlds, Mathematics (3D
Shapes), Virtual Stage Science Atomic/ Molecular Structure Class
Social Science HIV/ AIDS Protection Spatial Relations Special project (3D Puzzle)
Study Skills Virtual Museum Various Summer Camp '91 and '92,
VRRV Entrée
65
65
Page 66
67
52
©1998 Institute for Defense Analyses
often used in subsequent classroom activities. Examples of some of the virtual worlds that have
been developed include a world to model physical changes in molecules (water to ice and
back), a world to model chemical changes of molecules (wood, coal, diamonds), a world to
reenact WWII battles, a world that teaches about the Titanic, and a world that provides a
walkthrough of a built to scale house plan created for a geometry class.
As previously mentioned, Evans Bay Elementary School is one of the places where the
most student-development of virtual worlds has so far occurred. In New Zealand, the national
curriculum pays special attention to communications technologies and includes programs
aimed at meeting objectives from various skill and content areas. At Evans Bay Elementary
School, some of these requirements are being met by using VR as a tool for presenting the
results of research efforts. Initially, VR technology was used to support units on Antarctica and
sports. (The first of these was summarized in Figure 6 on page 44). In the second case, as part
of their study on sports, students examined a local debate as to whether or not the city council
should build a big sports stadium or upgrade the sewage system. To aid in their discussions,
some of the students developed a virtual sports stadium. Later work looked at how oral and
written language skills could be enhanced by encouraging students to explore new ways of
expressing their ideas. In this case, students worked cooperatively to create virtual
environments and develop stories within them. One group of students created a walkthrough of
the attic described in The Diary of Anne Frank. Another group constructed a world where the
participant finds himself alone in a futuristic prison and has to find out what has happened. The
story involved a complex hoax that fooled the prison warden into evacuating the prison.
More recently, in a math class at Evans Bay Elementary School, students were tasked to
develop set designs for a school performance of an abridged version of Macbeth. The staging
units available to work with were large wooden cubes and platforms of varying sizes. As shown
in Figure 8, the instructions required the students to work to scale, documenting their work
appropriately in their math books. Other efforts have included the creation of one virtual world
to present research information in a museum layout, and creation of another world to present a
classification and fitting epitaphs for famous and infamous people in history [Carey 1996a].
Another practical effort involving student development of virtual worlds is ongoing at
Haywood Community College, as part of the Exploring Literature in VR course. The objectives
of this course are to explore how writers use words, images, symbols, and settings to create a
mood, develop characters, and dramatize a story's theme or focus; discover the possibilities
that VR offers for providing a new perspective on a literary work; become acquainted with
various VR tools and strategies so that fictional worlds can be constructed; and freely express
reactions to a literary work and VR both in class discussions and in writing. The course requires
students to develop simple walkthrough worlds that depict major events in assigned stories.
Examples include scenes from My Kinsman, Major Molineux by Nathaniel Hawthorne, The
Yellow Wall-Paper by Charlotte Perkins Gilman, and A Jury of Her Peers by Susan Glaspell.
66
66
Page 67
68
53
©1998 Institute for Defense Analyses
East Carolina University provides instruction in how to use VR technology as an
educational tool. As part of a new VR concentration in the Master of the Arts in Education
Degree program, Course 6242 requires students to work with local teachers to identify a useful
context for using VR, develop the application, and then conduct a practical evaluation. So far,
this course has only been offered as a self-study class. While some students are currently
enrolled, none has yet completed the course.
Other efforts have been research oriented, investigating questions such as how student
development of worlds improves recognition and manipulation of 3D shapes, helps students
understand scientific concepts, or simply improves student motivation and general class
performance. Some of these efforts integrated world development into regular class activities
and others used special projects. The HITL has conducted most of this research, generally using
special projects. Examples include the VR program at the Pacific Science Center Summer
Camps in '91 and '92, a special course on Human Immune Virus/ Acquired Immune Deficiency
Figure 8. Student Tasking for Development of a Virtual Stage
Extension Maths
The set for Macbeth: We are staging our abridged version of Macbeth and need a series of set designs to con-sider.
We want you to prepare three design possibilities. The scene that you will be considering is the witches
scene attached to the back of this briefing sheet. Take note of the requirements of the scene. Where will actors
enter from and exit to? Where will they stand?
You have all used the VRML software to design things. This time you will use it to scale and I will be wanting
to see evidence that all the work has been done to true scale. You will need to show me written work in your
math books -where you will draw sketches and record measurements.
Begin by measuring the Multimedia room accurately.
What scale will you use?
How will you ensure that your measurements are accurate?
How will you measure the ceiling height?
Also measure the staging units that are in the media room. There are three different types.
Now use the VRML to build the media room. Put the preferences to centimeters and save this as the default.
As you work in any view there is a constant feedback of actual measurements.
Build the staging units in VRML to the same scale and keep these separate from the room.
Now save the file several times on the hard drive but give it different names, e. g. plan1, plan2, plan3.
Go to the first file and lay the staging units into the media room. Position them carefully and then place ava-tars
at the positions they would be standing in at asterix #1 on the script.
Now go to the second file and lay the room out differently. Position the avatars for asterix #2 on the script.
Repeat the process for asterix # 3 on the script.
67
67
Page 68
69
54
©1998 Institute for Defense Analyses
Syndrome (HIV/ AIDS) protection, and the VRRV Entrée program. These researchers have
supported students in developing virtual worlds meeting many themes. Examples of worlds
created by the students include Planetscape!! where two characters move with the participant
through a futuristic landscape, and Virtual Valley where a visual depiction of a valley enclosed
by mountains is supported by audio that reflects, for example, when an object is grabbed. In
Summer Camp '92, students tasked to develop a virtual world along an emotional theme
developed worlds called Peaceful Rainforest Peaceful, World Emotion, World Relaxation,
Spike World Intense Fear, Inca City Precarious, Free Space Free, Space Paradise Terror/ Joy,
and Future Dreams Relaxation/ Confusion/ Curiosity. As part of the VRRV Entrée program,
students developed a virtual world consisting of a space station where waste materials can be
recycled, and a rain forest where over-exploitation results in ecological disaster. The HITL
Wetlands Ecology work at Kellogg Middle School straddles the line between a special project
and regular class activities. This effort served as a VRRV pilot project and was designed to test
researchers' assumptions about the educational value of bringing VR technology to students.
The focus of this effort was to compare a constructivist pedagogical approach, using VR
technology, with traditional types of classroom activities.
Teachers and researchers at H. B. Sugg Elementary School, James Cook University, and the
Slaton Independent School District have all incorporated research activities into the regular
curriculum. In their math class, students at H. B. Sugg Elementary School created and
manipulated virtual pyramids as part of an investigation into whether such activities could
improve students' ability to compare, classify, and draw pyramid shapes. Researchers from
James Cook University used students in a primary school math class to help them look at the
effectiveness of virtual shape creation and manipulation in improving students' ability to
recognize and draw various 3D shapes. At Slaton High School, VR technology has been used
in two separate efforts. In the first, students researched atomic and molecular structures and
then developed virtual worlds demonstrating the structure of a selected atom or molecule. In
the second effort, students selected a particular geographical location where they would like to
live and researched its climate, natural flora and fauna, and various other conditions relevant
to living in that area. They then demonstrated their understanding by developing virtual worlds
that provided walkthroughs of houses designed to pay special attention to energy conservation
in heating and cooling.
3.2.4 Support for Students with Special Needs
The only identified effort that has considered how student development of virtual worlds
could support education for students with special needs has been conducted by the University
of Washington's HITL.
This research effort looked at how development of a virtual world could enhance spatial
processing skills in neurologically impaired children or, more specifically, at abilities in spatial
68
68
Page 69
70
55
©1998 Institute for Defense Analyses
relations, sequencing, classification, transformation and rotation, whole-to-part relationships,
visualization, and creative problem solving. The work was conducted as a special, one-week
intensive VR class at the end of the students' regular summer school program. Students were
given a workbook containing schedule information, visualization exercises, a reference card
for the software used to build the 3D objects, a set of 3D exercises, drawing paper, and writing
paper for making journal entries. Each day was divided into activity and thinking periods, with
the first two days including several group discussions. The children chose to work largely
independently, each creating puzzle pieces for an individual world. These objects were then
combined by HITL researchers in an overall virtual world that the children could visit.
3.2.5 Hardware and Software Issues
For each of the efforts where students develop virtual worlds, Table 14 and Table 15
summarize key aspects of the hardware and software platforms used. As would be expected
under the constraints of available financial resources, the majority of student world
development has been conducted on desktop machines, both PC and Macintosh computers.
The situation has been a little different with the HITL work where researchers visited summer
camps or various schools. Citing the lack of appropriate general use software, these researchers
had students develop virtual object models and behaviors on standard classroom computers
using Swivel-3D. They then took these world components back to the HITL where they were
combined to form worlds, and the students then travelled to the HITL to experience these
worlds.
Table 14. Hardware Support for Student-Developed Worlds
Organization
Platform Display Special I/ O
SGI Division
Workstation PC Macintosh Monitor Monitor w/ Shutter HMD Projection Screen CyberScope Spatialized Sound Hand Device
East C arolina U niversity 3 3 3 3 3 3
Evans Bay Intermediate School 3 3 3
Haywood Community College 3 3 3 3
Kelly Walsh High School 3 3 3 3 3
James Cook Univ., School of Education 3 3 3 3
Slaton Independent School District 3 3 3 3 3
University o f W ashington, HITL 3 3 3 3 3 33
H. B. Sugg Elementary School 3 3
69
69
Page 70
71
56
©1998 Institute for Defense Analyses
Again, probably driven by cost concerns, a much smaller proportion of the efforts employ
HMDs compared to the prevalence of HMDs with pre-developed educational VR applications.
Only half of the efforts used HMDs, the remaining efforts rely on desktop monitors either with
or without shutter glasses. Two efforts (one using an HMD and the other shutter glasses) also
used a projection screen and passive glasses as an alternative viewing mode.
Another way in which these
efforts differ from pre-developed
applications is in the reliance on
the visual display. There is no
evidence of speech I/ O or haptic
displays. The HITL did use
spatialized sound in one session
of the Summer Camp '91 effort
when two students returned for a
second session. Although these
students were successful in their
use of the technology, the
amount of effort required by both
the students and the researchers
prevented its continued use in
later sessions. The only
specialized hand input devices
used are wands or joysticks. There are probably several reasons for this minimal use of special
I/ O devices. Likely the most important factor is that extra skills are required to utilize
specialized devices, and students cannot be expected to master these extra skills in the limited
time they generally have available for developing virtual worlds. The fact that these devices
also require additional processing resources, quite significant resources in some cases, could
also be very relevant.
Virtus WalkThrough and VRDS are among the least expensive VR software development
packages for which support is available. They are also primarily focused at the PC level and
relatively easy to use. Accordingly, these packages have been the most popular choice for
student use. It will be interesting to see how increased use of more powerful PCs, or graphical
workstations, impacts this balance over the next few years; that is, whether the benefits of ease
of use and low cost continue to outweigh the increased development power offered by more
expensive products. Researchers at James Cook University, School of Education, are
conducting an informal study looking at the ease of use of Virtus WalkThrough and VRDS for
6th and 7th graders. When VREAM's new product, VR Creator, becomes available, these
researchers expect to compare the ease of use of Virtus WalkThrough Pro and VR Creator.
Table 15. VR Development Software for Student-Developed
Worlds
Organization
Superscape
/VRT
Virtus
WalkThrough/
Pro
VRDS REND
386
VRML Other East Carolina University 3 3 3
Evans Bay Intermediate School 3 3 3
Haywood Community College 3
Kelly Walsh High School 3 3 3
James Cook University, School of Education 3 3
Slaton Independent School District 3 3
University of Washington, HITL 3
H. B. Sugg Elementary School 3
70
70
Page 71
72
57
©1998 Institute for Defense Analyses
Other software used to support student development of virtual worlds includes Design it3D, 3D
Design Center, MacroMedia, Extreme 3D, and Hyperstudio.
Among the efforts discussed here, VRML has only been used by students at Evans Bay
Elementary School. Its use at this school, however, has been fairly extensive and supported by
the Virtus VRML 1. 0 software development package [Carey 1996b]. Once students had
familiarized themselves with this software, they began using VRML as a medium for research
presentations in several projects. This usage is still classed as exploratory, but the results so far
are encouraging; see Section 4.2 for data on evaluations of these efforts.
3.3 Multiuser, Distributed Worlds
In this section, the Narrative, Immersive, Constructionist/ Collaborative Environments for
Learning in Virtual Reality (NICE) and Virtual Physics applications are used as illustrative
examples of the type of work being conducted with multiuser, distributed educational VR
applications. These examples are shown in Figure 9 and Figure 10. The NICE project is a joint
development effort by the Interactive Computing Environments Laboratory and the Electronic
Visualization Laboratory at the University of Illinois at Chicago. The primary goal of this effort
is to study the effectiveness of a virtual environment as a conceptual learning and evaluation
medium. The Virtual Physics application is being developed by researchers at the University
of Lancaster in England, working with colleagues at University College London and
Nottingham University, on the Distributed Extensible Virtual Reality Laboratory (DEVRL)
project. In this case, researchers have been focusing on issues of collaborative learning. Since
only three applications in the category of multiuser, distributed educational VR applications are
discussed, NICE and Virtual Physics come close to defining the entire field. A summary of the
characteristics and usage of these three applications is given in Table 16.
As in the previous sections, only information for those VR applications that have been
developed or are currently under development is presented. However, it is useful to note some
plans for future development of multiuser, distributed educational VR applications.
Researchers at the SIMLAB at Carnegie Mellon University are planning the development of
an application called Collaboratory; they expect to work with educators and the Massachusetts
State Board of Education to develop a series of virtual worlds that will support collaborative
learning to meet curricula objectives in math, science, art, and music. The series of
ScienceSpace worlds is planned to be extended to support multiple users to enable research into
issues such as whether collaboration via users' avatars in a shared virtual world can support a
wider range of pedagogical strategies, and whether such environments will be effective
learning tools for students who are most motivated when intellectual content is contextualized
in a social setting. Researchers at the HITL are extending their Global Change application to
support multiple users so that children at Seattle's Childrens' Hospital can collaborate with
children in a nearby school. Finally, Learning Sites, Inc. plans to support distance education
71
71
Page 72
73
58
©1998 Institute for Defense Analyses
Figure 9. NICE
time with plants growing and animals populating newly formed ecosystems or migrating to other areas. In ad-dition to planting seeds and manipulating such variables as rainfall (by pulling a cloud over the land they want
to water), students can scale and po-sition parts of an ecosystem or factor
time to observe quickly and directly the effects of changes they make.
Stories that are created while inter-acting with the virtual world are au-tomatically
parsed to look like a picture book and placed on a Web
site. Avatars represent the group of children in each CAVE. These have
separate hands, body, and head that are mapped to a child's arm and head
to allow gestural interaction between participants as well as object selec-tion
and manipulation. Students can see their avatar bodies reflected, e. g.,
in water. Intelligent guides or genies provide guidance (e. g., a talking
signpost) or follow the children around sharing knowledge (e. g., So-fia
the friendly owl). Teachers can assume the role of genies in a manner
transparent to other participants. Focus of Current Work:
-Evaluation studies, including the creation of a real garden by children participating in school and com-munity
projects, in combination with the collabora-tive construction of the virtual ecosystem.
-Refine the interaction. -Develop an authoring tool, in the form of a simple vi-sual
language, to provide a user-programmable envi-ronment.
-Investigating issues of self-representation and non-verbal communication.
User-Programmable Environment: Will allow, for example, children to define a model of humidity and
growth for a particular plant or ecosystem, or construct new imaginary plants with their own set of rules.
Project Goal: To create a virtual learning en-vironment that is based on current educational
theories of constructivism, narrative, and col-laboration, while fostering creativity within a
motivating and engaging context.
NICE Description: The setting is a virtual is-land where children can search for empty
space and build their own ecosystems. Sym-bolic representations of various environmen-tal
elements are used to facilitate childrens understanding of complex ecological interre-lationships.
The microworlds evolve over
72
72
Page 73
74
59
©1998 Institute for Defense Analyses
Figure 10. Virtual Physics
-3D Pivot World. World consists of a table resting on a pivot (so that it moves in the x and z dimensions), and a number of objects of differing masses lying on the table. The task is to level the table by moving the ob-jects.
-Friction World. World consists of a snooker table with 10 differently colored balls rolling around. Unlabeled sliders control the extent to which the world obeys the laws of Conservation of Momentum and Conserva-tion
of Energy. Users can also adjust the elasticity of balls and coefficient of friction between table and balls, and induce impulse forces and repulsion forces between the balls. The task is to adjust the environment so
that the world behaves in a "real" manner and decide what the unlabeled controls do.
-Bowls World. World consists of a bowling green, a ramp for which slope and direction can be changed, jack ball, and three bowls for each player. Two sliders control the acceleration due to gravity and the frictional
coefficient of the green. Task is to roll bowls down the ramp to try and position them as near the jack as possible, building an understanding of the relationship between friction and weight.
Physics Laboratory (under development): -Provides empty lab where users can perform a
predefined experiment or define and conduct their own experiments in order to investigate
phenomenon.
-User enters an empty lab and selects from available objects. Behaviors are also selected
and used in their initial forms (real-world physics) or adapted to an imaginary world.
-Behaviors are expressed in mathematical for-mat as functions that amend object properties.
They may be built up in layers for each object.
-The lab has its own set of properties (e. g., grav-ity) which may be adapted in the same manner,
thus providing a set of laws for overall lab be-havior.
-Modes of intended use include students collab-orating on some investigative experiment and
using the lab as a communication tool for dem-onstrating rather than describing physical/ me-chanical
models.
Focus of Current Work: -Development of communication interfaces that integrate
effectively with simulations of the physical world.
-Study of how behavior is handled in the Physics Labora-tory to guide development of next generation VR toolkits.
Project Goals: To investigate how collabora-tive virtual worlds can be designed to provide
improved support for conceptual learning of physics.
Virtual Physics Description: A number of scientific worlds can be entered from a central
space. These worlds have been selected as providing motivation for collaboration. Cur-rent
worlds and their associated tasks are:
-Cannon World. World consists of a wall, tar-get, cannon with adjustable firing positions,
and cannonballs that are acted on by a uni-form gravitational field. One participant is
placed next to the cannon, and the second linked to the cannonball. The task is to hit a
target when the participant by the cannon cannot see it.
73
73
Page 74
75
60
©1998 Institute for Defense Analyses
Table
16.
Characteristics
and
Usage
of
Multiuser,
Distributed
VR
Applications
Developer
Application
Description
Learning
Objectives Supported
Intended Audience
Disp l a y
Usage
User
Organization
Date
of
Use
Co mp u t er M u s e u m
Network
Racer
A
game
where
3
people
work
cooperatively
to
move
a
packet
of
critical
information
from
Boston
to
Sydney
across
a
network,
choosing
different
routes
and
communication
methods.
There
are
3
stations:
2
net
racers
and
1
map
user.
Explain
networking concepts such as how data moves through many computers and how this is affected by speed of connections.
All
ages
Desk t o p
Exhibition
(part
of
The Networked Planet
Exhibit)
Computer
Museum,
Boston,
MA
1994 onward
Unive r sit y of Il lin ois at Ch icag o, I n t e r a c t i v e C o m p u t i n g L a b o r a t o r y a n d
E l e c t r o n i c V i s u a l i z a t i o n L a b o r a t o r y
NICE
Participants
explore
a
fantasy
island
with
a
range
of
soil,
altitude,
and
weather
conditions.
They
decide
where
to
plant
and
populate,
collaboratively
crafting
stories
by
building
small,
local
ecosystems
(such
as
vineyards
and
rainforests)
and
monitoring
them
as
they
develop.
The
world
can
continue
to
develop
even
when
there
is
no
interaction.
Engage
children
in
constructing
models,
say,
for
ecological interrelationships,
and
see
effects
of
changing
model
attributes.
Ages
6-10
CA VE w/ Glasses
Demonstration
SIGGRAPH
'96,
New
Orleans,
GA
August 1996
Evaluation
of
usability
and
educational effectiveness
Elementary
school
in
Urbana,
IL,
Oak
Park
School
and
hispanic
community
activities
in
Chicago,
IL
1996 ongoing
Demonstration
ThinkQuest, Washington,
DC
November 1996
Demonstration
SuperComputing
'96,
Pittsburg
November 1996
U n i v e r s i t y o f L a n c a s t e r , C o mp ut i n g Depa r t m e nt
Virtual
Physics
Number
of
scientific
worlds
entered
from
a
central
space:
3D
Pivot
World,
Cannon
World,
Bowls
World,
Friction
World,
and
Physics
Laboratory.
A
user
is
assigned
a
task,
such
as
refining
the
basic
physics
of
the
environment
to
model
the
real
world
(Friction),
or
defining
and
conducting
his
own
experiment
(Physics
Laboratory).
Develop
various
physics
concepts
via
construction
of
non-
symbolic
models.
Ages
16-
18,
college students
Desk t o p
Evaluation
of:
(1)
participant interaction, and (2) the level of physical knowledge that can be assimilated
University
of
Lancaster,
various
science
departments
1996
-
1997
74
74
Page 75
76
61
©1998 Institute for Defense Analyses
and distributed education with its Vari House application. The application will be made
available over the Internet and on CDs so that remote instructors can teach an entire class in
which all participants are immersed in a virtual world.
Teachers using VR technology at Evans Bay Intermediate School are hoping to set up a
virtual classroom, building on the concepts and work of an earlier project that looked at
telecommunications-enabled linking of learners. At the Correspondence School in New
Zealand, teachers also are intending to develop a virtual classroom application that supports, in
this case, second language learning.
3.3.1 Type of Use
There is little practical use of multiuser, distributed educational
VR applications to report. The only application that has seen such
use, Network Racer, has been available as an exhibit at the Boston
Computer Museum since October 1994. No additional venues for this
application are expected. The developers of the Virtual Physic
application are seeking funding to develop a practical use version of
their application, but have no firm plans for such additional work at this time. Whether NICE
is expected to transition into practical use at some time in the future is not known.
3.3.2 Educational Subjects Supported
Table 18 shows the different topics that this small
group of efforts support. As can be seen, they all focus on
science education rather than the arts.
3.3.3 Pedagogical Support
These applications all involve more than student walkthroughs of a virtual world. Instead,
students are required to actively collaborate in the virtual experience, often taking different
roles. For example, in the Network Racer application, three participants can change between
roles of a net racer and a map user in playing a game involving moving critical data across an
international network. This game is intended to support learning basic networking concepts.
Table 19 lists the different types of pedagogy supported by the three applications discussed in
this section.
NICE and Virtual Physics both have strong
pedagogical underpinnings that support investigating
the capabilities of collaborative VR as a learning
tool. NICE embodies principles of constructivism,
collaboration between real and synthetic users, problem solving, and authentic experiences to
support a distributed participatory theatre [Roussos et al., 1997]. More specifically,
Table 17. Classification of Applications
Practical Exhibition Use Network Racer
Research Vehicles NICE
Virtual Physics
Table 18. Educational Subjects
Environmental Science NICE Data networking Network Racer
Physics Virtual Physics
Table 19. Type of Pedagogical Support
Network Racer Guided-inquiry NICE Story-building, guided-inquiry,
collaboration Virtual Physics Guided-inquiry, collaboration
75
75
Page 76
77
62
©1998 Institute for Defense Analyses
constructivism is supported by activities that teach concepts involved in the creation and
maintenance of small local ecosystems. Students explore an island and select open land to plant
and populate. The behavior of different types of flowers, trees, plants, and animals in these
ecosystems is based on rules that, for example, dictate what happens to a particular plant when
it is close to other plants and given certain amounts of sunlight, rainfall, and weeding. Each
ecosystem continues to develop in the absence of the students at a predetermined rate. The
collaborative element of NICE allows students, or groups of students, working on separate VR
systems to interact in their different activities. Students can also interact with intelligent guides,
called genies, that not only provide information but interact with the students to help them
make decisions. Narrative plays a major role in NICE. As the students construct their
ecosystems, and even when events occur during the students' absences, every action is
recorded in the form of simple sentences, such as "Amy pulls a cloud over her carrot patch and
waters it." This recording is then parsed, replacing certain words with representative icons so
that the end product is a form of storybook. These books are made available on the students'
Web pages, and students can take home the story and reflect on it.
The approach taken for Virtual Physics was different. Again using principles of
constructivist learning and collaboration, the individual Virtual Physics worlds are each
designed to support the participants in cooperatively developing non-symbolic models of the
relevant physics concepts [Brna and Aspin, 1997]. This use of collaboration is based on
previous work investigating the educational utility of collaboration, in particular the
importance of maintaining a group's mutual understanding of the set of goals and how these
may be solved [Roschelle and Teasley, 1995; Burton and Brna, 1996]. The initial set of worlds
in Virtual Physics was specifically selected to motivate collaboration by providing students
with tasks that were difficult to perform independently. The most recent work focused on the
development of a Physics Laboratory for the Virtual Physics application. Using this new world,
researchers started to investigate issues in modeling behaviors in virtual worlds. The
researchers hope to continue investigating issues of collaboration and conceptual learning, and
to continue looking at topics such as how the interaction between representational fidelity,
immediacy of control, and presence impacts conceptual learning.
3.3.4 Hardware and Software Issues
The networking methods these applications use are of particular interest. Virtual Physics
uses support provided by the Internet and this most likely represents the trend of future
multiuser, distributed educational VR applications. NICE, on the other hand, is built on the
Graphical User Learning Landscapes in VR (GULLIVR) architecture that allows multiple
GULLIVRs running on separate VR systems to be connected via a centralized database that
ensures consistency across the environments. Multicasting is used to broadcast positional and
orientation information about each avatar, and the Transmission Control Protocol/ Internet
Protocol (TCP/ IP) is used to broadcast state information between the participants and the
76
76
Page 77
78
63
©1998 Institute for Defense Analyses
behavior system. A Java interface is currently under development to allow children with access
to the Web to use a special 2D version of NICE to interact with participants using CAVEs. A
simpler approach was taken for the Network Racer by exploiting the shared disk capabilities of
Windows for Workgroups 3.1 to build an application that would survive the loss of individual
nodes at runtime.
A summary of the hardware platforms, displays, and special I/ O devices used by these
applications is given in Table 20. Perhaps reflecting their intended use as practical applications
or research vehicles, Network Racer uses PCs, while the other two efforts both employ SGIs.
NICE is one of the few
applications that is using a
CAVE display to allow several
children to participate as a
group in a session with the
virtual world. Alternatively,
participants can use a more
limited version of a CAVE
(called the Immersadesk) that
uses a single projection screen
and passive glasses to support
stereoscopic viewing. Virtual
Physics supports both
immersive and desktop use.
Network Racer is a desktop
application. As yet, none of these applications has included any haptic or spatialized sound
displays. There is only limited use of speech I/ O (either speech recognition or digitized speech
output; no application supports both).
In terms of software support, Network Racer uses WorldToolKit while the others both use
less common VR development systems. Virtual Physics was developed using the Distributed
Interactive Virtual Environment (DIVE) system available from SICS in Sweden. In addition to
supporting the development of immersive, multiuser virtual worlds that can be networked over
the Internet, DIVE supports meetings between participants where each is represented by an
avatar. NICE is being developed using the facilities of GULLIVR that was designed to run in
a CAVE environment.
Table 20. Hardware Support for Multiuser, Distributed Worlds
Developer
Platform Display Special I/ O
SGI PC Monitor HMD CAVE
w/
Glasses
Speech
I/
O
Hand
Device
The Computer Museum 3 3 3 a
a. Digitized speech output only.
3
University of Illinois, ICE Laboratory and EVL 3 333 b
b. Speech recognition only.
3
University of Lancaster, Computing Department 3 33
77
77
Page 78
79
64
©1998 Institute for Defense Analyses
78
78
Page 79
80
65
©1998 Institute for Defense Analyses
4. Evaluations of VR Usage
This section reviews the evaluations that researchers and teachers have conducted on their
various uses of VE technology. As before, pre-developed applications, student development of
virtual worlds, and multiuser worlds are treated separately, in Sections 4.1, 4.2, and 4.3,
respectively.
In total, 35 completed evaluations have been performed on the identified efforts. Over 20
additional evaluations are currently underway or already planned. Just as the majority of
educational uses of VR technology have involved pre-developed applications, so have the
majority of evaluations to date been performed on this category of applications. The
proportions of completed evaluations performed in each category of pre-developed, student-developed,
and multiuser VR are 24: 10: 1, showing a higher predominance for evaluations of
pre-developed applications compared to those of student-developed virtual worlds. When
looking at only the number of efforts in each category that have been evaluated (19: 10: 1), the
proportions change only slightly. Work with multiuser, distributed applications began later than
that in other categories which is the primary reason for the low proportion of completed
evaluations of this type of application and, indeed, the low number of efforts themselves.
All the efforts that are primarily research oriented have been the subject of at least one
evaluation; this, of course, is to be expected. However, while over half of the pre-developed
applications in practical use have been evaluated, only two of the eleven identified practical
efforts where students developed worlds have seen a similar evaluation. While these figures
seem low, it must be remembered that all current educational uses of VR are, at least to some
extent, exploratory, and even where no explicit evaluations have been performed the
researchers and teachers are forming their own opinions of the value of the technology.
4.1 Evaluations of Student Use of Pre-Developed Virtual Worlds
Information on the set of completed evaluations of pre-developed virtual worlds is given in
Table 21. As this table shows, the majority of the pre-developed applications have been used
to conduct a single study and then laid aside. Applications that are the focus of a series of ongo-ing
evaluations are Virtual Environment Science Laboratory (VESL), Greek Villa, Newton-World,
MaxwellWorld, PaulingWorld, LAKE, Makaton World, Life Skills World, and Global
Change. At the current time, however, more than one evaluation has only been completed for
VESL, NewtonWorld, and MaxwellWorld. Overall, the evaluations have varied widely with
79
79
Page 80
81
66
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-Developed
Virtual
Worlds
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
ER G E n g i n e e r i n g ,
In c.
Cell
Biology
Evaluation
of
impact
of
immersion
(monoscopic) and interactivity on educational effectiveness
Informal
study
based
on
interviews
and
observations.
Subjects
viewed
material
using
HMD,
desktop,
or
video
tape.
Assessed
impact
on
symbolic
and
function
retention
of
human
cell
organelle
information,
time
spent,
enjoyment,
and
increased
interest
in
subject
matter.
-
Retention
overall
was
low,
but
immersive
VR
performed
best
for
symbolic
retention
and
non-immersive
VR
best
for
function
retention.
-
Time
spent
varied
the
most
for
immersive
users
and
was
consistently
underestimated
by
both
VR
groups.
-
Immersive
subjects
expressed
more
enjoyment
and
greater
likelihood
of
taking
a
free
biology
class.
G e o r g i a I n s t i t u t e o f T e c h n o l o g y , GVU Cen t er
Virtual
Gorilla
Exhibit
Formative
evaluation
Informal
study
where
students
were
observed
interacting
with
the
application,
and
subsequently
asked
questions
about
their
explorations.
-
VR
was
an
effective
tool
for
its
educational
objectives,
allowing
each
student
to
customize
their
learning
experience
to
best
suit
themselves
while
still
making
sure
they
were
exposed
to
most
important
facts.
Examples
of
potential
improvements
include
providing
students
with
an
automated
guide
to
answer
questions,
a
representation
of
a
virtual
body,
and
a
peer
to
interact
with.
-
Students
experienced
a
sense
of
presence
and
enjoyed
their
explorations. -Students had some
localization
problems
with
constant
amplitude
sounds.
CDS
Comparative
evaluation
of
immersive
design
as
a
concept
and
of
particular
user
interface
tools
Informal
study
based
on
evaluator
observations
and
subject
interviews.
Graduate
students
used
CDS
to
help
complete
a
10-week
architectural
design
project.
Compared
immersive
design
to
paper
or
desktop
computer-based
design.
Also
noted
successes/
problems
with
virtual
interface
techniques
such
as
virtual
pull-down
menus,
3D
positioning,
and
navigation.
-
Spatial
understanding
of
architectural
spaces
increased.
-
Overall
productivity
was
low,
but
specific
interaction
techniques
showed
promise.
-
Virtual
tools
adapted
from
the
desktop
metaphor
were
well
received
and
easy
to
use.
-
Small
changes
in
position,
size,
color,
or
texture
were
simple
to
make
and
provided
immediate
feedback.
-
Large
changes
to
geometry
and
creation
of
conceptual
designs
were
generally
difficult
and
cumbersome.
-
A
CAD
system
combining
traditional
desktop
modeling
with
an
immersive
option
would
be
useful
and
allow
the
strengths
of
both
environments
to
be
exploited.
In t e rface T ech nol o g i e s
C o r p o r a t i o n
VESL
Subjective
effectiveness evaluation
Informal
study
where
subjects
worked
with
VESL
and
then
completed
questions
about
its
effectiveness.
-
Students
consistently
rated
lessons
positively,
a
mean
of
7.0
(on
a
9-point
scale).
-
Teachers
gave
an
overall
educational
effectiveness
rating
mean
of
7.9,
with
particularly
high
scores
for
the
effectiveness
in
presenting
the
material
(8.7)
and
potential
for
integrating
VESL
in
the
classroom
(8.
7).
Subjective
usability
evaluation
Informal
study
where
subjects
worked
with
VESL
and
then
answered
questions
addressing
such
topics
as
aesthetics,
usability,
immersion,
update
rates,
graphics
quality,
quality
of
the
speech
and
sound
system,
and
utility
of
controls.
-
Mean
ratings
were
significantly
greater
than
5
(on
a
9-
point)
scale
for
each
of
aesthetics,
usability,
and
immersion.
Other
aspects
rated
equally
well.
The
overall
mean
rating
of
usability
was
6.
8.
80
80
Page 81
82
67
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-Developed
Virtual
Worlds
(continued)
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
In terfa ce T e c h n o l o g i e s
Co rp o r atio n ( c ont i n ued )
VESL (continued)
Effectiveness
evaluation
Used
multiple
choice
pre-/ post-
testing
of
knowledge
of
specific
physics
concepts.
Subjects
had
use
of
VESL
for
30
minutes
between
the
tests.
-
The
post-test
score
were
significantly
higher
that
pre-test
scores.
Ja mes Coo k Un i v ersity , S c h o o l o f E d u c a t i o n ,
Room
World
Evaluation
of
impact
of
immersion
on
recall
Used
post-testing
to
assess
recall
about
a
simulated
scene
after
exploring
that
scene
vs.
studying
18
photographs
of
the
scene.
-
VR
did
not
outperform
photos
in
ability
to
remember
objects
and
object
colors,
but
did
outperform
photos
in
ability
to
recall
numbers
of
each
object
and
object
location.
Great
Pyramid (from the Virtus Archaeological Gallery)
Subjective
effectiveness evaluation
Pilot
study
that
assessed
how
exploration
of
a
simple
virtual
pyramid
supported
learning
about
ancient
Egyptian
pyramids.
Used
videotape
recordings
and
subject
interviews.
-
Students
reported
that
books
were
more
successful
in
teaching
about
pyramids,
stating
that
the
virtual
pyramid
lacked
the
detail
and
life-
like
textures,
whereas
they
liked
the
book
pictures
and
explanations.
-
The
major
problem
was
in
navigating
sloping
passage
ways. -Students
particularly
appreciated
the
ability
to
see
the
design
view
of
the
corresponding
walkthrough
area.
No rth Caro l i n a S t at e Uni v ersi t y ,
Unive r sit y of No rth Caro l i n a
M e d i c a l S c h o o l
Street
World
Evaluation
of
whether
autistic
children
could
benefit
from
a
VR-based
learning
environment
Informal
study,
based
on
interviews,
that
assessed
2
autistic
children's
tolerance
of
an
HMD,
ability
to
identify
and
follow
a
car,
and
ability
to
move
to
and
stop
at
a
"stop"
sign.
-
Children
were
able
to
navigate
in
the
world
and
identify
objects.
They
demonstrated
learning
of
assigned
tasks.
Oreg on S t at e Un i v ersity ,
S c h o o l o f E d u c a t i o n
Spatial Relations
World
Evaluation
of
impact
of
immersion
on
spatial
problem
solving
abilities
Formal
study
based
on
pre/
post-testing.
Using
VR
and
desktop
Auto
CAD,
children
practiced
visualization,
displacement
and
transformation,
creative
thinking;
also
the
creation,
manipulation,
and
utilization
of
mental
images
in
solving
spatially
related
problems.
-
VR
aided
development
of
visualization,
displacement,
and
transformation
abilities.
Although
these
can
influence
spatially-
related
problem-
solving,
the
evidence
to
support
a
relationship
between
perceived
realism
was
inconclusive.
S h ef fi el d Ha l l a m
U n i v e r s i t y
Greek
Villa
Educational
effectiveness
Field
study
based
on
dialog
analysis,
questionnaires,
written
reports.
Assessed
how
collaborative
exploration
of
a
virtual
Greek
residence
can
support
learning.
-
Early
analysis
of
data
indicates
that
the
VR
experience
did
promote
"learning
talk"
between
students.
Analysis
is
ongoing.
81
81
Page 82
83
68
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-Developed
Virtual
Worlds
(continued)
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
Unive r sit y of Hou s ton , Geor g e M a so n Un i v ersity ,
an d NAS A JS C
NewtonWorld
Formative
usability
evaluation
Informal
studies
based
on
observations,
thinking
out
loud
protocols,
and
interviews.
Subjects
used
4
variations
of
the
interface
(menu-based,
gesture-based,
voice-based,
multimodal).
Assessed
task
completion,
task
times,
error
rates,
simulator
sickness
rates
and
nature;
and
subjective
ratings
of
task
difficulties
and
learner
motivation.
-
Users
were
comfortable
with
the
virtual
hand
and
bouncing
ball
metaphors.
-
The
majority
of
users
had
problems
navigating.
-
The
majority
of
users
ranked
the
multimodal
interface
(voice,
gestures,
and
menus)
above
the
others.
-
Voice
was
the
preferred
interaction
method
and
the
most
error-free.
Menus
were
also
well
liked
although
users
had
difficulty
in
selecting
menu
items.
Gestures
were
unreliable
and
the
least
preferred
interaction
method.
-
All
users
reported
slight
to
moderate
levels
of
discomfort/
eyestrain
after
wearing
HMD
for
approx.
1
1/
4
hours.
-
User
comments
suggested
that
the
ability
to
observe
phenomena
from
multiple
viewpoints
was
motivating
and
crucial
to
understanding.
Formative
subjective
effectiveness
evaluation
Informal
study
based
on
a
survey.
Participants
viewed
a
demo
and
then
experienced
the
world.
Survey
focused
attention
on
the
interactive
experience,
recommendations
for
improvements,
and
perceptions
of
potential
effectiveness.
-
A
large
majority
felt
that
NewtonWorld
would
be
an
effective
tool,
found
basic
activities
easy
to
perform,
and
were
enthusiastic
about
3D
nature
of
learning
environment
and
the
ability
to
observe
phenomena
from
different
viewpoints. -Some participants raised
concerns
regarding
limitations
of
prototype
and
encouraged
expanding
activities,
environmental
controls,
and
sensory
cues
provided.
-
Participants
experienced
some
difficulty
using
menus,
several
recommended
a
broader
field-of-
view,
and
some
had
difficulty
focusing
HMD
optics.
Evaluation
of
impact
of
multi-
sensory
interface
on
effectiveness
Formal
experiment
based
on
observations,
student
comments
and
predictions,
interviews,
usability
questionnaires,
and
pre-
and
post-tests.
Focused
on
both
the
importance
of
multi-sensory
experience
and
reference
frame
usage
in
learning.
Used
3
groups
of
subjects
differentiated
by
controlling
visual,
haptic,
and
auditory
cues.
-
Single
session
usage
was
not
enough
to
dramatically
improve
users'
mental
models.
-
Students
receiving
sound/
haptic
cues
rated
the
application
as
easier
to
use
and
the
egocentric
reference
frame
as
more
meaningful
than
those
receiving
visual
cues
only.
-
Students
who
received
haptic
cues,
in
addition
to
sound
and
visual
cues,
performed
slightly
better
on
questions
relating
to
velocity
and
acceleration,
but
worse
on
predicting
system
behavior.
-
Several
users
experienced
difficulty
with
eye
strain,
navigating,
menu
usage;
these
problems
interfered
significantly
with
the
learning
task.
-
Students
suggested
improving
the
learning
experience
by
expanding
features
and
representations
used,
and
by
adding
more
variety
to
the
nature
of
learning
activities.
82
82
Page 83
84
69
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-
Developed
Virtual
Worlds
(continued)
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
U n i v e r s i t y o f H o u s t o n , Geo r g e M a son Un iversity ,
an d NAS A J S C (c o n t i n u ed )
MaxwellWorld
Formative
usability,
learnability,
and
effectiveness
evaluation
For
usability
and
learning
studied
observations,
students'
predictions
and
comments,
questionnaires,
and
interview
feedback.
Formal
evaluation
of
effectiveness
based
on
pre-
and
post
tests.
Evaluated
the
application
as
(1)
a
tool
for
remediating
misconceptions
about
electric
fields,
and
(2)
teaching
concepts
about
electric
fields.
Tested
for
retention
of
material
over
time
by
conducting
a
third
session
approximately
2
weeks
after
2
prior
sessions.
-
Majority
of
students
commented
that
they
felt
MaxwellWorld
was
a
more
effective
way
to
learn
about
electric
fields
than
either
textbooks
or
lectures.
-
The
3D
representations,
interactivity,
ability
to
navigate
to
multiple
perspectives,
and
the
use
of
color
were
cited
as
important
characteristics
that
aided
learning.
-
Students
demonstrated
improved
understanding
of
physics
concepts,
learning,
for
example,
the
ability
to
describe
distribution
of
forces
in
an
electric
field,
and
identify
and
interpret
equipotential
surfaces.
-
Although
the
world
helped
students
qualitatively
understand
3D
superposition,
they
had
difficulty
applying
superposition
when
solving
post-test
problems.
Comparative
effectiveness
evaluation
Formal
experiment
based
on
pre/
post-
tests
and
questionnaires.
Compared
MaxwellWorld
to
the
EM
Field
computer-
based
simulator
on
the
extent
to
which
representational
aspects
(2D
vs.
3D
and
quantitative
vs.
qualitative)
of
the
simulations
influenced
learning
outcomes.
Lessons
used
utilized
only
those
features
of
MaxwellWorld
that
had
counterparts
in
EM
Field,
thus
focusing
on
electric
fields
and
electrical
potential.
Impact
of
multisensory
cues
(via
a
haptic
vest)
in
MaxwellWorld
was
also
examined.
-
All
students
demonstrated
a
better
overall
understanding
of
topics
on
the
post-
test,
though
students
with
an
initial
moderate
knowledge
benefiting
less
than
those
starting
with
little
or
no
knowledge
at
pre-test.
More
advanced
students
had
difficulty
overcoming
misconceptions.
-
In
most
areas,
each
group
of
students
performed
similarly.
The
MaxwellWorld
group,
however,
was
better
at
describing
the
3D
nature
of
electric
fields,
potentials,
and
their
respective
representations;
EM
Field
students
typically
restricted
answers
to
a
single
plane.
-
Students
rated
MaxwellWorld
as
easier
to
understand,
but
EM
Field
as
easier
to
use.
MaxwellWorld
was
rated
as
more
rewarding,
stimulating,
and
informative.
-
Students
who
had
difficulty
with
concepts
found
multisensory
cues
helped
them
understand
representations. -After 5 months,
both
groups
performed
similarly
on
retention
tests,
with
MaxwellWorld
students
showing
a
slight
advantage
in
their
ability
to
describe
electronic
fields
in
3D.
U n i v e r s i t y o f I o an ni na , Dep t . o f
P r i m a r y E d u c a t i o n
LAKE
Evaluation
of
usability,
and
comparison
of
navigation
devices
Informal
study
based
on
questionnaires
and
discussions.
Subjects
spent
up
to
1
hour
in
a
virtual
office
world
and
then
in
LAKE.
-
Students
preferred
using
a
traditional
mouse
as
the
navigation
device
and
did
not
try
very
hard
to
use
the
other
devices
(joystick
or
a
spaceball/
spacemouse).
-
Students
ranked
the
mouse
higher
for
ease
in
learning
and
use;
the
joystick
and
spaceball/
spacemouse
were
ranked
the
same. -Half the students
stated
that
they
had
immersive
experiences
such
as
"I
had
the
feeling
the
I
was
moving
in
the
real
world"
even
in
this
desktop
application.
83
83
Page 84
85
70
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-Developed
Virtual
Worlds
(continued)
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
U n i v e r s i t y o f M i c h i g a n , D e p t . o f C h e m i c a l E n g i n e e r i n g
Vicher
(I
and
II)
Formative
user
evaluation
(I)
Informal
study
based
on
questionnaires.
Users
answered
engineering
questions
and,
after
using
the
Vicher,
were
given
the
opportunity
to
modify
their
responses;
they
also
responded
to
general
questions
about
the
virtual
experience.
-
User
responses
after
Vicher
were
more
accurate,
more
complete,
and
showed
a
better
understanding
of
engineering
concepts.
-
Over
80%
responded
that
they
felt
they
had
learned
something
from
the
experience.
Safety
World
Formative
user
evaluation
Informal
study
based
on
evaluation
forms.
Users
rated
model
components
and
the
system
as
a
whole,
and
gave
short
answers
for
suggestions
on
improvements
and
other
feedback.
One
group
evaluated
the
safety
and
hazards
of
the
plant
from
a
written
description
and
then
evaluated
the
VR
representation.
The
other
group
used
both
the
VR
experience
and
the
written
description
in
judging
the
safety
and
hazards
present
and
then
evaluated
the
VR
system.
-
Most
of
the
students
rated
the
current
value
of
the
system
as
medium
to
low,
but
rated
its
potential
very
highly.
The
same
trend
was
observed
for
ratings
on
the
help
system
and
HMD
usage.
-
Student
rankings
on
their
understanding
of
the
chemical
process
and
its
hazards
increased
as
a
result
of
the
virtual
experience.
Across
the
students,
these
rankings
formed
a
Bell
curve
centered
between
2
and
3
(on
a
5
point
scale).
-
Most
common
complaint
was
difficulty
in
navigation.
-
Comparison
between
the
2
groups
did
not
show
any
significant
variations
in
their
evaluations
of
the
model,
the
equipment,
the
help
system,
or
the
future
potential
of
these.
U n i v e r s i t y o f N o t t i n g h a m , VIRAR T Gro u p
Makaton
World
Pilot
study
Informal
study
looking
at
how
well
the
system
was
received
by
students
and
teachers.
-
While
some
students
had
difficulty
in
exploring
the
individual
warehouses
and
identifying
correct
objects
in
the
reward
warehouse,
most
students
were
able
to
recognize
at
least
some
of
the
objects
in
the
reward
warehouse. -Most of the
students
recognized
the
hand
signs
displayed
with
each
different
object
and
immediately
mimicked
the
sign. -Some
students
had
difficulty
manipulating
either
a
mouse
or
spaceball,
preferring
to
use
a
touchscreen
to
point
to
objects
to
be
selected.
Life
Skills Wor l ds
Evaluation
of
skill
transfer
to
real
world
(Virtual
Supermarket)
and
promotion
of
self-directed
activity
Formal
experiment
based
on
examination
of
performance
during
training
with
VE,
videotapes
of
staff
and
student
interactions
while
using
the
VE,
and
pre/
post-testing
at
a
supermarket.
-
Students
who
used
the
virtual
supermarket
were
significantly
faster
and
significantly
more
accurate
on
the
return
to
the
real
supermarket
than
those
in
the
control
group. -While using
the
VE,
there
was
a
significant
decrease
over
time
of
teachers'
activities.
-
Teachers'
activities
decreased
at
a
faster
rate
for
didactic
categories
(such
as
instruction,
physical
guidance)
than
for
more
open-ended
assistance,
such
as
suggestion.
-
Students
showed
a
significant
increase
in
learning
speed
for
each
of
the
categories
into
which
their
self-directed
activities
were
coded.
84
84
Page 85
86
71
©1998 Institute for Defense Analyses
Table
21.
Completed
Evaluations
on
the
Use
of
Pre-
Developed
Virtual
Worlds
(continued)
Performing Organization
VR
Application/ World
Purpose
of
Evaluation
Description
Major
Findings
Unive r sit y of W a shing t on , HIT L
Atom
World
Evaluation
of
impact
of
immersion
and
interactivity
on
educational
effectiveness
Used
written
and
oral
pre/ post-
testing
to
assess
how
immersive
and
non-immersive
VR,
non-interactive
computer
program,
videotape,
and
no-
treatment
control
instruction
facilitated
factual
recall
and
comprehension
of
principles.
-
For
recall,
the
immersive
VR
outperformed
the
control
group
only.
-
For
comprehension,
the
advantage
of
VR
lay
in
the
interactivity
it
offered,
not
in
immersion.
-
For
long-
term
retention,
although
immersive
VR
scores
dropped
from
initial
post-test
level,
immersive
VR
maintained
a
significant
improvement
over
pre-test
scores.
Zengo
Sayu
Comparative
educational
effectiveness
Used
pre/ post-
testing
to
assess
how
VR
and
non
VR-based
whole
language
learning
compared
with
existing
computer-based
instruction
given
in
English.
-
Instruction
based
in
Japanese
(Zengo
Sayu
and
the
equivalent
non-
VR
instruction)
gave
a
significant
performance
improvement
over
the
English
computer-
based
instruction.
VRRV
Hors
d'Oeuvre
Assess
usability
and
motivational
impact
of
VR
educational
applications
Used
exit
questionnaire
to
assess
the
ease
of
use
during
short
experiences
with
VR,
students'
enjoyment
of
the
experience,
and
students'
sense
of
presence.
-
Students
rated
their
enjoyment
as
very
high
with
a
negligible
number
of
reports
of
queasiness.
-
Difficulty
in
navigating
around
the
world,
and
interacting
with
it,
decreased
with
age.
-
Enjoyment
and
sense
of
presence
decreased
with
age.
85
85
Page 86
87
72
©1998 Institute for Defense Analyses
respect to their formality. Some have included both informal subjective measures as well as
more formal pre-and post-testing.
In general, the types of
issues being addressed in
evaluations fall into two broad
groups: those relating to
effectiveness, and those
addressing usability. Table 22
identifies the applications
evaluated in each category,
and shows how these
categories can be sub-divided
to further distinguish the focus
of different evaluation efforts.
The remainder of this section
follows the hierarchy given in
Table 22 to structure the
discussion of particular evaluations.
4.1.1 Evaluation of Effectiveness
4.1.1.1 General Educational Effectiveness
One of the largest development and evaluation efforts is that being performed on the
ScienceSpace series of worlds by researchers at University of Houston-Downtown, George
Mason University, and NASA's Johnson Space Center (JSC). The major goal of this work is to
examine whether the type of immersion and multisensory communication available from VR
technology can help students construct accurate mental models of abstract science concepts and
remediate deeply rooted misconceptions about the relationships among mass, force, motion,
acceleration and velocity.
The first ScienceSpace world, NewtonWorld, has been the subject of three effectiveness
evaluations to date. The first was a subjective educational effectiveness evaluation by 100
physics educators and researchers attending a conference. After watching a 10-minute
demonstration of NewtonWorld, these participants were guided through a number of activities
and then asked to complete questionnaires. A second, this time formal, evaluation looked at the
impact of a multisensory interface (providing auditory and haptic cues in additional to visual
cues) on educational effectiveness. Here, in the course of a two and a half to three hour
instructional session, students spent one and a quarter hours inside NewtonWorld. During their
interaction with the virtual world, students predicted relationships among factors and behaviors
Table 22. Types of Completed Evaluations
Effectiveness Evaluations
General educational effectivenessVirtual Gorilla Exhibit, VESL, Greek Villa, NewtonWorld, MaxwellWorld,
Vicher (I and II), Safety World
Effectiveness for learning disabled students Street World, Makaton World, Life Skills World
Comparative educational effectiveness CDS, Great Pyramid, Zengo Sayu
Impact of immersionCell Biology, MaxwellWorld, Spatial Relations World, Atom World, Room
World
Usability Evaluations
General usabilityCDS, NewtonWorld, MaxwellWorld, LAKE
Usability for physically and learning disabled students VESL, Makaton World, Street World
Sense of presence, ease of navigation, enjoyment Virtual Gorilla Exhibit, Safety World, VRRV Hors d'Oeuvre
86
86
Page 87
88
73
©1998 Institute for Defense Analyses
of bouncing balls and then compared their predictions to virtual world observations. Analysis
of the collected data failed to show any significant increase in knowledge as a result of the
instruction, and the researchers suggest that multiple sessions with NewtonWorld may be
needed to overcome deeply seated science misconceptions [Salzman et al., 1995b]. With
respect to the use of multisensory cues, sound and haptic cues seemed to engage learners and
direct their attention to important behaviors and relationships more than visual cues alone.
Currently, this group of researchers is conducting an evaluation of the content and lesson
structure of NewtonWorld. Additional evaluations that will investigate the impact of age group,
ego versus exocentric viewpoints, and, again, multisensory interfaces on effectiveness will be
started in the fall of 1997. After these studies, the final version of NewtonWorld is expected to
undergo field testing.
The first formal evaluation of the educational effectiveness of MaxwellWorld provided
students with three 2-hour sessions in the world and resulted in more positive evidence of the
learning effectiveness than was found with single-session use of NewtonWorld [Dede 1997b].
Here pre-and post-tests showed that students improved their understanding of the distribution
of forces in an electric field and the use of test charges and field lines. Manipulating the field
in 3D appeared to play an important part in this learning. MaxwellWorld helped students
qualitatively understand 3D superposition, and though students had some difficulty in applying
superposition in the post-test problems, overall post-test performance was good with all
students demonstrating an understanding of concepts such as Gauss's Law, field versus flux,
and directional flux. A more recent evaluation of MaxwellWorld compared the effectiveness of
MaxwellWorld with that of a non-immersive application providing similar functionality; this
is discussed in Section 4.1.1.4. The researchers plan to begin field testing the application in
1998.
The work with the ScienceSpace worlds provides the best example of where evaluations
have been effectively used to support the development of educational VR applications. This
process used what the researchers term a learner-centered approach, defined as a special case
of user-centered design with the needs of learners and the ability of technology to support the
learner taken into consideration. Accordingly, the development process consists of an iterative
process of design and evaluation, with the evaluation addressing issues of both usability and
educational effectiveness. Table 13, based on data from Salzman et al. [1995a, 1995b], shows
how the results of a series of evaluations led to progressive design refinements. A similar cycle
of iterative design and evaluation is being performed for MaxwellWorld, and is expected for
PaulingWorld. Based on their work to date with the ScienceSpace worlds, the researchers are
developing design heuristics, assessment methodologies, and insights that they hope will be
generalizable to a wide range of educational environments.
Another application that, like NewtonWorld and MaxwellWorld, has been the subject of a
series of evaluations is VESL. The goal of the VESL's developers is to provide an application
87
87
Page 88
89
74
©1998 Institute for Defense Analyses
Table 23. Impact of Findings on Refinement of NewtonWorld
First evaluation— formative usability and learning
Positive Findings Negative Findings
-Students work well with the bouncing ball metaphor and
catch-throw activities -The virtual hand metaphor worked well
-The majority of students preferred the multimodal inter-face
(voice, gestures, and menus) to any single mode -Voice commands were easy to use, most error-free, and
preferred method of interaction
-Menus were well liked, but selecting items was difficult -Students found that having multiple viewpoints of phe-nomena
was motivating and crucial to understanding
-Additional visual, auditory, or tactile cues were needed to smooth the interaction and help focus on important
information
-Navigation (by pointing the hand in the desired direction
of travel) presented some difficulties -Hand gestures were unreliable and the least-liked inter-action
mode
-The least-liked interface characteristic was the inability to focus the HMD optics
-All participants indicated slight to moderate levels of
eyestrain and discomfort after using the application for over 1 hour
-Students interpreted the size of the ball as a cue for mass,
reinforcing the misconception that larger objects are more massive
Changes: -Expanded number of viewpoints from 2 to 5 with a "beaming: method for moving between views
-Incorporated a standard navigation training procedure for the introduction to NewtonWorld
-Eliminated gesture-based commands and explored feasibility of incorporating voice commands (these were simulated for the experiment)
-Provided sound cues to supplement visual cues
Second evaluation— subjective effectiveness educational evaluation
Positive Findings Negative Findings
-A large majority of participants felt that NewtonWorld
would be an effective teaching tool -A large majority found basic activities, including naviga-tion,
easy to perform
-Many were enthusiastic about the 3D nature of Newton-World and the multiple viewpoints for observing phe-nomena
-Many participants experienced difficulty using the
menus -Several felt that a broader field-of-view would have
improved their experience
-Many had difficulty focusing the HMD optics -Several expressed concerns regarding the limitations of
the prototype and encouraged expanding activities, envi-ronmental
controls, and adding more sensory cues
Changes: -Expanded interface to include a haptic vest and more extensive visual and auditory cues
-Refined menus to make item selection easier -Changed menu bar to a small 3-ball icon to increase visual field and improve experience of motion
Third evaluation— impact of multisensory interface on educational effectiveness
Positive Findings Negative Findings
-Students appeared more engaged in activities when more multisensory cues were provided
-Students receiving sound and haptic cues rated the world easier to use than those receiving visual cues only
-The more cues used, the higher the rating for ease of use
with which students could understanding what was hap-pening from the egocentric frame of reference
-Students receiving haptic cues in addition to visual and
auditory cues performed slightly better on questions relating to velocity and acceleration
-Single, short session usage of NewtonWorld did not sig-nificantly transform students' misconceptions
-Students receiving haptic cues in addition to visual and auditory cues performed slightly worse on predicting the
behavior of the system
-Several students experienced difficulty with eye strain, navigating, and selecting menu items; these problems
interfered with the learning task
Changes:
-Moved menu from fixed location in HMD field-of-view to user's second virtual hand -Refining auditory and tactile cues to provide richer information and to allow turning on/ off for individual lessons
-Expanding NewtonWorld to include a broader range of learning activities
88
88
Page 89
90
75
©1998 Institute for Defense Analyses
that can be used by students with physical disabilities, such as cerebral palsy and spinal cord
injuries, as well as by non-disabled students. Accordingly, usability has been a key concern, as
is discussed in Section 4.1.2. In terms of educational effectiveness, VESL has been the subject
of two evaluations. The first was an informal subjective evaluation where both students and
teachers participated [Nemire 1995b]. Both groups rated VESL highly on such topics as
educational effectiveness and potential for integrating VESL into the classroom curriculum. In
the second effectiveness evaluation, a small number of students were given a 30-minute session
with VESL and, based on their pre-and post-test scores, gained a significantly better
understanding of the physics concepts covered by VESL.
A formative user evaluation provided researchers at Georgia Institute of Technology's
GVU Center with useful information about an initial design of the Virtual Gorilla Exhibit. The
goal of these researchers' work is to improve their understanding of how VR can be used as an
educational tool to provide general knowledge to children (as opposed to providing task
training to adults). The informal evaluation provided feedback that suggested some potential
improvements to the application. For example, providing a guide to answer questions and
speed the learning process, providing students with a virtual body so they can see themselves
in the virtual world, and providing students with a peer to play with instead of just being at the
bottom of the gorilla social ladder. At present, the focus of this work is more on the
technological side of VR, and user comments and qualitative observations have been used to
determine what was working and how well it was working. Future work is expected to look
more formally at the educational effectiveness of the exhibit.
The evaluation of Greek Villa is similar to those performed on the ScienceSpace worlds in
focusing on pedagogical issues. In this case, the researcher undertaking the work is interested
in using VR to encourage group working and, in particular, is using VR to investigate the
possible role of "learning talk." Accordingly, the informal evaluation of Greek Villa focused
on the communication between students, and the results indicate that learning was apparent in
the talk that took place within the groups using VR [Grove 1996]. Further evaluation is
expected to investigate these issues more formally.
The development of the Vicher and Safety World applications at the University of Michigan
has been part of a research effort that has three goals:
° Providing educational modules with as much practical use to as many students as
possible.
° Determining what educational situations in engineering will benefit most from VR
technology, and
° Developing techniques for the display of, and interaction with, scientific and
technological information.
89
89
Page 90
91
76
©1998 Institute for Defense Analyses
The program of evaluations underway and planned for the Vicher and Safety World
applications is designed to support these goals. The initial evaluation of each of Vicher I and
Safety World were formative in nature, that is, intended to guide further development of each
application. For Vicher I, students completed a questionnaire soliciting answers to specific
engineering questions both before and after exposure to Vicher, and also provided some
general information about the virtual experience [Bell and Fogler, 1995]. User responses after
the experience with the virtual world were more accurate and more complete, and showed an
improved understanding of the pertinent engineering concepts. Most of the students felt that
they had learned through the experience. One feature that was particularly appreciated was the
3D color graph of reaction kinetics and its relationship to the packed bed reactor; students
cited this graph as an example of how the virtual world gave them a more tangible grasp on
the meaning behind the relevant equations. In the case of Safety World, students conducted a
safety and hazard analysis using the virtual world and then completed evaluation forms [Bell
and Fogler, 1996]. While students' analyses continue to be studied, initial findings are
available. Students ranked the current value of the safety world simulation, help system, and
HMD usage as quite low but, in each case, over 75% of the students saw potential values as 4
or 5 on a 5-point scale. The applications were refined based on the feedback received from
these evaluations and data from the next two formative evaluations (this time including Vicher
II) are already being analyzed. Meanwhile, off-site beta testing has begun.
In addition to the ongoing and planned evaluations of the ScienceSpace worlds, Vicher I
and II, and Safety World already mentioned, other evaluations of educational effectiveness are
currently underway. The HITL is completing data analysis on the evaluation of the educational
effectiveness of its Global Change application, field testing of Vari House is being conducted,
and Crossing Street is being used to assess the transfer of skill-related knowledge gained in a
virtual world to the real world.
4.1.1.2 Effectiveness for Learning Disabled Students
The VIRART group at the University of Nottingham has conducted evaluations looking at
the effectiveness of educational VR applications for learning disabled and autistic students. Its
first work was with Makaton World. An early, informal pilot study found that most students
were able to recognize and identify objects and hand signs. This study was followed by a more
formal evaluation of the potential benefits of the application and its role in the school
curriculum. Details on this later evaluation are not yet available.
The Life Skills World application, designed to promote self-directed activity in learning
disabled students, is also the subject of continuing assessment. An initial evaluation examined
how skills learned in a virtual supermarket transferred to a real supermarket [Cromby et al.,
1996]. In the real shopping task, students were asked to find four items (identified by pictures
on cards), put these items in their trolley, and take them to the checkout. Compared to their
90
90
Page 91
92
77
©1998 Institute for Defense Analyses
initial baseline performance, those students who received training in the virtual supermarket
performed significantly better than the control group of students who had experienced different
worlds from the Life Skills Worlds application. Students in the experimental group performed
the task faster, and collected more correct items. The researchers note that it is possible that the
advantage of the experimental group could have been caused by a greater familiarity with the
shopping task, or the fact that the experiences with the Virtual Supermarket were more
structured than those of the control group. To help resolve such uncertainty, the researchers are
currently investigating changes in sub-skills such as memorizing items on the shopping list.
They are also looking to develop measures for assessing generalized skills such as autonomy
and decision making. Meanwhile, a second evaluation looking at the effectiveness of the Life
Skills Worlds' Virtual House world for learning simple household tasks is ongoing. Evaluation
of VIRART's AVATAR House is also ongoing.
Finally, an evaluation of Street World showed that autistic children could learn to perform
simple tasks in a VE. This study demonstrated that students were able to master tasks of
recognizing and identifying a moving car, and finding and walking towards a stop sign.
Building on these positive results, these researchers are developing a second application,
Object World, that will be used to investigate whether VR can be a useful medium for teaching
autistic children to recognize classroom objects as compared to conventional teaching
techniques.
4.1.1.3 Comparative Educational Effectiveness
Additional studies have compared the effectiveness of VR-based instruction with existing
non-VR teaching practices. These studies have been on a smaller scale than those just
discussed, but still provide useful evidence for the benefits of VR technology.
Researchers at Georgia Institute of Technology, GVU Center, used CDS to look at whether
immersive VR aided the development of architectural design skills. CDS allows some
architectural design activities to be accomplished from within the virtual world, for example,
changing the color or texture of objects, and adjusting object orientation and position. Rather
than progressing from 2D drawings to 3D models, it allows students to start with a type of 3D
sketchpad. In an informal study, graduate students from the College of Architecture used CDS
to complete a 10-week architectural design project. When compared with designs developed
using paper or more traditional desktop design packages, the products of these students showed
increased spatial understanding of architectural spaces. While the ability to make small design
changes on-the-fly was valuable, it was difficult to make large changes. As a result, the
researchers recommend combining traditional desktop modeling with an immersive option to
allow the strengths of both environments to be exploited.
Researchers at James Cook University, School of Education, informally compared the
effectiveness of navigating through a virtual world against the use of textbooks when learning
91
91
Page 92
93
78
©1998 Institute for Defense Analyses
about Egyptian pyramids. This study used a simplified version of the Great Pyramid virtual
world included in the Virtus Archaeological Gallery. A small number of 7th grade students
were given an introductory lesson on Ancient Egypt and shown some high quality pictures and
diagrams of the pyramid. The students then explored the virtual Great Pyramid, identifying
those aspects of the virtual world that they liked and making suggestions for improvements
[Ainge 1996c]. Overall, the students stated their belief that the books were more successful for
teaching them about the pyramid. While the students felt that the virtual world gave them a
good feel for the size of the pyramid and the narrowness of the passageways, this particular
world lacked detail and life-like texture. Problems in navigation and the unrestricted movement
allowed by the absence of collision detection were often frustrating obstacles to explorations.
The evaluation of Zengo Sayu compared the use of a VR Japanese-based second-language
learning approach with a similar approach using real-world objects, and with a traditional
computer text-based approach. The key research questions being asked were: (1) Can students
using Zengo Sayu learn some of the Japanese language? and (2) How does Zengo Sayu
compare to other teaching methods in terms of language gains? All three treatments covered
the same content (five colors, two nouns, two verbs, and five prepositions) and were designed
to present instruction and offer rehearsal as consistently as possible. The main finding was that
the instruction based in Japanese (both the Zengo Sayu and real-world treatments) gave a
significant performance improvement over the English text-based instruction, and these two
treatments themselves did not provide significantly different results [Rose and Billinghurst,
1996]. Thus indicating that, at least for Zengo Sayu, the VR-based instruction can be as
effective as that provided by physical instructors.
Additional evaluations comparing VR-based instructional approaches with conventional
classroom methods are underway at the University of Washington, HITL, using Phase World,
and at North Carolina State University, using Object World.
4.1.1.4 Impact of Immersion
Cell Biology serves as another example of the effective use of formative evaluations and as
an example of a study focusing on the impact of immersion and interactivity on effectiveness
of VR-based instruction [Gay 1994]. The initial application was designed much like a textbook
in that users first learned about cell requirements and cell functions before building a cell and
testing its structure. In the first part of the evaluation, the researchers found that it took users a
while to get used to the VR interface which distracted users from paying attention to the
content. In addition, users were unable to remember the organelle function information that
they needed to repair incorrectly structured cells. Consequently, Cell Biology was redesigned.
One of the changes allowed users an opportunity to get acquainted with the interface before any
learning began. Another change was to remove the reliance on remembering cell requirements
by having organelles explain their function when selected and providing instant feedback on
92
92
Page 93
94
79
©1998 Institute for Defense Analyses
their correct usage. In the second part of the evaluation, researchers again used museum visitors
to compare the impact of immersive, desktop, and video-tape viewing of the refined
application. Overall, the interactive (immersive and desktop) users scored higher on post-testing
of symbolic and graphic retention. But desktop users performed slightly better than the
immersive users, although the researchers suggest that the differing resolution between the
HMD used and monitor might account for this difference. However, immersive users did report
more enjoyment, and more of these users stated that they would take a free biology class, as
compared to users in the other two groups.
The ScienceSpace researchers compared the effectiveness of MaxwellWorld with that of a
commercial 2D microworld called EM Field (EMF) [Salzman et al., 1997a, 1997b]. This
commercial application is widely used and covers much of the same material as
MaxwellWorld. The evaluation was performed in two stages focusing, respectively, on the
impact of the different visualizations (2D versus 3D) and the use of multisensory cues. Lessons
were designed to provide the same content and learning activities using each of the
applications, focusing on concepts pertaining to the distribution of force and energy in electric
fields. At the end of the first stage, both groups demonstrated significantly improved
conceptual understanding, with MaxwellWorld students better able to define concepts than
students who had used EMF. MaxwellWorld students showed comparable performance to EMF
students in sketching concepts, performed better in demonstrating concepts in 3D, were better
able to predict how changes to a source charge would affect the electric field, and could
recognize symmetries in the field. In the second stage of the evaluation, a subset of the students
was given an additional lesson in MaxwellWorld, this time supported by auditory and haptic
cues. The results showed that students gained a significantly better understanding of concepts,
and improved their ability to demonstrate these concepts in 2D and 3D representations. In
particular, students who had previously experienced difficulty with the concepts found that the
multisensory cues helped them to better understand the visual representations.
Overall, these results suggest that immersive 3D multisensory worlds can aid students in
developing appropriate mental models better than 2D representations. The evaluation also
provided insight into the nature of misconceptions that students hold. For example, the
researchers found that after using MaxwellWorld or EMF, students were unclear how the
electric field influenced charged objects and the relationship between potential and force.
Future work will investigate whether the continuing misconceptions are remediated when the
full power of the VR application is used, including multisensory representation and allowing a
user to experience phenomena from the perspective of a test charge [Dede et al., 1996a].
The impact of immersion and interactivity on effectiveness has also been investigated by
HITL researchers. Here an evaluation of an early version of the Atom World was conducted to
look at the impact of these factors on student learning about atomic and molecular structure at
the high school level [Byrne 1996]. Four treatments were used: high immersion and high
93
93
Page 94
95
80
©1998 Institute for Defense Analyses
interactivity (using Atom World), low immersion and high interactivity (a desktop chemistry
world), and low immersion and low interactivity (using (1) a video recording of an Atom World
session, and (2) a run-through of a session with the desktop chemistry world, both of these
accompanied by supporting narrative). The obvious additional treatment of high immersion
and low interactivity was not used because of the difficulty of eliminating interactivity from an
immersion scenario and because of the expectation that such a VR scenario would be more
likely to induce simulator sickness symptoms. The basic task that the students had to perform,
or watch, was building two hydrogen and one oxygen atoms and combining these to form a
water molecule. The major finding of this evaluation was that the groups of students who used
Atom World and the desktop chemistry world both showed significant improvements between
their pre-test and post-test scores, and these scores were not significantly different from each
other. Thus, as in the Cell Biology evaluation, immersion did not have a significant impact on
students' learning. The researchers suggest that lack of familiarity with the VR interface, as
evidenced by the number of errors students made with the controls, a failure to fully exploit the
potential of VR technology, and a low resolution HMD might have contributed to this finding.
Again, as was found with Cell Biology, interaction was a positive factor, because the Atom
World and Mac-based chemistry world groups outperformed students in the non-interactive
treatments and students in a no-treatment control group.
An early study, conducted by researchers at Oregon State University, looking at the impact
of perceived realism (immersion) on childrens' spatially related problem solving ability had
similar findings. Again an immersive VR treatment was compared with a 2D, non-immersive
desktop treatment. The results showed that immersion was not significantly related to spatially
related problem-solving skills. These results were unexpected, and the researchers hypothesize
they might be due to an insufficient number of experimental trials, difference in the number of
trials for each treatment, and unreliability of one of the measures used [Merickel 1994]. It is
likely that the rather different activities that students performed in each of the treatments also
served to cloud the data.
These studies showed mixed findings for the impact of immersion on student learning. One
study that examined the effects of immersion on one particular learning-related issue also
raised interesting questions. In this case, researchers from James Cook University, School of
Education, conducted a small study investigating the impact of immersion on recall [Ainge
1996a]. Here, the researchers assessed whether 6th grade students remembered more details
from a simulated scene (the Room World) than from photographs of that scene. The results
showed that VR did not help students recall which objects were in the room or object colors,
although it significantly enhanced recall for the numbers of each type of object and the objects'
locations.
94
94
Page 95
96
81
©1998 Institute for Defense Analyses
4.1.2 Evaluation of Usability
4.1.2.1 General Usability
Several of the groups that have developed educational VR applications have paid special
attention to issues of usability. Most have focused on the usability of specific worlds, although
a couple have also addressed general issues pertaining to the usability of VR technology itself.
Of course, one area where usability has been a critical issue is in evaluation of applications
intended for use by physically or learning disabled students.
As mentioned in the previous section, the ScienceSpace worlds are being developed
through a series of formative evaluations that have all considered usability. The recent
evaluation of MaxwellWorld that compared the effectiveness of this application with EMF also
considered usability. Students' ratings indicated that they found the various features of
MaxwellWorld more difficult to use than those of EMF, but MaxwellWorld representations
were easier to understand. However, neither of these differences were significant. Overall,
students described MaxwellWorld as easy to use, interesting, and informative, and indicated
that it was easier to remain attentive with MaxwellWorld than with the 2D counterpart
[Salzman et al., 1997a].
One notable feature about the evaluations of the ScienceSpace worlds is that they have
addressed the issue of multisensory and multimodal interfaces. One general finding is that
multisensory (spatialized sound and haptic) cues can smooth interaction. The researchers
suggest that these types of cues can prevent interaction errors through feedback cues and
enhance the perceived ease of use. Multimodal (voice command, gesture, menu, virtual control,
and physical control) interfaces can also ease interaction. The researchers found that
multimodal interaction seems to enhance learning by offering participants the flexibility to
adapt the interaction to suit their own preferences, and also facilitates distributing attention
when performing activities in the virtual worlds. Overall, students have shown noticeable
individual differences in their interaction styles, abilities to interact with a 3D world, and
susceptibility to simulator sickness. The most recent evaluation of MaxwellWorld provides
some specific data on the occurrence of simulator sickness symptoms, comparing student
responses to MaxwellWorld and desktop EMF [Salzman 1997a]. While students who used
MaxwellWorld did score significantly higher on a simulator sickness questionnaire for
disorientation and oculamotor discomfort than those students who used EMF, there was no
significant difference between the groups for symptoms of nausea. The overall simulator
sickness score did not significantly predict learning outcomes.
While non-spatialized sound has been successfully used in several other applications, the
evaluation of Virtual Gorilla Exhibit demonstrates that some care is needed in how non-spatialized
sounds are used [Allison et al., 1996]. Virtual Gorilla Exhibit uses monaural audio
presented to the participant, with additional sound feedback provided by a subwoofer
95
95
Page 96
97
82
©1998 Institute for Defense Analyses
concealed under the platform on which the participant stands. This audio is used to add realism
and provide cues as to a gorilla's emotional state (contented, annoyed, or angry). In the
prototype version of this application, the audio is played continuously at a constant volume,
regardless of the participant's location with respect to the virtual gorillas. Students found that
the constant volume confused them as to a gorilla's whereabouts. As a next step, the researchers
plan to disable sounds when the sound source is a certain distance away from the participant,
or when the participant is inside a building. Depending on the success of this approach, a
spatialized sound interface may be incorporated in the application.
The usability evaluation of LAKE has also provided useful feedback to the application
developers [Mikropoulos et al., 1997]. Subjects recommended introducing more visual realism
into the virtual worlds, even at the expense of a slower frame rate, and the use of sound to
further increase the realism. They also gave opinions on, for example, the use of virtual buttons
and other means of navigation, and on the representations used to depict such things as diluted
oxygen and phytoplankton. One surprising finding was that half the subjects stated that they
experienced immersive feelings even though LAKE is a desktop application. Overall, the
subjects, who were all future teachers, reported positive feelings for the use of VR technology
in the classroom.
4.1.2.2 Usability for Physically and Learning-Disabled Students
The majority of work investigating usability issues for physically disabled students has
been performed by Interface Technologies Corporation as part of the development of VESL.
VESL is intended for use by students with cerebral palsy, multiple sclerosis, and muscular
dystrophy, as well as by non-disabled students. The development of VESL began with a needs
analysis that took account of the capabilities of students with a physical disability and guided
the selection of hardware and software that was used [Nemire et al., 1994]. This was followed
by an investigation where persons with cerebral palsy participated in an evaluation of the
usability of a spatial tracking system (electromagnetic trackers placed on the back of the hand
and forearm) that allowed manual interaction to be accomplished by students with minimal arm
or finger control. Two different forms of predictive software were assessed for their ability to
assist participants in "touching" a virtual target. Both forms increased participants' speed in
touching the target and one also increased accuracy [Nemire 1995a]. The results of this
evaluation were used to develop a refined set of prediction software that was then used in
VESL. The prototype also supports speech commands.
A later formative evaluation of VESL itself examined a wider range of usability issues
[Nemire 1995b]. Students (with and without disabilities), teachers, assistive device specialists,
occupational therapists, and human factors engineers participated in this study. The questions
asked pertained to the aesthetics of the virtual world (for example, the appeal of the colors and
sounds used), usability (for example, the ease of navigation, enjoyability, and the utility of
96
96
Page 97
98
83
©1998 Institute for Defense Analyses
virtual controls), and immersion. Participants gave aesthetics an overall mean rating of 7.4 on
a 9-point scale. The mean ratings for the different usability questions varied more than in other
categories, ranging from 5 to 8 points, with an overall mean rating of 6.8. The participants
seemed to experience a sense of presence, giving a mean rating of 7.2 for immersion.
The other developer of educational VR applications designed for use by both physically
disabled and non-disabled students is Oregon Research Institute. Researchers here are
currently conducting a pilot study intended to support the design of a controlled study
investigating how well the Science Education World works for orthopedically impaired and
regular students at the middle and secondary level compared to traditional non-VR science
instruction.
Researchers at the University of North Carolina Medical School, North Carolina State
University, Computer Science and Computer Engineering Departments, and TEACHC have
investigated usability questions specifically related to autistic children. Using their Street
World application, these researchers conducted an informal study to demonstrated that such
children are able to tolerate an HMD and to navigate through a simple street scene by walking
in the appropriate direction.
The VIRART researchers have examined the usability of particular input devices for
severely learning-disabled students. The early Makaton pilot study showed that some learning-disabled
students had difficulty in using a SpaceBall or mouse for world navigation.
Subsequently, a number of population stereotype studies were conducted that looked more
closely at the usability of particular navigation devices (SpaceBall and joystick) and interaction
devices (mouse, glove, and touch screen) for severely learning-disabled students [Brown et al.,
1995]. The researchers found that a joystick was the most appropriate navigation device in
terms of facilitating control without leading to excessive levels of frustration, and a mouse or
touchscreen was the most appropriate interaction device, depending on the level of disability.
4.1.2.3 Sense of Presence, Ease of Navigation, and Enjoyment
Information on student enjoyment of VR experiences, the sense of presence that students
experienced, and their ease of navigating virtual worlds are provided from analysis of data
collected in the VRRV program. Data was collected from nearly 3,000 students in grades 4
through 12. As reported by Winn [1995], all students indicated that they enjoyed their use of
VR technology. However, the reported levels of enjoyment decreased with student age, and
boys reported more enjoyment that girls. Findings on the feeling of presence experienced by
the students were similar in that while high levels of presence were reported, these also
decreased with age. Findings on the ease of navigation and interaction were less positive.
Younger students had trouble using the wand device used to specify navigation commands,
although these difficulties did decrease with age, as did ratings of disorientation when inside
97
97
Page 98
99
84
©1998 Institute for Defense Analyses
and leaving the virtual worlds. There were very few reports of other simulator sickness
symptoms.
Additional findings on presence and navigation are provided by some of the efforts that
have conducted evaluations of particular pre-developed VR applications. For example,
students who used the Virtual Gorilla Exhibit seem to have experienced minimal presence. For
example, although the researchers conducting the evaluation had expected participants to make
physical gestures and approach the gorillas, many participants stayed in one spot and only
occasionally turned to look or move towards something behind them. Those participants who
did move through the virtual world acted as they would in the real world, choosing not to cross
moats, run into rocks, or fly through the environment. On the other hand, some feelings of
presence are implied by the participants' stated disappointment on the lack of haptic feedback
that would allow them to touch the virtual gorillas.
The first formative evaluation of Safety World provided feedback on a number of usability
concerns [Bell and Fogler, 1996]. When asked about HMDs, as opposed to desktop viewing,
students rated the current benefits of HMDs as 3 or below, on a 5-point scale, but saw the
potential of these devices as 4 to 5 points. The researchers found that the chief usability
problem occurred in navigating around the virtual world. They noted the need to present
information about joystick operation visually rather than textually. In general, the students did
not read the provided text-based operating instructions, implying a general need for operating
information to be made visually available in a virtual world. Another navigation issue was
raised in the early formative evaluations of Vicher I [Bell and Fogler, 1995]. Many students had
difficulty travelling through hallways and reported that this type of realism added little to the
overall VR experience. Instead, teleports were the preferred method for navigating between
rooms.
The informal evaluation of CDS investigated the success and problems with navigation and
other interface techniques such as pull-down menus. The researchers reached the conclusion
that virtual tools adopted from the desktop metaphor were well received and students found
them easy to use.
4.2 Evaluations of Student-Development of Virtual Worlds
There are only a small number of evaluations to report on student development of virtual
worlds. The majority of these have been conducted as part of efforts previously identified as
research oriented. Only one evaluation has been identified where student development of
worlds is an ongoing classroom activity. Details of the evaluations that have been conducted
are given in Table 24.
98
98
Page 99
100
85
©1998 Institute for Defense Analyses
Table
24.
Completed
Evaluations
Using
Student
Development
of
Virtual
Worlds
Performing Organization
VR
World/
Program
Purpose
of
Evaluation
Description
Major
Findings
Ev a n s Bay E l em ent a r y
S c h ool
Virtual
Museum
Effectiveness
of
VR
as
a
research
presentation medium
Informal
study
where
one
group
worked
with
VR
and
the
other
group
with
paper.
-
Students
working
with
VR
were
more
motivated,
enjoyed
and
seemed
challenged
by
this
use
of
the
technology. -Designing and
problem-solving
discussions
were
the
most
valuable
outcome.
H. B. S u gg E l e m e n t a r y
S c h o o l
Virtual
Pyramid
Educational
effectiveness
Used
pre-/
post-testing
to
assess
how
exam-
ining
a
virtual
pyramid
from
different
per-
spectives
and
moving
objects
into
it
improved
ability
to
compare,
classify,
and
create
pyramid
models
and
understand
their
graphical
manipulation.
-
Students'
ability
to
identify
and
draw
correct
per-
spectives
of
a
pyramid
improved.
James Co ok Unive r si ty ,
S c ho ol of E d u c a t i o n
3D
Shapes
Comparative
educational
effectiveness
Compared
construction
and
exploration
of
3D
shapes
worlds
with
construction
and
examination
of
3D
shapes
built
with
card
nets.
Used
pre/ post-
testing
to
assess
impact
on
drawing
shape
appearances
from
differ-
ent
viewpoints,
recognition
of
shapes
in
the
environment,
and
writing
shape
names.
-
VR
significantly
outperformed
the
card
nets
in
ability
to
recognize
shapes.
-
VR
made
no
strong
impact
on
shape
visualization
from
different
perspectives.
-
For
all
tasks,
a
greater
percentage
of
VR
students
improved
their
scores
than
control
(card
net)
stu-
dents.
K e l l y W a l s h H i g h S c h o o l
Computer Programming Class
Comparative
effectiveness
for
correcting
science
mis-
conceptions
Formal
study
based
on
interviews
and
video-
tapes.
Students
from
2
high
schools
worked
with
students
at
3
elementary
schools.
One
group
used
VR,
another
did
not
use
VR,
but
both
followed
a
constructivist
learning
approach.
A
third
group
attended
usual
classroom
activities.
-
All
three
groups
increased
learning
by
roughly
the
same
amount.
U n i v e r s i t y o f W a s h i n g t o n , H I T L
Pacific
Science Center Summer Camp '91
Assess
ability
to
work
cre-
atively
with
VR
and
enjoy
that
work
Pilot
study
based
on
opinion
survey,
video
tape
of
student
activities,
and
informal
observations.
Looked
at
student
activities
and
social
behavior
during
world
building
and
world
visiting,
broad
patterns
of
student
response
to
VR,
and
at
students'
personal
responses
to
their
experiences.
-
Students
demonstrated
the
ability
to
use
VR
con-
structively
to
build
expressions
of
their
knowledge
and
imagination. -Students enjoyed
the
experience
of
VR.
-
Students
accommodated
quickly
to
orientation
and
navigation,
and
collaborated
well.
Pacific
Science Center Summer Camp '92
Assess
impact
of
gender,
race,
and
scholarship
status
on
ability
to
work
creatively
and
enjoyably
Pilot
study
based
on
opinion
survey
and
informal
observations.
Looked
at
impact
of
factors
on
issues
such
as
sense
of
immersion,
enjoyment
of
designing
and
building
a
vir-
tual
world,
desire
to
build
another
world,
desire
to
experience
VR
again,
and
enjoy-
ment
of
the
camp
as
a
whole.
-
Race
and
scholarship
categories
were
highly
corre-
lated. -Gender
was
significant
only
on
the
sense
of
immer-
sion
experienced,
with
boys
reporting
a
greater
sense
of
immersion
than
girls.
-
Race/
scholarship
was
significant
only
on
reported
enjoyment,
with
non-
scholarship
students
reporting
more
favorable
opinions
about
VR.
99
99
Page 100
101
86
©1998 Institute for Defense Analyses
Table
24.
Completed
Evaluations
Using
Student
Development
of
Virtual
Worlds
(continued)
Performing Organization
VR
World/
Program
Purpose
of
Evaluation
Description
Major
Findings
U n i v e r s i t y o f W a s h i n g t o n , H I T L (C o n t i n u e d )
HIV/ AIDS
Assess
effectiveness
of
VR
for
'"
at
risk"
students,
and
assess
VR's
role
within
a
curriculum
Informal
study
based
on
anecdotal
and
per-
sonal
observation.
Looked
at
how
well
stu-
dents
had
learned
about
AIDS
and
computers
(as
evidenced
by
ability
to
con-
struct
HIV/ AIDS
world),
their
experience
in
visiting
the
world,
and
general
motivation.
-
Students
showed
higher
motivation
in
attending
class.
-
VR
portion
did
not
conflict
with
classroom
balance
and
brought
focus
to
topic
studied.
Puzzle
World
To
evaluate
the
impact
of
designing
and
experienc-
ing
virtual
worlds
as
a
spa-
tial
processing
skill
enhancement
method
Used
pre-/
post-
testing
and
interviews
to
assess
ability
in
spatial
relations,
sequenc-
ing,
classification,
transformation
and
rota-
tion,
whole-to-
part
relationships,
mental
imagery,
and
creative
problem-solving
as
evidenced
by
Inventory
of
Piaget's
Develop-
mental
Tasks
test.
-
Although
mean
scores
for
each
subtest
did
not
vary
sig-
nificantly,
the
total
mean
scores
did
indicate
significant
improvement
occurred.
-
Each
subtest
showed
some
improvement,
with
skills
in
classification
showing
the
strongest
improvement.
Wetlands Ecology
Assess
the
effectiveness
of
developing
wetlands
ecol-
ogy
worlds
to
visiting
pre-
built
worlds
and
tradi-
tional
instruction
Used
pre-/
post-
testing
to
assess
factual
recall
and
the
ability
to
draw
mental
models
of,
say,
the
nitrogen
cycle.
-
Low-
ability
students
who
did
world
building
signifi-
cantly
outperformed
those
studying
in
the
traditional
way. -High-
ability
students'
performance
was
not
affected.
VRRV
Entrée
Determine:
(1)
whether
students
learn
from
designing
and
building
their
own
virtual
worlds,
(2)
whether
constructivist pedagogy with VR leads to different outcomes to traditional forms of instruction, and (3) student characteristics that affect how well they learned from world building.
Looked
at
impact
of
general
ability,
spatial
ability
(secondary
school
students
only),
and
gender.
Where
possible,
compared
educa-
tional
effectiveness
of
developing
worlds
with
traditional
methods
for
learning
content
Used
exit
questionnaire,
pre-/
post-
testing,
interviews,
video
tapes
of
student
activities,
and
questionnaire
assessing
attitudes
towards
science
and
computers.
-
Students
who
built
virtual
worlds
did
learn
the
content
they
were
expected
to.
-
Students
who
built
virtual
worlds
did
equally
well,
regardless
of
general
ability.
-
Students
who
built
virtual
worlds
had
consistently
better
attitudes
towards
science
and
computers
after
experi-
ence. -Students
learned
more
and
enjoyed
the
project
more
who
used
3D
models
to
visualize
their
world
before
they
built
objects
in
the
computer;
were
easily
able
to
find
the
object
they
had
made
when
visiting
their
world;
and
reported
experiencing
a
high
degree
of
presence.
-
Students
who
had
difficulty
navigating
or
who
lacked
a
clear
understanding
of
tasks
to
be
performed
in
the
world
learned
less
and
enjoyed
the
experience
less.
-
At
the
elementary
level,
boys
reported
they
had
learned
more
about
VR
than
girls
and
needed
less
time
to
build
their
worlds.
At
the
secondary
level,
boys
enjoyed
the
world
building
more
than
girls
and
spent
more
time
on
the
computers. -High spatial ability
was
correlated
with
enjoyment,
learning,
and
feeling
of
presence.
100
100
Page 101
102
87
©1998 Institute for Defense Analyses
As shown in Table 25, the com-pleted
evaluations for this type of
use of VR technology all focus on
effectiveness. These evaluations are
equally split between more formal
experiments and informal studies.
There have been no evaluations of
usability and, in general, this issue
is less relevant for most of this cate-gory
of efforts.
4.2.1 General Educational Effectiveness
Only one formal study that focused on the effectiveness of student world building activities
in isolation of other issues has been identified. In this case, teachers at H. B. Sugg Elementary
School investigated whether world building could improve students' spatial skills. They found
that having 5th grade students work in groups to build a virtual pyramid improved the students'
abilities to identify and draw correct perspectives of a pyramid. In general, the greatest
improvement was found for drawing the front view of a pyramid; from the total group of 19
students, the students who correctly drew the figure went from 3 in the pre-test to 13 in the post-test.
Researchers at the University of Washington, HITL have addressed the educational
effectiveness of virtual world building activities for students in special populations. In
collaboration with the Seattle Public Schools' Interagency, an informal study investigated
whether building a virtual world helped the students classed as "at risk" learn about HIV/ AIDS
prevention, and improve their motivation and self-esteem. In this investigation, after receiving
instruction on HIV/ AIDS and on the software to be used, a class of students was tasked to
developed a VR game about HIV/ AIDS. After brainstorming a game concept, different groups
of students developed different objects. Both the game concepts and the objects were then used
by HITL researchers to create the actual game that the students could experience. The main
findings of this study were that the VR portion of the curriculum worked well in the classroom
setting, bringing focus to AIDS issues, and that students showed higher motivation in attending
class [Byrne et al., 1994]. Substantial improvement in self-esteem was demonstrated by some
students choosing to lecture about the project at other schools and volunteering to join a city-wide
AIDS peer education project.
Finally, while the VESAMOTEX effort at Slaton High School has not included any
structured evaluations of the effectiveness of student development of virtual worlds (and so is
not included in Table 25), the teacher involved has compared grade scores for the students in
physical science classes both before VR technology was introduced and afterwards. The
Table 25. Types of Completed Effectiveness Evaluations
General educational effectivenessVirtual Pyramid, HIV/ AIDS
Effectiveness for Learning Disabled Students Puzzle World
Comparative educational
effectiveness
Virtual Museum, Computer Pro-gramming
Class, 3D Shapes, Wet-lands Ecology
Ability to work creatively/ enjoyPacific Science Summer Camp '91
Characteristics that impact learning Pacific Science Summer Camp '92, VRRV Entrée
101
101
Page 102
103
88
©1998 Institute for Defense Analyses
students in these classes have a very diverse set of abilities, ranging from those classified as
having special education needs to students identified as gifted and talented. These classes also
include students with personal problems and difficulties such as drug abuse and teenage
parenting. VR was first used in the spring of 1996. A comparison of grades at the end of this
semester with those of the previous semester for over 70 students shows that more than 85% of
the students increased their overall grade, in several cases by over 20%. These results are
largely attributed to the increased motivation that VR technology seemed to generate in the
students.
4.2.2 Effectiveness for Learning Disabled Students
As part of their work in investigating the effectiveness of VR technology for students in
special populations, HITL researchers also have looked at the potential of virtual world
building for helping neurologically impaired children develop 3D spatialization skills. The
initial plan had been for the students to work together in choosing a virtual world to develop,
but these students chose to work independently. While each child developed object pieces for
the world he had conceived, the HITL researchers were still able to combine all these
components in a single virtual world for viewing. This was a formal evaluation effort and, using
the Inventory of Piaget's Developmental Tasks, the results showed that the intensive training
in 3D thinking did significantly improve overall mean scores related to spatialization [Osberg
1993]. For five of the more difficult subtests, the students had pre-tested at below 6th grade
level for three, and slightly above average for the other two. After the one-week course, the
group had improved beyond the 6th grade level with some students showing abilities at the 8th
garde level. The researchers note that the data does not indicate which aspect of the course did
the most good, that is, whether or not it was the world development activities that led to the
results.
4.2.3 Comparative Educational Effectiveness
Teachers at two schools have undertaken studies that compared the effectiveness of student
world building activities with regular classroom practices. In the assessment of the value of VR
as a research presentation medium at Evans Bay Elementary School, students were tasked to
select a research topic and conduct research. The research information then had to be organized
as if it were in a museum: using hall and alcove layout to show categorization and hierarchy of
information, and using corridors and doors to reflect the links between the pieces of
information. One group worked with paper and the other with VR. In comparing the work of
the two groups, the VR group seemed more motivated, searching more widely for resources
and utilizing computers in quite a sophisticated way. The VR group also engaged in more
discussion and group cooperation.
102
102
Page 103
104
89
©1998 Institute for Defense Analyses
At Kelly Walsh High School, participating students in a computer programming class each
worked with two or three 6th graders at a local elementary school to develop virtual worlds.
Students from chemistry classes in another high school worked with 6th graders from a
different elementary school but did not use VR technology, while students at a third elementary
school were taught a science unit following regular practices. Different topics were covered by
each of the groups, though all addressed chemical and physical processes and changes at the
atomic and molecular levels. All groups showed a small but significant increase in conceptual
understanding by the end of the period of instruction. However, there were no significant
differences in the knowledge gain displayed by the different groups [Moore and McClurg,
1995].
Two groups of researchers also have investigated the comparative effectiveness of this type
of VR-based approach. Researchers at the James Cook University, School of Education,
compared the development of 3D virtual shapes with the existing practice of building such
shapes from cardboard cutouts on 6th and 7th graders' subsequent ability to identify these
shapes [Ainge 1996b]. The subjects were drawn from two schools that have a majority of
aboriginal students who are recognized as a disadvantaged group. Students were tested on
visualizing the appearance of shapes from different viewpoints, recognizing shapes in the
environment, and writing shape names. The shapes studied were the cube, rectangular prism,
triangular prism, square-based pyramid, triangular-based pyramid, cone, cylinder, and sphere.
Although VR had little impact on shape visualization, it was found to significantly enhance
shape recognition and, on all tasks, a greater percentage of VR students improved their scores
than did those students who built cardboard models. Another study by researchers at James
Cook University is just starting. Here the researchers are interested in the comparative
effectiveness of VR instruction in the area of learning about historical cultures. The work will
include comparing traditional teaching methods with a VR approach and comparing the use of
researcher-developed materials and student-developed materials. To this end, the types of
materials that will be used will include historical pictures, a pre-developed virtual scene,
student-developed virtual scenes, and student-build models (for example, cardboard models).
A study by University of Washington, HITL researchers conducted at Kellogg Middle
School compared the effectiveness of a constructivist pedagogy that included student world
development with traditional instructional practices. Although the HITL researchers had
expected the VR treatment to be more effective than the non-VR treatment, there was no
significant difference in learning between the two treatments [Osberg et al., 1997]. Four
wetlands ecology cycles were studied across four classes. In each class, each student learned
about one wetland ecology cycle using the constructivist/ VR approach, two other cycles using
a traditional classroom instructional approach, and had a no-instruction treatment for the fourth
cycle (that is, instruction on some unrelated subject). The analysis was based on pre-and post-testing,
and preparation of concept maps (both before and after the treatments) collected on the
103
103
Page 104
105
90
©1998 Institute for Defense Analyses
world they were assigned and a chosen world they wished to represent. Interview and survey
information was also collected. The data provided no evidence that the constructivist approach,
combined with world building, was more effective than the traditional instructional approach
although, unlike the traditional approach, the constructivist approach did show a significant
improvement over the no-instruction treatment. The researchers suggest that one reason for
these unexpected results was that the students who participated were already experienced in the
constructivist paradigm, although not world building, and this experience was applied even in
the traditional instruction treatment. Nonetheless, the teachers involved in this project did feel
that world building had some positive effects. For example, the students' language and
presentation techniques evolved, they began to talk in terms of "perspectives" and "rotating the
view," and seemed more willing to consider part-to-whole relationships. These changes were
reflected in other classes and even personal relationships.
4.2.4 Ability to Work Creatively and Enjoyably
The University of Washington HITL's participation in the Pacific Science Center Summer
Camps in 1991 was the first evaluation of educational use of VR technology reported in the
public literature. During this summer camp, HITL researchers conducted a pilot study that
focused on the ability of 10-15 year olds to use VR technology constructively to demonstrate
certain knowledge or express their imaginations, and whether they enjoyed such work. More
specifically, the researchers were interested in seeing what students were motivated to do given
access to the technology and guidance in the world building process. The largely subjective
results showed that, indeed, students could use this technology creatively and enjoyed their
work. The researchers felt that students demonstrated rapid comprehension of needed concepts
and skills, they were willing to focus significant effort toward developing their worlds, and that
collaboration between students was very successful [Bricken 1992]. With respect to world
construction, the students learned enough about the modeling software in 10-15 minutes to
begin creating objects and later showed a fast accommodation to moving around in a virtual
world and the nature of virtual objects. Object interaction was more difficult, perhaps due to
the low resolution Virtual EyePhones HMD that was used. The students themselves showed
great satisfaction with their accomplishments and consistently rated their appreciation of the
technology extremely highly in an opinion survey. Even though low levels of disorientation
were reported when experiencing the virtual worlds, the students reported that they felt that VR
worlds were good learning environments as well as good places to play and work. As shown in
Table 15, although still positive, student ratings on the world-building tools were less
favorable. The unnatural nature of gesture-based interaction using a data glove also was a
concern.
104
104
Page 105
106
91
©1998 Institute for Defense Analyses
4.2.5 Student Characteristics that Impact Learning
The students at the Pacific Science Center Summer Camp in 1991 were predominantly
computer-literate, male Caucasians who had access to a fairly expensive summer camp.
Consequently, the following summer, scholarships were provided to help students from other
groups and backgrounds attend the camp. This allowed the HITL researchers to investigate
whether factors of gender, race, or scholarship impacted students' interaction with and
enjoyment of VR [Byrne 1993]. Since the researchers were also interested in looking at the use
of VR in a curriculum-like setting, this time students were instructed to develop worlds with
emotional themes. These students also reported positive VR experiences. With respect to
gender, the only significant difference was found in response to a question on awareness of
physical surroundings while immersed in a virtual world, with the boys reporting a greater
sense of immersion. Race and scholarship categories were quite highly correlated, with
significant differences being reported for questions about how much students enjoyed
designing and building a virtual world, how much they would like to build another world, and
how much they would like to be in a virtual world again. Although the ratings given by the
Table 26. Student Opinions About VR Technology During a Summer Camp (1991) a
VR Experience
How did you feel about experiencing VR?
(1: did not enjoy -7: enjoyed extremely
6.5
Do you want to experience VR again? (1: not at all -7: very much 6.8
Would you rather: go into a virtual world (1)
see a virtual world on a computer screen (0)
go into a virtual world (1) play a video game (0)
go into a virtual world (1)
watch T. V. (0)
go into a virtual world (1)
use your favorite computer program (0)
(forced choice) .95
.98
.96
.98
World-building Tools
How did you feel about building Swivel worlds? Do you want to learn more about building Swivel worlds?
Do you want to learn to program VR worlds?
(1: did not enjoy -7: enjoyed extremely
5.8 5.7
5.6
Would you rather: build a Swivel world and go into it (1)
go into a world that has already been built (0)
(forced choice)
.76
a. Based on data in [Bricken 1992].
105
105
Page 106
107
92
©1998 Institute for Defense Analyses
scholarships students were less favorable than those of their colleagues, the lowest mean score
for these students was 8.28 out of a possible 10, indicating that the scholarships students still
ranked the VR experience highly.
Data collected during the VRRV Entrée program was also analyzed to look at the issue of
gender. At the elementary level, self-reports on the amount of learning that took place indicated
that boys benefited more that girls from world building activities. At the secondary level, boys
reported more enjoyment from the use of VR technology than did girls. With respect to general
abilities, the VRRV data indicated that students performed equally well. However, students
with high spatial ability reported more enjoyment, learning, and a feeling of presence than did
those with low spatial ability [Winn 1995].
4.3 Evaluations of Multiuser, Distributed Virtual Worlds
As shown in Chapter 3, there have been few developments of multiuser, network-based
educational VR applications. Consequently, there are only two evaluations to report: a
completed evaluation for Virtual Physics and an ongoing evaluation for NICE. Details on the
evaluation of Virtual Physics is given in Table 27. While these studies have looked at issues of
educational effectiveness and usability, a significant focus has been on the role of collaboration
in learning.
In the ongoing study with the research vehicle NICE, researchers are investigating
interface, orientation, immersion and presence, motivation and engagement, as well as
cognitive and conceptual learning issues. They are in the process of developing an evaluation
framework that takes into account the measurement of both usability as well as conceptual
learning attributes of the application. Because the VR medium is still new, these researchers
have chosen to conduct their evaluations as a series of case studies using a small number of
children rather than perform a formal evaluation with an entire class. The studies themselves
follow a limited quantitative perspective and a highly qualitative approach. So far, children for
the experiments have been selected on the basis of gender and their expertise in computer/ video
game playing. While only very preliminary and informal observations can be yet made, it
seems that children who play video games need no instruction to begin using the VR
application and feel more at ease when interacting with the virtual world. Indeed, all the
children seem to gain the skills needed to the use the VR application quickly, but the ones
inexperienced with video games are less animated in the virtual space. In terms of conceptual
learning, initial observations reveal a high degree of diversity in how the children work with
their small ecosystems, and in how they collaborate with each other or the virtual genies.
The goal behind the DEVRL project is to investigate the effectiveness of collaborative VEs
for conceptual learning and developing skills in scientific problem solving. The Virtual Physics
application developed to support this goal is itself designed to help students develop non-symbolic
models of various physics concepts. The completed, informal experiment with this
106
106
Page 107
108
93
©1998 Institute for Defense Analyses
Table
27.
Completed
Evaluations
Using
Distributed
Virtual
Worlds
Performing Organization
VR
World/
Program
Purpose
of
Evaluation
Description
Major
Findings
U n i v e r s i t y o f L a n c a s t e r , C o mp ut i n g De par t me nt
Virtual
Physics
Evaluation
of
(1)
participant
interaction,
and
(2)
the
level
of
physical
knowledge
that
can
be
assimilated
Informal,
subjective
study
based
on
videotapes
and
observations.
Pairs
of
participants
working
simulta-
neously
used
the
environment
to
perform
tasks.
After
a
familiariza-
tion
session
using
the
3D
Pivot
world,
subjects
were
assigned
tasks
in
the
Cannon,
Bowls,
and
Friction
worlds.
Some
students
used
an
immersive
interface
and
others
a
desktop
graphical
user
interface.
-
Role
playing
in
collaboration
can
promote
effective
collaboration
if
the
roles
can
be
associated
with
appro-
priate
cognitive
processes.
-
During
their
collaboration,
each
pair
of
students
tended
to
persist
in
a
given
role.
-
The
immersive
interface
seemed
the
one
most
easily
understood,
providing
better
visual
cues
and
a
more
intuitive
interface.
-
The
desktop
interface
presented
navigation
difficulties
when
users
tried
to
track
moving
objects.
-
The
worlds
supported
"reperception"
of
problems
by
allowing
viewpoints
to
be
varied
and
allowing
a
mov-
ing
perspective
that
supported
different
frames
of
refer-
ence. -Most
participants
displayed
little
or
no
logical
method,
controls
were
rarely
moved
in
isolation,
and
model
building
techniques
appeared
to
be
poorly
developed.
107
107
Page 108
109
94
©1998 Institute for Defense Analyses
application evaluated both the level of physical knowledge that could be gained and types of
participant interaction. The testing was performed using the Cannon, Bowls, and Friction
worlds. Participants in the experiments were given a minimal introduction so that they would
be free to choose their own form of interaction. For example, using the Friction world,
participants were shown the world, introduced to the method of moving balls over the table,
and then asked to determine the function of two unlabeled controls. Most pairs of participants
seemed to treat the tasks posed in the experiments as a simple game, using trial and error to
solve the problems, rather than building an hypothesis about the system. Interestingly, the
abstract nature of the Bowls world, and the posing of the problem as a friendly competition
rather than a collaboration, presented the best results. While the approaches taken by the groups
were very different, the solutions always resulted in the development of a model for the motion
of a ball over a surface [Brna and Aspin, 1997].
With respect to the types of participant interaction that were evidenced, researchers found
that most participants displayed little or no logical method and their conceptual model building
techniques were poorly developed. Task solving activities invariably broke down into a
dominant leader and passive follower mode of interaction, and the VR interface did little to
mediate between different personality types. As a result of this experimentation, the researchers
have defined three types of collaborative learning environments: task division, the game, and
the mentor/ pupil model. In the first case, a task that may normally be performed by one user is
needlessly divided into a multiuser task, for example, one participant becomes little more than
a camera for the other participant( s). The DEVRL researchers noted that there is a danger that
all collaborative educational VR applications can degenerate to this. The second form of
collaboration is the game, where they found that simple games, such as bowls, can present
powerful learning devices for simple concepts, and the slightly competitive nature of a game
can add to the collaboration. Finally, the mentor/ pupil model is more suited to teaching
complex material. While the mentor may be initially external to the virtual world, this role can
be quickly assimilated into the participant group where it is passed between, or among, the
participants as discoveries are made and passed on to other participants.
108
108
Page 109
110
95
©1998 Institute for Defense Analyses
5. Conclusions
In the current transition from an industrial society to an information society, traditional
instructional approaches based on the use of textbooks in classrooms have been called into
question. Instead of memorizing facts, more emphasis is being placed on the higher-level
thinking skills needed to construct and apply knowledge. Students must learn to locate,
interpret, and creatively combine information, and to isolate, define, and solve problems.
Additionally, education is no longer seen as something limited to a classroom or to a certain
period in a person's life. Instead, education will be lifelong and must meet the needs of a
flexible workforce.
This paper has reviewed some of the uses of VR technology that attempt to support these
new visions. Before discussing any conclusions, it must be pointed out that VR technology, and
its application to education, is still maturing. Moreover, almost exclusively, the studies have
concerned one-time use of virtual worlds by any particular group of students, and there is no
information on how students respond to the technology over the long term. Therefore, the
conclusions given below present a snap-shot of the current state of affairs that will, hopefully,
serve to guide further research on the optimal use of VR technology in education.
The remainder of this section returns to the questions raised in Section 1 to see what has
been learned.
Questions relating to effectiveness of VR-based education:
° Does learning in virtual worlds provide something valuable that is not otherwise
available? Unique capabilities of VR technology include allowing students to see the effect of
changing physical laws, observe events at an atomic or planetary scale, visualize abstract
concepts, and visit environments and interact with events that distance, time, or safety factors
normally preclude. These capabilities allow virtual worlds to support a wide range of types of
experiential learning and guided-inquiry that are otherwise unavailable. Other benefits include
the ability to incorporate acknowledged good practices such as providing multiple
representations and placing at least some instruction under the learner's control. While these
latter attributes are not unique to VR technology, the technology does facilitate their use more
than many traditional educational practices.
The work reported here provides initial findings that are suggestive of the value of these
capabilities. The majority of uses of the technology have included aspects of constructivist
109
109
Page 110
111
96
©1998 Institute for Defense Analyses
learning and it is impossible to determine whether positive results are due to the use of this
learning method, the use of a virtual world, or some combination of the two. However, this
distinction is probably unimportant since, in the long term, the most significant impact of the
technology is likely to be the support it provides for this constructivist learning.
° How does the effectiveness of student use of pre-developed virtual worlds compare with
traditional instruction practices? Two formal evaluations have looked at the effectiveness of
interacting with an immersive virtual world as compared with traditional learning activities.
Both studies were in very different areas of curricula, with students of different ages and
applications of varying levels of interaction and complexity. These studies also used different
pedagogical approaches. The formal comparison of MaxwellWorld with a widely used roughly
equivalent 2D computer application showed that the MaxwellWorld group did demonstrate
superior learning in some areas and this advantage was retained after several months. The other
formal evaluation found Zengo Sayu to be more effective in teaching Japanese than traditional
computer-based instruction, and as effective as instruction provided by a human instructor.
The findings of one informal study were also positive, with students who used immersive
VR developing a better understanding of architectural spaces that those who used paper or
traditional CAD-based tools. Another informal study, this time with a non-immersive, simple
walkthrough application, had less positive results. In this case, students reported that textbook
instruction was more successful in teaching about pyramid structure than navigating through a
virtual pyramid. This negative finding may have been influenced by the considerable difficulty
students experienced in navigating through the narrow and sloping passageways inside the
virtual pyramid.
These types of evaluations can support decisions as to whether a particular application
warrants further development or introduction into practical use. As yet, however, researchers
have not tried to identify specific characteristics that make one form of instruction more
effective than another. There is a need for a series of evaluations that control the variables in
question to try and pin down such characteristics. This type of information would provide
guidance as to when a particular type of VR application should be considered and then help to
guide the development of the application to ensure its effectiveness.
° How does the effectiveness of student development of virtual worlds compare with other
instructional practices? The three studies that have addressed this question were, again,
conducted in very different areas of curricula, with students in different age groups, and used
different hardware and software development platforms. The Wetlands Ecology effort where
some students developed immersive virtual worlds did find a significant advantage for low-ability
students but there was no significant difference in the knowledge gain for high-ability
students. The HITL researchers who conducted the Wetlands Ecology effort note two factors
that could have contributed to the lack of a significant advantage for some students who used
VR technology. First, there was considerable overlap of concepts among the different wetlands
110
110
Page 111
112
97
©1998 Institute for Defense Analyses
cycles studied and, secondly, all the students who participated in the Wetlands Ecology effort
were already familiar with constructivist learning practices. The Computer Programming Class
effort reported a similar finding for lack of a significance in the learning between the students
who developed immersive virtual worlds and those participating in regular classroom
activities. In this case, however, the several groups of students involved in the effort were free
to address different topics within a defined area of the science curriculum and this makes
interpretation of this finding difficult. The study at James Cook University was more
controlled. Here researchers compared the value of creating non-immersive virtual worlds that
contained 3D shapes with the construction of card net shapes. This study had positive results
with respect to student recognition of the shapes but failed to show any strong advantage of the
use of VR technology for helping students to visualize shapes from different perspectives.
While simple, this use of the technology seems highly appropriate and the mixed results are
surprising. It would be interesting to see whether use of immersive VR technology would lead
to similar findings.
No conclusion on the effectiveness of student world development activities, compared with
traditional classroom practices, can be made based on the mixed results of these particular
studies. Many additional studies are required to isolate the impact of factors such as pedagogy,
curriculum, and the method of student development of worlds and determine when and how
student development of virtual worlds should form part of classroom activities.
° How does the effectiveness of student use of pre-developed virtual worlds compare with
that of student development of virtual worlds? The only evaluation that has addressed this
question is the Wetlands Ecology study performed by the University of Washington HITL. In
this effort, each group of students not only developed an immersive virtual world depicting one
wetland cycle and received more traditional classroom instruction for a second cycle, they also
visited the virtual world for a different wetlands cycle developed by another group. As before,
no significant differences in the knowledge gained were found when students visited a
predeveloped virtual world as compared with developing their own world. Again, the overlap
in concepts among the different cycles could have contributed to this finding. Note, moreover,
that all "predeveloped" virtual worlds used in this evaluation were themselves developed by
students and had no significant pedagogical component.
In general, it may not be appropriate to compare the learning effect of working with pre-developed
applications and student development of virtual worlds because some curricula
elements may benefit by one type of use and some by the other. The teachers and researchers
who have led efforts where students developed virtual worlds as part of their learning activities
believe that learning primarily occurs as a consequence of the research, world design, and
world construction activities rather than experiencing the usually quite simple developed
worlds. On the other hand, pre-developed applications typically address more complex subjects
and provide the student with quite sophisticated methods of guided-inquiry for knowledge
111
111
Page 112
113
98
©1998 Institute for Defense Analyses
construction. Already the current sets of pre-developed applications and world building efforts
are starting to show fairly consistent patterns of usage. Pre-developed applications are typically
used for those circumstances where students must manipulate the virtual environment and
perform experiments in order to learn basic and complex concepts. Whereas the goal
underlying student development of virtual worlds is for students to demonstrate knowledge
they have acquired.
° How do the effectiveness of immersive and non-immersive virtual worlds compare? Several
groups have compared the effectiveness of immersive VR with non-immersive approaches.
Often these assessments have used 2D computer worlds or simulations that are quite different
than the immersive application. For example, CDS and Spatial Relations World were both
compared against the use of computer-aided design (CAD) packages (in both, positive findings
for immersion were found). Only three studies that compared immersive and non-immersive
viewing of the same, or similar, virtual worlds were identified. Two of these studies used the
Cell Biology and Atom World applications and found that the important factor for performance
was interactivity, not immersion. Although in the case of Cell Biology, the researchers did find
the immersive VR performed best for symbolic retention, non-immersive VR was better for
function retention. The third study compared immersive MaxwellWorld with an equivalent 2D
computer simulation that provides similar levels of interaction. Here, while each group of
students performed similarly in most cases, some benefits for the 3D viewing supported by
immersion were found. In particular, the students in the immersive condition were better at
describing the 3D nature of electric fields.
While the evaluations show uncertain learning benefits for immersion, it is important to
note that the participants in the immersive conditions in these studies expressed more
enjoyment and motivation to learn than those in the non-immersive conditions.
° How well does VR technology support collaborative learning between students? Is this
collaboration educationally effective? Some researchers suggest that collaborative learning
can be achieved by having two or more students work together with "single-user" pre-developed
applications by taking turns to guide the interaction, record observations, or
experience the virtual world. However, there are no reports on how successful this approach is
in practice. On the other hand, the majority of student-development of virtual worlds has taken
place with students working in groups. In these cases, the teachers or researchers involved have
observed greater levels of meaningful discussion between the students though there is no data
on whether such collaboration impacted educational objectives.
Multi-user, distributed applications are specifically intended to support collaborative
activities within virtual worlds. Of the three such applications for which details are available,
Network Racer has well defined roles for each participant and participants appear to have had
no difficulty in using these roles to guide their collaboration. The interactions between students
in NICE are primarily of a social nature and, as yet, the researchers have drawn no conclusions
112
112
Page 113
114
99
©1998 Institute for Defense Analyses
about the effectiveness of this interaction. The informal evaluation of Virtual Physics does
provide some interesting points, notably that role playing can support collaborative learning if
the roles can be associated with appropriate cognitive processes, otherwise there is a tendency
for collaboration to degenerate into a basic follower-leader paradigm.
The potential of VR technology to support collaborative learning seems very high, but there
is a lack of knowledge on how to exploit the technology to actually support this type of
learning. This shortcoming is not a reflection on the technology itself because little is known
about collaborative learning. Once the characteristics and benefits of different types of
collaboration are better understood, then it will be possible to assess the advantages that VR
technology might bring to bear. Of course, collaborative VR applications may themselves
prove useful tools for conducting such research.
° Is VR-supported learning cost-effective? It is too early to attempt to answer this question.
As this paper has shown, there is some data on educational effectiveness, but it is sparse and
case specific. Data on the financial costs of developing VR applications is not publicly avail-able
and there is a lack of data on related costs, such as maintaining VR equipment.
Questions concerning where VR technology should and, equally important, should not
be used (considering both educational content and student characteristics):
° For what type of educational objectives or material is VR technology best suited? Where is
it not suited? It is easy to say that VR technology is suited for those situations where students
can be guided in the construction of knowledge or where they need to learn concepts that are
highly visual in nature, and that it is not suited for predominantly text-based materials. Such
general statements, however, are insufficient to select and guide appropriate uses of the tech-nology.
The technology already has been used for a wide variety of educational subjects and
these applications provide some indications of where it is suited. However, there is very little
data as to what characteristics of the technology support particular types of instruction. For
example, why did building a set of basic 3D geometric shapes help students to recognize those
shapes but did not help them to visualize and draw the shapes? Or why did students receiving
auditory and haptic cues, in addition to visual cues, perform better on questions relating to
velocity and acceleration, but worse on predicting system behavior? Answers to these types of
questions are needed to determine the strengths and weaknesses of the technology and those
situations that are most apt to benefit from its use.
° Are there specific student characteristics that indicate whether VR-based education is
appropriate? Does the technology benefit only certain categories of students? It has been
proposed that VR-based instruction will particularly benefit those students who are visually
based. This is very likely, but there is no hard evidence that such is the case. The evaluation of
the Wetlands Ecology effort suggested that less-gifted students and those starting with less
subject-related knowledge benefit most from use of the technology. However, analysis of the
113
113
Page 114
115
100
©1998 Institute for Defense Analyses
subjective data collected from the many students who participated in the VRRV Entrée
program indicated that students performed equally well, regardless of general ability. This data
also indicated that students with high spatial ability performed better and enjoyed the use of
VR technology more than those with a low spatial ability. It must be noted, however, that all
this data was collected during student world development activities, and equivalent data is not
available for instances where students have used pre-developed VR applications.
VRRV data also indicates a worrisome influence of gender, in that boys at the elementary
level reported higher levels of educational effectiveness and took less time to build virtual
worlds than did girls. At the secondary level, boys reported more enjoyment of world building
levels. Also, in the VR program at the Pacific Science Center Summer Camp in 1992, boys
reported experiencing a greater sense of immersion than did girls.
Additional research must be performed to take a closer look at the possible influence of
gender and other student characteristics on the effectiveness of different types of educational
uses of VR technology. For example, do boys typically have a greater experience with using
computers or playing video games which gives them some advantage for using VR
technology? It may be that different groups of students require different types of introduction
to the technology or prior training in skills such as spatial awareness. Investigation of these
issues is not only needed to ensure effective use of VR technology, but also its equitable use.
With respect to the potential educational effectiveness of VR technology for learning-disabled
students, two formal studies by the VIRART group have shown that the use of pre-developed
VR applications can help in teaching a special communication language and
everyday life skills. A study by the TEACHC group showed that such applications can also
help autistic children learn to recognize objects. HITL researchers demonstrated that world
building can help to develop spatial skills in neurologically-impaired students. There results are
encouraging. However, as shown by the 3D Letter World effort, interacting with most virtual
worlds does require some minimum level of hand-eye coordination and spatial awareness skills
that some special needs students may lack.
Questions concerning potential student and teacher acceptance of VR learning environ-ments:
° Do students find VR interfaces easy to work with? Overall, students' reports on usability
indicate that navigating through virtual worlds is one of the major problems confronting the use
of VR technology. Because navigation is a fundamental activity in virtual worlds, this is a
crucial area of concern. It seems unlikely that anything in advance of the current set of devices
and metaphors can be done to improve navigation in non-immersive virtual worlds, although
there is scope for improvement in immersive virtual worlds. The difficulty with navigation is,
however, an indicator of a larger problem. That is, that the current interaction paradigms of
command lines and graphical user interfaces used for interacting with a 2D space are
114
114
Page 115
116
101
©1998 Institute for Defense Analyses
insufficient to cover the wide range of interactions required with virtual worlds. In the absence
of a general 3D interaction paradigm there is little commonality between VR application
interfaces and the variety of types of input devices being used leads to additional differences in
how interaction commands are given.
There is some evidence about the types of VR interfaces that students prefer. Several
evaluations have demonstrated that students like multimodal interfaces, though some
modalities are preferred over others. For example, in the few cases where gestures were used
to control the interaction with the virtual world, students reported that they liked this type of
interaction less than others (such as menu-based interaction). Evaluations of NewtonWorld and
MaxwellWorld showed that students liked the use of multisensory interfaces and that the use
of auditory and haptic feedback can aid in learning. Yet multisensory learning is an area where
little is known and research is needed to ensure appropriate and effective use of different types
of sensory feedback. Perhaps the most important conclusion that can be reached from the
studies reported here is that students vary widely in their interaction styles and in their ability
to interact with a virtual world.
The low resolution of current display devices means that text does not fare well in virtual
worlds and the usual workaround is to take over most of the display to present needed text, at
the expense of displaying the virtual world itself. While most applications need to include some
text to present information, speech technologies could be used to minimize the reliance on text
for interaction. Unfortunately, the use of voice communications has been minimal so far. More
research into how to accommodate voice interaction in a virtual world is needed. This is
particularly critical for multi-user applications.
The usability of HMDs is a topic of wide concern. Though little data on this issue was
collected in the evaluations reported here, there are some noteworthy points to make. First of
all, few students commented on the limited resolution provided by most current HMDs. The
problems that were reported concerned difficulties with focusing the display. HMDs were used
for extended periods, although some students reported eyestrain after 90 minutes of use. The
weight and cumbersome nature of HMDs were not mentioned, and even autistic children were
able to use this type of display.
Another important issue relating to immersive applications is simulator sickness. Here,
gain, little data has been collected. Only one evaluation specifically looked at this question and
in this case users of MaxwellWorld did report more occurrences of disorientation and ocular
discomfort than students using a non-immersive 2D counterpart, although there was no
significant difference in reports of nausea. Overall simulator sickness scores did not predict
learning outcomes. In addition to this single study, other researchers have noted that the
occurrence of nausea is rare, and the most common symptom of simulator sickness is
disorientation. More data on the frequency and severity of such symptoms is needed.
115
115
Page 116
117
102
©1998 Institute for Defense Analyses
There has been some commendable work looking at the usability of VR interfaces by
students with physical disabilities and learning disabilities. This type of research is to be
encouraged. One of the goals should be developing a set of interface and experiential
characteristics that need to be supported for different groups of learners, along with guidelines
for building such characteristics into educational VR applications.
° Does the effective use of VR technology change the teacher's role in the classroom? The
HITL researchers have worked with a larger number of teachers than any other single group.
These researchers report that teachers found that, whether they used pre-developed
applications or had students develop their own virtual worlds, their roles in the classroom
changed. Instead of being a teacher with all the answers, they became facilitators who
supported students in their discovery of worlds and building ideas based on information gained
from those worlds. This change in the teacher's role is one of the ways in which VR technology
has long been expected to influence current educational practices and points out the need to
prepare teachers for these new types of activities. East Carolina University offers a course for
education students in how to develop and use VR applications. Also a few developers of pre-developed
applications provide some teacher training for their specific applications. But wider
questions as to how teachers should be prepared for this new role, and what different types of
resources they need, remain to be investigated.
There are additional issues to consider for immersive "single-user" VR applications. The
fact that it is difficult for teachers to monitor students' moment-by-moment activities can
present challenges for lesson administration. Also, current applications do not provide teachers
with assistance for assessing a student's learning or recognizing particular problems a student
may have with the material. The integration of intelligent tutors into educational VR
applications seems a logical next step that should help to resolve some of these problems.
Indeed, given the sophistication of some current VR applications, it is surprising that no
evidence of such integration was found. (A couple of applications do include intelligent guides,
but none of these maintain models of student knowledge that can be used to guide further
instruction or provide teacher feedback.)
° What are student and teacher reactions to the use of this technology? Based on data
collected from thousands of students of different ages, using different applications with
differing interfaces, there is overwhelming evidence that students enjoy both experiencing pre-developed
applications and developing their own virtual worlds. Use of VR technology seems
to serve as a valuable motivating factor. For example, in the VESAMOTEX project, the
introduction of VR to the classroom improved attendance and reduced the number of negative
progress reports that had to be sent to students' parents. Even more telling, several "at risk"
students who participated in a HITL effort where they developed a virtual world designed to
increase awareness of HIV/ AIDS prevention, subsequently volunteered to become involved in
activities to educate their peers about the danger of the HIV/ AIDS virus. Does this enjoyment
116
116
Page 117
118
103
©1998 Institute for Defense Analyses
and increased motivation last? There is no data to answer this question, although the
ScienceSpace researchers, who have had students use an application up to three times, feel that
learner motivation will remain high, even when the novelty factor of VR technology has worn
off.
The single teacher who appears to have the most experience with the use of VR technology
in the classroom is at Evans Bay Intermediate School where students have been very active in
the development of virtual worlds. This teacher points out that he is not a trained researcher and
he is working out his own methodology for optimum use of the technology through trial and
error. His impressions so far are that: (1) 3D spatial relationship concepts are rapidly
accelerated, (2) higher-level thinking can be stimulated if the tasks assigned to students are
challenging and of a problem-solving nature, and (3) again, motivation for many students is
extremely high. There are no reports of teachers who, having tried the technology in their
classrooms, decided not to use it. The largest problem seems to be a lack of resources that limit
how the technology is used.
Two surveys designed to aid in defining VR's role in education also provide some input on
teachers' reactions to the technology. While the first of these surveys focused on the use of VR
technology in environmental education [Taylor 1994] and the second looked more broadly at
K-12 education [Yu 1996], these surveys asked similar questions. In both cases, a majority of
respondents (over 200 for each survey) said that they would use VR technology if it were
affordable, available, and easy to use for students and teachers. When asked what type of
research they thought was necessary, respondents first of all recommended that educators and
VR developers should work closely together in developing educational programs, and
suggested areas of research including: studying the advantages for learning from a VR
representation as compared to learning from a 2D representation, studying what constitutes an
effective virtual learning environment, and creating standards for both building and measuring
the effectiveness of educational VR applications. Areas perceived as most beneficial were the
ability for students to experience situations that were not accessible in the real world (including
changes in scaling and/ or time), programmable participation, and enhancing the education of
students with disabilities. It is encouraging to see that work in most of these recommended
areas is underway. For example, the development of the MAEL program and several VR
applications have been guided by teacher input, and Global Change, the ScienceSpace Worlds,
and Atom World are just a few of the applications that allow students to experience situations
that are otherwise inaccessible. However, as previously mentioned, work has yet to start in
trying to characterize what constitutes an effective learning environment and, at this time, it
seems premature to start developing standards for building and measuring the effectiveness of
educational VE applications.
117
117
Page 118
119
104
©1998 Institute for Defense Analyses
Practical questions:
° Are the hardware platforms and minimum set of interface devices required affordable to
most schools? Virtually any PC can be used for the development or desktop viewing of non-immersive
virtual worlds. Consequently, this basic level of technology should be within the
reach of most school budgets. If 3D viewing of non-immersive virtual worlds is desired, then
additional graphics processing power and special interface devices such as shutter glasses or
3D projectors and passive glasses are required and these can incur significant additional costs.
The basic level of technology required for immersive virtual worlds is more expensive.
Here the lowest-end hardware platform that can be reasonably used is a Pentium-level PC
augmented by special graphic accelerator boards. The least expensive immersive visual display
is an HMD and the price of this type of device starts at just under $1,000. Additional special
interface devices, such as a head tracking system and six degrees-of-freedom mouse, are
usually required. Together, this equipment puts the current price of an entry-level immersive
VR hardware platform at around $10,000. A more realistic figure for a system suitable for
practical, every-day use and capable of supporting timely interaction with complex worlds is
in excess of $25,000. Note, moreover, that these are figures for a single platform and efficient
classroom use of the technology usually requires several platforms. This expense is beyond
most elementary, middle, and high school budgets. Some of the costs will decrease as the
technology itself and its market matures. But since technology research and development is still
required in many areas, major cost reductions will be slow in coming.
° Are the needed software development tools available? A wide range of VR software
development packages are available at a range of costs. The more powerful ones can require
training to use effectively and can cost several thousand dollars; these packages are best suited
for professional world developers. There are half a dozen or so simple, inexpensive
development packages suitable for use by students, primarily for the development of non-immersive
virtual worlds. (Some studies looking at the comparative ease of use of three such
packages are underway.) But there is a shortage of mid-range products that provide a
comprehensive set of easy-to-use tools for developing immersive applications at a reasonable
cost. In the interim, several researchers are using their own custom-build tools or tools not
specifically designed to support the development of virtual worlds, for example, various CAD
and other modeling packages.
In this area, the development of VRML is a factor to watch. Already most development
packages provide support for VRML and VRML is starting to be used to distribute virtual
worlds over the web. At least one VRML-based development package has been brought onto
the market, although this event is too recent to allow any estimates on how widely used this
product may become. The current version of VRML is still very limited in many important
ways but the language has considerable support in the VR community.
118
118
Page 119
120
105
©1998 Institute for Defense Analyses
° Is the technology currently mature enough for practical use? By and large, the technology
needed for non-immersive virtual worlds is mature and becomingly widely used in many
different application areas. The major outstanding problem is the difficulty experienced in
navigating around a virtual world and the best solution is most likely not technological but
practice to develop needed skills.
With respect to immersive virtual worlds, the fact that the technology is already is limited
practical use in a variety of application areas does not necessarily demonstrate its maturity.
Current uses of immersive VR technology tend to be isolated examples of what proponents of
the technology can achieve. There are several technological problems that stand in the way of
widespread use of the technology. Affordable, higher resolution immersive displays are needed
and, in the case of HMDs, wider fields of view and more comfortable devices. Delays in scene
updates and tracker inaccuracies can also be problems when head-tracking is used. New
metaphors that facilitate interacting with immersive virtual worlds and, in particular, that
support multimodal and multisensory interfaces are required. Likewise, advances in integrating
VR technology with networking technologies are needed to support collaborative immersive
virtual worlds. While there is ongoing research in all these areas, the advances required to
achieve the level of technology maturity needed for widespread, practical use of educational
immersive virtual worlds is unlikely to occur within the next five years. Less urgent, but also
desirable, is the development of affordable spatialized sound displays that could increase the
realism of virtual worlds. Inexpensive haptic feedback vests have already demonstrated their
usefulness in the ScienceSpace applications, but there is a lack of affordable, general-purpose
high-resolution haptic feedback devices. Here technology development is proceeding at a
slower pace.
Purchasing special VR-related devices and acquiring continued support for them are
additional areas of concern. Until very recently, the developers of these devices have tended to
be small companies who were competing for a small amount of business. This led to an
unstable marketplace where several companies have closed or changed their area of business
over the last several years. This situation seems to be changing as more buy-outs and
consolidations between organizations are taking place and these changes should result in
greater market stability. Nonetheless, few VR interfaces devices provide a high level of
robustness and the potential for reliable customer support must be considered during product
acquisition.
A final issue concerns the integration of VR technology and the Web. Already there are
several browsers on the web that can be used for non-immersive viewing of virtual worlds,
although these browsers only support very minimal interaction with the worlds. However, the
ultimate goal here is to allow students to collaborate in educational activities using an
immersive virtual world, regardless of their geographical location. While several groups are
planning to conduct research in this area, there is no progress to report as yet. Meanwhile,
119
119
Page 120
121
106
©1998 Institute for Defense Analyses
studies looking at the bandwidth and connection requirements for collaborative systems that
support immersive viewing with voice, sound, and additional modalities (such as haptic
feedback) are needed to determine how well the evolving Web is likely to support projected VR
technology capabilities.
120
120
Page 121
122
107
©1998 Institute for Defense Analyses
List of References
Ainge, D. J. 1996a. "Grade-Six Students' Recall of Detail: VR Compared with Photographs."
VR in the Schools, 1( 4), pp. 1-4.
Ainge, D. J. 1996b. Upper Primary Students Constructing and Exploring Three Dimensional
Shapes: A Comparison of Virtual Reality with Card Nets. James Cook University,
School of Education, Townsville, Australia.
Ainge, D. J. 1996c. "Introducing Primary Students to VR with Virtus WalkThrough: Two Pilot
Studies." VR in the Schools, 2( 1), pp. 10-15.
Allison, D., B. Wills, L. F. Hodges, and J. Wineman. 1996. Gorillas in the Bits. Georgia Insti-tute
of Technology, Atlanta, GA. Technical paper GIT-GVU-96-16.
Bell, J. T. and H. S. Fogler. 1996. "Preliminary Testing of a Virtual Reality Based Educational
Module for Safety and Hazard Evaluation Training." In Proc. American Society for
Engineering Education Annual Conference, Indiana Sectional Meeting, Peoria, IL.
Bell, J. T. and H. S. Fogler. 1995. "The Investigation and Application of Virtual Reality as an
Educational Tool." In Proc. American Society for Engineering Education Annual Con-ference,
Session 2513, June, Anaheim, CA.
Bricken, M. 1992. Summer Students in Virtual Reality: A Pilot Study on Educational Applica-tions
of Virtual Reality Technology. University of Washington, Human Interface Tech-nology
Laboratory of the Washington Technology Center, Seattle, WA. Technical
Publication R-92-1.
Brown, D. J., S. V. G. Cobb, and R. M Eastgate. 1995. "Learning in Virtual Environments
(LIVE)." In Virtual Reality Applications, pp. 245-252. Academic Press: San Diego,
CA.
Brna, P. and R. Aspin. 1997. "Collaboration in a Virtual World: Support for Conceptual Learn-ing."
To appear in Proc. 1st International Conference on Virtual Reality in Education
and Training, June 24-26, Loughborough, UK.
Burton, M. and P. Brna. 1996. "Clarissa: An Exploration of Collaboration Through Agent-Based
Dialogue Games." In Proceedings of the European Conference on Artificial
Intelligence in Education, September 30-October 2, Lisbon, Portugal, pp. 393-400.
121
121
Page 122
123
108
©1998 Institute for Defense Analyses
Byrne, C. 1993. Virtual Reality and Education. University of Washington, Human Interface
Technology Laboratory of the Washington Technology Center, Seattle, WA. Technical
Publication R-93-6.
Byrne, C., C. Holland, D. Moffit, S. Hodas, and T. Furness. 1994. Virtual Reality and "At Risk
Students." University of Washington, Human Interface Technology Laboratory of the
Washington Technology Center, Seattle, WA. Technical Report R-94-5.
Byrne, C. M. 1996. Water on Tap: The Use of Virtual Reality as an Educational Tool. Ph. D.
Dissertation. University of Washington, Seattle, WA.
Carey, B. 1996a. Applying Low End Virtual Reality in the Primary Classroom. Evans Bay Ele-mentary
School, Wellington, NZ.
Carey, B. 1996b. Virtual Reality Software as a Presentation Medium for Students' Research.
Evans Bay Elementary School, Wellington, NZ.
Carey, B. 1997. Personal Communciation.
Cromby, J. J., P. J. Standen, J. Newman, and H. Tasker. 1996. "Successful Transfer to the Real
World of Skills Practiced in a Virtual Environment by Students with Severe Learning
Difficulties." In Proc. 1st European Conference on Disability, Virtual Reality, and
Associated Technologies, July 8-10, Maidenhead, Berkshire, UK, pp. 103-107.
Dede, C., M. C. Salzman, and R. Bowen Loftin. 1996a. "MaxwellWorld: Learning Complex
Scientific Concepts via Immersion in Virtual Reality." In Proc. 2nd International Con-ference
on Learning Sciences, Charlottesville, VA, pp. 22-29.
Dede, C., M. C. Salzman, and R. Bowen Loftin. 1996b. "ScienceSpace: Virtual Realities for
Learning Complex and Abstract Scientific Concepts." In Proc. IEEE Virtual Reality
Annual International Symposium, pp. 246-253.
Dede, C. 1997a. Personal communication.
Dede, C., M. Salzman, R. Bowen Loftin, and K. Ash. 1997b. "Using Virtual Reality Technol-ogy
to Convey Abstract Scientific Concepts." To appear in M. J. Jacobson and R. B.
Kozma (Eds.), Learning the Sciences of the 21st Century: Research, Design, and
Implementing Advanced Technology Learning Environments. Lawrence Erlbaum:
Hillsdale, NJ.
Gay, E. 1994. "Is Virtual Reality a Good Teaching Tool?" Virtual Reality Special Report, Win-ter,
pp. 51-59.
Grove, J. 1996. "VR and History -Some Findings and Thoughts." VR in the Schools, 2( 1), pp.
3-9.
Jonassen, D. H. 1994. "Thinking Technology." Educational Technology, April, pp. 34-37.
122
122
Page 123
124
109
©1998 Institute for Defense Analyses
Merickel, M. L. 1994. "The Relationship Between Perceived Realism and the Cognitive Abili-ties
of Children." Journal of Research on Computing in Education, 26( 3), pp. 371-381.
Mikropoulos, T. A., A. Chalkidis, A. Katsikis, and A. Emvalotis. 1997. Students' Attitude
toward Educational Virtual Environments. University of Ioannina, Department of Pri-mary
Education, Ionnina, Greece.
Moore, A. D. and P. A. McClurg. June 1995. Casper Virtual Reality Project: Preliminary
Report. University of Wyoming, Laramie, WY.
Nemire, K., A. Burke, and R. Jacoby. 1994. "Human Factors Engineering of a Virtual Labora-tory
for Students with Physical Disabilities." Presence, 3( 3), pp. 216-226.
Nemire, K. N. 1995a. "Designing a Virtual Science Laboratory to Accommodate Needs of Stu-dents
with Cerebral Palsy." In Proc. 3rd Annual International Conference on Virtual
Reality and Persons with Disabilities, San Francisco, CA, August 2-4.
Nemire, K. N. 1995b. "Learning in a Virtual Environment." In Proc. 10th Annual International
Conference on Technology and Persons with Disabilities, March 14-18, Los Angeles,
CA.
Osberg, K. M. 1993. Spatial Cognition in the Virtual Environment. University of Washington,
College of Education, Seattle, WA.
Osberg, K. M., W. Winn, H. Rose, A. Hollander, H. Hoffman, and P. Char. 1997. "The Effect
of Having Grade Seven Students Construct Virtual Environments on their Comprehen-sion
of Science." In Proc. Annual Meeting of the American Educational Research Asso-ciation,
March, Chicago, IL.
Roschelle, J. and S. Teasley. 1995. "The Construction of Shared Knowledge in Collaborative
Problem Solving." In O'Malley, C. E. (Ed.), Computer Supported Collaborative Learn-ing.
Springer-Verlag: Heidelberg, Germany.
Rose, H. and M. Billinghurst. 1996. Zengo Sayu: An Immersive Educational Environment for
Learning Japanese. University of Washington, Human Interface Technology Labora-tory
of the Washington Technology Center, Seattle, WA.
Roussos, M., A. E. Johnson, J. Leigh, C. R. Barnes, C. A. Vasilakis, and T. G. Moher. 1997. The
NICE Project: Narrative, Immersive, Constructionist/ Collaborative Environments for
Learning in Virtual Reality. University of Illinois, Chicago, IL.
Salzman, M. C., C. J. Dede, and R. Bowen Loftin. 1995a. "Learner-Centered Design of Senso-rily
Immersive Microworlds Using a Virtual Reality Interface." In Proc. 7th Interna-tional
Conference on Artificial Intelligence and Education, Alexandria, VA, pp. 554-564.
123
123
Page 124
125
110
©1998 Institute for Defense Analyses
Salzman, M. C., C. J. Dede, and R. Bowen Loftin. 1995b. "Usability and Learning in Education-al
Virtual Realities." In Proc. Human Factors and Ergonomics Society 39th Annual
Meeting, San Diego, CA, October 10-15, pp. 486-490.
Salzman, M. C., C. Dede, D. Sprague, and R. Bowen Loftin. 1997a. "Tools for Teaching
Abstract Science: Can Virtual Reality Help?" In Proc. Conference on Computer-Human
Interaction.
Salzman, M. C., C. Dede, and D. Sprague. 1997b. "Assessing Virtual Reality's Potential for
Teaching Abstract Science." To appear in Proc. Human Factors and Ergonomics Soci-ety
41st Annual Meeting.
Tiffin, J. and L. Rajasingham. 1995. In Search of the Virtual Class. Routlege Publishing: Lon-don,
UK.
Turoff, M. 1995. "Designing a Virtual Classroom." In Proc. 1995 International Conference on
Computer Assisted Instruction, March 7-10, Hsinchu, Taiwan.
Wilson, J. R. 1996. "Effects of Participating in Virtual Environments: A Review of Current
Knowledge." Safety Science, 23( 1), pp. 39-51.
Winn, W. 1993. A Conceptual Basis for Educational Applications of Virtual Reality. Univer-sity
of Washington, Human Interface Technology Laboratory of the Washington Tech-nology
Center, Seattle, WA. Technical Publication R-93-9.
Winn, W. 1995. "The Virtual Reality Roving Vehicle Project." T. H. E. Journal (Technological
Horizons in Education), 23( 5), pp. 70-75.
Yu, Y. C. 1996. Virtual Reality and K-12 Education. (Posted at sci. virtual worlds newsgroup.)
124
124
Page 125
126
111
©1998 Institute for Defense Analyses
List of Acronyms
1D One-dimensional
2D Two-dimensional
3D Three-dimensional
AAAS American Association for the Advancement of Science
AIDS Acquired Immune Deficiency Syndrome
BTEC Business Technician Education Council
CAD Computer-Aided Design
CAVE Cave Automatic Virtual Environment
CD Compact Disc
CDS Conceptual Design Space
DEVRL Distributed Extensible Virtual Reality Laboratory
DIVE Distributed Interactive Virtual Environment
DOF Degrees of Freedom
EMF EM Field
GULLIVR Graphical User Learning Landscapes in Virtual Reality
GVU Graphics Visualization and Usability
HCC Haywood Community College
HITL Human Interface Technology Laboratory
HIV Human Immune Virus
HMD Head-Mounted Display
HP Hewlett-Packard
I/ O Input/ Output
IEC International Electrotechnical Committee
ISO International Organization for Standardization
JSC Johnson Space Center
LAKE virtuaL Approach to the Kernel of Eutrophication
LIVE Learning in Virtual Environments
M. A. Master of Arts
125
125
Page 126
127
112
©1998 Institute for Defense Analyses
MAED Master of Arts in Education
MAEL Mobile Aeronautics Education Laboratory
MOO Multi-User Domain or Dungeon (MUD) Object-Oriented
MUD Multi-User Domain or Dungeon
N/ A Not Applicable
NASA National Aeronautics and Space Administration
NCSA National Center for Supercomputing Applications
NCREL North Central Region Educational Laboratory
NICE Narrative, Immersive, Constructionist/ Collaborative Environments for
Learning in Virtual Reality
NRC National Research Council
NSF National Science Foundation
PC Personal Computer
RSE Resource for Science Education
SEMAA Science, Engineering, Math, and Aerospace Academy
SGI Silicon Graphics, Inc.
TBD To Be Determined
TCP/ IP Transmission Control Protocol/ Internet Protocol
TEACHC Treatment and Education of Autistic and Other Communications Handi-capped
Children
TRI Transition Research Institute
TPR Total Physical Response
UK United Kingdom
VESAMOTEX Virtual Education -Science and Math of Texas
VE Virtual Environment
VESL Virtual Environment Science Laboratory
VIRART Virtual Reality Applications Research Team
VR Virtual Reality
VRDS Virtual Reality Development System
VREL Virtual Reality and Education Laboratory
VRML Virtual Reality Modeling Language
VRRV Virtual Reality Roving Vehicle
126
126
Page 127
128
113
©1998 Institute for Defense Analyses
Appendix A. Participating Researchers and Teachers
Dr. David Ainge James Cook University
David. Ainge@ jcu. edu. au School of Education
Townsville 4811, Australia
Prof. Bill Winn University of Washington
billwinn@ hitl. washington. edu Human Interface Technology Laboratory
Seattle, WA 98195-2142
Mr. Don Allison Georgia Institute of Technology
don@ cc. gatech. edu Graphics Visualization and Usability Center
801 Atlanta Drive
Atlanta, GA 30332-0280
Mr. Rob Aspin University of Lancaster
aspinr@ comp. lancs. ac. uk B22 Secams Building
Bailrigg Campus
Lancaster LA1 4YR, UK
Dr. John Bell University of Michigan
JohnBell@ umich. edu Dept. of Chemical Engineering
3704 H. H. Dow Building
Ann Arbor, MI 48109-2136
Ms. Marj Bisiar Natrona School County District
marjbi@ aol. com Casper, WY
Dr. David Brown University of Nottingham
epzdjb@ epl1. maneng. nottingham. ac. uk Department of Manufacturing Engineering and Operations
University Park
Nottingham NG7 2RD, UK
Mr. Thomas Bulka Northern High School
tbulka@ husky. northern-hs. ga. k12. md 86 Pride Parkway
Accident, MD 21520
Prof. Lowry Burgess Carnegie Mellon University
lb30@ andrew. cmu. edu NASA/ Robotics Engineering Consortium
Pittsburgh, PA 15123-3890
Dr. Brice Carey Evans Bay Intermediate School
bruce. carey@ vuw. ac. nz Kemp Street
Kilburnie, Wellington, New Zealand
127
127
Page 128
129
114
©1998 Institute for Defense Analyses
Dr. John Cromby University of Nottingham Medical School
mvzjjc@ mvn1. 1disab. nottingham. ac. uk Department of Learning Disabilities
Queens Medical Center
Nottingham, NG7 2UH, UK
Dr. Chris Dede George Mason University
cdede@ gmu. edu Graduate School of Education
4400 University Drive, Fairfax, VA 22030-4444
Mr. Tim Dedula NASA Lewis Research Center
wtdedula@ lerc. nasa. gov Technical Services Directorate
21000 BrookPark Road Cleveland, OH 44135-3191
Dr. Tony Gaddis Haywood Community College
tony@ daystrom. haywood. cc. nc Regional High Tech Center
10 Industrial Park Drive
Waynesville, NC 28786
Dr. Eben Gay ERG Engineering, Inc.
erg@ world. std. com 2 Moore Road
Southboro, MA 01772
Mr. Jonathan Grove Sheffield Hallam University
j. m. grove@ shu. ac. uk Communications and Information Research Group
Mundella House
Collegiate Crescent Campus
Sheffield S10 2BP, UK
Prof. Larry Hodges Georgia Institute of Technology
larry. hodges@ cc. gatech. edu Graphic, Visualization, and Usability Center
801 Atlanta Drive
Atlanta, GA 30332-0280
Dr. Lynn Holden (Egyptian Museum/ Rosacrucian Park)
(408) 947-3600 200 Button Street, Apt. 79
Santa Cruz, CA 95060
Dr. Dean Inman Oregon Research Institute
deani@ ori. org 1715 Franklin Blvd.
Eugene, OR 97403
Prof. Bowen Loftin University of Houston
bowen@ gothamcity. jsc. nasa. gov Dept. of Natural Sciences
One Main Street
Houston, TX 77002
Dr. Gail Ludwig University of Missouri
gailmo@ showme. missouri. edu Dept. of Geography
Columbia, MO 65211
128
128
Page 129
130
115
©1998 Institute for Defense Analyses
Prof. Tassos Mikropoulos University of Ioannina
amikrop@ cc. uoi. gr Dept. of Primary Education, EARTH team
Doboli 30, 45110 Ioannina, Greece
Dr. Ken Nemire Interface Technologies Corporation
knemire@ netcom. com 1840 Forty-First Avenue, Suite 102
Capitola, CA 95101
Ms. Eugenia Nikolou University of Ioannina
me00010@ cc. uoi. gr Dept. of Primary Education, EARTH team
Doboli 30, 45110 Ioannina, Greece
Prof. Veronica Pantelidis East Carolina University
lspantel@ ecuvm. cis. ecu. edu School of Education
Virtual Reality and Education Laboratory
Greenville, NC 27858
Dr. Kim Osberg/ Dr. Howard Rose Firsthand LLC
kmo@ firsthand. com Seattle, WA
hrose@ firsthand. com
Dr. Maria Roussos University of Illinois at Chicago
mroussos@ eecs. uic. edu Interactive Computing Laboratory and Electronic
Visualization Laboratory
851 S. Morgan Street, Room 1120
Chicago, IL 60607-7053
Dr. Donald Sanders Learning Sites, Inc.
dsanders@ berkshire. net 151 Bridges Road
Williamstown, MA 01267
Ms. Marilyn Salzman George Mason University
msalzman@ gmu. edu Graduate School of Education
4400 University Drive, Fairfax, VA 22030-4444
Dr. Dorothy Strickland North Carolina State University
dorothy@ adm. csc. ncsu. edu Computer Science Department, Box 8206
Raleigh, NC 27695
Mrs. Marcia Talkmitt Slaton Independent School District
mtalkmit@ tenent. edu (VESAMOTEX)
Box 10
Wilson, TX 79381
Dr. Umesh Thakkar University of Illinois at Champaign-Urbana
uthakkar@ ncsa. uiuc. edu National Center for Supercomputing Applications
152 Computing Applications Building
605 E. Springfield Avenue
Champaign, IL 61820
129
129
Page 130
131
116
©1998 Institute for Defense Analyses
Mr. Bob Trueman Computers in Education
Bob_ Trueman@ nybe. northyork. on. ca c/ o Northview Secondary School
550 Finch Avenue West
North York, Ontario MZR 1N6, Canada
Ms. Becky Underwood Kelly Walsh High School
75117.773@ CompuServe. com 3500 East 12th
Casper, WY 82609
Mr. Terry Wolfe Educational Service Unit #3
twolfe@ esu3. esu3. k12. ne. us 6949 So. 110th Street
Omaha, NE 68128-5722
Mr. Peter Zohrab Correspondence School
zohrab@ ihug. co. nz Languages 1 Department, Private Bag
Wellington, New Zealand
130
130
Page 131
Public Reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington
Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project
(0704-0188), Washington, DC 20503.
REPORT DOCUMENTATION PAGE
1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
6. AUTHOR( S)
7. PERFORMING ORGANIZATION NAME( S) AND ADDRESS( ES) 8. PERFORMING ORGANIZATION
REPORT NUMBER
10. SPONSORING/ MONITORING
AGENCY REPORT NUMBER
9. SPONSORING/ MONITORING AGENCY NAME( S) AND ADDRESS( ES)
11. SUPPLEMENTARY NOTES
12a. DISTRIBUTION/ AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE
13. ABSTRACT (Maximum 180 words)
14. SUBJECT TERMS 15. NUMBER OF PAGES
16. PRICE CODE
20. LIMITATION OF ABSTRACT 19. SECURITY CLASSIFICATION
OF ABSTRACT
18. SECURITY CLASSIFICATION
OF THIS PAGE
17. SECURITY CLASSIFICATION
OF REPORT
NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std. Z39-18
298-102
Form Approved
OMB No. 0704-0188
DASW01 94 C 0054 DARPA Task A-189
Institute for Defense Analyses 1801 N. Beauregard St.
Alexandria, VA 22311-1772
UNCLASSIFIED UNCLASSIFIED UNCLASSIFIED SAR
January 1998
Christine Youngblut
IDA Document D-2128
Educational Uses of Virtual Reality Technology
Defense Advanced Research Projects Agency 3701 N. Fairfax Drive
Arlington, Virginia 22203-1714
FinalÑ January 1996Ð December 1997
The potential of VR technology for supporting education is widely recognized. It has already seen practical use in an estimated 20 or more public schools and colleges, and many more have been involved in evaluation or research
efforts. This document reviews current efforts that are developing, evaluating, or using VR technology in education. It builds a picture of the states of the art and practice, and reviews some of the critical questions that are being
addressed. Educational uses of the technology are broadly distinguished as those where students interact with pre-developed VR applications and those where students develop their own virtual worlds in the course of research-ing,
understanding, and demonstrating their grasp of some subject matter. Forty-three efforts in the category of pre-developed applications and another 21 efforts in the category of student virtual world development are reported in this
paper. The results of 35 evaluations that have been completed on these efforts are summarized. Another 20 ongoing or planned evaluations are identified.
Virtual Reality, Virtual Environments, Education 123
Approved for public release; distribution unlimited. 131