The National Science Foundation (NSF) funded the Human Interface
Technology Laboratory (HITL) at the University of Washington to
hold a three-day workshop on "The Impact of Three Dimensional
Immersive Virtual Environments on Modern Pedagogy" in Seattle,
Washington, and a three-hour mini-workshop at a Virtual Reality
in Education and Training (VRET '97) Conference in Loughborough,
England. These workshops were held in May and June, 1997, respectively.
The HITL invited people with backgrounds in educational technology,
cognitive psychology, global change and K-12 education. Relevant
industrial participants were also invited but only one was able
to attend. The workshop brought together 35 people from across
the US and it was held at Battelle Conference Center. This informal
format provided opportunities to exchange ideas in and out of
sessions. There was a mixture of general plenary discussions,
individual and group presentations, and small group break out
sessions.
Overall the feedback from the participants was positive. There
was a sense that bringing together representatives from the four
groups of people at the Seattle workshop was an effective way
to get at the issues, though some problems did arise--namely the
lack of time to draw more concrete conclusions among and between
the groups. Introducing some of the concepts presented in the
Seattle workshop to European counterparts at VRET '97 showed that
there were more similarities than differences between the problems
faced by researchers and developers working with VR on both sides
of the Atlantic.
This final report primarily is our effort to synthesize, distill
and summarize the discussions and comments made by participants
in the Seattle workshop. Where relevant, comments from our European
participants are also included and noted. No clear consensus was
reached regarding the recommendations to NSF. Hence, the recommendations
are not listed in any priority order and merely reflect the various
points brought up during the discussions.
We would like to thank Janice DeCosmo at the University of Washington, who provided significant input on the global change content areas. Bev Lynds and Ron Kantor also provide helpful insights on prior drafts of the final report.
PART 1. OVERVIEW
I. Introduction
The National Science Foundation (NSF) funded the Human Interface
Technology Laboratory (HITL) at the University of Washington to
hold a three-day workshop on "The Impact of Three Dimensional
Immersive Virtual Environments on Modern Pedagogy" in Seattle,
Washington, and a three-hour mini-workshop at a Virtual Reality
in Education and Training (VRET '97) Conference in Loughborough,
England. These workshops were held in May and June, 1997, respectively.
The Seattle workshop brought together 35 educational technologists,
teachers, cognitive scientists and specialists in global change
from across the United States. A list of those in attendance is
p[rovided in the Appendix to this report. The VRET mini-workshop
brought together approximately 50 people who's background were
primarily in educational technology. This report is based on the
discussions, comments and recommendations that arose at both workshops.
The purpose of the workshops was to identify the key questions
surrounding the use of advanced immersive and desktop computer
technologies to teach complex science material, exemplified by
global change, in K-12 education. From these questions, the workshops
were charged with developing an agenda for conducting research
on the relative merits and effectiveness of these technologies.
To date, virtual reality technology has been used primarily in
the military, training, and entertainment markets. Now, however,
the means to produce immersive, interactive virtual environments
have developed to the point where it is feasible to use virtual
environments in schools to help students learn. However, there
has been no careful and systematic study of the capability of
virtual environments, relative to other often cheaper technologies
and their attendant pedagogies, for helping students learn. Part
of the challenge of research focusing upon the development and
testing of such environments is finding ways to integrate them
within existing school culture. The adequate testing of such technologies
requires an approach that synthesizes VR technologies with new
theories of learning.
There is a concern that children graduate from high school without
an understanding of science that is sufficient for them to enter
the workforce or to continue their education in college. This
is largely because arriving at even a rudimentary understanding
of the true complexity of natural science defies traditional teaching
strategies that oversimplify natural phenomena to make them teachable
and emphasizes decontextualized assessment and declarative knowledge.
Global change was chosen as the science content because there
is a consensus that children need to understand the ramifications
of continued misuse of the Earth's land, air and oceans. The complexity
of the processes that affect the world through global change meet
the criterion of complexity of the science content to study.
No single laboratory can be expected to have all the expertise
in virtual reality, instructional technology and science content
to be able to develop and execute a program of research into the
characteristics of the most effective ways teach science. An interdisciplinary
and collaborative approach is needed, especially at the planning
stage, and a model of research and development, recently referred
to as "design experiments", should be adopted that places
researchers in vivo, inside classrooms on an ongoing basis.
II. Workshop Objectives
The objectives of the workshops were to:
III. Workshop Focus.
To give focus to the workshop, participants received two documents before arriving in Seattle. The first was a list of questions designed to draw attention to the key issues the workshop was expected to address. The second was a "White Paper", prepared by Bill Winn, that elaborated on the questions and provided more context for the workshop's deliberations. (This paper may be found at HITL's web site, http://www.hitl.washington.edu/publications/index.html as document R-97-15.)
The questions for discussion fell into five groups:
Attributes of virtual reality
1. What particular attributes of virtual environments, desktop
multimedia programs and simulations and other strategies for teaching
about global change are likely to be the most successful?
2. Is immersive VR sufficiently different from its precursors
to permit entirely new learning experiences that might lead to
large improvements in students' understanding of Global Change?
If VR is to make innovative and unique contributions to learning,
it is critical to start by identifying what the technology is
uniquely capable of offering to educators.
Global change as subject matter
3. What concepts and principles of Global Change are important
for students to understand while at the same time serving as good
learning tasks for research.
4. What common misconceptions do students have about Global Change?
Which of these might the various technologies and strategies we
discuss be best suited to correct?
5. What new misconceptions might VR, or other technologies, create
in students' understanding of Global Change?
Although it is to be hoped that research will identify the usefulness
and pitfalls of using VR for all subject areas, Global Change
is a particularly useful domain to begin with. It is topical and
helping students understand how it works is of critical importance
to preserving our standard of living for suture generations. It
is sufficiently complex to provide a rigorous test of the capabilities
of VR for learning. It is a content area that lends itself well
to students learning by exploration and by applying the scientific
method.
Approaches to learning using technology
6. What teaching and learning strategies are best matched with
which technology to be most successful in helping students understand
Global Change?
7. Are strategies based on constructivist theories of learning
particular effective for teaching about Global Change using technology?
It is likely that new technologies will not attain their maximum
potential in education if they are only used to implement current
strategies for teaching and learning. It is reasonable to expect
that VR is capable of supporting innovative pedagogy (as well
as existing pedagogy). A starting point for pedagogical innovation
is likely to be emerging theories of knowledge construction by
students.
Student characteristics
8. Is it possible, or necessary, to match strategies and technologies
that teach about Global Change to particular student characteristics?
9. How much knowledge of Global Change and basic skills in Math
and Science must students have before working with virtual environments,
simulations and other modes of instruction?
Past experience with computer-based education has shown that learning
with technology is not the best road for all students to follow.
It is likely that VR also will be more effective with some students
than with others. It is also possible that that effectiveness
of VR will vary according to how much knowledge of the subject
students bring to their learning experiences in virtual environments.
The risks of advocacy
10. What research methodologies and designs should be used to
provide rigorous and objective tests of the relative merits of
the various technologies and strategies to teach about Global
Change?
11. How much can we generalize from research findings about Global
Change to other areas of science and other disciplines?
VR has caught the public's imagination and has raised expectations
among some that it will solve a number of educational problems.
Such advocacy has, in the case of past technologies, led to disappointment
and to the dismissal by educators of even those technologies that
can be effective. It is therefore important for us to use appropriate
methods to study VR's effectiveness and to determine the subject
areas to which it can make the most contribution.
IV. Workshop Assessment.
The most important products from the Seattle workshop were the
deliberations and recommendations that are reported in Part 2
of this report. The quantity and quality of these attest to the
overall success of the workshop. Formal and informal feedback
showed that participants were generally positive towards the activities,
although some felt that there had not been enough time to complete
the work and one or two felt that they had not been able to integrate
with their working groups. However, most judged the collaboration
of scientists, cognitive scientists, teachers and educational
technologists to have been a productive feature of the workshop.
The mini-workshop, conducted by Rose Yu and Bill Winn at the VRET
Conference in Loughborough, was, likewise, well received and judged
a success. The interaction among European and North American scholars
working on educational applications of VR led to some insights
that would not otherwise have arisen. While it is somewhat risky
to generalize, the greater pragmatism of several European colleagues
led to some interesting discussions during the group sessions.
PART 2. WORKSHOP DELIBERATIONS AND RECOMMENDATIONS
I. Organization of Part 2
This part of the report summarizes of the discussions and recommendations
from both workshops. The information that follows is organized
around three themes -- Learning, Global Change and Virtual Reality
-- and, more importantly, around the intersections of these themes,
shown in the following diagram.
The report does not include discussion of the intersection of
VR with Global Change. We consider this area to be of minor relevance
to the study of VR in teaching and learning global change concepts.
II. Learning
To fully account for theories of how people learn is far beyond
the scope of this report. For that reason, we present some general
principles that apply to all learning and then some that apply
specifically to learning complex science content that are most
likely to be important when learning is supported by some advanced
technology.
General principles:
Attention is a prerequisite for learning.
We must get students involved in their learning processes. It
is obvious that if students are not paying attention to the material
they are to learn they cannot begin to learn it. Our instruction
must therefore gain and hold students' attention. Research has
demonstrated that the newer technologies, including VR and multimedia,
can get students' attention.
What is learned persists if it is practiced.
A quick visit to a scientific subject is not sufficient for learning.
This is true of cognitive as well as of psychomotor learning.
Computer-based technologies are ideal for allowing students to
repeat activities, with or without variations, for as long as
it takes for them to master concepts, principles and skills.
What is learned persists if it is learned in context.
A great deal of recent research has shown that much of what is
now learned in schools is learned without a context that makes
application of knowledge easy, effective and useful. Knowledge
learned this way is "inert". Technology-based instructional
environments, as indeed real environments, can provide the needed
context.
People forget things they do not use.
Repeated visits to the subject are essential. One thing to consider
when constructing contexts for learning is that, without the opportunity
to apply new knowledge and skill immediately and to keep using
it, new knowledge is quickly forgotten even if it contextualized.
Technologies can provide such opportunities up to a point. However,
we need to keep in mind the need to facilitate transfer from the
classroom to the real world as we develop instructional environments
for science.
Experiential Learning
Students learn best when they enjoy a rich, often multi-modal,
experience of the educational material. Learning by looking at
books or computer screens, or listening to a teacher, is only
partly effective. Students need to experience the concepts and
principles contained in the content as much and as directly as
possible. Learning experiences can be improved by observing a
number of principles listed in the following that involve ownership,
engagement and social context:
Intrinsic motivation is increased if the learner has "ownership"
of experience and control over or personal involvement in the
experience. Often in class, students receive information that
has been provided by others - the teacher, textbook writer, video
producer. Students may learn this material and be rewarded by
getting a good grade, but they do not feel that the knowledge
they acquire from it is their own. On the other hand, students
who can construct their own ideas directly from experience, without
the mediation of a third party, are more interested and excited
in what they learn. Motivation is now largely intrinsic, interest
higher and retention longer.
Effective experiential learning requires complete mental and
physical engagement, appropriate amounts of challenge and a cohesive
narrative framework for the learning experience. All dimensions
of the experience should be available to students. If the real-world
experience is multi-sensory, then the learning environment must
replicate all of these sensations. If the real-world experience
is challenging, then the learning environment must reflect this
challenge, not water it down simply to make it easier to learn.
None of this will be effective unless the structure of the experience
is clear to the student. If the experience extends over time,
then the "story" it tells must be coherent, the focus
of the activity clear and the end goal evident.
Learning experiences must be valid within a social context
established by group interaction. All learning experiences
involve more than just one person, directly or indirectly. This
is true at the time of learning when students interact with at
least one other person and increasingly with other students and,
over the Internet, with people outside the immediately classroom
community. It is also true at the time when the student uses what
is learned. Therefore, the learning experiences and activities
must make sense to the group within which the student learns.
The final outcomes of learning must be valid within the broader
community within which they will be applied. The judgment of the
learning experience's validity, within both of these contexts,
is determined socially.
Points of View of Knowledge
Knowledge construction involves the development and manipulation
of different epistemological frameworks. The ways in which we
know and come to know about the natural world vary very considerably.
Traditional "book" learning and experiential learning,
which we have just discussed, are two such variations. Constructivist
approaches to learning science and the new highly interactive
and usually multi-modal technologies that support learning environments
make it possible to present many different epistemological frameworks.
For example:
Students may easily compare and contrast different models of content.
Thanks to the improving power of computers to create accurate
and pedagogically viable simulations of global change, students
can experience a variety of different theories of how the environment
is being altered by natural phenomena and by human activity. The
student may, for example, experience simulations based on different
models of El Ni“o and decide, on the basis of personal experience
or other evidence, which is the most valid. In this way, the student
may reconcile discrepancies between personal mental models of
environmental change and scientific evidence. Misconceptions about
the natural world develop in early childhood and often persist
into adulthood.
Students may generate and test hypotheses concerning experienced learning environments and the representational models they contain.
Extending the ideas from the preceding point, students may now
use simulations to test their own ideas about global phenomena.
Here, students consider the world created in the learning environment
as a scientist might the real world. What is more, the "student-scientists"
can make manipulations within the learning environment that cannot
be made in the real world. For instance, students can observe
what would happen if the tropical rain forests were instantaneously
restored to their 19th century state.
Students can use authentic tools for interacting with the learning environment.
The best but often least practical way to do this is to "apprentice"
the student to an environmental scientist working in the real
world. Students make observations about such things as the weather
or the occurrence of animal and bird species and provide this
information to the scientist researching those phenomena. However,
much of the effectiveness of this approach can be achieved through
working in computer-based learning environments. In VEs, for example,
students may use virtual thermometers to measure air and ocean
temperature. In desktop simulations, students may drag and drop
instruments for measuring air pollution onto a map of the country
to make observations in particular places. In all of this, actual
navigation through the environment must be intuitive so as not
to intrude.
Students have the time and opportunity to think about their experiences.
This reflection should include thinking about the information
itself as well as about the social implications of what the simulation
reveals. Reflecting on global change from this point of view completes
the move from simply reflecting on the content itself, to reflecting
on it like a scientist, to reflecting on it like a responsible
scientist or citizen.
Symbol Systems
All information in learning environments is presented through
symbols of one sort or another. Different symbol systems reflect
different epistemological stances. Therefore the way in which
information is presented to students needs careful consideration.
Variation in symbols can have radical and sometimes unpredictable
effects on understanding. This is because different symbol systems
engage perceptual and cognitive processes in different ways.
Different symbol systems activate different mental models in memory.
The modality in which one acquires new knowledge has some impact
on the mental models that arise from that knowledge, how that
knowledge is stored in memory, how it is retrieved and how it
is used. The main division between symbol systems falls between
those that represent information pictorially and those that use
language to do so. Pictorial, or "iconic", symbols resemble
what they stand for. This means that they are easily interpreted
and encoded as mental images. Linguistic, or "digital",
symbols have an arbitrary relation to what they represent and
therefore have to be learned before they can be interpreted.
The structure of representations increases the likelihood of certain interpretations and reduces the likelihood of others.
How information is arranged in a learning environment can have
far-reaching consequences in how that information is understood,
encoded and retrieved. Even at the most basic pre-attentive levels
of perception, before meaning and context have been ascribed to
messages, such factors as grouping of objects, symmetry, color
coding and devices that influence eyeflow constrain the number
of possible interpretations that students will subsequently make.
The design and construction of the interface to the learning environment
and the way in which the environment itself is constructed are
therefore critical to learning.
Some, less "abstract", symbol systems allow more direct construction of conceptual and propositional knowledge.
At the heart of experiential learning is the importance of direct
interaction with the environment. A student might take ocean temperature
by clicking on a box containing the word "thermometer"
and reading the temperature in a pop-up window. Or the student
might grab a virtual thermometer, dip it into the virtual sea,
and read the temperature off the thermometer's scale. More generally,
the learning environment can be a realistic and convincing facsimile
of the real world in which the student feels a high level of "presence".
Or the environment can be a series of scales and dialog boxes
on a desktop computer screen that sit on top of exactly the same
simulation program and allow the same interactions with it that
vary only in the manner of their execution. In most cases (but
see the next point), the former, realistic representation will
lead to more direct and more robust knowledge construction.
Skill in symbol systems varies among learners.
Some students do learn better from more abstract symbol systems.
For example, some students learn about the environment just as
well if not better from reading text than from running a simulation
or visiting a VE. On the other hand, students who tend to do less
well in a classroom that relies heavily on language-based symbol
systems - texts and lectures - may learn more and retain more
if they work in learning environments that faithfully simulate
the real world and allow direct interaction with it. It is important
not to assume that learning through desktop simulations or in
VEs is best for all students.
III. Global Change
This section briefly summarizes of some of the important aspects
of global change and earth system science. It is an attempt to
introduce a context for the subject matter, and is neither exhaustive
nor based on learning goals for any specific educational level.
The content related to global change may be broken down into 3
major categories as below. Any curriculum on global change will
include some focus on these major categories. The depth of understanding
built through the curriculum for the elements within these categories
will depend on age-appropriate learning goals and applicable science
standards. Some detail is included here; of course, most K-12
students will be able to explore only a limited subset of the
basic concepts outlined below.
Natural processes relevant to global change
This category includes physical, biological, geophysical and chemical
processes which together make up the earth system. Traditionally,
these subjects have been taught independently of one another.
The recent advent of "Earth System Science" as its own
discipline has arisen out of the necessity of viewing the system
as a whole to better analyze and predict global change. This category
can again be subdivided.
"Building block" concepts
These are closest to what might be taught in "traditional"
disciplines: the life cycle of plants and animals in biology;
volcanism and the movement of tectonic plates in geology; global
atmospheric and oceanic circulation and redistribution of energy;
small scale atmospheric circulation patterns and weather; the
atmospheric greenhouse effect on earth and other planets; chemical
exchange between land, biosphere, ocean and atmosphere, etc.
Interdependence of processes - feedback mechanisms
This includes consideration of the effects of the processes listed
above on one another, such as in complex processes of: desertification,
El Nino, ice-albedo feedback, sea level change, etc.
Multi-dimensional, multi-scale processes
global change is manifested at local scales in a system described
by multiple variables in multiple dimensions. Mechanisms driving
global change may be global or local scale; interactions of the
properties of the system at different spatial and time scales
produce secondary (and beyond) feedback.
Dynamic equilibrium
The earth system will continuously evolve to achieve equilibrium
with respect to dominant processes, such as energy exchange. Perturbation
of parameters within the system may force a new equilibrium state.
A new equilibrium state may not be achieved through slow gradual
steps but may be approached discontinuously.
Human influence on global change
It is important to understand the complexity and variability inherent
in the natural system. Human influence on the system, then, may
be understood as a mechanism that may drive the system to a new
equilibrium state.
Humans as part of the natural system
Through natural life processes, humans participate in the biochemical cycles that constitute elements of the earth system. Other human activities also produce profound effects on the earth system. In particular, industrialization, agricultural practices and human population growth have significantly affected the earth.
Atmospheric composition and human influence
Industrialization and agricultural practices have caused significant
changes in the concentrations of trace gases in the atmosphere.
Some of these gases are contributors to the atmospheric greenhouse
effect, which is a natural process that keeps our planet within
the "habitable zone" for life. However, as the concentrations
of these gases increase through human influence, the energy balance
of the system is significantly affected, and many feedback mechanisms
are set in motion.
Other gases and particulates produced by human activity affect
the suitability of the lower atmosphere for sustaining plant and
animal life (acid rain, ozone and particulate concentrations).
They also affect the upper atmosphere's ability to filter out
solar ultraviolet radiation through depletion of stratospheric
ozone and the energy balance of the earth system through increased
reflection of solar radiation by microscopic particles.
Human impact on land cover
Human influence on land cover through clearing of forests, burning
of agricultural debris, development of urban areas, re-routing
of rivers, etc. has profoundly affected earth's energy balance
and availability of resources.
Population growth and natural resource consumption
Humans require natural resources to thrive. Population growth
strains natural resources such as energy and water. Continued
growth, and unequal consumption of resources affects the earth
system and our quality of life.
Assessment of change and predictions for the future
How do we assess global change? This is a complex issue involving
analysis of the aggregate of change at local and intermediate
spatial scales, averaged over suitable time periods. It necessitates
an understanding of the natural variability of the system. To
analyze such a complex system, computer models are created to
reflect current understanding of the relevant processes and to
keep track of the effects of feedback mechanisms.
Natural variability of the system
To understand when global change is occurring, we must characterize
the level of natural variability inherent in the system. An unusually
hot summer on a hemispheric scale may be a natural departure from
"normal" - several of these (how many?) in a row, however,
may constitute the hemispheric response to global change.
Use of global models to analyze and predict change
Global atmosphere-ocean-land models are run on supercomputers
to analyze past climates and to predict global change. These models
represent the most sophisticated tools we have, however, they
are approximations of reality and are limited by the current state
of knowledge and availability of data. Analysis of model-produced
data is also complex, since many feedback responses resulting
from model iterations may not be well-understood even by the creators
of the model representations.
Scientific predictions and societal decision-making
Global models are used to make the best predictions we can about
global change. These results inform policy-makers and citizens
about action that may be taken to alleviate/moderate predicted
effects. However, no model is a perfect replica of reality, and
decision-makers must recognize the evolutionary nature of scientific
knowledge. Decision-makers strive to balance probable outcomes
with the effort required for implementation of action plans.
IV. Attributes of Virtual Reality
Autonomy, Presence, Interaction
According to Zeltzer, instructional technologies
may vary in the extent to which they are autonomous, induce presence,
and allow meaningful interaction between user and system. Autonomy
refers to the extent to which the VE is capable of performing
its own actions without intervention from or attention to the
user. A VE that rates high on this dimension would be dynamic,
following its own paths to goals. The user's intervention might
or might not change the course of events. High-fidelity real-time
simulators are examples of autonomous VEs. A simulated oil spill
might move inexorably to a pollution disaster in spite of a student's
attempts to prevent it. The key to creating autonomy in a VE lies
in the goodness of the computational models that create system's
actions and govern its behavior.
Presence is the experience the user has of being in a real place
when visiting a virtual one. It depends on the effectiveness of
the physical interface. For presence to be high, the interface
must allow the user to interact with the VE in natural, intuitive
ways. Any environment conveys to people who visit it ways of interacting
with it that are intrinsic to it and therefore appear natural
- its "affordances". Thus, a virtual thermometer in
a VE should be as "graspable", manipulable and useful
as a real one. And the ability to use the virtual thermometer
should be intuitively evident to the user. This clearly requires
a great deal of attention to and success in interface design.
But when successful, presence is high and the sense that the user
is interacting with a computer at all disappears. The interface
itself vanishes.
The third dimension is interaction. While the physical interface
and the presence it engenders may be improved by interactions
involving the natural affordances of the VE, we are concerned
here with the logical interface to the VE. This means the extent
to which the VE responds correctly to the student's actions. For
example, a VE that simulates smog in a large city must correctly
simulate the causes that the student may manipulate and the effects
the student may observe. When the student enacts policy that requires
cars to be non-polluting, the smog level must be seen to decrease.
Variation in a simulation's autonomy, presence and interaction
is likely to have a profound effect on both the information the
VE conveys and what the student learns from it. Research into
the effectiveness of VR and other learning environments for learning
must pay careful attention to these three factors.
Dimensionality
The extent to which these three factors vary is determined in
part by the degree of immersion that virtual environments afford.
VR researchers generally distinguish between three levels of immersion
which describe different degrees to which users are enabled to
interact with data in two- or three-dimensional space. The first
level is in fact two-dimensional. Here, users work with data displayed
on a typical computer screen in ways with which anyone who has
used a computer is familiar. The user may move the data up and
down and left to right. This level is referred to as "2D".
The next level still confines the user to a regular computer screen.
However, the data are displayed in such a way that the user can
appear to move around the screen in three dimensions. That is,
the user may move "into the screen" along the "z"
axis and see objects fly by on all sides as if moving through
them. This creates an illusion of three-dimensions on a two-dimensional
display. The term "two-and-a-half D -- 2.5D" reflects
this compromise. Most arcade-like computer games and VRML "flythroughs"
fall into this category. Finally, the user may don helmet and
gloves and "enter" a fully three-dimensional VE. With
head and hand tracking, the user is surrounded on all sides by
objects representing data. This condition is referred to as "3D".
Egocentric and exocentric perspectives
Yet another factor that comes into play is the point of view of
the user vis-‡-vis the display. If the data are displayed
to reflect the user's point of view and to keep up as this changes,
then the VE is said to be "egocentric". This puts the
user firmly in the driver's seat as it were, allowing a first-person
experience of the VE. It is also possible for a user to observe
the VE more objectively, to watch someone else's activities in
what is sometimes referred to as a "God's eye view".
In this case, the VE is said to be "exocentric". Most
films, for example, are exocentric. The viewer is an observer
of other people's actions.
The combination of the dimensionality of the experience (2D, 2.5D or 3D) and point of view (egocentric or exocentric) yields six major ways in which VEs can vary.
| Egocentric | Exocentric | |
|---|---|---|
| 2D | ||
| 2.5D | ||
| 3D |
We believe that studying the effects of varying these six factors
on learning and performance in VEs, along with studying variations
in autonomy, presence and interaction, is a useful way to structure
a research agenda.
Other attributes
So far, we have considered very general attributes of VR. More
particular attributes of VR are also likely to influence learning.
We believe that the study of variation in these is also likely
to be profitable for researchers. Their relevance is, naturally,
to more specific applications of VEs in education.
V. Learning and Global Change
The topic of global change presents significant challenges to
teaching. Students approach the subject matter with pre-(mis-)conceptions
related to many of the "building block" concepts as
well as the more complex subject matter of the interdependence
of earth processes that produce global change. Interdependence
of processes and the concepts of different spatial and time scales
are difficult to grasp. Media coverage of this topic is extensive
and problematic. Many of the popular media presentations related
to global change include inappropriate use of model predictions,
inaccurate representations of physical processes, false associations
of unrelated phenomena, and use of emotion-laden language. Techniques
need to be developed to enable students to confront these preconceived
ideas as they construct their understanding of the topic.
A sampling of these challenges (undoubtedly just the tip of the
iceberg) to learning about global change is presented here.
Conceptualizing relevant processes: Making Visible the Invisible
The challenges of teaching some of the basic science content related
to global change are numerous. Many processes - those involving
atmospheric gases, for example - cannot be directly sensed by
humans. The study of fluids - atmosphere and ocean - is filled
with difficulties of visualization, counter-intuitive effects
and complex scale interactions. Transformation and redistribution
of energy also requires students to form abstract concepts and
to appreciate the global scale along with the more accessible
local effects. Tools which enable students to better visualize
or to gain direct experience with these phenomena would increase
their ability to understand the system.
Importance of scales
Global change is often represented by one number, the change in
global average surface temperature. This is misleading, as this
number may not be representative of any single place on earth,
nor may it be the most significant parameter to characterize specific
local effects. Change is manifested on the local scale; local
change is evaluated for the globe and over time. Forcing is also
scale-dependent. Students can benefit from exploration of the
connection between local and global scale, from evaluating the
interactions between processes at different spatial and temporal
scales, and by examining model predictions at various scales.
Natural variability, human impact and global change
The earth system is dynamic and has experienced dramatic change
over time. In the absence of human impact, the climate has both
warmed and cooled, species have appeared and become extinct and
land cover has changed. Human activity currently has a significant
impact on the earth system. How does it compare with known natural
variability? In the past, and in the future? (Paleontologist Peter
Ward says that the disappearance of the greatest number of species
from earth has resulted from two types of catastrophic events:
comet bombardment and the dawn of man).
Understanding the natural variability of the system, both on the
long and short time scales, leads to greater understanding of
the effects of human activity on the earth. Assessment of long-term
global trends is not a simple extrapolation of short-term changes.
Students will benefit from exploration of the statistical relationships
between long- and short-term trends, natural variability, and
effects of human impact on that variability. These phenomena can
be introduced simply in the K-12 curriculum.
Predicting global change
Numerical models are used to explore the earth system, examine
interdependent processes and scale-dependencies within the system,
and make predictions about future states of the system. These
computer models, from the simple to the complex, are useful in
enabling students to build successful mental models of the earth
system and its component processes. Students who have access to
model-building tools learn about the earth system while acquiring
experience with the advantages and limitations of using computer-based
(or VR) models to represent real world processes. Model building
enables understanding of the dominant processes in the system,
and the ways in which those processes interact to affect the outcome.
Using a ready-made model can also help to build understanding
of system response to various levels of forcing.
It is important that students learn about model limitations -
that computer models reflect at best the "state of the art"
knowledge of the phenomena represented in the model; that the
accuracy of model predictions is limited by the level of sophistication
of the model processes and by the data used to initialize it;
that continuous variables are represented by values at discrete
points; etc.
Common misconceptions
Selective absorption of radiation by gases in the atmosphere,
or the greenhouse effect, is a major concept that involves several
common misconceptions. Complex interdependence of oceanic and
atmospheric circulations, and land cover feedback onto earth's
radiation budget are also fundamental concepts that students typically
struggle to grasp. (We need to be aware that a new visualization
or immersion tool - VR - may also introduce new misconceptions).
Since many students will enter the classroom with ideas gained
from popular media or discussions with parents and others who
have been exposed to these information sources, it is important
to address some of the major misconceptions prevalent in this
domain. One of the most widespread problems is the false association
of unrelated phenomena - the confounding of the ozone hole and
the greenhouse effect, for example. Another problem is the assignment
of values to physical phenomena: the greenhouse effect and radiation
are "bad"; ozone is "good" (and this is doubly
complicated because of the various roles that ozone plays in the
atmosphere).
Media presentations are often based on simple answers or analyses
of processes that are by nature complex. It is best to confront
students with this in a straightforward way, and provide tools
that allow them to explore the complexities of the system and
assess the probability of the existence of simple answers.
Global vs. local/personal perspectives
Global change curricula must address at some level the issue of
personal responsibility and action. Students who are engaged in
earnest reflection on this issue may become truly distressed by
it. They will need to focus on solutions that they can relate
to. A model building tool that can be used to predict the success
of remedial action would be a powerful way for students to visualize
the potential effects of their impact projected onto global scale
phenomena.
VI. Learning and VR
General Principle
In this section, we report what is known and speculate about what
might be the case when VR is used to help students learn. From
the previous sections on learning and on the attributes of VR,
we may derive a general principle that VR improves learning, when
it does, by providing learners with new, direct experiences of
phenomena they could not have experienced before either in direct
interaction with the real world or using other technologies. It
is when these affordances of VR are most likely to help students
that we expect VR to provide the greatest "added value"
over traditional learning systems.
Other Principles
Beyond this general principle, we propose the following additional
principles which are either axiomatic or for which their is either
reasonable theoretical or empirical reasons for believing:
Learners can easily and without effort visit places and view objects from different points of view.
This is pretty much the same idea that has persisted ever since
media have been used by educators. Film, audio and pictures can
provide students with vicarious experiences of places they would
not other wise be able to visit. Granted, these experiences are
constrained by the technology used to provide them. Yet VR, it
is believed, will allow a greater variety and eventually more
realistic experiences of this kind. Also, given the attributes
of VR that concern its great flexibility to vary scale, interaction,
points of view, movement through virtual space and so on, VR can
promote learning by letting students examine properties and behaviors
of VEs in many different ways, from outside or within, at large
or small scales, from an egocentric or exocentric point of view
and under varying degrees of student control. This power and flexibility
is something new for learning that VR brings.
VR is ideal for letting students explore things, and therefore potentially very effective for student-directed knowledge construction.
The most effective way to use VEs for learning is to let students
explore them rather than telling them exactly what to do. Students
should be given goals to attain, such as, "Find out what
happens over the next two centuries if automobile emissions are
cut one percent annually for the next twenty years." However,
for the unique attributes of VEs, described earlier, to be used
to their best advantage, how students actually do this should
not be specified beyond telling them what tools they have at their
disposal. In this way, VEs can support all of the well-documented
advantages of constructivist approaches to learning.
Any data, however abstract their referents, can serve to create surrogate objects and places with which learners may interact directly in authentic and quasi-authentic ways.
Since VEs are created entirely from data, anything that can be
described as data and placed in a database in a computer can be
used to create a VE. It is difficult to imagine any object or
phenomenon that cannot be described in this way. This means that
a VE can contain objects that represent anything in the known
universe and can exhibit any imaginable behavior. These objects
can be manipulated and observed in exactly the same ways as objects
that represent concrete and more familiar things from the real
world. The trick, of course, is to represent abstract concepts
and principles using metaphors that afford natural properties
and means of action.
In time, it is likely that some of these affordances will become
conventional. But for now they are not. We need to study and develop
a repertoire of metaphors for building VEs that represent the
hitherto unrepresented. The same is true of the ways in which
students are to interact with VEs. Just how does one "pick
up" a neutron and place it in an atom? How does one directly
manipulate the amount of CO2 in the atmosphere? While our goal
is to enable students to understand and experience what environmental
scientists do, the new powers that VR provides to learners requires
that new modes of interaction be developed that nonetheless conform
to what it is reasonable to assume scientists would do if they
had the tools that VEs provide.
Exposure to VEs will enable the development of mental skills
needed to learn from them. There is evidence, particularly
from research on the impact of television, that exposure to a
medium develops within the student the mental skills needed to
understand and act on the messages the medium communicates. The
symbol systems of the technology become internalized and can then
serve as cognitive and perceptual tools for thinking about the
information the medium brings. It is reasonable to expect that
experience in VEs will likewise enable students to develop the
perceptual and cognitive skills required for learning within them.
Because VEs are multimodal, skills required for learning in any
modality supported by the VE can develop. These will certainly
include visual and auditory spatial skills. But they will also
doubtless include particular skills for working in VEs which have
not yet been identified. Navigation skills and particularly situation
awareness are likely to be inmportant. Other entirely new abilities
may develop in users of VEs.
Interfaces are becoming increasingly intuitive.
There are two sides to this issue. One is a matter of design and
has nothing directly to do with learning. As our knowledge of
VR improves and our technologies get better, we become capable
of building better interfaces to VEs that allow input using natural
behaviors. Pressing buttons on wands and making arcane hand gestures
when wearing gloves can be replaced by spoken commands and more
natural gestures. On the learning side, we are seeing a complementary
development. With some practice users can learn how to use awkward
interfaces. For example children who have used our VE interfaces
find using wands and glove gestures to be intuitive.
The learner can experiment by manipulating variables that cannot be manipulated in the real world, like gravity, and observe the results as if they were the consequences of that manipulation in the real world.
The general principle stated at the beginning of this section
is especially important for learning complex concepts and principles
of science. In a well-designed VE, students may directly manipulate
objects and, through appropriate metaphors, processes in the natural
world. Their observation of the results which may lead to the
construction and testing of hypotheses and let students conduct
significant experiments that they could not conduct in the real
world. In this way, they are also learning to think inductively
and to act like scientists.
VEs can teach complex topics with less need to simplify them.
One clear advantage of VEs is that the simulations that lie behind
them can be as complex as possible. But, because the student never
sees into the simulation itself but only works with a virtual
interface to it, that complexity need never confuse students.
There is a tendency among teachers and instructional designers
to simplify content in order to make it easier to teach. This
leads to the development of a number of the misconceptions about
the environment mentioned in the previous section. The key to
understanding global change is that it is indeed complex and somewhat
unpredictable. To simplify it is therefore to seriously misrepresent
its basic assumptions and principles. In a VE, students can interact
with and observe the causes and effects of global change in all
their complexity.
VR is engaging and seductive.
As we saw in a previous section, one attribute of VEs is that
they gain and hold students' attention. This assists their learning
in two ways. First, it motivates them to learn. Students enjoy
working in VEs and will continue to do so if they are given the
opportunity. Second, the nature of VEs is to keep attention focused
on the matter at hand. You cannot look away. Therefore students'
attention cannot wander to other distractions at times when they
are engaged in learning.
VR will succeed only as far as the feedback (reaction to learner actions) it provides succeeds in guiding students' knowledge construction.
This principle is a direct consequence of the requirement that
VEs react logically to user input. The consequences of a student
action must make sense within the conceptual model that drives
the simulation and within the mental model that the student in
constructing. When students make mistakes as the result of misconceptions
in a faulty mental model, the feedback from the VE may vary just
as it does in any other learning situation. The VE may in fact
ignore errors, in which case a "bug" now exists in the
student's mental model that will be revealed later. The student
then has to backtrack to find and correct the earlier mistake.
The VE may say the student has committed an error without saying
what the error was. Or the VE may correct the error for the student.
In each case, the feedback from the VE provides an opportunity
for the student to learn.
VEs will succeed only as far as representations and interactions are effectively designed and fit together.
Nice-looking worlds or highly interactive ones will not work on
their own. An effective VE therefore must have high levels of
both presence and interaction (and autonomy when simulations are
time-critical).
VII. Learning, VR and Global Change
There are three parts to this section. In the first, we identify
what students need to achieve cognitively in order to learn about
global change. In the second, we summarize those attributes of
VR that support learning. In the third section, we revisit the
five "challenges" for learning about global change,
outlined above, in order to illustrate how VR might be used to
teach about complex science material.
Summary: Cognitive Requirements for Learning about
Global Change
In order to learn complex content, as exemplified by global change,
students need to achieve the following skills and abilities:
Fluency with symbol systems
Students need to learn to recognize objects that represent concrete
objects in the real world. (This is not always as easy as it seems
in VEs given the current low level resolution of most HMDs.) They
must learn the metaphors used to represent abstract concepts and
principles regardless of the medium through which they are learning.
They must internalize the symbol system so that its interpretation
and use are automatic and do not intrude on learning content.
Discrimination ability
Students must learn the fine discriminations of color, size and
shape that characterize changes in the perceptible environment
as the result of changes in global warming.
Concept recognition
Students must learn to correctly classify the phenomena they observe.
Application of principles
Students must learn, through guided induction, the processes that
operate in the environment that reflect global change.
Accurate mental models
As a consequence, students should develop valid mental models
that contain the concepts and principles of global change. These
models may be idiosyncratic provided they remain within the bounds
of "correct science".
Transfer
Students' mental models should be contextualized so that they
can be applied in settings other than the VE in which they were
acquired. These settings may include the purely academic, when,
for example, students are required to take exams or write essays
about global change. They may also include the real world, where
students find themselves having to explain to others about global
change, to the purely practical, where the behavior of students
vis-á-vis the environment is appropriate and thoughtful,
and where their criticism and direction of others' behavior is
likewise reflective of what they have learned.
Motivation
Students need to be motivated to learn about global change and
motivated to apply what they have learned.
Retention
Students must remember what they have learned.
Problem solving skill
Students must develop the ability to define the problems they
need to solve, to gather data about global change from whatever
environment they are working within, to interpret those data and
to draw inferences that lead them to testable hypotheses, to test
those hypotheses and determine their tenability.
Summary: Relevant attributes of VR
The attributes of VEs that are likely to foster the above cognitive
requirements may be summarized as follows:
Data representation
VEs can represent data about any imaginable phenomenon in any
imaginable format.
Social learning and collaboration
VEs are becoming multi-participant making it possible for discussion
and interaction to take place inside the VE.
Immediacy of feedback
The consequences of student actions are immediately apparent.
Appropriateness of feedback
The feedback the student receives (none, knowledge of results,
correction of errors) can be adapted to fit the students' learning
styles and current mastery of content.
User control over VE itself
Students may interactively modify the nature and behavior of VEs.
They can even build their own VEs to embody their own understanding
of particular phenomena.
Nature of interactions
Interactions between the user and the VE may vary. They can be
direct, as when a student action directly impacts on some aspect
of the VE. They can be mediated by other users, as when one student
is delegated by a group to perform certain tasks within the VE.
They can be mediated by language, where a student communicates
indirectly with the VE. Or they can be mediated by an agent within
the VE who acts on behalf of the student.
Multisensory experience
VEs provide visual, auditory and to some extent haptic and tactile
experiences.
Constraints
There are of course constraints that need to be borne in mind
when making decisions about using VEs. A trade-off between these
and the added value of VEs is an important action to take. These
constraints include: Cost, difficulty of running immersive VEs
on currently available equipment, difficulty of maintaining hardware
and software, lack of curricular materials, and the high cost
of developing materials.
Using VR to Learn about Global Change
Five "challenges" to learning about global change were
presented in an earlier section on learning about global change.
These were a sample of the kinds of issues that face students
and teachers of this material. We return to these now and suggest
how VR might be used to help students meet each challenge. Again,
these suggestions are a small sample of potential ideas.
Conceptualization of relevant processes
Many of the processes at work within the phenomenon of global
change cannot be sensed by humans. Teachers face significant problems
about how to visualize these processes. Many of the cause-effect
relationships are counter-intuitive.
VEs, whether immersive or not, are particularly effective at visualizing
abstract processes. Once these processes are accessible to students'
senses, the necessary discriminations can be made, the concepts
assigned to their appropriate categories and the principles that
link them understood. The counter-intuitiveness of processes as
observed without understanding can be mastered by iterative observation,
hypothesis-building and testing hypotheses in a VE. All of the
potential of immersive VEs for developing this scientific problem-solving
skill and for applying it once developed come into play.
Importance of Scale
Global change operates locally, globally and at scales in between.
Many students have difficulty understanding how global change's
local manifestations, like declines in salmon stock, may have
global causes and impact far beyond the student's local environment.
Simulations that allow easy transition from local to global points
of view can make it easier for them to understand the relationships
among local and global phenomena. The ability of VEs to allow
students to instantaneously change points of view may be useful
here.
Natural Variability, Human Impact and Global Change
Global climate has changed over the earth's history before humans
had any impact on the environment. Today, there is evidence of
human impact. Students need to understand the extent to which
global change is a natural phenomenon and the extent to which
it is affected by human activity.
In a VE, students can make observations and measurements related
to global change at any point in history. They can travel forward
in time to see what the future holds with or without changing
the amount of pollution humans put into the atmosphere. With a
reasonably accurate database, they can also travel back in time
to observe and measure climatic fluctuations throughout earth's
history. They will observe radical changes in climate during the
ice ages and shifts in composition of the earth's atmosphere during
the planet's early evolution and at subsequent periods. From these
observations made in the future and in the past, they will be
able at least to reason about the relative contributions of natural
cycles and human impact to global change without necessarily "coming
up with the right answers".
Predicting Global Change
We have previously talked about students making predictions about
where global change will lead us and testing those predictions
in VEs. We now emphasize the difficulties environmental scientists
themselves have in making these predictions. The difficulty arises
from the complexity and incompleteness of the numerical models
scientists use for this purpose and the difficulty that non-specialists
have understanding them.
The issue of prediction from models is an important part of the
environmental education curriculum. Having students use modeling
software to build their own models from theory and data, to build
simulations that run the models and make predications, and to
build their own VEs that allow easier interaction with and observation
of the models' effects will help students develop an appreciation
of the difficulty of predicting global change.
Common Misconceptions
Students come to science classes with preconceptions about global
change. These are often misconceptions derived from at best oversimplified
and at worst incorrect accounts of the phenomena in the popular
media. Provided misconceptions make intuitive sense - they mostly
do --, then students' mental models for global change will be
robust and resistance to change even in the face of compelling
evidence.
Working in VEs allows students to develop ownership of what they
study. The evidence they derive from observation and problem-solving
in the VE is theirs, not evidence provided by a teacher, a textbook
or an expert. It is therefore more likely to be believed. The
ability to manipulate VEs in radical ways also allows students
to gather information in "what if" scenarios that are
beyond the power of scientists to explore in the real world. As
we saw above, VEs can still be understandable and useable even
when they do not oversimplify the content. It is therefore likely
that misconceptions about global change can be replaced by accurate
understanding by learning in VEs.
Personal Points of View
Finally, science education aims at students taking personal responsibility
for their actions on the environment. Students who understand
the critical problems facing our planet are sincerely distressed
by them. Using simulations so that students may see the future
consequences of their actions or of their inaction can help them
become more responsible citizens. Doing so within a compelling
virtual environment will likely heighten their motivation to act
responsibly.
VIII. Recommendations for Research
In this section, we provide a list of research questions, hypotheses
and related issues that arose from the two workshops. Since the
preceding sections of the report provide a framework and rationale
for these, and since they are a distillation from a significant
amount of debate, we present them without discussion.
We assumed that a basic goal of science education is to develop
students' appreciation of science in all its complexity and to
increase students' ability to use the scientific method. The following
research questions address this goal.
Research questions
A question whose answer must preempt the answers to all of the
following questions is: "What is wrong with the way students
learn about global change and science generally using the methods
and technologies we currently have?" If it turns out that
in answering this question we discover that we are doing perfectly
well now, then it will be difficult to justify investment in VR.
On the assumption that there is still room for improving science
education, we suggest the following research questions.
Can We Increase Motivation?
Does VR help increase students' motivation to learn in the short
and long term? Why or why not? Is VR measurably better than current
technologies and strategies for increasing motivation?
What is the relationship of engagement to motivation? How do VR
effects and the technical means to achieve them relate to engagement?
What is the combined effect? Does VR create a better level of
engagement than other methods? How do these apply to different
topics?
Can We Help Students Make Sense of Real World?
What aspects of the "real world" do we want students
to comprehend? Does VR facilitate the acquisition of knowledge
(facts, concepts, principles, etc.) better than current methods?
What is the measurable benefit of making the invisible visible,
the impossible possible and the abstract concrete?
Does VR address phenomena that are inherently 3D better than current
methods?
How closely do the models have to resemble reality and behave
according to students' expectations in order for learning to take
place?
Does VR help students to build trust in their observations and
reasoning and to overcome misconceptions better than current methods?
Does VR Help Students Build Mental Models?
How does VR facilitate the building of mental models? Is VR more
intuitive for developing understanding than other methods?
Does VR help integrate the sciences with other disciplines and
encourage collaboration better than current methods?
Does VR increase metacognitive ability?
Can VR Develop Ability to Synthesize, Abstract,
Predict and Transfer?
Does VR help students improve the use of their mental models to
synthesize their existing knowledge, abstract to similar situations,
predict outcomes and hence transfer their skills and knowledge
to a similar situation or domain?
Can VR Increase Retention Rate
Does VR help students retain skills and knowledge better than
current methods? Does VR provide improved methods for feedback?
More General Questions
What have we learned or can we learn from where VR has been successfully
used elsewhere (e.g. in training for pilots, surgeons, in battlefield
simulations; in providing entertainment; in doing 3D design)?
Given that the interface devices are still not very intuitive
and sometimes even get in the way, how do we design a research
program whose findings about learning are not likely to be confounded
with interface issues?
What are the key problems in K-12 science education? How does VR help address those issues?
What is the best instructional role for augmented reality? What
is the role of augmented reality in science education?
What content areas (or types of learning outcome) are best learned
with VR?
Can we introduce relatively complex science topics earlier into
the curriculum (at lower grade levels) if students learn about
them in VR?
What are the misconceptions that VR is best used for correcting?
What are new misconceptions that VR is likely to create? How do
we avoid creating them?
How does the level of immersion in a VE affect learning outcomes?
Do VEs best support part-to-whole or whole-to-part models of teaching?
What is the value of role-playing in VEs?
Will the affordance of anonymity and the use of character templates
in VEs make it possible for students to overcome stereotypes?
Can VR be used to embed educational content in an enticing form
within a narrative context?
When should VEs present concrete experiences and when abstractions?
Sample Testable Hypotheses (How we might answer the research
questions.)
VR leads to greater improvement on divided attention tasks than
single, forced attention tasks.
Students retain what they discover in VEs longer and better than
they do discoveries made in other ways.
Exposure to VR improves students' spatial reasoning and spatial
orientation abilities.
Students constructing and inhabiting 3D immersive VEs representing
global change acquire and retain greater understanding of the
system parameters and are able to synthesize and predict future
outcomes better than students who use the best non-immersive methods
for world building and viewing.
A learner who experiences multiple levels of immersion will learn
more.
A learner who experiences multiple levels of immersion will learn
more about global-local interactions.
A learner will trust experiences in VEs sufficiently to modify
misconceptions.
Interacting with other students in a VE will improve learning.
Student assessment of and performance in VR is positively related
to complexity, fantasy and degree of realism.
Manipulating the system is more effective for learning than not
manipulating the system.
Having a student participate actively in the behavior of a VE
will lead to greater understanding of certain concepts and principles
than having them observe the VE from within, which will in turn
be more effective than having them observe from without.
Students who learn science in VEs will have a more positive attitude
towards science and technology.
Independent Variables to Manipulate Across Technologies
and Treatments
Population type (based on age, gender, ethnicity, mental and physical
disabilities, etc.).
Levels of immersion and concomitant ability to manipulate space,
time, scale, frames of reference (ego- vs. exocentric).
Extent of use of spatialized sound, force feedback, etc.
Extent of the ability to communicate, role-play.
Cohesiveness and obviousness of story-line.
Relative emphasis on entertainment value.
Sense of ownership operationally defined as building vs. visiting
worlds.
Science expertise of the teacher.
Other issues affecting research and implementation
of technology in schools
Time to learn and use technology effectively in classroom, to
integrate curriculum to meet goals.
Cost to acquire, maintain, network and update systems.
Flexibility to meet different needs of teachers in various disciplines
and ability levels.
Obsolescence.
Dan Barstow
TERC
Janice DeCosmo
University of Washington, WA Sea Grant
Chris Dede
National Science Foundation
Daniel Edelson
Northwestern University
Richard Edgerton
Seattle School District
Tom Furness
University of Washington, HITL
Ken Galluppi
Educator
Bob Gotwals
Shodor Educational Foundation
Bill Hastie
Washington State University, Olympia
Kathleen Heidenreich
Local Science Educator
Jeff Hendricks
Stanwood Middle School
Hunter Hoffman
University of Washington, HITL
Earl Hunt
University of Washington, Psychology
Ronald Kantor
University of Houston, Clear Lake
Walter Keenan
NOAA
Bob Kozma
Stanford Research Institute
Lynn Liben
Penn State University
Chien Liang "Jonathan" Liu
Washington State University
Bowen Loftin
University of Houston
Miles Logsdon
University of Washington, Oceanography
Beverly Lynds
UCAR
Patricia Morse
National Science Foundation
Michael Moshell
University of Central Florida
Joan Piper
Museum of Flight
Howard Rose
Firsthand LLC
Nora Sabelli
National Science Foundation
Perry Samson
University of Michigan
Tim Schmidt
Stanwood Middle School
Jim Slotta
UC Berkeley
John Smith
University of Washington, Education
Mike Spranger
University of Washington, WA Sea Grant
Mark Stoermer
University of Washington, APL
Trav Stratton
Pacific Northwest National Labs
Bill Winn
University of Washington
Raul Zaritsky
National Center for Supercomputing Applications, U. of Illinois