| Publications Page | HITL Home |
Virtual reality (VR) refers to a new computational paradigm that fundamentally redefines the interface between humans and computers. Paradigm shifts change totally the way we think about a technology. What follows is a brief redefinition of the computer revolution.
MORE THAN REALITY
We describe innovations in terms of what they replace. Only after decades do we come to understand the pervasive impact of new technologies on our culture. The automobile was first the horseless carriage. It replaced the carriage, looked like a carriage, and moved at the speed of a horse. Decades later, the automobile has transformed our landscapes, the pace of our travels, and our concept of space. The television replaced the radio. Television programs were first radio programs with pictures. Decades later, the television has transformed our evenings, the pace of our senses, and our concepts of news and entertainment.
The computer is first a symbol processor. Although decades have barely passed, it is transforming our concepts of information and information processing. But the computer has yet to be understood for what it is of itself, we still view it from the impoverished model of what it replaces. McLuhan said that computers extend our central nervous system. But our CNS is not a symbol processor, it is a reality generator. The essence of the computer revolution is yet to come, computers are essentially generators of realities. Cyberspace, virtual reality, embodies the fundamental nature of computers, the creation of a diversity of realities. We have seen the shell, the narrowness of sequential computation, in the processing of one dimensional strings of symbols. Zeros and ones. We have seen the image, the flatness of pixel computation, in the desktop metaphor. Icons and mouse-clicks. Now we can prepare to see the meat, the sensory surround of situated computation, in cyberspace. Inclusion and unconstrained realities.
Symbolic realities are low bandwidth externalized worlds structured by syntax and semantics. Computers have evolved from a series of vacuum tubes that represented binary states to graphics generators that create photorealistic images. Virtual reality is the next step in this evolutionary path. The user is placed inside the image; the generated image is assigned properties which make it act as if real. The user becomes a participant within the computational space.
VIRTUAL REALITY
A virtual reality computer generates a direct experience computational environment. The participant wears hardware which senses his natural behavior, and displays from his personal perspective. The best way to think about the experience of VR is to look around the physical reality each of us inhabit. Physical reality surrounds us, we see it from a perspective. We do not see the back of our head. When we turn our head, the world holds still while we redirect our perception in the new direction. VR has the same inclusive quality. In physical reality, there are objects, localities of contiguous mass. Most objects, and most of what people see, are manufactured. In VR, everything is manufactured. VR has objects, but they do not necessarily have mass. VR objects are programmable, their properties can be arbitrarily changed. VR includes software for the construction of, maintenance of, and interaction with arbitrary databases with visual semantics.
Here is a list of the changes that define the paradigm shift accompanying VR technology:
symbol processing --> reality generation viewing a monitor --> wearing a computer symbolic --> experiential observer --> participant interface --> inclusion physical --> programmable visual --> multimodal metaphor --> virtuality Wearing a computer means that input devices are directly coupled to the natural physical actions of the participant. Like clothes, VR input devices become invisible. Our natural behavior, without intervening metaphor, achieves that results we wish to achieve by initiating action. Thus, VR calls for a total re-engineering of the user interface. In fact, interface as a concept becomes obsoleted. We do not interface with reality, we interface with particular objects. VR is an inclusion, an environment.
Generating applications for VR in education, curricula, or most any other human activity is easy and fun. Just substitute the virtual for the actual, then get rid of the constraints of the actual. What follows are five somewhat deeper educational issues posed by VR.
PROGRAMMABLE PARTICIPATION
The characteristics of VR are the same as those of good teaching. The teacher wants to create an environment which is programmable (curricula) and which the students participate (Dewey). "The most important principle of classroom activity design is that the student's actions determine what will be learned." (Walker, Fundamentals of Curriculum). That is, attention comes first, learning comes after attention is focused. And learning is primarily action.
Arbitrary Environments: With VR we can build arbitrary environments. Every aspect of the curriculum is enhanced.
History: visit a virtual reconstruction of ancient Greece. Not words, not pictures, but Athens as best we know it. Geology: peel back the strata of the Earth Chemistry: swim along with the molecules, enter into chemical reactions Ecology: become a raindrop and participate in the Earth's water cycle. ...
The idea is simple, everything we do to educate with words and with pictures can be provided as virtual experience. We can vary location, scale, density of information, interactivity and responsiveness, time, degree of participation.
Individualized Instruction: Programmable environments are personalized worlds. They are at the call of the participant, they can accommodate to prior actions or to specified preferences. The art of user-modeling has been weak because context has been omitted. VR provides a fully controllable, empirical context. Imagine that each VR object stores its history, activities and interactions with the student. Imagine that each object has access to statistical and classification algorithms. A teacher could ask for a synopsis of each students work in LogicLand; could toss tasks into a student's environment, each task calibrated to an appropriate skill level; could observe by inhabiting the task, by being the challenge.
Intelligent Training: Imagine assembling a carburetor composed of 15 intelligent virtual parts. As you try to put one piece into another, the piece could refuse, gently guiding your hand to the correct position. A part could squeak out if mishandled, could record the attempts to place it in a wrong position and offer diagnosis and immediate negative reinforcement.
The idea is not that all jobs become easy, rather it is that we have an idealized training environment . The effort is still in specifying and knowledge engineering the task. All easy, all clerical tasks become easy to teach. VR cannot show an Art student where to aesthetically place the brush stroke.
What-if Scenarios: Yes, we are in fantasy land, we can branch to whatever fantasies we specify. What if water flowed uphill? What if two and three were six? What if we cleared all the forests?
NATURAL SEMANTICS
VR input is coupled to natural behavior. The rule of thumb is that a child should be able to command the system. No command lines or mouse clicks, rather, simple walking and pointing and speaking and grasping.
VR makes sense when what a participant sees and hears has a meaning that does not require explanation. Consider a house. A textual description requires reading skills, a procedural database (lists of coordinates) requires decoding, a picture can be recognized immediately but is not interactive. A house in VR is most like a physical house, you can look at it while walking around it, you can walk in the front door and explore each room. A physical house has natural semantics, no one needs to explain it. Natural semantics is what a child learns before symbolic schooling.
Most sciences have natural semantics, most symbolic studies (the three Rs) do not. But, except for graduate school, almost all symbolic studies are an attempt to refer to natural (visceral) semantics. We read in order to build a picture of the world, we write in order to describe our world to others. Mathematics is a tool for solving measurement problems in the world. In the study of naive physics, folks are shown to have unrealistic (simplistic) models of behavior. Most of these studies actually measure a person's understanding of symbolic representations of physics. We should expect confusion. Put a kid on a baseball field and see if he ignores the concept of momentum.
Text does not fair well in VR, it is not constructed for interaction. The VR analog for text is natural speech. Mathematics can be transcribed into VR easily, either by the embodiment of problems in an experiential context, or by the representation of abstractions by concrete images.
Affective Education: One surprising result from VR research is that participants have a strong positive emotional reaction. They feel free, empowered, as if superman. VR is compelling. Kids have the same reaction when they are empowered in the classroom. Its not surprising that participation and emotion are related, consider school sports and other extracurricular activities. VR offers a path to the emotions, for this reason VR researchers are stepping very carefully.
Developmental Sensitivity: By tying a VR curriculum to pre-school skills of movement, we can avoid the Piagetian shock of premature symbolic abstraction. Why is there avoidance of mathematics?
CONSTRUCTIVISM
Virtual environments are not constrained to only viewing. The student can interact with objects and spaces in VR. The student can use tools to create new environments, to modify old ones, to take simulation exams, to fix errors, to play.
Rather than teaching a structure of symbols (such as algebra) and then teaching the meaning of that structure, in VR we will first teach meaning through experience, then (if necessary) teach the symbolic abstraction of our experiences. But the computer is an ideal tool for manipulating symbolic abstractions. Rather than teaching the abstraction, we may just teach how to use the VR tool, a natural interface with abstractions. VR is not a simulation of reality, it is a superset of reality, it is more than reality. this is easy to see from the programming perspective. To introduce gravity into VR, we introduce a property (mass) and then constrain (limit) the objects to a particular relationship between their masses. To introduce solidity, we constrain boundaries so that the insides of two objects do not occupy the same space. Simulation of physical reality in VR is always an act of decreasing its flexibility. One of the joys of VR is that it permits us the freedom to escape the bounds of the physical.
VR teaches active construction of the environment. Data is not an abstract list of numerals, data is what we perceive in our environment. Learning is not an abstract list of textbook words, it is what we do in our environment. The hidden curriculum of VR is: make your world and take care of it. Try experiments, safely. Experience consequences, then choose from knowledge.
COGNITIVE PRESENCE
In VR, each object can be inhabited as a virtual body. Students are not merely coparticipants, bringing their perspective within the same context of an object. Rather students can become the object, see and act in the virtual world as if the object. How do we approach a technology that explicitly permits modification of body image, of self-image. What is it like to swap eyes with someone? Is empathy learnable? How do students structure realities and their presence within? How do cognitive models become manifest? What is it like to have expanded powers? Can we integrate three or thirty concurrent eyes? Are there any questions that a new reality leaves unasked?
MULTIPLE PARTICIPANTS
Foremost, VR is for multiple participants. It gets boring pretty fast when you interact only with algorithms. So take the choices offered by one VR and fill it with the entire school. Since each participant is within her own computational environment, the assumption of a shared communal environment is not necessary. Rather than assuming that we all are included in the same environment (a dictate of physical reality), assume that we are each in a unique environment. (Studies of situated action conclude that each person has a unique world view. No seventh grade algebra student has the same erroneous model of how algebra works, nor do erroneous models stay consistent over time for a single student.)
Uniqueness in VR means that each participant has a unique world. All worlds are conceptually, but not visually, superimposed. Communality is negotiated between participants, they agree to maintain common images. Differences can be maintained through the use of imaginary variables. The philosophical implications are substantial: we collapse the virtual potentia by mutual observation/measurement, otherwise we maintain a macro-quantum physics.
NEGATIVES
Avoid the hype, wait ten years for the first prototype systems to reach the schools. Wait another ten years for common usage.
No one has any idea what extended exposure to high-quality VR is like.
We have in the past mishandled technological power.
We really don't know much about minds and mental processes and image addiction and void spaces and ....
Our intellectual community is not really prepared for the study of comparative realities, although the physicists have been wrestling with mathematical models of multiple concurrent worlds. Linguistics is also familiar with multiple concurrent interpretations.
VR is coming, make of it what you may.