Virtual Reality: Directions of Growth
Notes from the SIGGRAPH '90 Panel
by William Bricken
9/10/90
I. INTRODUCTION
Virtual reality (VR) systems were introduced to the general public by
VPL and by Autodesk on June 6, 1989, VR Day, at two trade shows.
This event was preceded by about four months of media coverage.
Since then, VR has captured the public's imagination. It is also in
the unique position of being commercially available before being
academically understood.
Any technology which has the audacity to call itself a variety of
reality must also propose a paradigm shift. In essence, a paradigm
shift expands the potential of an entire discipline. For me, VR has
expanded every aspect of Computer Science, and is providing a base
for a very satisfying philosophy as well.
And just what is the paradigm shift? Computers are not only symbol
processors, they are reality generators. Until recently, computers
have generated only one dimensional symbolic strings. Text and
numbers. Text is a code which, when read, generates images of
reality in our minds. During the 80s, we enhanced the expressability
of computation by adding space and time dimensions to the realities
being generated. Two dimensional windows, 2D animation, solid
modeling, simulation. Now, in the 1990s, computer systems can
generate virtual environments, entire multisensory worlds which
include us as interactive participants. Digital information can seem
as-if-real, changing our notions of computation, symbolism, meaning,
metaphysics, self, and culture. Virtual realities are more than
real.
The potential for VR to contribute to societal infrastructures such
as manufacturing, marketing, telecommunications, science,
entertainment, art, education, medicine, and media, suggests an
economic impact that rivals the Gross National Product. We live in
two superimposed worlds, the one of mass and the one of information.
The huge accumulation of difficult to access words on paper indicates
that the world of mass is not particularly well-suited for dealing
with information. As our culture matures into an information
society, we are now discovering the virtual world, an ideal place for
interacting with information.
What follows is a wide ranging discussion of interesting growth areas
for VR. I'll define VR, point to some active areas of research, tell
you about virtual world tools, outline some things we have learned
from working in the field, and discuss some risks and philosophies
engendered by VR techniques.
II. THE RESEARCH SUITE
VR is the body of techniques that apply computation to the generation
of experientially valid realities. HITL is forming its research
agenda around a suite of three interrelated technologies:
Behavior Transducers
hardware interface devices
Inclusive Computation
software interaction techniques
Intentional Psychology
biological constraints and plasticity
Behavior transducers map natural behavior onto digital streams.
Natural behavior is what two year olds do: point, grab, issue single
word commands, look around, toddle around. Behavior transducing
interface devices include body trackers, voice recognizers, spatial
sensors, kinesthetic feedback devices, and subjective audio and video
displays. Transducers work in both directions, physical behavior to
digital information (the virtual body) and virtual display to
subjective experience (the physiological model).
Inclusive software provides tools for construction of, management of,
and interaction with digital environments which surround a
participant/user. The central design issue for VR is getting
behavior transducers and virtual environments to feel good to a
participant. The intentional psychology of VR will require a deep
knowledge of how we work, our physiology, our sensations, our
cognition. We must refocus the effort of interface from the needs of
symbol processors to the needs of people.
III. THE ESSENCE IS INCLUSION
We at HITL believe that the primary defining characteristic of VR is
inclusion, being surrounded by an environment. VR places the
participant inside information. Some of the changes in perspective:
picture --> place
observe --> experience
use --> participate
interface --> inhabit
When we extend our field of view onto a computational environment
beyond about 60 degrees, a remarkable phenomenon occurs. We shift
from a feeling of viewing a picture to a feeling of being in a place.
This shift is accompanied by an emotional response. It seems as
though the unification of our symbolic processes with our visual
processes creates a feeling of wholeness, of empowerment. We shift
from external users (exercising rights) to internal participants
(exercising responsibilities), from being observers to having
experiences, from interfacing with a display to inhabiting an
environment.
My colleague Meredith Bricken and I have collected videotaped
behavior and exit interviews from over 500 people that we have guided
through initial VR experiences. We have seen overwhelmingly
positive responses, eagerness to return to "that place", willing
suspension of disbelief.
IV. VIRTUAL WORLD PROJECTS AT HITL
Our knowledge about VR and about how people respond to the VR
experience is being extended at HITL through several active projects:
information database, sci.virtual-worlds
simulation laboratory
virtual environment operating shell
laser microscanner display techniques
design and construction of worlds
3D audio display
instrument display prototypes
multiple participant worlds
educational experiences and environments
virtual prostheses
The information database is a project for NASA to follow the
development of VR and to serve as a clearinghouse for references and
research in the field. Sci.virtual-worlds is a moderated USENET
newsgroup for the discussion of VR issues.
The simulation laboratory provides a research environment for
prototyping VR hardware and for testing and evaluating effects on
human sensory, perceptual and psychomotor behavior.
The Virtual Environment Operating Shell is a software suite currently
written in C that wraps around the UNIX operating system. VEOS
provides resource and communication management for coordination of
the modules which make a VR system:
i/o hardware, behavior transducing input and display devices
world construction kits, CAD packages
dynamic simulation kits, for interaction and animation
virtual world tools
computational and display processors
The laser microscanner is a hardware research project to design a
high performance, low cost virtual display. Rather than creating an
aerial image using cathode-ray tubes or matrix element devices, the
laser microscanner scans a color image directly onto the retina. We
don't think in terms of addressing pixels, we think in terms of
addressing rods and cones directly. The head-mounted unit will
integrate 3D visual and audio display, voice recognition, and head
and eye tracking.
We build worlds for presentation, evaluation, and experimentation.
Our interest is the design of comfortable, functional worlds.
For Boeing, we are exploring 3D audio display techniques, and
building prototypes for design and display of complex instrument
panels and machines, in essence simulating the design of aircraft
cockpits.
We are working on the implementation of multiple participant worlds
for an application to telecommunications. You can think of VR as a
very sophisticated replacement for the telephone.
Education and industrial training are natural applications of VR
techniques. We are designing virtual environments conducive to
learning, we're studying the transfer of skills between virtual and
actual tasks, and we're exploring the implications of VR for
educational theory and practice.
And we have great interest in the application of VR to prostheses for
the handicapped, for providing virtual bodies which extend individual
capacities, for providing alternative control devices for
interaction in virtual worlds.
V. OTHER RESEARCH AREAS
VR has intersected other areas of research in some surprising ways:
audio modeling
teleoperation, telepresence
image integration, HDTV
interactive drama
military simulation
3D audio hardware is commercially available, we should expect to hear
of inclusive sound systems in the stores soon. Audio theorists are
interested in specification languages for 3D music, in audio lenses
and icons (earcons), and in modeling ambience, the analog of
ray-tracing for sound.
Telepresence, the development of remotely controlled robots, requires
the same interface techniques as VR. The primary difference between
these disciplines is that teleoperation looks at interaction with
real (usually inaccessable) images, VR looks at virtual images.
Both want inclusive, interactive environments. The possibility of
inhabiting real worlds shook me out of a self-imposed computer
graphics narrowness. We can apply VR interaction and hardware
techniques to explore anywhere we can place a probe. We can inhabit
a remote undersea vehicle, processing digitized images into worlds
that mix the actual with the virtual. We can swallow a miniaturized
transmitter and explore our own stomach. We can build artificial
bees with fiber optic visual links and micromotors for dancing and
for rubbing antennae. We can then put our virtual bee-selves into
the physical hive and interact with real bees in their home
environment. I can hardly wait.
The multimedia community is very interested in digital images. It
seems only natural that we should port these flatlander tools into
VR. We could tile polygons with TV. More importantly, automated
conversion of images to 3D objects (the image recognition problem)
would permit a seamless integration of video-real with
graphic-virtual.
Hypertext has raised the question of interactive fiction. The
theatrical community is working to install plot and character into
virtual worlds, creating interactive drama. What do a good story and
a good experience have in common? Can we construct participatory
plots, guided experiences, autonomous characters?
Actually, VR grew up in the military. The first substantive
application of VR was to help Air Force pilots improve their ability
to aim missiles. The most refined and widely distributed VR
environment today is SIMNET, a large scale, simulated tank combat
system. Recently, I saw a paper on training close combat fighters in
VR. Sort of reminds me of the video arcade.
VI. VIRTUAL WORLD TOOLS
To give you an idea of what work and play will be like in VR, I'll
describe some of the tools we're designing at HITL:
the wand
the virtual body
virtual home, virtual community
concurrent inconsistent worlds
autonomous entities
concrete mathematics, experiential programming
The Wand is an evolution of the Mouse. It is a simple physical
device with a wide diversity of uses, ideal characteristics for a
tool. Physically, the wand is a spatial position and orientation
sensor on a handheld stick. In software, the Wand emanates a ray
which can be used for pointing at virtual objects. Coupled with
voice commands, the Wand can be used to identify objects, to attach
to and move objects, to bring things closer or place them at a
distance, to indicate a direction for flying, to identify a location
to teleport to, to measure distance, as a pen for drawing, as a
knife, as a switch, as a spotlight. Lots of functionality from a
little hardware.
People achieve presence in VR by inhabiting a virtual body. The
virtual body is a software toolkit for associating an arbitrary suite
of behavior transducers (such as wands, voice command systems,
headtracking, etc.) to a display of self in a virtual world. What we
do physically is sensed and converted to virtual behavior. Don't
think that the virtual body is necessarily in the shape of our
physical body; any object in VR can be inhabited. If you are
controlling a physical robot, you may prefer your virtual body to be
the shape of that robot. If you are navigating a data terrain, you
may prefer to have a virtual body shaped like a jeep or an airplane.
The virtual body can filter and map physical behavior onto superhuman
capacities. One of the first things we did to figure out how a
virtual body might be used was to search the old comic books for
super powers.
The virtual home is an environment designed for personalized comfort,
for work and for play. My virtual home will have a cozy chair, a
fireplace, some cats, and a cabinet full of virtual tools and toys,
essentially what I now have at (physical) home. Physical reality is
a great starting model for virtual reality. Take what we like and
delete what we don't.
Virtual homes will be customized, personalized environments. The
virtual home extends to a virtual community. People we work with are
not organized by some cryptic email address that is basically a
program to tell the network where to find them. They are organized
in close proximity in space. In a virtual community, friends have
virtual homes that are visible from our own virtual home. They are
our neighbors. We visit them by pointing to their home and saying
"jack me there". Less frequent acquaintances may be down the road or
over the hill. The idea is to organize virtual space to accommodate
to human culture.
One profound capability in VR is to maintain inconsistent views for
different participants, to intermix personal realities. In physical
reality, mass has a way of being unarguable. We quickly default to
assuming a consistent, objective reality that is communal to
everyone. Consistency is an assumption and is widely
overgeneralized. Each person in physical reality, for example, has a
viewpoint, each viewpoint is necessarily in a different physical
place, each perspective provides different information about the
inclusive environment. Every experience is unique. We agree to
suppress our differences for massive objects, but the line is always
fuzzy. We certainly tolerate differences within the domain of
conversation. How we talk is an excellent example of concurrent
inconsistent worlds.
In VR, communality can be negotiated rather than assumed. In VR,
the color of my shirt can appear to be green to me, but blue to you.
So long as we do not talk about or interact with the color of the
shirt, how it is rendered to each of us is irrelevant. Carry this a
bit further: I can be sitting in my virtual home next to an empty
chair. You jack a duplicate of your virtual body into that chair.
From my perspective, you are visiting me. Now, from your
perspective, you are still sitting in your virtual home, in your
customized environment. You have an empty chair, and I jack a
duplicate of my virtual body into it. We are now sitting in two
totally different environments while sharing a mutual conversation.
For me, you are in my home, for you, I am in your home. So long as
the inconsistencies in our environments are not items of contention
or confusion, the differences will not interfer with communication.
When they do interfer, the explicit differences become subject to
negotiated resolution.
But the pluralism of VR is much deeper. It is possible to maintain
inconsistencies directly, without resolution, using a mathematical
technique called the imaginary boolean value. We could choose to
represent the color of my shirt as ambiguous, as context dependent.
Both green and blue. We can then discuss the color of the shirt as
being inconsistent, as information about which we simply do not see
eye to eye. I bring up these ideas from an esoteric branch of
representation theory to illustrate a fundamental point. VR is not
bounded by the assumptions of physical reality. We can have whatever
we can formally specify.
The HITL architecture specifies that every object in VR, including
space itself, have processing and memory resources. Entities are
objects with the capabilities of operating systems. Every entity is
a system, every entity is a variant of the same system. This means
that we can use the same editing, debugging, and interaction tools
for modifying each entity. Entities are running a sense-process-act
loop; in artificial intelligence terms, each entity is an agent, an
actor. This means that VR is inhabited with artificial life. Every
entity is capable of independent action, in response to
environmental changes, in response to internal memory or process
changes, or in response to changes in the rules, the disposition,
specifying that entity's internal processes. Each entity is an
expert system using pattern-matching on its input to trigger
disposition rules and metarules which generate outputs to the
context. The environment itself is just another entity, one that
includes other entities within it. All cyberspace is Toontown.
We have been able to demonstrate that mathematics itself (in
particular logic, integers, and sets) can be expressed concretely,
using 3D arrangements of physical things, such as blocks on a table,
doors open or shut, rockwalls that respond to gravity, the things of
everyday life. String-based symbolic representations of mathematical
concepts are typographically convenient, but tokens are not at all
essential to mathematical expression. VR makes it convenient to
express abstract ideas using spatial configurations of familiar
objects. One benefit of this approach is that we can build visual
programs, set them on a virtual table, and watch them work. We can
experience programs as other entities rather than as dumps of text.
Bugs would manifest as structural anomalies, as visual
irregularities. Architectural design has a sensual, experiential
semantics. It is but a quirk of typography that we have ignored the
experiential semantics of computational languages. More
fundamentally, experiential computing unites our spatial and our
symbolic cognitive skills, permitting mathematical visualization,
analytic gestalt, whole brain processing.
VII. EDUCATIONAL APPROACHES
VR provides an exciting educational medium for exploring worlds and
for exploring ourselves. It provides a training environment that is
rich, replicable, and responsive. It permits direct evaluation of
educational theory. The central educational issue for VR is one of
transfer of experience. Do skills and habits learned in VR transfer
to the physical world? Here are some educational issues:
Constructivism
"Human knowledge is essentially active." Piaget
Natural Semantics
non-symbolic, preoperational interaction
Programmable Participation
conducive and responsive environments
Cognitive Presence
modifiable self-concepts, learning by becoming
Social Reality
unique concurrent worlds
Educational psychologists have long known that people actively
construct their experience of reality. In VR, students will
construct their knowledge, then dwell within it, exploring their
understanding.
Natural semantics means that the computational environment hides
symbolism in favor of displaying information in an innately
recognizable form. The two-year-old criterion: if a kid recognizes
it, its natural. The three Rs, all symbolic, will become the three
ACTs: enact, interact, and abstract.
VR provides the potential for completely customized, individualized
learning. Educational environments will uniquely respond to the
participant-learner, in terms of both needs and preferences. A
student model will not be necessary, instead the teacher and student
will modify the environment in support of student behavior.
We also have a tool for affective education, for sharing
perspectives, for mapping perspectives into broader contexts, for
changing self-image, for remapping capabilities.
Education is inherently social. Explicitly shared worlds and
multiple concurrent agreements provide the opportunity for groupwork,
social consensus, and the construction of functional,
multiparticipant environments.
In general, everything we do to educate with words and pictures can
be provided as virtual experience.
VIII. LESSONS LEARNED
Tom Furness, the Director of HITL, has over twenty years experience
in VR. He pioneered most of the hardware interface devices we use
today, in the extremely demanding environment of military aircraft.
Personally, I have worked on VR related projects for six years,
beginning at Atari Research Labs in 1984. Meredith Bricken designed
Autodesk's worlds, built Virtual Seattle for CHI'90, and has
pioneered research into the design of comfortable virtual
environments. Over the years we have learned some lessons:
Psychology is the Physics of VR.
Our body is our interface.
Knowledge is in experience.
Data is in the environment.
Scale and time are explorable dimensions.
One experience is worth a trillion bits.
Realism is not necessary.
A major theme of VR research is that Psychology, in the broad sense
of behavior, perception, cognition and intention, provides the rules
and the constraints of virtual worlds. Psychology is the Physics of
VR.
This may come as a shock, it is one of those truths that is obvious
after it is said, but elusive before it is stated explicitly. Our
body is our interface. Interface is not something that is out there,
in some machine. Interface is a boundary which both connects and
separates, interface takes place at the surface of our skin. From
the perspective of VR, interface is physiology, interaction is
natural behavior. We simply want to use the power of computers to
make computation invisible.
Knowledge is in experience, it is not in some abstract, symbolic
representation. Data is in the environment, it is not stored away in
some memory array. These observations serve to remind us that we are
not the computer. To understand computation, we should participate
within it, rather than writing programs to dominate it. Humans have
a great skill for projecting outward, for becoming the tool we are
handling. We need reminding that we are creatures who dwell inside
an environment.
VR is inherently multidimensional. As well as freedom of translation
and rotation, in VR we can travel in scale and in time. Think of
scale as simply another direction; when we traverse scale, size
instead of location changes. We can also travel through time using
any of the techniques of film editing, including slow-motion, fast
forward, and temporal discontinuity.
There is a tremendous compression ratio between digital information
and human experience. Very approximately, it takes a hundred million
polygons to simulate what we see in one scene. Add duration,
multisensory channels, and interaction, and you get a lot of digital
information being transacted with each moment of consciousness.
Computation will not come close to this bandwidth for a long time.
Fortunately, virtual world experience does not require the
information density of physical reality.
Because our minds provide such tremendous flexibility in interpreting
what is outside of us, realism in VR is simply not necessary. Our
cognitive plasticity permits even simple cartoon worlds of 500
polygons to be experientially satisfying. We must design worlds that
respect our physiological needs. For example, we conceptualize
perspective in physical space as having six degrees of freedom, three
in translation and three in rotation. But our bodies have roughly
four and one-twelth degrees of freedom. We move easily in all
directions on a plane, forward and to the side, but not up, off the
surface we stand on. We rotate freely around the vertical by
turning, but our natural rotation forward, around our waist by
leaning, is at best 270 degrees (3/4 of a full 360 degree rotation).
And our ability to bend side to side is only about 120 degrees,
one-third of a full rotation. This adds up to a little more than
four degrees of physiological freedom. Input devices which permit
complete freedom of translation and rotation usually get people lost
in space. The dimensionality of our abstract perspective does not
match that of our physical construction. We must also differentiate
that which is innate from that which is learned. Pilots, for
example, have learned to fly in all six degrees of freedom. Realism
is both physiological necessity and cognitive interpretation. In VR,
world design that conforms to physiological necessity frees our minds
to furnish the rest of our reality.
IX. COMING ATTRACTIONS
Here are the coming attractions, what I believe will be available by
the end of the decade:
public domain VR software
massive database access
fabric of space
negotiable group space
conversational programming
artificial life
crossvalidation of realities
HITL is electing to distribute its software in the public domain. We
hope to create a context for the growth of an industry and for the
understanding of alternative realities. We hope to encourage the
evolution of a shared software and hardware environment which will
permit researchers to share progress and results. The commercial
marketplace can then improve on public work, selling value-added
features like customer support, prebuilt worlds, faster hardware,
better algorithms, realer time.
VR requires a new approach to database management. We want to access
massive databases such as Landsat as a function of our perspective,
our location in the database. We expect to see interactive databases
which we can explore through movement. Already waiting is the entire
Earth to one meter resolution, the location of every aircraft and
ship, large hunks of the Moon, the human body down to the resolution
of a cell, the flow of the economy, the network of computation. We
have digital worlds to explore.
I have mentioned that space is an entity. Many interactions between
entities can be expressed as internal processes of the spaces which
include them. Gravity is a primary example; we can implement
simplistic local gravity by decrementing the Z component of the
velocity vector of each entity in a space at each time tick. Rules
that apply uniformly to every entity in a space instead can be
ascribed once to the space itself. The inclusive space enacts local
gravity by owning the locations of the entities it includes. We want
to be able to place fields in space, to have space maintain its local
version of continuity, gradient, and metric, to build space-filling
logics which branch as a function of location.
One advantage of customized environments is that we will have to be
explicit about what is shared. VR suggests an approach to
cooperative work in a computational environment: rather than assume
communality and specify differences, assume complete difference and
specify what is common. It may turn out to be fun to build communal,
consensual contexts, to negotiate the group space.
One consequence of autonomous entities is that they can respond to
our communications. With voice recognition, we will be able to speak
to virtual entities as a means of programming their structure and
behavior. "I want the green cube I'm looking at to double in size."
The cube has a sensor for voice. Its rulebased disposition matches
the vocal input to its own identity and to its size changing
function. If you have permission, it changes itself to your
specification.
Another consequence of autonomous entities is that they may have
their own agenda. The coupling of the behaviors of several entities
could determine events. Rulebases that support emergent behavior are
tremendously difficult to construct. We hope that the programmable
environment of VR will provide autonomous entities with a context for
the growth of interesting virtual life.
Fundamentally, VR forms a new reality, at least to the extent that we
are willing to relax our minds. We will need to calibrate the
effects of transfer across worlds and across realities. VR is the
first empirical tool of metaphysics, it permits us to compare
realities, to ask which alternative reality is preferable for which
tasks.
X. RISKS
Do virtual worlds pose significant risks? I have prepared a list of
what I believe are the issues and problems for VR:
descriptive confusion
lack of experience
cognitive remodeling
fluid self
sensory overload, sensory ecstasy
power and control
cultural adaptability
VR is seeking definition, it could be anything from email to a fully
surrounding, multi-sensory environment. We are struggling with
appropriate comparisons. VR is not a drug and is not physically
addictive. Drugs change our perspective from inside the body, VR
changes our external environment. VR may well be psychologically
addictive (that is, entertaining), just like all good media
experiences can be. And there is that constant tension between
physical responsibility and cognitive exploration. Is VR escapist?
Escapism means seeking diversion from physical reality. VR cannot
escape being escapism, VR is perfect escapism. Is VR theater, or
interactive drama, or is it more than art? Is it scientific
visualization, or physical simulation, or is it more than science?
Is it financial modeling, or the perfect sales tool, or is it more
than economics? It's a good idea to spend some time figuring out
what VR is.
To me, the greatest problem is that we have virtually no experience
in VR. There are perhaps around ten thousand VR non-virgins. But I
estimate that there are no more than fifty people who have spent
twenty hours in VR. All of this excitement is purely conceptually,
we have very little experience with what we are talking about. The
first item on the VR agenda must be to construct and distribute
hundreds of systems, so that many people can contribute to our
understanding. We should know at least something about the cognitive
effects of VR before it is a consumer item with the distribution of
Nintendo. When a representative of MCC asked the lab the best way to
invest two million research dollars in VR, the answer was clear: give
away forty $50,000 systems.
The most complex, and potentially dangerous, risk is what we are
calling cognitive remodeling. Those who spend a lot of time in VR
bring back to physical reality some strange habits, like navigating
across a room by pointing, like bumping into walls cause they aren't
just images, like dreaming in polygons. VR effects dreaming
strongly, it seems to provide tools for control of the dreamlife from
within the dream. VR changes mental models. Now, it is not
dangerous that this is happening, cause all intense work produces
similar effects. Anyone who has programmed all night will know that
the programming slips into dreams. The problem is not that these
things happen, it is that we don't have the faintest clue what is
going on. We do not know the borders between virtual and actual. We
have not yet had the opportunity to evaluate current theories of
reality crossing.
And how will we react when we are able to redefine our bodies, swap
our perspectives, mix our senses. We will have the ability to map
arbitrarily across sensory input, self-image, and behavioral output.
What will a fluid self be like? We will need to understand the
cognitive and behavioral effects of transportable perspectives, of
programmable bodies, of synesthetic sensations, of exchangable body
parts, of inhabiting arbitrary objects, of masslessness, of
negotiable communality, of complete empowerment.
Are there limits to the degree of warpage our senses can tolerate?
This is, of course, an empirical question. What are the functional
constraints of sensory modification for enhanced productivity, for
enhanced enjoyment? Are there sensory pathways to insanity or to
ecstasy? Just which side of the monitor do you stand on?
We have been discussing a domain which emphasizes personal freedom.
VR could be used for horrible purposes, but that negative assumes
that we are strapped to a chair. So long as each individual has the
freedom to reach up and turn off the experience, VR itself is quite
benign. But how will authority respond to this frontier? VR is
interactive, but will I have the right to remove the virtual arches
in my prebuilt reality given away with each hamburger? Are
advertisements from the creator necessarily non-interactive? Where
are the edges of property and ownership in a world which is digital?
Will there be commodities? What are the rights of autonomous
computational entities? Will there be stability? Will there be a
Virtual Environmental Protection Agency? I don't know, but I
certainly look forward to negotiating the communal rules of personal
responsibility in cyberspace.
The biggest issue is how our culture will respond to this new
reality. We have amassed hundreds of years of favoritism for the
objective, the scientific. Our values, ethics, and aesthetics are
predisposed toward Objectivism. Is VR a better place for transacting
information? How will physical reality react to competition? What
will socialization without material consequence be like? What kind
of intimacy will arise from explicitly penetrating world views? What
kind of cultures will arise when the VR network is standardized? Are
we like Columbus, discovering a completely new land in an unexpected
place? Is living in VR necessarily pathological? These are indeed
exciting times.
XI. EVOLVING PHILOSOPHIES
We have come very close to talking about philosophy, so here are some
comments on philosophical concepts:
situated semantics
pervasion
immaterial realism
constructivism
boundary mathematics
more than reality
Situated activity is a growing school of thought in AI. The idea is
simple: what we do depends on our environment as well as our
internal state. We react and respond, constantly bringing external
context into our interpretation of the moment. Currently, symbolic
logic is split in half, between syntax (representation) and
semantics (meaning). Syntax is strictly formal, it has no basis in
experience. Semantics attempts to connect syntactic symbols to the
reality of the world by mapping representation onto meaning. The
problem is that it does so without regard to context external to the
formal symbols. Since environments necessarily introduce external
unknowns, standard semantics is just too literal. Situated activity
is an attempt to build a theory of context.
VR, in comparison, is totally situated. By defining natural behavior
as the rules of interaction, by displaying recognizable spatial
structures as output, by providing context in toto, and by including
the participant, VR redefines the relation between syntax and
semantics. Semantics, what we consider to be anchored to reality, is
displayed directly as (virtual) reality. Syntax, the symbols that
guide computational activity, is hidden in the background, out of
sight.
Environments include their participants, they pervade their contents.
Pervasion is a non-dualistic concept, more familiar to Buddhism than
to Christianity. A pervasive space is one which is diffused
throughout every part of itself, including those parts occupied by
other spaces and objects. Objects themselves are those boundaries of
spaces that we can sense. When we look at the container of an
environment, from the outside, we see that it surrounds a portion of
space. The important point to understand is that environments focus
our attention on a particular portion of space, they do not separate
space into two opposite parts. The outside space still pervades the
inside space. For VR, the physical pervades the virtual. When we
enter a virtual world, we always bring our physical body. VR is not
a separate reality, in a dualistic sense, it is a pervaded reality.
The shift from duality to pervasion is from networks to maps, from
separation to unity, from confrontation to cooperation, from male to
female, from one to zero.
The Copernican revolution introduced a physics that differed
fundamentally from appearance. VR introduces a metaphysics that
differs fundamentally from the material. At the foundation of
Objectivism is an attempt to be realistic about the material world.
VR calls for immaterial realism, for being realistic about
information. The currency of VR is organization, not possession, not
accumulation, not territory. All laws are transmutable, we can
satisfy fantasy rather than fact. It is science itself that is
redefined. In VR, we can choose to be reductionalist, but at the
bottom of it all, there is not Mass or Nature, there is the Void.
VR is representational, but not a priori rational, empirical, or
verifiable. VR is illogical positivism: if you can specify it, it
is meaningful. All empirical hypotheses are true.
Another fundamental philosophical position engendered by VR is that
of constructivism, that our minds and our bodies coparticipate in
defining reality. Objectivism places an overemphasis on the input of
the physical body. Solipsism overemphasizes the mind.
Constructivism recognizes that reality is like light, it is both
particle and wave, both objective and cognitive, both observation and
participation. In VR, we cannot escape the realization that we are
the architects of our environment.
We are applying boundary mathematics to VR in three different ways:
as a foundational mathematics, as a technique for logical deduction
and maintenance of inconsistency, and as a spatial embodiment of
abstract concepts.
Boundary mathematics is a calculus of inclusion. The essence of VR
is inclusion, the relationship between an environment and a
participant. The primitives of boundary mathematics are also
participant and environment. Let ( ) represent an environment, and
let i represent a participant. A variation of Spencer Brown's Laws
of Form provides the axiomatic basis:
Observe: i ( ) = ( )
Participate: ( i ) =
The left-hand-side of each equation is descriptive (objective),
explicitly mentioning the participant. The right-hand-side is
experiential (participatory), implicitly using the participant's
perspective. We read the left-hand-side from our traditional
externalized, objective perspective. The right-hand-side refers to
our experience, from the subjective perspective. When we observe an
empty environment, we perceive its boundary. When we are included in
an otherwise empty environment, we perceive emptiness. That's all
there is at the foundations of experience.
The most important thing to realize about VR is that it is more than
reality, more than a simulation of reality. You add physical realism
to a virtual world by adding constraints that reduce the
possibilities in that world. Native VR lets you walk through walls,
we add collision detection to disallow this power. Native VR has no
gravity, we add gravitational equations to simulate a gravitational
reality. Reality simulation is a subset of potential VR experiences.
The least elaborated virtual world is the Void.
We describe innovations in terms of what they replace. Only after
decades do we come to understand the pervasive impact of new
technologies on our culture. The automobile was first the horseless
carriage. It replaced the carriage, looked like a carriage, and
moved at the speed of a horse. Decades later, the automobile has
transformed our landscapes, the pace of our travels, and our concepts
of time and space. The television replaced the radio. Television
programs were first radio programs with pictures. Decades later, the
television has transformed our evenings, the pace of our senses, and
our concepts of news and entertainment.
The computer is first a symbol processor. Although decades have
barely passed, it is transforming our concepts of information and
calculation. Computers are thought to replace typewriters and
desktops and filing cabinets. But the computer has yet to be
understood for what it is of itself, we still view it from the
impoverished model of what it replaces. McLuhan said that computers
extend our central nervous system. Our CNS is not a symbol
processor, it is a generator of personal realities. VR marks the end
of the infancy of computation, the essence of the computer revolution
is yet to come. Essentially computers are reality generators.
And reality is in the eye of the participant.