Facing interface issues; Special Report: Virtual Reality

Computer Graphics World
April, 1992


by Brill, Louis M.

   Currently, participants must rely on one or more of a variety of
tools to explore virtual environments. Some of these include visual
aids such as head-mount displays or goggles Binocular
Omni-Orientational Monitors (BOOMs), and direct eye-scanning devices;
body-movement and gesture-tracking devices such as sensor-laced
gloves, body suits, and TV or infrared trackers; mobility devices such
as stationary bikes, trackballs, flying mice, and treadmills; and
interaction via spoken commands.

   Of all these interfaces, the headmount display coupled with the
sensor-laced glove is the interface most strongly associated with VR.
Certainly, it is the interface that has received the most media hype
over the last few years. But while the head-mount and glove do provide
access to virtual environments, they are not without their pitfalls.

   "When you have to wear a headmount display, you incur ergonomic and
functional costs in terms of comfort and extended use," says Dr. Steve
Ellis, group leader for the NASA Ames Advanced Displays and Spatial
Perception research team (Mountain View, CA).  "Ultimately, anyone who
has to wear a helmet for a living learns to dislike them after a
while."

   Ben Delaney, publisher of CyberEdge Journal, an industry trade
newsletter based in Mill Valley, California, concurs, pointing out
that head-mount technology has far to go in terms of userfriendliness
and realism.

   "Currently, comfort is a big part of creating the ideal head-mount.
After wearing one for about 15 to 20 minutes, they literally become a
pain in the neck due to weight and generally poor ergonomics," he
says. "Furthermore, the cable [connecting the] computer to [the
display] restricts the user's movements to a limited circle.

   "Another problem is the lack of high-resolution imagery for sharp
detail or photo-realistic capabilities," Delaney continues. "We are at
least one to two years away from a head-mount that would even approach
the clarity of a decent monitor, much less a high-end workstation
display."

   Sensor-laced gloves have also had problems. With most current
technology, a participant's movements are not registered by the VR
system precisely as these movements are made; on the contrary, there
is a lag in time between a participant's movements and the
registration and viewing of those movements on the VR system.

   Vendors of head-mount displays acknowledge that the issues of
discomfort, poor resolution, and time lag are important concerns, and
they are working on addressing them in future product releases.  For
example, VPL Research (Redwood City, CA), a supplier of a variety of
different VR equipment, reports that newer models of its head-mount
displays weigh half as much as their predecessors.

   But some are quick to add that, in many cases, these issues are
real problems only in certain applications. According to Ann
LaskoHarvill, director of product design for VPL, helmet weight is not
an issue, in, for instance, VR architectural walkthroughs because
participants generally do not spend much time reviewing a building
interior.

   Time lag is also application-dependent, she says. For example, in
architectural walkthroughs, time lag isn't as much an issue as it
could be in other applications because the participant interacts very
little with the environment.

   "Time lag really becomes important when you're trying to [perform]
real-time operations such as telepresence, or use virtual controls, or
manipulate small, delicate objects such as surgical tools," she says.
"In these uses, we need techniques that don't [create] lag problems."

   But even as VPL and others are working hard to improve the
headmount/glove interface, other vendors are working equally hard on
alternative interface devices. For example, SimGraphics Engineering
Corp. (Pasadena, CA) has developed the Performance Animation System
(PAS), a "real-time performance animation cartoon," according to
company president Steve Tice, in which animated cartoon characters
projected onto a large viewing screen interact with members of a live
audience.

   This is how PAS works. Live actors positioned behind the projection
screen wear special face masks that record their eyebrow, cheek, head,
chin, and lip movements. The masks transform these movements through
digital encoders into signals, which drive the faces of the computer
characters; if an actor smiles, his cartoon character will smile as
well. To help them express the cartoon characters' physical movements,
the actors use flying mice, joysticks, and foot pedal controls.

   Behind the screen with the actors are TV monitors focused on the
audience, so the actors can see the audience members. If a person in
the audience acts a certain way toward a cartoon character, the actor
will see this in the monitor and be able to react to the person's
actions. This enables the cartoon characters to banter with the
audience and create live, spontaneous performances.
 
Other PAS Uses

   According to Tice, PAS is more than just an entertainment device.
It can also be used in such engineering applications as the study of
human presence in hostile environments. For instance, says Tice, "You
could create simulations of a space station construction crew.  Actors
could drive their computer animation counterparts to perform specific
tasks in a simulated environment and then analyze the results of that
interaction.

   "It could provide studies as to how tools respond in job
situations, how people work in teams, or how their spacesuits would
hold up in collusion situations where they might bang into each other
or parts of the station structure," he adds.

   As PAS becomes more popular as a simulated theater environment,
Tice plans to improve the system's interface connections so the actors
can manipulate their virtual counterparts unencumbered via a form of
facial and body animation that remotely maps an actor's face using TV
cameras and image/object recognition software. In this scenario, the
camera would focus on an actor, read his facial and body movements,
and transform those movements through digital encoders into
corresponding movements for the cartoon creatures.

   Although SimGraphics' "unencumbered VR interface" is only on the
drawing board at this point, the idea of being able to explore virtual
worlds unencumbered, without first having to don gogles, gloves, or
face masks, is clearly catching on.

   For instance, Fake Space Labs (Menlo Park, CA) has developed a VR
tool called the Binocular Omni-Orientational Monitor, or BOOM2, a
periscopic device with a high-resolution binocular viewer on one side
and a counter-balance on the other.  According to the company, BOOM2
users can see in any direction, walk around or through its range, and
encounter virtually any world view as easily as they would with a
standard headmount--except BOOM2 provides images with higher
resolution and faster tracking.

   BOOM2 operators connect physically with their virtual world via
button pads placed on the sides of the unit. Using the buttons,
participants can transport objects into and navigate through the
virtual world or, in specialized situations, use one hand to work the
BOOM2 while the other hand manages a glove or trackball device.

   According to Mark Bolas, president of Fake Space Labs and a former
researcher at NASA Ames Research Center, the BOOM2's appendage-like
stanchion may not be as sexy as its head-mount cousin, but when you're
in and out of virtual worlds all day long, immediate accessibility
becomes a big issue.

   "The BOOM2 is essentially an unencumbered device that allows you,
within seconds, to walk from a real-world situation and, by just
grasping its handle and peering through the binocular viewer, to enter
unobtrusively into a virtual world.

   "We're getting real strong interest from people who've spent at
least one year or more working with a head-mount and now want a more
comfortable viewing situation," he continues. "It seems that when
users are spending inordinate amounts of time with headmounts, it
takes them away from their primary intent of just getting their work
accomplished in a timely fashion."

   The BOOM2's main unit functions as a viewport to client-specific
virtual worlds. It is versatile enough that, in a modified form, it
can also function as a desktop workstation. Further along, Bolas
envisions a form of the BOOM2 as an entertainment device. In fact, a
space flight game has already been designed for it (it was shown in
prototype form at a VR event called Cyberthon in San Francisco 18
months ago), and according to the company, it is just a matter of time
before the BOOM2 finds a home in an arcade or theme park.

   The BOOM2 can also be applied as a remote observer in a
telepresence situation. In this scenario, the BOOM2 consists of the
viewing unit and an off-site TV camera joined to a mobile robotic
device.  The robot and camera can be inserted into a hostile
environment such as a fire or nuclear power plant, so they can be
inspected without harm to the operator.
 
BOOM Benefits

   Says Bolas of the device, "Surprisingly, we have found people are
more engaged in their virtual worlds through the BOOM2 than other
interface devices.  People seem to be more willing to suspend
disbelief when they use a BOOM than when they use a head-mount.  They
don't feel trapped as they might when they're under a headmount
because they can let go at any time. They're not encumbered by it and
they feel more in control.

   "The BOOM gives you a lot of additional advantages for free," he
adds. "On the operation side, its CRTs provide high resolution,
excellent gray-scale, and the capability to merge alphanumerics with
visual data. Another important aspect is motion display, as it
provides instantaneous tracking, which means there is no time lag when
the BOOM is moved through its virtual world. As important are its
convenience factors of instant accessibility and that it's easy to
share among users."

   Bolas' enthusiasm is paralleled by the BOOM's expanding customer
base. For instance, the US Army Corps of Engineers is using it for a
waterways visualization project. Meanwhile, SRI Research is using the
unit along with custom software to explore 3D virtual representations
of molecular models.

   The NASA Amqes Research Center is using the device in an even more
unique way.  At NASA Ames, scientist Creon Levit and his research
group have converted a typical wind tunnel environment into a virtual
laboratory for conducting aerodynamic flow studies of, among other
things, aircraft.

   In traditional wind tunnel applications, researchers place aircraft
of various sizes in large, aerodynamically sealed chambers which
simulate flow pressures incurred against the aircraft when they are
flying at different altitudes and during take-off or landing. Levit
and his colleagues used the BOOM to create a virtual wind tunnel that
allows them to test virtual aircraft under extreme flight and
atmospheric conditions, without having to construct any physical
models of the craft beforehand.

   "In our set-up, the researcher can enter this model environment as
a degrees ghost' and, without disturbing the model, act as an
influence in studying its resultant aerodynamic situation," he says.

   "The researcher can also expand his contact within the BOOM
environment by wearing a special data glove," he continues. "By
carefully positioning himself in the virtual space, he can walk around
the aircraft and, at appropriate moments, emit virtual degrees smoke'
into the environment. The smoke streams off the fingertips of the
glove as a dye emitter that swirls around the aircraft model . . .
and indicates vertices or turbulence the plane might react to from
whatever particular atmospheric simulation it encounters."

   One software firm that seems particularly convinced of VR's future
pole in the design process is Autodesk (Sausalito, CA), a leading
developer of computer-aided design software and a major investor in VR
technology. According to the company, many of its customers use its
Autocad design program in combination with VR tools to enable their
clients to "visit" proposed structures. Using appropriate VR software,
architects can transform blueprints of a structure into a virtual
structure complete with stairs to climb, rooms with doors that open
and close, hallways to walk through, center columns, and windows with
views.  Then, both architect and client don headgear and mount a
treadmill; strolling about, the architect can escort the client
through a computerized virtual rendition of the proposed structure. If
the client doesn't like something, it's much easier to change the
design with the cursor, than to change it with a jackhammer after the
building has been constructed.

   Thus far, most of the focus on VR technology has been to create a
realistic virtual environment. But one firm that has begun to merge
these virtual worlds with the real world is The Boeing Company
(Seattle). The result is something that Boeing researchers David
Mizell and Thomas Caudell call "augmented reality." They see it as an
application for conventional manufacturing processes.

   In the augmented reality scenario, users wear a Heads-Up,
seethrough display headset (HUD set-up), a sort of visor that,
combined with head-positioning sensors and workplace registration
systems, augments the visual field of the user. The visor superimposes
a virtual floating window display that acts as an electronic
associate, providing image projections of assembly instructions or
blueprints to guide workers during their manufacturing tasks.

   Thus, in a manufacturing situation, as a worker is preparing to
drill a metal structure he can, at the appropriate moment, call up a
floating text window through his visor. The visor displays these
virtual instructions and diagrams which register against the
fabricated piece and indicate to the worker the exact
three-dimensional location for a drill hole, accompanied by a drill
size and specific depth information.

   The visor could also be used in the assembly of a wire frame. As
the worker fits wire cables into the frame, he can study template
outlines which overlap the actual metal structure and designate how
and where cable sets are fitted onto a wire formboard.

   Virtual and real environments can be combined in other applications
as well.  For instance, wearing an augmented headset, a surgeon could
access appropriate, real-time medical imaging data, such as X-ray or
computer-aided tomography scan information, on his patient. This data
could then guide his efforts during the patient's operation.

   Reflection Technology (Waltham, MA) makes a product incorporating a
technology similar to that used in the HUD visor. Called the Private
Eye, the interface is a sort of miniature TV screen, that, when
mounted on a headset, can be dropped into viewing position as desired
so it doesn't block the user's field of view. The Private Eye can also
be worn on the wrist or around the neck.

   With each step vendors take to develop VR interfaces that resolve
current problems with weight, resolution, and time lag, they get
closer and closer to the ideal VR experience as envisioned by VR
pioneer Myron Krueger. Krueger, who has been designing what he calls
"artificial realities" for 22 years, feels participants should be able
to explore artificial environments without the encumberments of
headmounts, gloves, joysticks, or any other accouterments.
Ultimately, he says participants should be able to walk through and
perceive these virtual worlds using their natural senses.
 
An "Unencumbered" Approach

   Indeed, Krueger has sponsored the "unencumbered approach" his
entire career.  Back in 1971, while still a graduate student at the
University of Wisconsin, Krueger created an artificial environment,
called Psychic Space, which participants reacted to by simply using
their feet.

   "Here, the focus was on getting the body involved," he says. To do
this, Krueger designed a gallery floor with special sensors that
detected the participants' movements. As the participants moved about
the room, they discovered their movements could drive a number of
different scenarios displayed on a wall; for instance, they could use
their feet to play music and to draw.

   They could even use their feet to make their way through a virtual
maze. "As participants followed the maze, if they attempted to cross a
boundary illegally, the maze would fight back," explains Krueger. "The
boundary walls reacted by stretching elastically, or the maze itself
might shift about.  Thus, the participant was caught within a smart
maze where the degrees no cheating' rules were strictly enforced."

   In his pursuits of artificial realities, Krueger has created at
least 20 different variations based on four themes: perception of a
person via a sensory floor; perception of a person as a full-body
immersion via a television camera; use of interactive software
incorporating a data tablet or a mouse; and interactivity from a
desktop environment with hand gestures from seated participants.

   In all of these scenarios, Krueger has sought to achieve the ideal
VR experience in terms of unencumbered interactivity within a virtual
or artificial environment.  Though it may be another 100 years or so
before we even get close to achieving the ideal VR environment
characterized by Star Trek's Holodeck, there can be no doubt the
effort to create better and better VR interfaces is moving ahead in
earnest. As that effort continues, the human access to virtual worlds
proceeds to improve dramatically.
 
Is There a Doctor in the (Virtual) House?

   As VR researchers and participants push forward into the virtual
realm, some have begun to voice concern over just how safe virtual
reality is on a daily or extended basis. According to British
scientist Robert Stone, the side effects experienced by people who are
immersed in a virtual environment for a long period of time are, to a
large extent, unknown, but over time, participants have been known to
be affected by "simulation sickness."

   The symptoms of simulation sickness include nausea, visual fatigue,
and spatial diosrientation.  Its root cause stems from "cue conflict,"
which occurs when the body's senses receive mismatched cues between
real-time reactions and the resultant physical motions and visuals of
the participating simulation. This refers particularly to when
participants are reacting in real time to an artificial situation, but
the simulated results are time-delayed, thus creating confusion
between the participants' actions and what appears as the results
thereof.

   Simulation sickness may not affect someone participating in a five-
to 10-minute VR experience, but a pilot training for several hours in
a fully operative military or commercial simulator could easily
succumb to some form of simulation sickness.  Thus, simulation
sickness depends on the kinds of interfaces the participants are
using, as well as tracking delays and how fully immersed the
individual is within the simulated environment.

   While simulation sickness occurs mostly from highly immersive,
motion-oriented simulation platforms, it is inevitable that, as more
responsive virtual reality systems become available to the public,
particularly those with greater realism, this malady will begun to
disappear. Thus, comments Stone, the development of computer-to-human
interfaces should be a two-way street: As much as interfaces are
designed to optimize human immersion, the results of these immersion
situations should be studied so users exiting from the virtual realm
are ultimately as safe as those who park and exit from automobiles.
 
 Virtual Reality  Providers

   Following are some providers of resources used to build  virtual reality 
computer systems:
 Ascension Technology POB 527 Burlington, VT 75402 802-655-7879
Tracking devices 

CAE Electronics Ltd.  C.P. 1800
Saint-Laurent Quebec, H4L 4X4 Canada 514-341-6780 Head-mount displays 

Division Ltd.  Quarry Road Chipping Sodbury Bristol, BS17
6AX England 44-0454-324527 Specialized transputers used for VR applications

Exos 8 Blanchard Road Burlington, MA 01803
617-229-2075 Hand-worn interface devices 

Fake Space Labs 935 Hamilton Avenue Menlo Park, CA 94025 415-688-1940
Modified VR display controller that functions as stand-alone and desktop
viewing unit


Myron Krueger Artificial Realty 55 Edith Vernon, CT
06066 203-871-1375 Custom-designed virtual world environments

Logitech 805 Veterans Boulevard Redwood City, CA 94063
 415-365-9852 x,y,z mice (flying mice) 

Polhemus 1 Hercules Drive, POB 560 Colchester, VT
05446 802-655-3159 Tracking devices

Pop-Optix Labs 241 Crescent Street Waltham, MA 02154 617-647-1395
Specialized optics for head-mount displays 

Sense8 1001 Bridgeway POB 477 Sausalito, CA 94965 415-331-6318 
VR authoring systems

SimGraphics Engineering Corp.  1137 Huntington Drive South Pasadena, CA 91030
213-255-0900 Systems configuration house/ OEM VR equipment supplier

StereoGraphics 2171-H east Francisco Blvd.  San Rafael, CA 94901
415-459-4500 Stereoscopic displays 

Straylight 150 Mount Bethal Road Warren, NJ 07050
908-580-0086 VR authoring systems

Subjective Technologies 1106 Second St., Suite 103 Encinitas, CA 92024
619-942-0928 Various tools for controlling virtual environments 

TiNi Alloy Co.  1144 65th St., Unit A Oakland, CA 94608
510-658-3172 Tactile feedback systems

Virtual Research 1313 Socorro Ave.  Sunnyvale, CA 94089
408-739-7114 Head-mount displays 

Virtual Technologies POB 5984 Stanford, CA 94309 415-599-2331
Instrumented gloves and clothing

The Vivid Group 317 Adelaide St. W., Suite 302 Toronto, Ontario M5V IP9
Canada 416-340-9290 VR authoring systems 

VPL Research 656 Blair Island Road Suite 304 Redwood City, CA 94063
415-361-1710 Head-mount displays, sensor-laced gloves, full body suits,
VR software 
VREAM 2568 N. Clark St., #250 Chicago, IL 60614 312-477-0425
VR authoring systems

Xtensory Inc.  140 Sunridge Drive Scolls Valley, CA 95066
408-439-0600 Tactile feedback systems