Down to Earth; Commercial applications of  virtual reality 

Special Report: Virtual Reality,  part 1
Computer Graphics World, March, 1992                      

Emmett, Arielle

   In Japan, where consumer electronics rival popular religion,
Matsushita Electric Works Ltd. (Osaka) employs a VPL Research
head-mounted display linked to a Silicon Graphics-based RB2
workstation to help prospective customers design their own kitchens.
"This application is by appointment only.  It's a very deep project,"
notes Jaron Lanier, the CEO of VPL Research (Redwood City, CA), which
develops and distributes VR systems internationally.

   Customers who don't see the kitchen of their dreams in the
Matsushita showroom can create their own initial design on a CAD
system and download the design to the SGI workstation.  The customer
can then strap on the VPL "eyephones"--a headset with telescreens
providing wrap-around stereoscopic views of cartoonish,
computer-generated appliances--and tour any number of virtual kitchen
worlds [the Japanese call them "system kitchens"].  Customers also don
the $ 8000 VPL data glove, which uses position sensors to detect the
movement of fingers as the virtual reality "traveler" points to
objects within the illusory system kitchen space.

   Matsushita customers can thus fly through rooms, do 360-degree
revolutions, and rearrange appliances at will.  "If the customer wants
to see other types of system kitchens, he can change the colors and
heights of cabinets," says Takashi Nashiyama, a researcher in
Matsushita's Artificial Intelligence Laboratory in Osaka.  "Virtual
reality helps them experience it as though they are there."

   Although the uses of VR around the world are not all quite so
commercialized, the technology is playing a game of catch-up,
overtaking earlier hype comparing it to fantastic drug-induced
journeys into computer-generated "cyberspace."

   "People's perception of what VR is and what it can do are much
stronger than what the field can actually produce now," says Dr. Peter
Tinker, a member of the technical staff of the Rockwell International
Science Center (Thousand Oaks, CA).  "It's an odd thing; people have a
very clear idea of what you're talking about when you describe VR.
[But] they tend to expect too much from it."

   On the other hand, the technology has advanced to the point where
it can be used as a practical tool in select applications.  Tinker
notes, for example, that VR has already "sold itself" in the aerospace
industry for prototyping and human factors evaluation.  Using VR early
in the design cycle "helps you make sure people feel comfortable in
the [proposed] environment; [prototyping] is part of the early design
stage." Human factors number crunching builds on virtual prototypes by
yielding detailed mathematical data about the fit and comfort level of
designs.

   According to Lanier, commercial applications are being pursued most
aggressively in Europe and Japan.  Aside from the Matsushita and Art +
Com examples, he cites Fujita of Japan, the largest Japanese
construction company, which has embarked on an ambitious project to
combine virtual reality equipment from VPL Research and Fake Space
Labs (Menlo Park, CA) with robotic "telepresence"--the science of
driving robots remotely and attaching video cameras to them to perform
building inspection and other tasks, including robotization of
construction.

   Fujita has a shortage of human building inspectors, Lanier
explains.  Rather than disperse the available manpower to construction
sites around the world, Fujita is counting on the use robots, which
will "live-tape" the sites and feed back the video information to
Japan.  There, human inspectors wearing VPL eyephones or utilizing a
stereoscopic "BOOM" (a box mounted on a desktop that floats like a
periscope and provides wrap-around three-dimensional imagery),
designed by Fake Space Labs, will "step into" the virtual world of the
video camera and "see" the sites as the robot sees them.  For example,
when the human inspector moves his head to see a particular
construction element, the robot's cameras will move into position
correspondingly.

   The inspector's virtual vision, however, will be enhanced by a
continuous flow of 3D construction data pumped across the VPL RB2
workstation and into the eyephones.  The data, represented as
graphical images, is superimposed over the video image.  By lining up
the graphical elements with the real video image, the inspector can
check each site for missing elements and construction disparities.
"If you track the images back to the eyephones, you have the sensation
of being where the robot is," Lanier says.

   In Europe and Japan, all kinds of commercial applications are
flowering," Lanier continues.  Virtual reality is seen not simply as
an integrated entity but as a source for spin-off technologies that
may or may not use traditional "wrap-around" virtual reality displays.

   According to Charles Grimsdale, CEO of Division Ltd. (Bristol,
England), makers of integrated virtual reality systems (known as
ProVision), most European applications are still in the R&D phase.
Although there are a few real end-users, Grimsdale says, "The majority
of customers are looking at end-use applications between 18 months and
five years from now." Those already taking the plunge include
University of Eindhoven's Calibre Institute (Holland), which is using
a ProVision system to convert Autocad DXF files into virtual rooms and
spaces for real-time architectural studies.  In addition, a multimedia
company called Virtual S (London) is using Division's virtual reality
systems to sell new product and service ideas to corporate clients.

   "They used our virtual reality product to sell a telephone query
system called CallScan to Pepsico," Grimsdale says.  Why virtual
reality?  Aside from the "sexiness" of the new technology, "VR is
ideal for showing clients products that are otherwise too expensive to
demonstrate," he declares.  In addition, the 3D sensory impact is
unmistakable.  "We firmly believe that to create an immersive
experience . . .  you've got to make the display sufficiently
compelling that it really hits them," he says.  A headmounted or BOOM
virtual reality display fits the bill, Grimsdale assets.  "Standard 3D
computer screens for mass presentations do not have the same impact."

   Additionally, the European automobile and aerospace CAD markets are
especially interested in virtual reality.  "We've seen an interest
across the board in aerospace, where one of the key issues is design
for maintainability--human factors," Grimsdale continues.  "In the
automobile industry, there is a definite interest in [using virtual
reality] to replace clay models.  European companies looking at the
problem see it as a competitive issue."

   Indeed, Mercedez-Benz has already begun a joint project with Art +
Com in Berlin to study viewer perception of depth in preparation for
building a "virtual reality car interior planning system," according
to Art + Com's Tranberend, who has designed custom software extensions
of VPL's RB2 system.  "We're looking at what kind of mechanism the
human brain has that allows it to get an impression of how big a room
is," he says.  "If a person has a feeling of a [virtual room], it's
possible to make predictions about . . .  the feeling he will have in
the real room when it's built.  To my knowledge, this has never been a
topic of research before," he says.

   In the UK, where "time-to-market" issues have become a matter of
competitive survival, virtual reality spin-offs may have a major
impact in the footwear industry, according to Terry Freer, manager of
Satra Footwear's management business area.  "Footwear is a fashion
item, and we have little time to do preproduction engineering to find
the most economical way to combine time, [operations], and materials,"
he says.  Satra consequently opted to build a VR application using
WorldToolkit virtual reality software produced by Sense8 Corporation.

   The application is still in the early stages of development.
"Basically, we're using Sense8 WorldToolKit to build real-time
three-dimensional models of shoe-making operations," he says.  "Within
the virtual world, we mimic hand operations on the stitching machine
to find the most economical way to carry out operations."

   Freer says his company chose WorldToolKit because of its
flexibility, specifically, the availability of a C library on which to
build custom applications, as well as the "extreme realism" of the 3D
graphics, including texturing of surfaces.  Interestingly enough, the
company has chosen to use a standard computer screen, rather than a
headmounted display, to simulate shoemaking operations.  "We're using
a spaceball to manipulate objects and alter perspectives," Freer says.
Strict VR displays are not needed.  "And we're still very much in the
learning stages."

   In the US, the feeling is similar: Virtual reality is still in a
nascent, "learning" stage.  Most of the applications interest is still
heavily concentrated in the university and government labs, especially
in the fields of medicine and aerospace, rather than in commercial
enterprises.  Indeed, unlike the Japanese, who reportedly are pouring
millions of dollars into VR applications research, most US enterprises
appear reluctant to make any serious investments.  Work is
consequently focused on developing prototypes and demos without talk
of immediate production goals.  Part of the problem appears to be an
age-old one: American industry's reliance on short-term gain.  Another
part of the problem has to do with disagreements about the direction
of VR--a promising, albeit unruly technology, observes Myron Krueger,
a pioneering independent inventor and virtual reality researcher.

   "To say that there are [commercial] applications today that are
pulling freight is a ruse," says Krueger, who remains skeptical even
of the reported European and Japanese commercial applications.
Krueger argues that too many tecnical problems associated with VR must
be solved--among them, the questions of the interface device, the
quality of head-mounted displays, the problem of position tracking,
the "delay" in response a VR traveler feels when he moves within a
virtual world, as well as many other networking and political
challenges.

   Nonetheless, work is proceeding.  NASA, for example, has a virtual
reality project in nearly every one of its divisions' both NASA and
the Air Force still support highly visible flight training and
helicopter simulation projects using VR technology--spin-offs, as it
were, of the now legendary "Super Cockpit" project of 1981 at
Wright-Patterson Air Force Base, which used high-resolution,
helmet-mounted position sensors to teach pilots how to fly F-16
Fighter Jets.

   Today, though, VR applications for flight simulation have given way
to more varied, if less grandiose, projects.  At NASA Ames Research
Center, for example, Dr.  Elizabeth Wenzel, a research psychologist,
is pursuing "virtual acoustic display" research for VR systems.
Wrap-around acoustic sound that features "spatialized" auditory cues
in 3D may one day help air-traffic controllers process confusing radio
transmissions from aircraft.

   Besides this application, NASA and other aerospace facilities are
concentrating their VR research efforts in areas such as human factors
engineering, virtual prototyping of buildings and military devices,
aerodynamic (gas flow) analysis, 3D data visualization, satellite
position fixing, and planetary exploration simulations.

   Human factors, for example, is the accent at the NASA George C.
Marshall Space Flight Center (Huntsville, AL).  There, engineer Joseph
P. Hale has been using a VPL Research eyephone/data glove system
coupled to a Macintosh computer (for number crunching) and Silicon
Graphics polygon engine to do computationally intensive ergonomic
analysis.  "VR is a difficult concept to sell to management," Hale
says.  "Many view it as just a video game." But in the shrinking
dollars game of federal budgets for aerospace, Hale says that VR may
prove a valuable human factors tool.

   Hale is currently developing number-intensive virtual reality
"mock-ups" to do what he describes as "operations development and
training." Rather than wait for designers to complete a proposed
design for a space vehicle and test it with a full-scale, expensive
foam-core mock-up, Hale says he can test design elements earlier in
the design cycle if they are created virtually.

   One project Jale is pursuing is ergonomic evaluation of a furnace
for growing crystals, which is being designed for NASA's Space
Station.  By entering the virtual world of three different designs for
a pivot used to access the furnace, Hale is able to perform realistic
"reach analysis" and determine which design works best.  One problem:
Objects are not yet programmed in the virtual world to behave as they
would in microgravity.  "You can grab the crystal [in the virtual
world], but it doesn't show it tumbling away from you," Hale says.

   Dr. Tinker points to similar limitations.  But he says most of the
real challenges are political, not technical.  "A big advantage [to
VR] is that you can make changes very quickly to designs and bring the
customer into the design loop." That, Tinker says, has been the
linchpin for VR: "It's a powerful marketing tool," he says.  "Perhaps
this is not what people want to hear, but it's powerful because you
put the customer in control.  Rather than showing the customer charts,
pictures, even video, you let the customer experience the new product."

   Perhaps nowhere is virtual experience more powerful than in the
emerging biomedical applications in which US laboratories are staking
leadership.  At the University of North Carolina at Chapel Hill, for
example, a group of computer scientists led by Frederick Brooks and
Henry Fuchs have created complex virtual visualizations of protein
molecules and their receptor sites.  Scientists who enter the "virtual
world" at the molecular level use head-mounted displays and a kind of
"force feedback" mechanism in free space that lets them "navigate" to
different sites on the molecule and "dock" (using a mechanical arm and
hand-grip with position sensors) when conformations are compatible.
During this process, scientists actually "feel" the push and pull of
molecular forces as receptor and drug compounds interact and "lock" in
the correct conformations.

   "When you build models out of plastic, you don't have any idea of
what the forces [between sites] are," explains Fuchs, the Federico Gil
professor of computer science at UNC who is working on several VR
projects.  In this molecular application, scientists program in
relevant information about the bonding energies of various atoms, as
well as ways to calculate the combination of intra- and intermolecular
forces.  The program works transparently to calculate attractions and
repulsions which are reflected in both the visual and "tactile"
sensations of the molecular journey.

   "Biochemists don't have to worry about how to calculate the
forces," he says.  "They just worry about the descriptions of the
molecules." The application helps them "see" at the molecular level
whether new drugs, organic molecules, or synthetic analogs will "fit"
biologically active receptor sites.  This process could represent a
new means of testing drugs and bringing molecular chemistry a quantum
leap forward.

   Fuchs, however, is not stopping with the receptor site problem.  He
and other scientists have been using VR techniques and supercomputers
to synthesize diagnostic images of a patients's body, to do
"predictive" modeling of radiation treatment using dynamic images
created by ultrasound, magnetic resonance imaging (MRI), and X-ray.
"We look inside a patient with an ultrasound probe and read back
echoes to know where a tumor is in three-space," he says.  But using
digital models of a patient provides a radiation therapist with more
complete information than any of the separate imaging techniques.  A
radiation therapist operating in a virtual world can view and expose a
tumor from every angle based on electronic representations of the
body, and then model specific doses and configurations of radiation
beams to target the tumor most effectively.

   With VR techniques, "We can get a continuum of multiple images
[defined] in space," Fuchs says.  "Virtual reality may enable a
physician to get a better visualization of the relationships of
various beams targeted on the patient's tumor than seeing it on a
conventional workstation screen.  Therefore, virtual reality will
enable the therapist to come up with better alternatives for treatment
than with conventional workstations."

   However, work is just beginning in this area, and the virtual
reality models are very simplistic.  Fuchs anticipates that VR
solutions in medical applications will become truly practical within a
20-year time frame.  He is counting on better-quality displays
(including tiny displays), better head-tracking capability, and
higher-quality graphics to help bring VR applications forward.

   Currently, though, there are medical challenges that seem much
closer to resolution.  A variety of VR techniques, for example, are
being used to help physicians visualize surgical problems--possibly to
"rehearse" operations before they actually occur.  Dr. Joseph M.
Rosen, an associate professor of surgery at Dartmouth Medical School
(Hanover, NH), for example, has collaborated with two software
engineers, Steve Pieper and Scott Delp, to create computational models
of the human face and the lower extremities to examine the effects and
outcomes of surgical procedures.

   "If you cut a muscle, the program will tell you what happens [to
the tissue and bone]," Rosen says.  "The simulation looks the same as
a volumetric model," he remarks.  "But the model is computational; it
understands what happens when you stretch a muscle or cut the bone."
Never before has a physician used a virtual reality system to evaluate
the forces acting on a portion of the body.  In three to five years,
however, surgical simulations may become routine--espe-cially to
rehearse the strategies for intricate and rare operations, many agree.

   In the meantime, though, small medical VR systems and applications
are beginning to evolve.  At Greenleaf Medical Systems (GMS; Palo
Alto, CA), for example, which licensed the medical rights to VPL
Research technology, neuroscientist Walter Greenleaf, Ph.D., is using
sensor-lined data gloves and data suits to obtain range-of-motion and
strength measurements of injured and disabled patients.  "Studies of
injured people are difficult to do--you need ways to measure angles,
momentum, and force," he explains.  Greenleaf Medical Systems is
consequently using the sensor-lined glove and suit to obtain the force
measurements.  Mathematical data from the sensors feed back to a
Macintosh workstation, which calculates and then visualizes impairment
ratings in graphical format to help a physician or patient evaluate
strength and mobility.

   Although the system uses no head-mounted display as yet-Greenleaf
explains that VR display devices are not sophisticated enough to hold
much clinical value--virtual reality may be a viable means of
visualization in the future.  Orthopedic physicians are already using
the examination system (known as Eval) in the clinical community.
"It's been very well-received," Greenleaf says.  A similar application
known as "gait analysis" will enable physicians to evaluate children
walking down a runway prior to orthopedic surgery.  The children will
wear data suits to measure motion; and virtual reality displays may be
used to portray results.

   GMS has also developed new rehabilitation devices for patients.
"GloveTalker"--a data glove sign-language device, due to hit the
market this year, "allows someone without a voice [such as a stroke or
celebral palsy patient] to make gestures the computer understands,"
Greenleaf says.  In effect, GloveTalker interprets signs and then
translates them into commands that appear on the computer screen as
written language, as well as synthesized speech.

   "The device can be used to help recovering hospital patients
communicate their needs to staff or family; or it can be adapted to
athome use," Greenleaf says.  Tested at Loma Linda Medical Center and
Cal State North Ridge, the device (or similar virtual-reality systems)
could soon be used for stroke rehabilitation.  "We could retrain
someone to walk using virtual reality if we turn down the gravity
vector," Greenleaf remarks.  "Using headmounted displays, a patient
could relearn how to open a door, walk, point, or turn around in
space."             

   Welcome brave new world.  If it all sounds like science fiction,
it's hardly surprising.  Virtual reality applications, even as they
are imagined today, do have an aura of the mythic and impossible about
them.  Though it's clear that practical, highly beneficial
applications may be just around the corner, popular acceptance may not
be enough.  The VR industry needs serious investors--knowledgeable,
aggressive investors who know the technology--who aren't afraid of the
long shot, and who can provide the one thing virtual reality sorely
lacks: cold, hard cash.

   Arielle Emmett is a freelance writer based in Media, Pennsylvania.