In the scene depicted on the cover of HITLab Review, the pilot in the cockpit,
and the two advisors to the left and right are represented by
avatars generated by a computer located at the third advisor (center
with HMD). All four participants are in different locations and
may represent, for example, a pilot, in a single seat fighter
being advised by an AWACS operator, a Forward Air Controller and
a ground controller. A version of the shared virtual environment
is present at each location, and is animated by gestural signals
generated at the various sites. In addition to tracking movements
of the hands and head, eye movements are also tracked and shown
as a circle where the pilot is currently looking. Other physiological
measures, such as galvanic skin response, electromyography, and
electrocardiography could also be added.
Each advisor is attempting to impact the situation awareness of
the pilot by transferring a mental model to the pilot. To do so
the advisors transmit information, and they transmit probes. Probes
are means of assessing the state of the pilot's knowledge. In
face-to-face communication we use probes ("OK?" "Know
what I mean?" "Yea?") and look for verbal and gestural
confirmation that the information being transmitted is understood.
This is an important aspect of intelligent communication. It prevents
the transfer of redundant information, it allows for changes in
pacing and repetition when there is ambiguity or lack of understanding,
and it builds confidence on the part of the transmitter and receiver.
The concept of telesavance draws from a large body of literature
on communication, from previous work at the HITLab on distributed
virtual worlds (GreenSpace), from work at the HITLab and elsewhere
on facial gesture recognition (Joey King's work),and from ideas
generated by the Virtual Pilot project. Telesavance represents
a unifying construct with the following potential advantages:
Some of the communication advantages represented by the ViP scenario
could be achieved by placing a video camera in the cockpit. However
the advantages of using avatars include lower communication bandwidth
(73,728,000 bits/second for video vs. 11,520 bits/second per sensor),
the ability to manipulate gestural data (e.g. to summate, exaggerate
or filter, and for machine recognition), and the flexibility of
information presentation (e.g. the see-through pilot shown on
the cover). The question of whether video is a sufficient solution
is one that needs to be answered empirically.
An experiment is being prepared to test the performance advantages
which can accrue from the communication of situation awareness
in virtual reality. This proof of concept study is being supported
by Armstrong Laboratory at Wright Patterson Air Force Base, and
the Air Force Office of Scientific Research (AFOSR). Future directions
for the program include the creation of design guidelines to identify
domains in which telesavance can play a role, and expansion into
other domains (e.g. ship borne, remotely piloted vehicles, and
training).
Read Also:
Brown, C. 1997. Research Isolates the Critical Parameters of Collaboration - Project Adds a Dash of Wisdom to External Reality. EE Times, July 28, Issue 964, Available at: http://www.techweb.com/se/directlink.cgi?EET19970728S0044