Virtual Pilot Environment The term savance comes from the noun "savant" meaning a wise, learned or knowledgeable person. The term "telesavance" refers to the transmission of the state of a person's "wisdom" or situation awareness using telecommunication techniques. The HITLab has been exploring the concept of telesavance as part of the evolution of the Virtual Pilot (ViP). The ViP was demonstrated in September 1996, and described in the HITLab Review. In the ViP demonstration two people played the role of pilot and copilot in a shared virtual cockpit. They communicated with voice and gesture as they collaborated on a demanding task. This concept has acquired some momentum in the form of telesavance. One embodiment of the concept is shown at right.

In the scene depicted on the cover of HITLab Review, the pilot in the cockpit, and the two advisors to the left and right are represented by avatars generated by a computer located at the third advisor (center with HMD). All four participants are in different locations and may represent, for example, a pilot, in a single seat fighter being advised by an AWACS operator, a Forward Air Controller and a ground controller. A version of the shared virtual environment is present at each location, and is animated by gestural signals generated at the various sites. In addition to tracking movements of the hands and head, eye movements are also tracked and shown as a circle where the pilot is currently looking. Other physiological measures, such as galvanic skin response, electromyography, and electrocardiography could also be added.

Each advisor is attempting to impact the situation awareness of the pilot by transferring a mental model to the pilot. To do so the advisors transmit information, and they transmit probes. Probes are means of assessing the state of the pilot's knowledge. In face-to-face communication we use probes ("OK?" "Know what I mean?" "Yea?") and look for verbal and gestural confirmation that the information being transmitted is understood. This is an important aspect of intelligent communication. It prevents the transfer of redundant information, it allows for changes in pacing and repetition when there is ambiguity or lack of understanding, and it builds confidence on the part of the transmitter and receiver.

The concept of telesavance draws from a large body of literature on communication, from previous work at the HITLab on distributed virtual worlds (GreenSpace), from work at the HITLab and elsewhere on facial gesture recognition (Joey King's work),and from ideas generated by the Virtual Pilot project. Telesavance represents a unifying construct with the following potential advantages:

Some of the communication advantages represented by the ViP scenario could be achieved by placing a video camera in the cockpit. However the advantages of using avatars include lower communication bandwidth (73,728,000 bits/second for video vs. 11,520 bits/second per sensor), the ability to manipulate gestural data (e.g. to summate, exaggerate or filter, and for machine recognition), and the flexibility of information presentation (e.g. the see-through pilot shown on the cover). The question of whether video is a sufficient solution is one that needs to be answered empirically.

An experiment is being prepared to test the performance advantages which can accrue from the communication of situation awareness in virtual reality. This proof of concept study is being supported by Armstrong Laboratory at Wright Patterson Air Force Base, and the Air Force Office of Scientific Research (AFOSR). Future directions for the program include the creation of design guidelines to identify domains in which telesavance can play a role, and expansion into other domains (e.g. ship borne, remotely piloted vehicles, and training).


Read Also:

Brown, C. 1997. Research Isolates the Critical Parameters of Collaboration - Project Adds a Dash of Wisdom to External Reality. EE Times, July 28, Issue 964, Available at: http://www.techweb.com/se/directlink.cgi?EET19970728S0044