Experimental Method

Participants

The 14 participants for this experiment were engineers, scientists, and mathematicians selected from the staff of the Johns Hopkins University Applied Physics Laboratory (JHU/APL). Each participant had experience in at least one of the following relevant areas; naval tactical displays, naval AAW operations, and personal computer or workstation use. Each participant was asked before the experiment to provide written consent to the terms of the experiment. After they completed their four trial runs, each participant also completed a questionnaire which helped determine the relevant characteristics of the participant population.

Equipment

The perspective display was presented on a Sony Model HDM 3830 38-inch diagonal high-definition television (HDTV) monitor with 1920 x 1024 pixel resolution, driven by a Silicon Graphics Skywriter 4D/440 IG2 graphics workstation with four MIPS R3000/3010 processors and a dual graphics pipeline. The plan-view display was driven by the same graphics workstation and was presented on a Hitachi CM2086A3S6 19-inch color monitor with 1280 x 1024 pixel resolution. The two displays were placed side-by-side. User input in both cases was provided via a standard three-button mechanical mouse.

The perspective and plan-views were presented on different monitors for two reasons. The 19-inch monitor with its nearly square aspect ratio is not appropriate for a perspective view. With a perspective view the depth dimension is foreshortened or compressed requiring less vertical screen space to present data. At the same time the width dimension is stretched in the foreground requiring more horizontal screen space to prevent objects in the periphery from being clipped. HDTV, with its wide aspect ratio, is well suited for perspective views. The square aspect offered by the 19" monitor is most appropriate for a plan-view which scales width and depth dimensions equally. Also, the HDTV could only be driven in an interlaced fashion. This produced an annoying flicker with the plan-view's associated text windows. Thus, the non-interlaced 19-inch monitor was used for the plan-view.

This equipment, and the software applications that drive it, are part of a project called Force Threat Evaluation and Weapon Assignment (FTEWA). FTEWA is part of the U. S. Navy-sponsored Force Anti-air warfare Coordination Technology (FACT) program. FTEWA is being developed at the Johns Hopkins University Applied Physics Laboratory as part of a continuing effort to improve anti-air warfare (AAW) coordination. For more information on FTEWA, see Dennehy (1994).

Scenarios

The participants were presented with two separate scenarios, each with 50 aircraft. In both scenarios, the aircraft were randomly distributed around a point in the Strait of Hormuz in the Persian Gulf which presented a mixed background of both land and water. For the perspective display, the point of view of the observer was at about 25,000 feet altitude and far enough away from the center point (about 150 miles) to keep all aircraft within the display's field of view. The plan-view display provided the observer with a view looking directly down on the center point.

The participants were not allowed to change the view point in either display type. With the perspective display, the scene animates between view points under the control of a six degree of freedom force ball called a Spaceball. Mastery of the Spaceball would have required more training time than was available. Also, during trial runs it was found that, because of the novelty of the device, participants could not resist experimenting with it and thereby distracting themselves from the task at hand. To be fair, participants were not allowed to use the range and offset view changing controls on the plan-view either. Though McGreevy (1991) concludes that motion of the point of view greatly enhances perspective display utility, fixed views on the perspective display were carefully chosen to not hide any descending aircraft. A fixed view on the plan-view was chosen that centered the scenario and assured no aircraft would move out of range as the scenario progressed.

All aircraft parameters were randomly selected within the following ranges: distance from the center point was from 0 to 100 nautical miles; altitude was from 10,000 to 20,000 feet; ground speed was from 150 to 300 knots; aircraft heading was from 001 to 360 degrees; and each aircraft was either ascending, descending, or maintaining altitude. The major distinction between the two scenarios was that only 10 of the 50 aircraft in Scenario A were descending, while 21 of the 50 aircraft in Scenario B were descenders. Aircraft did not perform any altitude maneuvers. An aircraft was either ascending, descending, or flying steady at the start of the scenario and remained so for the duration of the experimental run. The aircraft moved across either display according to their initial heading, ground speed, altitude, and climb rate. Therefore, for the perspective display, apparent aircraft position in three dimensions appeared to change, while ground speed, climb rate, and heading remained constant. For the plan-view display, position of the aircraft symbols only changed in two dimensions. Movement in the third dimension (altitude) could only be determined by "hooking" the aircraft and reading a numerical display of altitude and climb rate.

A number of standard Navy Tactical Data System (NTDS) symbols were used on the plan-view display to represent the different types of aircraft. A half circle represented any type of friend airframe, a full circle represented a friend ship, a half box represented both commercial and unknown aircraft. All symbols had the same orientation and were positioned on the display at their ground projection point. A line protruding from the center of the symbol, called a velocity leader, indicated position, heading, and velocity. The endpoint at the center of the symbol indicated current position. The other endpoint indicated a predicted future position after a fixed period of time expired. The symbols did not embody any altitude or climb rate information. Symbols had to be hooked to obtain such information. It has been suggested that naval plan-view symbols be augmented with some ascent/descent encoding. Because comparison with an existing symbol set was desired, a myriad of symbol encodings already exist that most users don't remember, and most existing systems require altitude to be monitored, a middle of the road approach was implemented. Ascent or descent could be determined by examining a single field, climb rate, in the amplifying text panel.

Aircraft were shown on the perspective display as shaded renderings of E-3A airborne warning and control aircraft, F-14 and F-18 fighters, A-6 strike aircraft, and 737-class airliners. Aircraft symbols on the perspective display were rotated to show heading, scaled to show relative distance from the observer, and given shadows directly below them to show their ground position and altitude. In addition to motion across the display, the length of the leaders attached to the shadows showed aircraft velocity. The shadows and leaders also served as a redundant cue for heading. This was especially useful for disambiguating the head-on case where examination of the symbol alone could not resolve whether the aircraft was moving toward or away from the viewer. To show ascent or descent, symbols were given a pitch angle. All ascenders were pointed upward at an angle of 25 degrees and all descenders were pointed downward at the same angle. Descent could be determined by the pitch angle alone for many symbols. In more difficult cases such as head-on symbols or inbound symbols in the periphery whose pitch angle appeared to flatten out because of the curvature of the earth, descent could be detected by observing that the aircraft and its shadow appear to be converging (i.e. lines drawn along the fuselages of the aircraft and its shadow will intersect in front of the symbol). Similarly, the aircraft symbol and its shadow run parallel for steady flight and appear to diverge for ascending flight.

Experimental Design

A 2x2 counterbalanced within-participants design was used for the experiment. Each participant performed a total of four tasks: two with the plan-view and two with the perspective view. Of each pair of tasks, one had 10 descending aircraft and the other had 21 descending aircraft. Half of the participants performed both plan-view tasks first, while the other half performed the perspective view tasks first. It was expected that this approach would minimize learning effects.

Procedure

The experiment was conducted in a partitioned area in a computer laboratory whose environmental conditions are designed to replicate a ship's Combat Information Center (CIC). After having the task explained to them verbally, each participant was tutored for about five minutes on the operation of the two displays. Particular attention was given to the interpretation of the visual cues on the perspective display and the amplifying text panel. Participants were told that they were being timed and that they must work quickly and continue working at the task until they completed it without errors. Participants were then allowed to practice for a couple of minutes on a sample scenario to become familiar with the mouse functions. Following the scenario, each participant was interviewed for subjective impressions.

The following procedure was used for the plan-view portion of the experiment. Using the mouse, the participants were asked to start an on-screen digital clock. Immediately, the participant would begin "hooking" the aircraft symbols one at a time by depressing the right mouse button while the on-screen cursor was directly over the desired symbol. When a symbol was hooked, a circle would appear around it, and a text window would appear in the upper-left corner of the display showing several aircraft parameters including climb rate. If the climb rate was negative, signifying a descending aircraft, the participant would depress the left mouse button to "mark" that aircraft's symbol. When the participants thought that all descending aircraft, and no others, were marked, they would use the mouse to stop the clock. If the task had indeed been completed correctly, the clock would stop. If there were errors in the participant's selections (either descenders were not marked, non-descenders were marked, or both), a message was displayed stating "You have made some error(s). Keep going." and the clock continued to run.

For the perspective display, the procedure was very similar. After starting the clock, the participants could immediately begin marking aircraft that appeared to be descending because of their apparent pitch attitude on the perspective display. If the participants were unsure about the pitch attitude of a particular aircraft, they still had the ability to hook a perspective aircraft symbol and receive the textual display of aircraft parameters including climb rate. Again, when the participants thought that they had correctly marked all descending aircraft, they would use the mouse to stop the clock.

Data Collection

The computer automatically produced a data file for each trial run by recording at five second intervals the time of day, time from the beginning of the run, number of symbols correctly marked, number of descenders not marked, and the number of non-descenders that were mismarked. Though participants could assume that completion time was being measured by virtue of the on-screen clock, they were not aware that data was being recorded every five seconds. Also, each time the participants thought they had successfully marked all tracks and depressed the stop button on the on-screen digital clock to stop the run, this action was recorded in the data file. These data files were parsed to produce files in a format that was readable by a statistics program.

Continue

Table of Contents