Direct Manipulation

Author: Michael Dennehy

Direct manipulation is a style of Human Machine Interaction (HMI) design which features a natural representation of task objects and actions promoting the notion of people performing a task themselves (directly) not through an intermediary like a computer. Virtual Reality can be viewed as a field which can draw upon the principles of direct manipulation for Human-Computer Interaction (HCI) design or as an example or extension of direct manipulation itself. In VR, not only can task objects and actions be naturally represented, the task environment can be naturally represented as well. With either view, an understanding of direct manipulation principles is essential for the successful design of human computer interfaces in virtual environments. The remainder of this article will discuss the characteristics and benefits of direct manipulation along with its relation to virtual environments and the foundation areas of computer science.

Direct manipulation is a topic in the interdisciplinary field of HCI. Computer science, psychology, linguistics, graphic design, and art all contribute to this field. The computer science foundation areas of computer architecture and operating systems provide you with an understanding of the machines upon which human computer interface styles such as direct manipulation are implemented. This understanding allows you to determine capabilities and limitations of computer platforms, providing boundaries for realistic HCI designs. For instance, though productive in a visionary sense, many purported VR HCI concepts are not practical on today's computer systems. As with any computer application development, direct manipulation interface development benefits greatly from the computer science foundation areas of algorithms and programming languages. The specialized field of computer graphics can also make a key contribution.

A favorite example to elucidate direct manipulation principles in contrast with the intermediary style of interaction (eg. traditional keyboard based, command driven interfaces), is travel in a car. With direct manipulation, you drive the car by manipulating the steering wheel and pedals. The car responds immediately to your actions, and these responses are immediately evident. If you are making a mistake such as turning too sharply, you can quickly recognize this and perform a corrective measure. With an intermediary style of interaction, you sit in the backseat of the car giving a stranger directions. Further, imagine the stranger possessing poor interpersonal skills and having a limited vocabulary. You've lost the feel for the road and you don't have a direct view of where you are going. Worse yet, you have to rely on a stranger who, if they don't receive explicit directions using particular phrases in a fixed order, idles in the middle of the road or takes you to unfamiliar places from which you don't know the way out.

In relation to interactive computer systems, a direct manipulation interface possesses several key characteristics. As mentioned earlier, a visual representation of objects and actions is presented to a person in contrast to traditional command line languages. Further, the visual representation usually takes the form of a metaphor related to the actual task being performed. For instance computer files and directories represented as documents and file cabinets in a desktop publishing system. The use of metaphors allows a person to tap their analogical reasoning power when determining what actions to take when executing a task on the computer. For example, this property is drawn upon heavily by desktop metaphors in their handling of windows like sheets of paper on a desk. With direct manipulation, actions are rapid, incremental, and reversible with results being immediately visible. This enhances the impression that the person is performing the task and is in control not that the computer is responding to requests while the person waits powerlessly wondering if the computer is doing the job correctly.

Given a thoughtful design and strong implementation, an interactive system employing direct manipulation principles can realize many benefits. Psychology literature cites the strengths of visual representations in terms of learning speed and retention. Direct manipulation harnesses these strengths resulting in systems whose operation is easy to learn and use and difficult to forget. Because complex syntax does not have to be remembered and analogical reasoning can be used, less errors are made. When they are made, they are easily corrected through reversible actions. Reversible actions also foster exploration because the fear of breaking something has been diminished. Also, a person can gain confidence and mastery because they are in control and because the system responses are predictable and immediate.

Because these benefits of direct manipulation are also desired in VR systems, direct manipulation principles should be drawn from when designing VR systems especially in the use of VR's special input devices. For instance, when using a data glove, a person should be able to select actions rapidly and easily by pointing and gesturing. Gestures should be natural and intuitive in the particular virtual environment. Actions should be represented visually and be incremental, immediate, and reversible to give a person the impression of acting directly in an environment. If voice recognition is employed, care must be taken to assist it with visual cues and complement it with hand gestures. Otherwise, the recreation of a complex command syntax minus the keyboard is a lurking danger.

References:

Norman, D. (1988), The Psychology of Everyday Things, Basic Books, Inc.