CHAPTER 4

INTERFACE STRATEGIES FOR A 3D COLLABORATIVE WORLD

Computer users have had the benefit of evolving human-computer interfaces ever since the first command structure was created. Human beings communicated with the first computers by way of switches and lights on a control panel. Soon thereafter, a terminal was attached to the CPU in order to provide a more dynamic interface. Users then could type characters on a keyboard and see the characters echoed to the terminal screen. Certain combinations of characters communicated a user’s request for the computer to perform a function on the user’s behalf. The results were then communicated back to the terminal when appropriate. Menus soon followed that allowed a user to pick a command from a list of possible choices without having to type the command.

As computer memory became less expensive and new monitor technologies were developed, computer systems could afford to express pictures on the monitor screen instead of simple characters. A pointing device was added to the user’s arsenal of input devices and he or she used it to more freely select pictures and text on the screen. Whole new graphical user interfaces (GUIs) were developed that consisted of popular user controls, often called widgets. These widgets became integral parts of computer operating systems.

Because these GUI controls were so universal, the first desktop Virtual Reality (VR) systems used them for their human-computer interface. In fact, all the multi-user

world clients discussed in Chapter 3 continue to show significant use of the GUI controls popular in 2D graphics. There seems to be incredible potential to invent new graphical interfaces that take advantage of the third dimension. Many 3D modeling technologies and 3D virtual worlds provide a third dimension in which interface components can be designed and implemented. This chapter looks at the traditional 2D GUIs and discusses possible new interface strategies for communicating with 3D multi-user worlds.

2D GUI Interface Controls

In a 2D world, interface components usually occupy a fixed location on the screen when enabled. This allows for inter-program consistency and easy remembering for the user. The evolution of interface components has always been toward easier use, less physical movement of the pointing device, and simpler task repeatability. As computer programs have become more complex with more available options to choose from, the GUI has become the subject of much conversation. 2D GUIs have been evolving toward simplification and reduced size. Small size is important in 3D environments as well in order to allow the user to see as much of the world as possible.

Initially, GUI controls consisted of check boxes, option button groups, buttons, text boxes, and list boxes. These objects are the cornerstone for human-computer dialogs in a windowed operating system. A GUI programmer bundled the controls into different pop-up windows that appeared to respond to the user’s actions. These same 2D GUI controls continue to be used heavily today in computer programs as well as on HTML-based Web pages. The HTML specification includes standard input attributes for check boxes, option buttons, single choice list boxes, multiple choice list boxes, text boxes, and simple buttons. Many list boxes take advantage of a drop-down feature where the user only sees the list of available choices after selecting a drop-down symbol to the immediate right of the box. Drop-down list boxes save precious screen real estate.

In an effort to cut down on the need to move the pointing device extensively, short-cut menus have become popular with the introduction of the Windows 95 operating system. In Windows 95 software, short-cut menus become available when a user right-mouse clicks on an object on the screen. The menu includes all context sensitive choices available based on the current state of the software in memory. If the choices are extensive, a cascading menu structure is used to organize the user’s path to the desired option. Short-cut menus save pointing device movement since the menu appears where the user clicks instead of requiring movement to a specific menu or toolbar location. They also eliminate unnecessary memorization of menu choices and toolbar icons as all appropriate menu choices appear with the pop-up menu.

Lately, more and more computer programs and Web pages are taking advantage of defining multiple areas within the main application window. Such areas are called frames or panes. Frames and panes provide independent navigation by the user in order that multi-purpose application tasks can be better distributed for user control. Multi-user 3D world clients are a good example of a multi-purpose application. In a multi-user environment, the user participates in chat, world navigation, and options selection simultaneously. With a frame environment, the option selection decision is always on-screen independent of where the user is in the 3D world or the current status of her chat sessions. Each frame can contain the traditional 2D GUI controls to allow the user to interact with the frame’s activity. Often, frame size is within user control.

New Opportunities with 3D

In 3D applications, the user is usually an active participant in the 3D scene and has a specific location and viewpoint in 3D space. This provides an opportunity for new human-computer controls. Traditional 2D GUI interfaces place the controls on the computer screen which is a barrier to the user’s actions. The user can’t move forward into the screen. With desktop 3D interfaces, the goal often is to reduce the user’s sense of the computer screen in order to help immerse the user in the computer program. The user is afforded the ability to move forward and backward relative to the screen. Using the popular 2D GUI controls and placing them on a fixed screen location reduces the user’s sense of immersion in many cases. In 3D environments, 2D GUI controls which are fixed in location change size based on the distance of the current viewpoint from the control. Placing 2D GUI controls in fixed locations in 3D space limits their availability to a subset of possible viewpoints. Is there a useful intermediate mode somewhere between always being on screen and fixation in a 3D location?

Also, in a 3D interface, the controls themselves can be three dimensional. Door knobs can open doors, file cabinet drawers can organize documents, and steering wheels can control vehicles. These controls are easily recognizable from many viewing angles. Traditional 2D GUI controls are flat and not easily manipulated from angles that are not perpendicular to the user’s line of sight. Besides, traditional 2D controls are not often encountered by a user in non-computing activities in the real world. They are constant reminders to the user that they are not immersed in a real situation but, in fact, are being provided a computer simulated virtual reality.

Current desktop multi-user 3D world clients are not considerate of an immersed collaborator. The clients take advantage of the best known 2D GUI controls and frames. In theory, this provides the user the best chance of immediately using the software effectively based on his or her accumulated knowledge of word processors, spreadsheets, and Web pages. But, if provided the same interface while immersed in the world with an HMD based VR system, the user would find the interface distracting. Providing a different interface for non-desktop users means retraining for the desktop user who for the first time gains access to VR immersive technology. There exists an opportunity to create one interface that works well for both desktop and sight immersed users.

3D environments provide the ability for a user to have virtual hands and feet which are not necessarily within the field of view at any time. A 3D interface can take advantage of that fact by placing sophisticated controls in the hands of the user and then giving the user the ability to move his or her hands up within view or down out of view. Perhaps even better is the opportunity to let a user choose from different input devices which are made available in the world. The user could choose their input devices and controls by picking up the objects they want to use. There is no reason a user could not choose where to put their controls. Initially, while learning, a user could put their controls (or help information) in a place where they are always within view, such as dangling from the front brim of a virtual hat. Then, they could move those objects to their virtual hands to be brought within view as necessary.

Today’s 3D Multi-user World Interface Solutions

As discussed in Chapter 3, the client is usually responsible for handling world navigation, text or voice communications, and personal user preferences. Each of these components require some interface design. The following section reviews choices made by the 3D multi-user world clients available today for download over the internet.

World Navigation

A virtual collaborator has six directions available to her when navigating a virtual world: up, down, forward, back, right, and left. In many worlds, the up and down directions are not available since the user is not intended to leave the ground terrain. In many clients, such as Sony Corporation’s CommunityPlace, blaxxun interactive’s Passport, OZ Interactive Inc.’s OZ Virtual and Netscape Communication Corporation’s Live3D, forward, backward, right and left movement is initiated with the arrow keys on the user’s keyboard or user’s mouse movement when the user is in a walk mode. The up arrow or mouse movement away from the user navigates forward in the world, the down arrow or mouse movement toward the user navigates backward in the world, the right arrow or right mouse movement takes the user to the right in the world, and the left arrow or left mouse movement takes the user to the left in the world. In SGI’s CosmoPlayer VRML viewer, these movements are available when the dashboard has been turned off. The dashboard contains visual navigation controls that require mouse interaction in order to navigate the user. The dashboard is a user preference that can be toggled on or off by selecting the Show Dashboard item on the short-cut menu that appears when the user right mouse clicks anywhere in the world.

When navigating with the keyboard, the user is immersed visually without any artificial cues in the world. When navigating with a pointing device such as a mouse, the user sees a pointing device cursor which leads the user in the direction she is traveling. The mouse does allow more control as the user need not travel in a pure east, west, north, or south direction, but instead can move at any angle along the ground plane by moving the mouse in that direction. Sony Corporation’s CommunityPlace viewer shows a tail connected to the cursor that defines the user’s current direction.

Vertical movement while in walk mode varies substantially between viewers. Live3D and CommunityPlace use the CTRL key on the keyboard as a request for vertical movement. When the user holds the CTRL key down, the up and down arrows on the keyboard and forward and backward mouse movement move the user vertically instead of forward and back. CommunityPlace also provides a jump button that takes the user a certain distance above the ground plane and changes the viewpoint orientation to face the spot from which the user jumped. OZ Interactive Inc.’s OZ Virtual viewer requires a user to move off of the ground plane in order to fly vertically. The user then can move vertically with the pointing device and keyboard arrows. To enable this intuitive movement, the world designer must be sure to leave gaps in the ground design from which the user can get airbound. CosmoPlayer invokes vertical movement through its dashboard. The right-most control lets the user move up and down when selecting it with the pointing device.

As these viewers get more sophisticated, they are expected to improve on automatic terrain following algorithms while in walk mode. With terrain following in place, the user uses the keyboard arrows or pointing device to move forward, back, right, and left and the viewer follows the ground plane defined as the first geometry that exists directly beneath the user’s virtual feet. OZ Interactive Inc.’s OZ Virtual viewer currently does a realistic job of terrain following when the gravity option is turned on by the user.

Virtual world clients typically allow the user to change the navigation mode in order to traverse the world in a different manner. Netscape Communications Corporation’s Live3D offers five explicit modes: walk, look, spin, slide, and point. These modes are available from a simple on-screen menu in the lower left corner. The user clicks on the menu to select the mode of navigation she prefers and that mode is active until another is selected or a world is entered that modifies the mode while loading in the computer’s memory. The look mode allows the user to move the viewpoint orientation without moving her location nor changing her current feet plane. Keys and mouse movement are used to look right, left, up and down. Spin mode allows the user to spin the world vertically or horizontally, but not around the forward-back axis. Spinning changes the user’s feet plane to reflect the change of scenery. Slide mode uses the arrow keys for movement left, right, up, and down. Point mode allows the user to click on an object in the distance and allow the viewer to select a viewpoint that brings the user closer to the object selected. These five modes allow for great flexibility of navigation for the user.

In CosmoPlayer, navigation modes are changed through the short-cut menu made available with a right-mouse click anywhere in the world. Navigation mode changes change the controls available on the dashboard. OZ Interactive Inc.’s OZ Virtual viewer allows the navigation mode to be changed from the menu bar that is part of the window frame. Navigation modes are not as explicit as in Live3D’s viewer as the user turns on and off different attributes such as gravity to create the mode desired. CommunityPlace provides different controls on a picture based control panel. These controls allow a user to choose actions typical of different modes such as pointing or sliding without explicitly changing the current mode.

All the viewers allow the user to enable and disable a default light in the world. The light, called a headlight or headlamp is a directional light that emanates from the user’s forehead. Some viewers disable the default light when the world dictates. In all the browsers reviewed here, default light choices are made from a menu. CommunityPlace and OZ Virtual include the default light option from an available menu bar menu. Live3D and CosmoPlayer include the option in the short-cut menu provided the user when he or she right-mouse clicks anywhere in the world. Live3D additionally shows a headlight option on the navigation menu seen in the lower left of the screen at all times.

Collision detection defines whether a user can walk through solid objects when navigating in a virtual world. Collision detection can be enabled or disabled on the same menu as the headlight choice. Yet, Live3D does not make the current collision detection decision explicit on its on-screen menu.

Current navigation speed is typically chosen by the viewer based on the complexity of the world model. Yet, the user modifies the speed by her actions. CosmoPlayer provide an explicit menu choice on its right-mouse click shortcut menu. A user can choose slow, normal, or fast navigation speed. In the others, speed is derived by the speed of any pointing device movement and the length of time a keyboard key is kept depressed. A user accelerates as a key is held down until some terminal velocity is reached.

A note of interest is the lack of 3D objects used for navigation assistance. Only CosmoPlayer uses 3D objects in its interface, providing 3D objects on its dashboard.

Integrated Communications

For worlds with voice communications, the communications interface can be quite simple. Both OZ Interactive Inc.’s OZ Virtual and OnLive! Technologies Inc.’s Traveler viewers provide nothing more than volume control and a mute button. Volume is adjusted through two simple arrow buttons where a click on the up arrow increases the volume by a fixed amount and a click on the down arrow decrease the volume by a fixed amount. The user clicks multiple times to change the volume dramatically. A user clicks on the mute button if he or she does not want to send or receive any voice communications. Then, the client has more resources available to world rendering.

For worlds with text chat communications, the text environment is controlled by the user through adjustable multi-line text boxes and a fixed single-line text box. The multi-line text boxes show the user the chats to which he or she is active. In Intel Corporation’s, blaxxun interactive’s, OZ Interactive Inc.’s, Circle of Fire Inc.’s, Worlds Inc.’s, and Sony Corporation’s multi-user clients, the user is always privy to a global chat and then made privy to any other chats in which the user gets involved. Each of these chat boxes can be resized by dragging the chat box border with the mouse. blaxxun interactive provides an attractive point and click menu from which a user can jump between chat sessions. The user then need only make a single multi-line text box big enough for comfortable reading.

blaxxun interactive provides another communications vehicle, the business card. Each user can save a local business card with information about themselves in the real world. Then, through a menu, the user can request and send business cards to other users.

In no client that I visited did text chat communications take advantage of the 3D world for interface presentation. I found the chat environment to be very natural and intuitive given my background with text editors and word processors. Only in Circle of Fire Inc.’s Active World did I encounter the concept of a homestead where users could put their own pictures and information in the world. Users could also add a mailbox to their homestead on which other users click to send email to that homestead’s creator. A next step would be to add a telephone to each homestead in order to initiate voice communications (via telephone or, if active in the world, a voice channel).

Personal Choices

In all the clients I used for multi-user communications, options are presented to the user via traditional 2D GUI controls and context sensitive use of the right mouse button. Controls are provided via HTML controls within frames or Java based interfaces existing within an HTML frame. I found no new, innovative uses of 2D controls and thus found making option choices very intuitive for anyone skilled in using popular graphical operating systems or Web browsers. I found much innovation in what world options are made available, but not in the controls that allow choices to be chosen.

Immersion Strategies

For an immersed collaborator, the screen no longer exists. For now though, internet connected, multi-user world collaborators are more often not immersed and the screen is used extensively with the interface design. Multi-user clients take advantage of a typical collaborator’s computing experience to provide typical mouse-based controls. Menus, GUI controls (text boxes, check boxes, drop down lists, etc.), toolbars, toolboxes, palettes, dialog boxes, and combination widgets are all used to let the collaborator interact with a 3D multi-user world. The client developer usually uses a multi-windowed approach to organize the 3D world navigation, communications, and personal options selection interfaces.

Traditional desktop interfaces undoubtedly provide ease of learning to first time users, but don’t support a natural environment for an immersed participant. 3D world interface designers have an opportunity to design with both the immersed and desktop collaborator in mind. I believe users who immerse themselves in a 3D world for the first time will have a more pleasant experience if they can interact with the world in a manner similar to how they have been interacting through the desktop. Yet, immersed behaviors need to be natural in order to get a user to believe they are physically in the world and not in a computer simulation.