============================================================================= Virtual Worlds Research at Columbia University Contact: Prof. Steven Feiner Department of Computer Science Columbia University 500 W. 120th St. New York, NY 10027 feiner@cs.columbia.edu 212-939-7083 212-666-0140 (fax) Description: Our research on virtual worlds is centered about the development of new metaphors for visualizing and interacting effectively with rich information spaces. This work is performed within the context of Columbia's Computer Graphics and User Interfaces Laboratory, whose research spans a wide gamut, including knowledge-based graphics, animation, rendering, visualization, visual languages, and hypermedia. Our virtual worlds research facilities include HP 9000/7XX workstations with high-performance 3D graphics accelerators, a VPL DataGlove, several 3D tracking systems (including a four-receiver extended range Ascension Flock of Birds and eight Logitech ultrasonic trackers), StereoGraphics CrystalEyes 3D stereo eyewear, a custom-built see-through head-mounted display, and a Crystal River Beachtron 3D sound processor. Students also have shared access to a 4-processor SGI Onyx with Reality Engine and an 8-process HP 735 cluster with FDDI interconnect. Our n-Vision visualization testbed [3,4] allows users to explore abstract virtual worlds populated by objects representing functions of large numbers of variables. A ``3D window system'' partitions the physical space in which users interact -- a volume containing objects that are viewed in stereo and manipulated using the DataGlove. One current application is an example of ``financial visualization'' in which users can determine the effect of market variables on the value of financial instruments. n-Vision incorporates a novel approach to visualizing higher-dimensional data that uses nested heterogeneous coordinate systems. We are currently developing a knowledge-based ``world-design'' component that will select appropriate interaction and presentation techniques from n-Vision's repertoire [1]. We have also been experimenting with an approach to user-interface design that embeds the physically small flat panel display of a portable workstation within a virtually large information surround. The information surround is presented on our see-through, head-mounted display. A mirror beam splitter merges the user's view of the regular screen with that of the surrounding virtual world. We refer to this approach as a ``hybrid user interface'' because it attempts to combine the strengths of heterogeneous user interface technologies. Our current prototype is an X11 window manager that allows the user to move windows between the flat panel and the surround [8]. This project is being carried out in conjunction with a larger multifaculty collaboration in which we are building the software infrastructure for a 2Mbit/sec wireless mobile computing network [2]. KARMA (Knowledge-based Augmented Reality for Maintenance Assistance) [6] designs virtual worlds that explain how to operate, maintain, and repair equipment. By creating such material automatically using AI techniques, we address the tremendous human effort that currently underlies the design of ``hand-crafted'' hypermedia and multimedia presentations. This work builds on our research on the knowledge-based generation of 3D graphics [9,10] and on a multifaculty collaboration in the coordinated generation of text and graphics [7]. KARMA uses our see-through head-mounted display to create an ``augmented reality'' in which a synthesized virtual world overlays the user's view of the physical world. Our experimental domain is simple end-user maintenance for a laser printer. We attached several 3D trackers to key components of the printer, allowing the system to monitor their position and orientation, so that the physical and virtual worlds can be registered. A modified version of the IBIS rule-based illustration generation system [10] interactively designs overlaid graphics and simple textual callouts that fulfill a set of goals that are input to the system. While many virtual environments researchers have worked with simple 2D windows and control panels, we have developed support for a full X11 window system server within our augmented reality testbed [5]. The user's head is tracked so that the display indexes into a large X bitmap, effectively placing the user inside a display space that is mapped onto part of a surrounding virtual sphere. By tracking the user's body, and interpreting head motion relative to it, we create a portable information surround that envelopes the user as they move about. We support three kinds of windows implemented on top of the X server: windows fixed to the head-mounted display, windows fixed to the information surround, and windows fixed to locations and objects in the 3D world. Objects can also be tracked, allowing windows to move with them. To demonstrate the utility of this model, we have developed a small hypermedia system that allows links to be made between windows and windows to be attached to objects. Thus, our hypermedia system can forge links between any combination of physical objects and virtual windows. Program: Opportunities to participate in our research projects are available as part of Columbia's undergraduate, M.S., and Ph.D. programs in Computer Science. References 1. Beshers, C. and Feiner, S. AutoVisual: Rule-based design of interactive multivariate visualizations. IEEE Computer Graphics and Applications, 13(4), July 1993, 41-49. 2. Duchamp, D., Feiner, S., and Maguire, G. Software technology for wireless mobile computing. In IEEE Network, 5(6), November 1991, 12-18. 3. Feiner, S. and Beshers, C. Visualizing n-dimensional virtual worlds with n-Vision. Computer Graphics, 24(2), March 1990 (Proc. 1990 Symp. on Interactive 3D Graphics, Snowbird, UT, March 25-28, 1990), 37-38. 4. Feiner, S. and Beshers, C. Worlds within worlds: Metaphors for exploring n-dimensional virtual worlds. Proc. UIST '90 (ACM Symp. on User Interface Software and Technology), Snowbird, UT, October 3-5, 1990, 76-83. 5. Feiner, S., MacIntyre, B., Haupt, M., and Solomon, E. Windows on the world: 2D windows for 3D augmented reality. Proc. UIST '93 (ACM Symp. on User Interface Software and Technology), Atlanta, GA, November 3-5, 1993, 145-155. 6. Feiner, S., MacIntyre, B., and Seligmann, D. Knowledge-based augmented reality. Communications of the ACM, 36(7), July 1993, 52-62. 7. Feiner, S. and McKeown, K. Automating the generation of coordinated multimedia explanations. In IEEE Computer, 24(10), October 1991, 33-41. 8. Feiner, S. and Shamash, A. Hybrid user interfaces: Breeding virtually bigger interfaces for physically smaller computers. Proc. UIST '91 (ACM Symp. on User Interface Software and Technology), Hilton Head, SC, November 11-13, 1991, 9-17. 9. Karp, P. and Feiner, S. Automated presentation planning of animation using task decomposition with heuristic reasoning. Proc. Graphics Interface '93, Toronto, Canada, May 17-21, 1993, 118-127. 10. Seligmann, D. and Feiner, S. Automated generation of intent-based 3D illustrations. Computer Graphics, 25(4), July 1991 (Proc. ACM SIGGRAPH '91, Las Vegas, NV, July 28-August 2, 1991), 123-132.