GreenSpace: Creating a Distributed Virtual Environment for Global Applications

Paper presented at IEEE Proceedings of the Networked Reality Workshop, October 26-28, 1995, Boston, MA

© 1995 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Contents:

Abstract

The GreenSpace program began in April, 1993 and continues at the present as a collaborative effort among the Human Interface Technology (HIT) Lab at the University of Washington, Seattle, USA, the Fujitsu Research Institute (FRI) of Tokyo, Japan and Fujitsu Limited of Japan. The purpose of the GreenSpace program is to develop and demonstrate a new global communications and information environment for the twenty-first century. We envision creating sensory-rich, highly interactive, virtual environments (GreenSpaces) that can link human senses and minds across the world. We believe a new communications medium of this type will: 1) help the peoples of the world transcend geographic, language and cultural differences; 2) facilitate global collaboration to solve pervasive problems and realize untapped economic opportunities; and 3) make communication more efficient and conserve energy resources by moving our minds and information around the world at the speed of light, instead of moving mass (paper, ourselves and so on).

The first phase of our GreenSpace work culminated in November, 1994 in a public (proof of concept) demonstration between Tokyo, Japan and Seattle, USA. The demonstration site in Tokyo was the NICOGRAPH Convention (the Japanese equivalent of SIGGRAPH), and the Seattle site was the HIT Lab. Four participants, two at each location, were able to interact within a shared virtual environment. We are now in the second phase of our GreenSpace work and are focusing on software infrastructure and application development. In this endeavor we have collaborated informally with the Fraunhofer Institute for Industrial Engineering (FhG-IAO) in Stuttgart, Germany.

We are taking a systems and concurrent engineering approach. We have established a cross-disciplinary team that includes technologists, interface designers, human factors scientists, application designers, content providers, users and business interests. Our principle objective is to build a prototype GreenSpace network that integrates emerging virtual interface technologies, multimedia, high performance computing and broadband telecommunications. The GreenSpace network will be our vehicle to explore the requirements for and feasibility of a global virtual community and its application to real-world problems and economic opportunities.

This paper overviews the GreenSpace concept, our previous GreenSpace work, a synopsis of related research projects, and closes with our future directions.

Introduction

The GreenSpace program began in April, 1993 and continues at the present as a collaborative effort among the Human Interface Technology (HIT) Lab at the University of Washington, Seattle, USA, the Fujitsu Research Institute (FRI) of Tokyo, Japan and Fujitsu Limited of Japan. The purpose of the GreenSpace program is to develop and demonstrate a new global communications and information environment for the twenty-first century. We envision sensory-rich, highly interactive, virtual environments (GreenSpaces) that can link human senses and minds across the world. We believe a new communications medium of this type will: 1) help the peoples of the world transcend geographic, language and cultural differences; 2) facilitate global collaboration to solve pervasive problems and realize untapped economic opportunities; and 3) make communication more efficient and conserve energy resources by moving our minds and information around the world at the speed of light, instead of moving mass (paper, ourselves and so on). In the following sections we overview the GreenSpace concept, our previous GreenSpace work, related research and other activities and conclude with our future directions.

GreenSpace Concept Definition

In this section we overview the goals, current environment and motivation for GreenSpace. We then introduce the concept of a virtual green or GreenSpace, overview system components necessary to implement the GreenSpace concept, and summarize our GreenSpace development strategy. Finally, we overview the concept of the GeenSpace Research Center -- itself a GreenSpace -- dedicated to furthering the development of information and communications technology and demonstrating solutions to global problems that can be solved by such technology.

Goals of the GreenSpace Program

The long term objectives of the GreenSpace program include:

Objective 1. Develop distributed virtual environment concepts, components and systems which will provide an experimental system for the demonstration and evaluation of an international virtual community.

Objective 2. Establish and demonstrate a GreenSpace network linking the United States, Japan and other selected countries.

Objective 3. Develop representative applications that clearly demonstrate the far reaching advantages and implications of integrating virtual interface technologies with broadband telecommunications.

Objective 4. Establish a center of excellence for advanced information and communication technology -- implemented as a distributed virtual world in which the participants themselves are linked by broadband communications technology.

Global Environment

We are poised at a special time in the history of our planet. There are pervasive problems affecting the citizens of the earth, some regional in nature (such as hunger and disease) whereas others affect us all (e.g. global warming, depletion of natural resources, pollution of our ecosphere, natural disasters such as earthquakes, and so on). Another area of current concern is the state of the world's economy. It is clear that for any nation's economy to grow and prosper, the market for their products must also be healthy. Additionally, there is an opportunity to recapture world citizens whose contributions to society are compromised by physical and cognitive disabilities.

The impetus for this project is to promote cooperation among world citizens to solve the pervasive problems that confront us and exploit the opportunities that lie before us. Discussion, mutual understanding and consensus in dealing with these issues must be preceded by greater awareness, education, and communication among nations. To gain this understanding and cooperation we must overcome a number of barriers which include: inability to communicate because of geographic remoteness, cultural differences, language barriers, and the limitations of existing computing, communication, information and interface technologies.

Need For A New Communication Environment

To overcome the above barriers and solve problems of global importance, a new environment for communication of ideas and understanding must be developed. Currently, the primary method for interactive communication is to move people from one place to another, to attend meetings, and so on. We are expending the energy resources of the earth to move mass around just to move information.

Fortunately, an evolving confluence of technologies (high performance computing, communication, information and human-machine interfaces) and its potential to solve real world problems is recognized at all levels in science, engineering, the government and society at large. For example, in the United States, the IEEE identifies in the following elements in its Grand Challenges in Electrotechnology for the Twenty-first Century:

In addition, the Federal High Performance Computing and Communications (HPCC) Program was created in 1991 to accelerate the development of future generations of high performance computers and networks. The HPCC program identifies both Grand Challenges and National Challenges [HPCC]. The Grand Challenges address scientific and engineering problems such as forecasting the weather, designing life saving drugs, and understanding how galaxies are formed. National Challenges are major societal needs in key areas such as health care, manufacturing, digital libraries, education and lifelong learning, electronic commerce, energy management, the environment, national security, and government services.

HPCC technologies provide a technical infrastructure with the potential to empower humans further in such areas as accelerating learning, enhancing our ability to create, extending communications and recapturing world citizens who have been lost due to physical disabilities and illiteracy. However, technology alone does not solve problems -- people solve problems, using the tools that are available. Most effective problem solving occurs when there can be a "melding" of minds, sharing of ideas, directing of enterprise. In this regard there is no substitute for coming together in a common place where understanding can be engendered and ideas exchanged.

The Concept of a Virtual Green

Throughout the ages we have all enjoyed the opportunity to come together in beautiful places such as parks and village greens. These places are often improved and landscaped for public use, often within a built-up area. These places are often used for peaceful enjoyment, relaxation and interaction with friends and acquaintances. We can also think of other types of commons such as: learning places such as classrooms, libraries and museums; business commons such as corporate conference rooms; entertainment commons such as golf courses and theaters. In each case, these are places where we come to rest, to recreate, improve ourselves or perhaps earn our livelihoods. Each of these commons require us to physically go to a place to experience the ambiance and interact with other participants in the common. We propose the concept of a virtual green or GreenSpace, defined as "a shared environment in which participants are immersed in a sensory medium which is electronically mediated using virtual interface components that are interconnected through a high bandwidth network." In a virtual common we can provide the mechanization of "being there" by sending bits of information rather than burning calories.

System Overview

From a highlevel system perspective, the construction and operation of GreenSpaces requires the following components:

Greenspace Virtual Environment Workstation: The GreenSpace (GS) workstation includes all the human-machine interface and computational components required to mediate the virtual experience for a single participant. In general the GS workstation architecture should accommodate sensory-rich multimodal interfaces that seamlessly integrates components such as: 3D stereo visual display, spatial sound, speech recognition and synthesis, position tracking, touch and gesture input with force feedback. The exact configuration and capability of a GS workstation will depend on the cost and performance requirements associated with the target application.

GreenSpace Network Operating System: The GreenSpace Network Operating System (GNOS) manages the local operation of each individual workstation and the operation among all the workstations that are connected over the GreenSpace network. In general, GNOS is a software environment built on top of and makes use of more generic distributed operating and communication facilities. From the perspective of the participant, GNOS must insure that the look, feel and behavior of the virtual experience is consistent with expectation. From the technical perspective, GNOS must manage real-time, concurrent objects on a large scale in a distributed computing environment. GNOS must transparently provide the communication services withthe Quality of Service required by the application (that is, insure bandwidth, latency, reliability, consistency, and so on).

World building tools and expert team: To build a virtual environment takes a rich suite of generic and application-specific tools: interface widgets, interface configuration, CAD modeling of spaces and structures, animation, behavior modeling and so on. World building is a very labor and expertise intensive undertaking and is likely to remain so for some time to come. Inexpensive and powerful tools are not generally available and the integration of tools is sorely lacking. In addition to tools, world building is inherently a cross-disciplinary enterprise. One must bring together the arts, sciences and engineering, users and business interests.

Application content: Content is modeled and implemented as objects with both attributes (data) and methods (behaviors). For example, content includes your avatar (your virtual self), the geometric model of the space your avatar is occupying and the allowable behavior between your avatar the space -- for example, opening a door. Application content must satisfy not only functional requirements in the technical sense but also be participant-friendly and aesthetically pleasing.

Technical infrastructure: The GS workstation, GNOS, World building tools and application content are all built on top of a technical foundation that includes: virtual interface concepts and components, multimedia, distributed computing, distributed objects and databases, computer networking and telecommunications.

Development Strategy

Our overall technical strategy is as follows:

The GreenSpace Research Center

We often set up "world centers of excellence" to bring together the best scholars, technologists, and government policy makers to study pervasive problems, formulate strategies, establish agenda and execute plans for building a better world. Conventional centers require relocation and transport of people and building of new facilities. This is expensive, disrupts the lives of participants, and often excludes some of the most qualified participants.

We propose to create a GreenSpace Research Center, a center of excellence implemented as a virtual environment. The mandate of this center would be to promote the further development of information and communications technology and demonstrate solutions to global problems that can be solved by such technology. A unique aspect of the GreenSpace Center is that -- because it is virtual -- it becomes the means to create other centers of excellence. The GreenSpace Research Center is virtual community where minds and senses may be linked with a singleness of purpose.

Summary of Previous GreenSpace Work

Our previous GreenSpace effort was divided into two distinct phases: GreenSpace I focused on the GreenSpace concept definition and demonstration. It began in April 1993 and culminated in November 1994 in a public (proof of concept) demonstration between Tokyo, Japan and Seattle, USA. GreenSpace II is work in progress and is currently focusing on software infrastructure and application development. It began in April 1995 and will be completed in March 1996. We are also currently working on our long-term development strategy that builds on GreenSpace I and II and will take us into the twenty-first century:

GreenSpace I

The main technical objective of GreenSpace I was to take the first step toward defining, developing and demonstrating an immersive communications medium where distant participants feel a sense of presence and are able to communicate in a shared virtual environment. In addition, we began to analyze and specify requirements and protocols that would enable collaboration at a distance among hundreds of participants who are immersed in an environment rich in visual, aural and tactile cues.

The GreenSpace I work culminated in a public (proof of concept) demonstration that linked Tokyo and Seattle within a real-time virtual environment during November 14-17, 1994. The demonstration site in Tokyo was the NICOGRAPH Convention (the Japanese equivalent of SIGGRAPH), and the Seattle site was the HIT Lab. Four participants, two at each location, were able to interact within a shared virtual environment.

Demonstration

Two participants at the Seattle site and two participants at the Tokyo site were immersed in a virtual three dimensional environment designed to play a simple game of herding animated creatures on a table top. The creatures were small animated figures that could move about autonomously and would move away from individual participants; they could be herded only if surrounded by all four participants. At each site the participants were immersed in a three dimensional virtual room and seated at a small table across from one another. Each participant saw a texture-mapped image (head and shoulders snapshot) of their local counterpart across from them and their remote counterparts on their respective left and right. The Seattle participants saw an oriental room and the Tokyo participants saw an occidental room. The rooms and tables had the same physical layout and scale but were texture-mapped differently to reflect the two different environments. All participants were headtracked and handtracked. Each participant had a virtual paddle to move the creatures. Exit questionnaires were taken.

Network configuration: The network configuration for the demonstration is shown below:

Network Configuration

Site Configuration: The site configuration for the demonstration is shown below:

Site Configuration

Lessons Learned

Below we summarize key issues associated with the Virtual Experience, the Experimental Method, Application Testbed, Development Testbed, and Scalable Environments. We defer discussion of other technical issues to the section System Issues.

The Virtual Experience

Presence

Quality

Communication

Experimental Method Although public demonstrations are important to visibility, they do not add significantly to our scientific and technical knowledge. Public relations demonstrations require a substantial effort and resources to do properly -- yet they yield little statistically meaningful information. They are primarily designed to accommodate a large volume of people (Disneyland style) rather than to conduct controlled experiments that yield statistically meaningful information.

Development and Application Testbed We need to establish a development testbed that will support rapid prototyping of concepts and implementations as well as an ongoing series of experimental trials to verify and validate the prototypes. It is necessary to focus on real applications and construct a prototype virtual environment that enables real users to do real things in real work settings. To truly advance our understanding and technology, we need to choose applications that are visionary in scope, whose requirements are representative, and that can be developed in an incremental manner.

Scalable Environments One must carefully consider which characteristics of an environment must be scalable (for example, the number of participants, the geographic proximity, the number of objects and behaviors, the types of interaction, and so on) and the implications of scalability on such things as real-time response, model consistency and reliabilty. It is clear that even relatively simple environments can require significant communication bandwidth even when only a few virtual objects and humans are involved (if they are rapidly changing state and all state change information is broadcast to all remote sites and participants). Different applications can have very different performance and functional requirements. It is clear that the optimal design of distributed virtual environments is both application and technology dependent. One size will not fit all.

GreenSpace II

The focus of GreenSpace II is: 1) to develop a communications software infrastructure for GreenSpace and 2) to develop and demonstrate a representative application. To that end we are developing GSnet and a virtual collaborative environment for Architectural Design:

GSnet is an object-oriented C++ class library that supports communication and a shared database among distributed applications. One of the primary design goals for GSnet is to support scalable, multi-participant virtual environments. As suggested by Macedonia [MAC] and others ([ROEHL], [PULKKA]), GSnet employs multicast groups and provides a mechanism that supports the concept of an Area of Interest Manager]:

"to logically partition virtual environments by associating spatial, temporal, and functionally related entity classes with network multicast groups ... and ... [to focus] an entity's processing and network resources to its area of interest via an Area of Interest Manager" [MAC].

While still experimental, GSnet is designed to use unicast and multicast IP and currently runs over MBone on the Internet. We are developing a local area testbed at the HIT Lab and are working with our international partners to establish a global testbed. GSnet is compatible with the dynamic creation, modification, and deletion of objects associated with multicast groups.

In addition to employing multicast groups, another important element to scalability is efficient distribution of objects and computation associated with operating virtual worlds. If worlds are large, it makes sense to distribute only that part of a world with which a participant actually interacts. Multiple client/single server architectures with centralized databases, while convenient from the programming point of view, do not scale well because of a communication and computation bottleneck associated with centralized resources. GSnet implements a peer-based, distributed database and computation philosophy that can support a range of application-specific architectures.

Using early mock-ups of GSnet for his Master's thesis, Pulkka demonstrated at the HIT Lab the concept of "Spatial Culling of Interpersonal Communication within Large-scale Multi-user Virtual Environments" [PULKKA]. This work demonstrated the control of interactions among multiple participants as they move about in three dimensional space based on switching between multicast groups associated with bounding boxes in a three dimensional space. The mock-up uses the OpenInventor environment and runs on Silicon Graphics workstations over the HIT Lab's local net and has been run over the campus network at the University of Washington between the Computer Science Department and the HIT Lab.

In addition to GSnet, we have selected Architectural Design as our first major application domain and are currently investigating the requirements for a Virtual Design Studio for the design of architectural structures and spaces. We discuss this further in the section Virtual Design Studio for Architectural Design. We have selected Architectural Designfor two reasons: first, because it is a good application of virtual environments in its own right and second, because the expertise and tools associated with designing physical structures in virtual space are -- to a large extent -- the same expertise and tools one needs to design virtual spaces in virtual space. Software tools and environments for a Virtual Design Studio are being integrated into the GreenSpace software infrastructure. (Note: The same argument holds for other content creation tools as well, for example, graphic arts, film, drama, communications, sound and music etc. All have an important role in developing virtual environments).

We are also developing advanced interface concepts. For example, when available, we plan to use advanced 3D visual interface technologies such as the Virtual Retinal Display [VRD] under development at the Human Interface Technology Lab. In addition, we plan to investigate the use of multimodal interfaces that include composite video and real-time graphics, spatial sound, speech recognition and synthesis, gesture recognition, and haptic devices.

GSnet Overview

We are using object-oriented methods to develop our software infrastructure. GSnet is a prototype object-oriented C++ class library that supports networked communication and a shared database among distributed applications. GSnet is designed to use both unicast and multicast IP and currently runs over MBone on the Internet. From the perspective of each participant or software client (local or remote), the GSnet presents an individual worldview of the shared GreenSpace world. The GreenSpace world, from the perspective of network communications, is made up of internal parts, those managed by Mr. N., and external parts, those managed by external protocols, such as Real-time Transport Protocol (RTP)-based services, Video Audio Tool (VAT) audio, Network Video (NV) video, the experimental Reliable Multicast Protocol (RMP), or any other external services. Mr. N. is a special process on each machine. Mr. N. is the only process which communicates outside the host.

The world data structure of a GreenSpace environment is based on groups, which are collections of chunks. Chunks are objects of specialized types that define the data and methods for a GreenSpace world. Chunks are organized into groups and clients subscribe to groups. A client process, which has a worldview of the collection of groups it is interested in, manipulates the chunks in that group. The client's actions are reflected to other local clients and to the network by Mr. N. If the client becomes interested in another group, Mr. N. will begin to cache it (if it hasn't already), and transfer the contents of the group to the client. The network of Mr. N.s insures database consistency among the clients. Each client caches only those groups of interest to it. Mr. N. caches the union of the groups subscribed to by its clients. The Software Hierarchy for GSnet is depicted below:

Software Hierarchy

Software Hierarchy The main elements:

The chunkCollections figure below illustrates how an application can organize and manage all the groups and chunks in its worldview by creating objects of the chunkCollection class. ChunkCollections determine the structure of a GSnet-based simulation. GSnet itself simply offers an amorphous blob of chunks and groups, linked together with special chunks called GANNCs which are references to other groups. (A GANNC is analogous to a URL on the World Wide Web). This is suitable for self-contained worlds, one world per group; essentially the equivalent of nonscalable multi-participant VRML. However, real scalability depends on factoring and distributing worlds in such a way that an application can dynamically cache, compute on and interact with only a subset of world state and behavior of interest to the application.

chunkCollections

Chunk Collections

Each participant might be located in one group, and have objects from surrounding groups in view. A chunkCollection might use special chunks in each group to relate virtual space to the network of groups. The collection would then provide a pseudo-group to the client, which hides the group structure from the client programmer. Pseudo-groups are useful for three reasons. First, a client application may have different worldviews, each tailored to a separate requirement of the application, and independent of the actual network worldview of the client. Second, the pseudo-groups are automatic: as chunks leave and enter the groups, and as the chunkCollection chooses to coalesce new groups into the pseudo-group, the matching chunks automatically appear and disappear from the pseudo-group. Finally, the network implementation of group organization is encapsulated, so that as the structure cues are improved and modified, the client software continues to work without modification.

Currently GSnet runs on SGI under IRIX and DEC/alpha under OSF/1. We have conducted initial local tests between SGIs, between Alphas and between SGIs and Alphas. We also established connection with the Fraunhofer Institute in Germany and demonstrated basic GSnet operation between the HIT Lab and the Fraunhofer Institute.

We are working on enhancements to GSnet and the software layers above GSnet to support the shared virtual environment for collaborative architectural design; enhancements include generic GS services and application specific functionality. Generic services include such things as database management, rendering, interaction protocols and device drivers. We are currently using OpenGL, OpenInventor, VRML and related software as our primary programming interfaces and development tools.

Virtual Design Studio for Architectural Design

The HIT Lab is fortunate to be able to collaborate with the College of Architecture and Urban Planning (CAUP) and the Community and Environmental Design Simulation Lab (CEDeS) at the University of Washington. We are jointly developing the application of GreenSpace to Architectural Design in the context of a Virtual Design Studio. Over the long run, this effort may involve other architectural schools including the University of British Columbia, the University of Hong Kong, MIT, and others.

We have selected Architectural Design not only because it is a good application of virtual environments in its own right but also because the expertise and tools associated with designing physical structures in virtual space are -- to a large extent -- the same expertise and tools one needs to design virtual spaces in virtual space. In addition, Architectural Design is a complex process and profession. For example, there are many different types of communication and collaboration, different tasks, different participants, different workstation capabilities, and different communication mediums:

  • 3D spatial immersion and presence is important to the aesthetics and human factors of design. While experienced architects may be good at imaging architectural spaces, novices, builders, and clients may not be. Since buildings are expensive and consume considerable time and physical resources to construct, sophisticated CAD tools -- and by extension, virtual environments -- are justified if they significantly improve the quality of the design, customer satisfaction and/or reduce costly rework of physical structures.

  • Fly throughs, walk throughs and dynamic "what if" changes are central to the understanding and "experiencing" of architectural spaces which is critical to the overall design process. Virtual architectural spaces may be simple or complex with respect to both their geometry, their autonomous behavior and their interaction with humans.

  • Different levels of expertise and types of collaboration are required among architects, clients and ancillary staff. Many different people are involved in a collaborative process.

  • Both qualitative and quantitative information is involved, as well as representations of abstract information and physical structures.

    Related Research Projects and Other Activities

    Research Projects There are a number of government and academic research projects around the world which are working specifically on distributed virtual (also called synthetic) environments. Foremost in the US is the work by the Department of Defense on SIMNET, DIS and NPSNET. A good place to start is at the home page of the Naval Post Graduate School [NPS]. A few good places to start in the academic world include [SICS], [NOTT] and [GS]. See also [HEN] for pointers to various Web sites.

    Commercial ventures There are number of commercial research and product ventures of interest. For a start see [VRML], [DIVE], [MAK], [DIV], [MERL], [WORLDa], and [WORLDb].

    Architectural Design There are a number of fascinating and important projects underway associated with applying virtual reality techniques to Architectural design, as well as the notion of a Virtual Design Studio. A good place to start is [PRES] and [WOJ]. A few good places to start on the World Wide Web include Architecture Schools at the University of Washington, MIT, UCLAand other: Virtual Sculptorhttp://caad.arch.ethz.ch/~kurmann/sculptor.html, VoxDesignLhttp://plato.informatik.hab-weimar.de:8080/VoxDesign/

    World Wide Web The World Wide Web is a happening. As limited as the current HTML and HTTP protocols are, we see the Web phenomena as the primitive beginning of cyberspace, a virtual environment that ultimately will support the kind of interactive virtual environments we envision for GreenSpace. The first rudimentary but significant steps are already being taken based on the Virtual Reality Modeling Language ([VRML]) and Java (Sun Microsystems, Inc.) [SUN] to extend the protocols to support 3D geometry and objects that can be dynamically distributed and executed. Conceptually, these are some of the key elements of the technical infrastructure that eventually will enable the construction and operation of GreenSpace virtual environments.

    Modeling and content creation: The Keystone Initiative Silicon Studio, an SGI spin-off company, with support from the world's leading makers of animation and content creation software, announced the technical specifications for the Keystone initiative. The Keystone initiative will result in an open environment for software tools that includes the definition of an application framework. The Keystone initiative breaks new ground by creating a common interface metaphor for 3D modeling and environments, dramatically broadening the acceptance of next-generation 3D technology in content creation.

    System Issues

    Even the simplest application of distributed virtual environments requires a complex and sophisticated integration of disparate concepts and technologies. The conception and construction of virtual environments is inherently multi-disciplinary and -- in the broadest sense -- is based on principles of perception, psychology, philosophy, and physics. In concrete terms, the definition and specification of a virtual environment begins with a thorough understanding of the application, the role of the humans, the role of the machines and the interaction of all the elements to achieve the goals of the application. We believe that the design and development of useful virtual environment technologies and systems should be driven by a thorough assessment of application requirements, devising new interface concepts that match the requirements and understanding the fundamental human factors associated with those interfaces. The science and technology of virtual environments is a new field that requires much work. Collectively, we have just begun to explore the basic human factors, interface protocols, component technologies and system concepts that optimally match and augment human capability through an electronically-mediated sensory experience.

    Components of technologies exist or are in development that either directly or indirectly address the conception, construction and operation of distributed virtual environments. These components are far from complete and there is by no means coherence among them. Invention or innovation is required in most, if not all areas. Below we summarize some of the key technical issues, current technology and related developments. See the writings of Maceonia [MAC], Brutzman [BRUT], Roehl [ROEHL], and [PRES] for overviews of some of the technical issues. See also [HEN] for the distributed VR mailing list and [VMRLa] for the VRML repository.

    Distributed operating system, objects and databases The technical base requires a distributed operating system for the efficient management of distributed objects. Such an operating system must support object creation, concurrent and parallel operation, and persistence, all on a large-scale. The operating system must provide real-time services and -- more generally -- guarantee quality of service among distributed objects. It must efficiently support synchronous and asynchronous services, as well as anisochronous and isochronous data and media streams. All these topics are active areas of much research and development. Currently we are far from the ideal operating system but progress is being made and it appears to be only a matter of time until the required technical base is in place.

    A standard Interface Definition Language is needed for specifying attributes and behaviors and interface services on each host that can interpret these specifications dynamically. This is also a very active area of research and substantial commercial development efforts exist that are motivated by transaction-based enterprise-wide business applications. In addition, interactive multimedia and interactive TV services dictate standards associated with media objects including streams, events and other data types. Existing and emerging standards such as CORBA, DCE, OpenLook, COM/OLE, Magic Cap/Telescript, Taligent , ScriptX, PREMO, Sun Java and others are addressing various aspects of this complex problem.

    Networking and telecommunications Athough there are still technical issues that need to be resolved (e.g. IP congestion control and IP multicasting), ATM (Asynchronous Transfer Mode) is currently the leading technology for Wide Area Networks (WANs). ATM typically runs on top of SONET (for example, an OC-48 rated fiber can carry 2.4Gb/s). Commercial companies are already planning to deploy wavelength mutiplexed 40 gigabit/sec backbones in the next few years and research is underway to develop Terabit/sec optic networks. ATM/SONET to the curb at OC-3 rates (155 Mb/s) is planned in the near future.

    LANs are still the domain of computer vendors, where ethernet (10Mb/s) and connectionless protocols like IP are standard. The transition to higher speed networks is full of competing technologies. For example: switched ethernet (10Mb/s ~ 30Mb/s), FDDI (Fiber Distributed Data Interchange, 100Mb/s), 100BaseT (100Mb/s) and 100VG-AnyLAN (100Mb/s). Fibre Channel is another emerging technology that is highly scalable to gigabit/sec rates and supports multiple protocols. At present there are no clear leaders in the high-speed LAN technology. Existing workstations can not sustain the 100+Mb/s transfer rates that these high speed networks promise. Typically, modern workstations can sustain 40Mb/s to 60Mb/s. We expect that the networking capabilities of participating sites to vary greatly, from the low end of the spectrum of phone lines (28.8Kb/s using modems), to ISDN (128Kb/s), T1, Frame Relay, all the way up to ATM over OC-3 (155Mb/s) or higher bit rates.

    Finally, whatever the physical medium employed, network communication depends on a variety of protocols to insure the necessary Quality of Service, including such protocols (and their successors) as: TCP, UDP, IP, Mbone, Reliable Multicast Protocol, Real-time Transport Protocol, Network Time Protocol, Real-time Control Protocol, and so on.

    Distributed Virtual Environments A distributed virtual environment operating system and development environment must support a rich set of object classes and behaviors that support complex geometry and interactions among humans and software entities. In addition, the environment should support distributed computing and communication that makes efficient use of computing resource and communication bandwidth Existing technology is a step in the right direction, but has a long to go (see [DIVE], [DIV], [SICS], [MERL], [NPS] for example).

    Distributed virtual environment operating systems and development environments are built on top of distributed object-oriented operating systems. Since the operating system technology of the type discussed above is not yet generally available, current virtual environment operating systems are built upon existing operating systems, primarily flavors of UNIX, but also PC/DOS, MS Windows/NT, IBM OS/2 and others. The limitations of existing operating systems significantly limits the performance and capability of existing distributed virtual environment operating systems and development environments. However, we expect that they will evolve to take advantage of new operating system architecture and services as they become available.

    Programming Standards Our modeling work has taught us the importance of human readable files which specify geometric objects. The Virtual Reality Modeling Language (VRML) is a new, evolving standard for 3D geometric modeling that has been adopted by the World Wide Web community. Currently VRML 1.0 does primarily address static geometry and links among VRML objects and lacks specification of object behaviors. However another recent development of importance is the release of Java (from Sun Microsystems). Java is an object-oriented language and environment designed to support the dynamic export and interpretation of object methods that is operating system and platfrom independent. It probably will be included with future VRML specifications. (See [VRML], [SUN], [DIVE], and [DIV].)

    Graphics hardware Currently our highest end graphics system is a Silicon Graphics Onyx RealityEngine 2. This Onyx is rated at just under one million textured, Z-buffered, Gouraud-shaded triangles per second (1 Mt/s). When the Onyx was introduced two years ago, the system cost well over $250,000. Today, one can buy an Onyx with the same performance for $70,000. Recently there have been several alliances of Computer Image Generator (CIG) manufacturers and game companies (for example, Silicon Graphics with Nintendo, Martin-Marietta with SEGA). It is expected that these collaborations will produce systems with performance in the 100Kt/s range for a very small fraction of the cost of an Onyx.

    As personal computers (based on chips from Intel, SGI, DEC, IBM/Motorola) approach modern workstations in performance (e.g. 100-200 million instructions per second), sophisticated operating systems can be supported (e.g. UNIX, Linux, WindowsNT, OS/2), and the CPU can be used to speed up the computationally intensive graphics tasks. The bus architecture on personal computers is also improving considerably: the PCI bus will be supported on most of the next generation personal computers, giving an increase in I/O throughput. Several third party board manufacturers are already offering PCI-bus based 3D graphics accelerator cards that have performance similar to $50,000 graphics workstations a few years ago. These boards range in price from about $600 to $10,000. One reason for the proliferation of inexpensive 3D accelerator boards is the development of the GLINT chip from 3Dlabs, which is fully compatible with OpenGL (Silicon Graphics open graphics API).

    Graphics Software OpenGL is a 3D graphics library from Silicon Graphics that is quickly becoming the most dominant programming toolkit in the industry, surpassing other software such as 3DR, PHIGS, PEX, and Renderware. The machines supported by OpenGL licensees constitute a large percentage of the graphics workstation marketplace, as well as the majority of the PC market. The recent merger of Silicon Graphics Inc. with 3D modeling giants Alias and Wavefront will help keep OpenGL as the toolkit of choice at present time. APIs on top of OpenGL such as OpenInventor (Silicon Graphics) are also important, especially because VRML is derived from OpenInventor.

    Other Hardware Advances

    Head mounted displays : The HIT Lab is currently working on the Virtual Retinal Display [VRD], a compact visual display based on directly scanning an image on to the retina of the eye. More conventional LCD-based displays have substantially improved in price/performance. One such device is the I-Glasses from Virtual I/O.

    Tracking devices : Currently priced at only $800, including a three degree-of-freedom (DOF) head tracking device, the Virtual I/O I-glasses offers the possibility of building a six DOF tracker for under $5,000. (Most six DOF trackers cost around $10,000).

    3D audio : Spatialized audio is an important part of a multisensory virtual experience. With the advent of cheap digital signal processing chips, boards capable of spatializing several simultaneous sound sources are under $1,000, down from $10,000-$20,000 a few years ago.

    Haptic interfaces : Force feedback devices are starting to come out of labs into the market, for a few thousand dollars.

    Summary and Future Research

    We propose the creation of a new global communications and information environment for the twenty-first century. We envision creating sensory-rich, highly interactive, virtual environments (GreenSpaces) that can link human senses and minds across the world. We believe a new communications medium of this type will: 1) help the peoples of the world transcend geographic, language and cultural differences; 2) facilitate global collaboration to solve pervasive problems and realize untapped economic opportunities; and 3) make communication more efficient and conserve energy resources by moving our minds and information around the world at the speed of light, instead of moving mass (paper, ourselves and so on). There are many challenges that must be met to achieve the future we envision. On the technical side, invention or innovation is required in all areas including virtual interface technologies, multimedia, computing, communication, information and system integration. All these areas are exciting and fertile research topics for the foreseeable future. On the application side, development of sophisticated virtual environments is expertise and labor intensive and requires coordinated effort among cross-disciplinary teams. In addition to the above, there are cultural, governmental and economic challenges. To address all these issues we propose the creation of the GreenSpace Research Center, a virtual environment that can bring together representatives from all walks of life and professions to define the new communication concepts and technology for the twenty-first century.

    From a development standpoint, we believe it is essential to take a concurrent engineering approach and establish a development testbed that will support technology innovation and applications development. We are focusing our efforts on those key enabling technologies and applications that build on the strengths of the HIT Lab and those of our partners. In particular, critical areas of research include: 1) interaction protocols (human to human and human to virtual objects); 2) advanced interface devices such as portable, high-definition 3D visual displays (such as the Virtual Retinal Display [VRD]); 3) multimodal interfaces; 4) world building tools; and 5) software infrastructure.

    We are currently formulating our plan to take us into the twenty-first century. Whatever the future holds, we are excited to be a part of inventing the future and to have the opportunity to work with our colleagues from all over the world to bring these concepts into reality.

    References

    [BRUT] Brutzman, Donald P., Macedonia, Michael R., and Zyda, Michael J. "Internetwork Infrastructure Requirements for Virtual Environments," submitted to the NII 2000 Forum of the Computer Science and Telecommunications Board, National Research Council, May 23-24 1995, Washington, DC. (http://www-npsnet.cs.nps.navy.mil/npsnet_publications.html).

    [DIV] "Division Limited Home Page" at http://www.division.co.uk/.

    [DIVE] "Dive Laboratories Inc." at http://www.divelabs.com/

    [GS] "GreenSpace Project" at http://www.hitl.washington.edu/projects/greenspace.

    [HEN] Hensbergen, Eric Van. "Distributed Virtual Reality Resourceshttp://www.csh.rit.edu/~airwick/dist.html ).

    [HPCC] "Technology for the National Information Infrastructure" at http://www.hpcc.gov/blue95.

    [MAC] Macedonia, Michael R. "Exploiting Reality with Multicast Groups: A Network Architecture for Large-scale Virtual Environments" in IEEE Computer Graphics and Applications, September, 1995. pp. 38-45. (HTML Document).

    [MAK] "Mak Technologies" at http://www.mak.com/.

    [MERL] "Mitsubishi Research Labs" at http://www.merl.com/.

    [NOTT] "Communications Research Group" at http://www.crg.cs.nott.ac.uk/.

    [NPS] "NPSNET Research Group" at http://www-npsnet.cs.nps.navy.mil/npsnet/index.html.

    [PULKKA] Pulkka, Aaron. "Spatial Culling of Interpersonal Communications within Large-Scale Multi-User Environments". Master of Science Thesis in Computer Science and Engineering at the University of Washington. 1995.

    [ROEHL] Roehl, Bernie. "Distributed Virtual Reality -- An Overview" at http://sunee.uwaterloo.ca/~broehl/distrib.html.

    [PRES] <Entire issue>. Presence, Vol 4, Num 3, Summer, 1995.

    [SICS] "Swedish Institute of Computer Science (SICS) Distributed Interactive Virtual Environment (DIVE)" at http://www.sics.se/dce/dive/dive.html.

    [SHAVE] "The Shave Project" at http://os.sri.com/research/shave.shtml.

    [SUN] "Java (tm): Programming for the Internet" at http://java.sun.com/.

    [VRD] "Virtual Retinal Display" at http://www.hitl.washington.edu/projects/vrd/.

    [VRML] "VRML Repository" at http://www.sdsc.edu/vrml/.

    [WORLDa] "Worlds Away" at http://www.worldsaway.com/.

    [WORLDb] "Worlds" at http://www.worlds.net/.

    [WOJ] Wojtowicz, Jerzy. Virtual Design Studio. Hong Kong Press. 1995.