"Another world: inside artificial reality. Ditlea, Steve PC-Computing v2, n11 Another World Inside Artificial Reality YOU CAN FLY TO THE MOON BY POINTING YOUR FINGER. With a flick of your wrist, see the world through the eyes of a child. Reach out and grasp furniture, windows, or walls that exist only within the silicon memory of a PC. Wave your hand to create virtual paper on an empty desktop, a simulated skid in a nonexistent car, or X-rays of the human body. Artificial reality was the stuff computer researchers' dreams were made of--until recently. This year it's become ... well, reality, thanks to a confluence of developments in new technologies emerging from the labs, including light-weight 3-D stereoscopic displays, magnetic positioning systems, advanced graphics chips, and continuing expansion of computer memory, along with novel interfaces between human and computer. What's most remarkable isn't that computerized artificial reality exists, but that it is emerging so quickly from the research labs into public reach. Already, prototype systems have migrated from customized graphics workstations to off-the-shelf PCs like the Compaq Deskpro 386/33. Within a year, PC users will be able to design their own artificial worlds for eye-popping presentations and realistic engineering and architectural modeling. The creative fervor in artificial reality is reminiscent of the atmosphere that prevailed a decade ago, when PCs were beginning to make their mark. A handful of researchers on both coasts are combining home-brewed and professional-grade components to process complex artificial environments. Such virtual worlds are meant to be perceived not only by a person's sight and hearing but also by touch, shattering the barrier between the computer screen and worlds beyond. Exploring Places That Aren't There Within 50 miles of one another along the San Francisco Bay are three innovative outposts of research on the frontiers of computerized reality: the NASA Ames Research Center at Moffett Field, VPL Research of Redwood City, and Autodesk in Sausalito. The NASA Ames Research Center anchors the southern tip of the bay and nearby Silicon Valley to the edge of the 21st century. Here, in a building near one of the world's largest experimental wind tunnels, is the modest lab of the Human Interface Research Branch of NASA's Aerospace Human Factors Research Division. From this room has emerged much of the technology currently associated with artificial reality. To experience the NASA Ames personal simulator, you don special headgear: a round metal frame bearing a wraparound visor and a rectangular aluminum enclosure the size of a tissue box. Look into this head-mounted display and you see a stereoscopic 3-D image of the lab in black and white, complete with walls, checkerboard floor, ceiling, furniture, desktop computers, and equipment racks. Move your head to the side or up and down; the computer shifts the display to realistically match your point of view. Next, slip a black lycra glove attached to strands of black cable onto your right hand. Move your hand to calibrate this DataGlove, then point your index and middle fingers while bending your third finger and pinkie. You see a disembodied image of the glove do the same; the display moves in the direction you indicate. Point straight up, and the room appears to fall away. With your feet still on the ground, you're flying. The Moon in Your Hand Research scientist Dr. Michael McGreevy, who initiated and guided NASA's journey into artificial reality, is about to make a lifelong dream come true, using this setup to simulate visits to the solar system. The balding, mustachioed scientist has secured funding for a project called Visualization for Planetary Exploration, which could result in virtual environments of the moon and the planets. "We'll use visual data recorded by space probes and satellites to create computer models of each planet," McGreevy explains. "When we're through, you'll be able to hold the moon or any planet in your hand and point to where you want to go on its surface. The computer will scale the environment back to life size and you can be virtually present at the indicated location. The planetary environment would seem to surround you. You'd feel like you were there." Using the same artificial reality application, known as virtual travel, you could take simulated trips to exotic locales, faraway resorts, or business meetings without moving from your sofa or desk. The democratization of space travel is one example of how artificial reality may provide greater access to every individual. "We've shown that a personal simulator is possible," McGreevy says, "but we don't want to be the high priests of artificial reality. We need to democratize the technology because that will unleash thousands of talented people who will collectively develop its rich promise." McGreevy can be credited with designing and implementing the first practical artificial reality system on an off-the-shelf computer system (albeit a specialized graphics computer, an Evans & Sutherland Picture System 2), which included an inexpensive 3-D head-mounted display. Considered an essential component of artificial reality technology by many developers, the headmounted display was not invented at NASA Ames, but researchers there refined it to a practical size and cost. In 1965, computer pioneer Ivan Sutherland started developing a helmet sporting a pair of CRTs with left- and right-eye views that were adjusted by computer according to the user's head movements. By 1982, Tom Furness of the Wright-Patterson Air Force Base, in Ohio, had built an elaborate aircraft-training simulator into what became known as the Darth Vader helmet. In 1984 McGreevy asked Furness to sell him a helmet for his experiments but was told it would cost a cool $1 million. A Lot Cheaper Than Darth's By cannibalizing two $79.95 Radio Shack pocket TVs with black-and-white LCDs and placing them in a $60 motorcycle helmet, McGreevy and hardware contractor Jim Humphries were able to assemble the first NASA Ames head-mounted display for a lot less--under $2,000. NASA's government-funded lab was later able to build on early artificial reality work at Atari Research, where an affordable head-mounted display, a glove input device, and educational and game software were developed but never marketed. After the first great video-game boom went bust, sending Atari into decline, several of its researchers joined forces with NASA: hardware "imagineer" Scott Fisher, software whiz Warren Robinett, and--under contract with VPL Research--glove inventor Thomas Zimmerman. Fisher is recognized as having brought the glove--now also considered of vital importance for interacting with artificial reality (see sidebar)--to the NASA Ames personal simulator system. Safer Than Being There According to Robinett, the NASA bureaucracy was confused at first about the usefulness of the Virtual Interface Environment Workstation (VIEW), the official name for its artificial reality system. Originally, it was seen as just a way to display operational data, essentially a virtual instrument panel. Notes Robinett, now a project manager for artificial reality research at the University of North Carolina, "the real application for the VIEW system had to be telepresence--artificial presence in hazardous environments like space. With the head-mounted display hooked up to remote cameras and the glove controlling robot arms, you've got what we call 'telerobotics.' For a lot of uses, it's safer than being there. You can assemble a space station without risking astronauts' lives." With video camera input or computer-generated graphics, telerobotics could also be useful at the bottom of the ocean or in handling nuclear materials. Microtelepresence, a variation using optical or computer-aided magnification, would allow the manipulation of materials at a microscopic or even molecular level. (This technology should not be confused with computer-generated simulations of molecules, an area that was pioneered at the University of North Carolina and is now being tested for the modeling and synthesis of pharmaceutical drugs.) At NASA Ames, Fisher and colleagues McGreevy, Humphries, and Dr. Beth Wenzel are now perfecting the next-generation VIEW system. The original one was driven by a computer with just 32K of memory, the second version used a Silicon Graphics IRIS workstation, and the newest system employs a Hewlett-Packard minicomputer and custom graphics boards to "set the standard for artificial reality modeling," says Fisher. When completed, the system will show stereoscopic images in color or high-resolution black and white that are generated by computer, supplied by video cameras, or replayed from videodiscs. The updated VIEW system will also incorporate 3-D sound positioning (based on research by Wenzel) and speech recognition and synthesis. "We can finally stop beating the drum for artificial reality," asserts Fisher. "After doing non-stop demos and public relations for virtual environments, we can start facing the serious issues that lie ahead of us." Virtual Beauty Contests To see the hottest advances in artificial reality, you once had to be privy to government, university, and private research labs, but on June 7, 1989, computerized environments went public. Two firms commercializing the technology held demonstrations in San Francisco and Anaheim. VPL Research proclaimed the occasion a holiday, Virtual Reality Day. Declared its press release, "Like Columbus Day, VR Day celebrates the opening of a new world. VR Day will be celebrated every year with a parade and virtual beauty contest inside Virtual Reality." Despite such whimsy, big business--as represented by Pacific Bell--is taking artificial reality seriously. In San Francisco's Brooks Hall, at the regional-telephone-company-sponsored Texpo '89 exhibit of telecommunications products, a place of honor is reserved for VPL Research's demonstration amid the applications that will be possible once broadband fiber-optic phone lines connect offices and homes. On display is the world's first shared virtual reality. Side-by-side color monitors suspended above eye level show two views of a colorful computer-generated child day-care center, as seen from the points of view of two participants seated below the screens, each wearing 3-D goggles and a DataGlove. Transmitting image data over fiber-optic cable, users in separate cities could manipulate the same computer-mediated environment--a welcome development for an architect and client, say, with offices in different parts of the country. For Pacific Bell, shared virtual reality carries the allure of profits from the use of its extensive fiber-optic cable networks. To enter this virtual reality, you put on a VPL DataGlove and a VPL-manufactured head-mounted color display called an EyePhone. You can see the day-care center's interior, in 3-D, complete with doors, windows, furnishings, and a mannequin representing your colleague. Look down to see your own effigy and a disembodied hand floating nearby. To move around, flex your index and middle fingers and "let your fingers do the walking." If you want to change the placement of the water fountain, reach out to it and make a fist; grasp and move it, then flatten your hand to release it. This kind of tactile interaction is one way artificial reality differs from computer simulations of the past. Experience the day-care center from a six-year-old's eye level; to get short, point down with your little finger. To get big again, point up. Reality Built for Two Presiding over this demonstration is Jaron Lanier, founder and CEO of VPL Research and originator of VR Day. Clad in a black tunic and topped by blond dreadlocks, he is introducing the first commercial shared virtual reality, RB2, which stands for Reality Built for Two. According to Lanier, "the essence of virtual reality is that it's shared." This complete system would cost you about $430,000 (or $225,000 for one person, if you insist). If you already have the four required Silicon Graphics IRIS computers (two per person, one for each eye), RB2 will seem like a bargain at $69,925, including an enhanced Macintosh II as a design/control workstation, two DataGlove systems with control unit, two EyePhone display systems with head tracking and control units, four display output adapter cards for the IRIS computers, Swivel 3-D software for designing the appearance of virtual objects on the Mac II, Body Electric software for defining object behavior, and ISAAC rendering software for the IRIS computers. For added input of body gestures, you can order a DataSuit for an extra $35,000 to $90,000. Even with sophisticated graphics workstations powering RB2, its synthetic environments are hardly detailed enough to be confused with the real world. And Lanier's "first new level of objectively shared reality available to humanity since the physical world" has other drawbacks: by the end of the first day's showings, one of the system demonstrators is experiencing motion sickness and has to take Dramamine before the next day's session. Such "simulator sickness" is common in flight training where the body reacts to conflicting sensory cues from simulation and reality. Virtual Juggling But the 29-year-old Lanier remains an effective proselytizer for artificial reality. Having learned to juggle by manipulating virtual objects in slow motion with a pair of DataGloves, he argues persuasively for tennis tournaments played across continents in virtual reality. Having worked closely with Greenleaf Medical Systems of Palo Alto to develop medical diagnostics of hand impairment using the DataGlove, and with Dr. Joseph Rosen, who is researching surgical simulation, Lanier convincingly describes the possibilities for virtual reality-aided surgery with CAT scan data superimposed on a patient's body to locate internal organs. Another veteran of game design at Atari, Lanier may have the most vivid imagination among the current practitioners of artificial reality. While putting the final touches on the day-care-center demo, he was also creating a more fantastic virtuality: a giant birth canal through which you travel and are born, only to find that you are the virtual woman giving birth to yourself and are reborn again and again (some of the women at VPL find this excessive and feel that once is quite enough). Lanier consistently reaches beyond the obvious for virtual worlds. "Everybody wants to see Toontown, like in Who Killed Roger Rabbit?," he remarks, "but how about a virtual mirror in which you can see yourself change species? In virtual reality you can visit the world of the dinosaur, then become a Tyrannosaurus. Not only can you see DNA, you can experience what it's like to be a molecule. In virtual reality, the entire universe is your body and physics is your language." On VR Day in San Francisco, reporter Nancy Steidtmann asks Lanier if he's afraid someone may steal his technology and misuse it. "Sure," he answers, "but I can't let that stop me. I can't worry about not doing things." He adds, "We're creating an entire new reality. It's too big for any one company to own." Famous last words? Cyberspace on a Desktop The current activity in perfecting artificial reality, much of it in California, is not unlike the development of the movies in Paris at the turn of the century. Two apparently opposite tendencies emerged then: a realistic, documentary bent, characterized by the Lumiere brothers, who filmed scenes of everyday life; and an imaginative, fictional inclination, exemplified by Georges Melies and his fancifully staged Voyage to the Moon. More recently, filmmaker Jean-Luc Godard argues that the two types of movies were much closer in execution than historians would have us believe. If Jaron Lanier, with his hallucinatory environments, is the Melies of artificial reality, then the developers at Sausalito-based Autodesk, working within the bounds of realistic computer-aided design software, are virtual reality's Lumiere brothers. Yet by drawing inspiration from the work of science fiction author William Gibson, they too are crossing the line between reality and fantasy. On Virtual Reality Day in Anaheim, just a gnome's throw from Disneyland, the members of the Autodesk Research Lab (see "One Company Stakes Its Future on Innovation," PC/Computing, September 1989) are demonstrating their version of artificial reality for invited guests in a hotel suite. One by one, visitors navigate within either of two colorful virtual worlds: a cityscape with a few buildings at an intersection surrounded by fields, a lake, and clouds; or a room interior complete with table, chair, and books that can be "grasped" and moved around. While these are strictly solo experiences rather than shared realities, they too rely on the VPL EyePhone and DataGlove as the connection between human and computer. What makes the Autodesk system significant is the computer running it--it's not some exotic combination of workstations with a six-figure price tag, but a high-end PC-class motherboard built around an Intel 80386 processor. For VR Day in June, a dozen add-on circuit boards, including a pair of Matrox graphics boards, were assembled along with the central processor inside a suitcase-size equipment rack. By August, all the necessary electronics were crammed into an off-the-shelf Compaq Deskpro 386/33, offering the prospect of a virtual reality engine for under $30,000 (not counting the VPL EyePhone and DataGlove, which add about $18,000 more). Don't look for an Autodesk computer on your dealer's shelf. The company has never been in the hardware business. As a software firm seeking to market the first PC-based virtual reality program, Autodesk had to put together a computer platform on which to prototype its product. Creating a commercial program will be up to the firm's product development and marketing people. Though William Bricken, lab director at the time of the demonstration, won't be involved in the finished product, he has confidence in the package. "Given our company's approach," he says, "you can expect a representational program, something functional that will work with existing AutoCAD files." Architectural Walkthrough Since Autodesk's mainstay design program is the world's most widely used CAD software, engineers and architects will soon be able to take advantage of artificial reality. "You'll be able to create a building design and move through it in any direction," Bricken says. "To reposition a design element like a window, you'll be able to reach out and shift it." Achieving architectural walkthrough has long been a goal of artificial reality researchers. The most advanced work is being done by Fred Brooks's team at the University of North Carolina, where a deluxe system shows high-resolution depictions of floor plans and 3-D full-color interiors, complete with real-time adjustment of the position of the sun and the ratio of direct/ambient light. To navigate through a virtual structure, you can use velocity-modulating joysticks (in what is called a helicopter metaphor), move a head-mounted display (the eyeball metaphor), or, in the closest thing to being there, walk on a treadmill you steer using handlebars (the shopping-cart metaphor). Because of hardware limitations, Autodesk's virtual reality program will be less detailed and somewhat slower in real time, but it nonetheless represents a major advance in PC-based software. In the absence of a pre-existing need or market, Autodesk's attempt at making a product from artificial reality is a commercial gamble. Who will spend $3,000 to $4,000 on a software package requiring a top-of-the-line PC and $20,000 in extras? Current AutoCAD users in architecture, design, and manufacturing may want to use artificial reality in demos, presentations, and sales pitches. Also expected are a new breed of virtuality hackers, using the Autodesk program to create their own alternative realities. Autodesk's product, due on the market by mid-1990, may be called Cyberspace, in homage to the term coined by William Gibson in his cyberpunk novel Neuromancer. Autodesk has filed for trademark protection, as a name for software, of the word, which Gibson uses to describe the nightmarish universal computer data network of tomorrow. Given to such punning coinages as Cyberia and cyburbia, Autodeskers may have gone a little overboard. Company founder John Walker, who initiated the firm's foray into virtual reality withi his internal paper titled "Through the Looking Glass," admits the unsuitability of cyberspace and other terms for artificial worlds beyond the screen. Gibson himself is amazed. Cyberspace "is the last term I expected to catch on," he admits. "When I came up with it, I remember thinking that it was just too obvious. It didn't seem clever enough." But What Will It Do for Me? At work on two film scripts and a novel set in a Victorian England in which Charles Babbage succeeds in building the first computer, Gibson confesses that he's not curious about current developments in virtual reality. "When you've imagined it in an evolved state," he says, "the art isn't interesting. Besides opening things up for paraplegics, I don't see what artificial reality will do for anyone. It won't help save the rain forests; in that respect, I would prefer seeing cold fusion happen. Artificial reality will just be a gadget for rich countries, for military applications. At the low end, it will just end up being used for better Nintendo games." With an input device like the VPL DataGlove priced at $8,800, artificial reality should remain a rarefied pursuit for some time to come. As of this writing, only a thousand or so people have ever experienced artificial reality directly. Thousands more have experienced Myron Krueger's Videoplace (see sidebar) in museums and special exhibits, but they had little idea what they were seeing. At the same time, the ability to plunge into virtual worlds beyond the computer screen is so compelling that the market can hardly wait for real-world technology to catch up, and Gibson's vision may already be turning to fact. By this Christmas, artificial reality components may find their way into as many as 1.5 million homes, thanks to a DataGlove-like plastic glove and a competing flat-panel sensor device, selling for $80 to $90 apiece as Nintendo computer game system peripherals. The orders are in from Toys R Us and other retailers: Mattel's Power Glove and Broderbund's U-Force are expected to be big hits. At first they will be rather expensive Nintendo joystick replacements, with the accompanying virtual reality software not expected until 1990. With 20 million home Nintendo systems, projections show an aftermarket for 7 million artificial reality add-ons. Even with deep discounts, a half-billion-dollar U.S. artificial reality industry seems inevitable within a few years. And that's just for lowest-common-denominator video games. PC hardware and software entrepreneurs, take note. Steve Ditlea, a columnist for Omni magazine, has been covering personal computer technology for ten years. He lives in Armonk, New York.