From: rick@cs.arizona.edu (Rick Schlichting) Subject: Kahaner Report: Virtual Reality research in Japan Date: 9 Oct 91 15:18:05 GMT [MODERATOR'S NOTE: This item is posted here by Rick Schlichting, moderator of comp.research.japan, one of the most solid and solidly useful newsgroups on the net. Rick suggested this article would be of special interest to sci.virtual-worlds. Thanks, Rick. -- Bob Jacobson] [Dr. David Kahaner is a numerical analyst visiting Japan for two-years under the auspices of the Office of Naval Research-Asia (ONR/Asia). The following is the professional opinion of David Kahaner and in no way has the blessing of the US Government or any agency of it. All information is dated and of limited life time. This disclaimer should be noted on ANY attribution.] [Copies of previous reports written by Kahaner can be obtained from host cs.arizona.edu using anonymous FTP.] To: Distribution From: David K. Kahaner ONR Asia, [kahaner@xroads.cc.u-tokyo.ac.jp] Re: Virtual Reality 9 Oct 1991 This file is named "vr.991" ABSTRACT. Comments on Virtual/Artificial Reality research in Japan I recently wrote a short note [vr.791, 5 Sept 1991] summarizing papers at a Symposium on Artificial Reality, held this July in Tokyo. Since then I have had the opportunity to visit and speak with two of the meeting organizers. Professor Yoshio Tsukio Professor of Mechanical Engineering Faulty of Engineering University of Tokyo Tokyo, Japan Tel: 3812-2111 x6366, Fax: 3818-0835 and Professor Michitaka Hirose Associate Prof of Mechanical Engineering Faulty of Engineering University of Tokyo 3-1 7-chome, Hongo, Bunkyo-ku Tokyo 113, Japan Tel: 3812-2111 x6367, Fax: 3818-0835 Email: HIROSE@IHL.T.U-TOKYO.AC.JP VR or AR is now a "hot" topic both in the U.S. as well as in Japan; results are appearing almost daily. Rather than try to wait until a substantial body appears I intend to distribute short notes on interesting activities as I learn about them. Most of the VR experiments I have read/seen involve some kind of hat or helmet covering the eyes of the wearer/user. Small lcd displays within give a three dimensional display of a computer generated scene. Transmitter/receivers on the helmet allow detectors (mounted in a fixed position nearby) to determine the exact location and orientation of the user's head. This information is passed to a computer that changes the helmet image in concert with the head's changing viewpoint. It is also common to allow the user some other means of manipulating the scene, for example by giving him/her a steering wheel or joystick, etc. Instead of a helmet some systems use a very large display such as would be viewed while looking out the window of a plane or ship, but smaller displays are far cheaper and have other advantages. At the moment the combination of resolution of displays and computing power on the systems generating the images leaves much to be desired in terms of image resolution, but this will improve rapidly. There is also a problem with time delay between head movement and scene movement, because a great deal of computing needs to be done to generate even small motions, and this obviously increases more than linearly with resolution improvements. Nevertheless, most users of even these low-resolution systems experience a strong sense of realism. (Wide-screen cinema can also disorient viewers, so this effect is not too surprising.) Many VR systems also incorporate a "data-glove", a fabric glove with sensors attached that a user will wear; a computer generated image of the glove is placed in the visual scene and is made to move in concert with the user's motion of the gloved hand. As the user moves his/her hand in space, the image glove can be made to grasp objects within the computer generated scene and move them around. Research in VR seems to fall into several broad categories. (1) Experimenting with the hardware, software and the computational models in order to enhance the sensations of realism to the user. This involves improving the display hardware, understanding the meaning of "realistic sensation" in terms of visual factors such as field of view, resolution, stereo, audio, etc. (2) Developing applications, (3) Developing tools to aid researchers and users. Applications are limited only by the imagination of the researchers; games are the obvious first ones, but there are many opportunities related to training, from pilots to surgeons. VR is at the intersection of computer graphics and human computer interface, and is a natural extension of both. For example, Yusen Marine Science has developed a simulator for maneuvering of large container ships. The simulator uses six workstations for setting various navigation environments, which are reproduced on a several large screens allowing a 240 degree field of vision. The screens are in a chamber patterned after a ship's bridge and the view on the screen is changed to conform with steering and speed changing operations. Human factors such as nervousness, misjudgments and misconceptions are incorporated into the system to enable trainees to get more realistic simulations. Tsukio and Hirose have developed a sophisticated laboratory at the University of Tokyo for experiments in VR. As at many other places (also in the West) theirs contains a collection of purchased and built-up equipment to allow them to (a) experiment with existing VR techniques and (b) build upon these for new work. Most of their purchased equipment is from the US, although I was told that they are considering changing to some UK products which they felt were more cost effective. They admitted that at the moment the US is furthest along in this area and has some of the most creative ideas. But this is certainly not the only lab studying VR in Japan, and not even the only one at the University. Hirose explained to me that most of the large Japanese companies have some hardware/software research in general VR topics. He mentioned specifically NTT's Human Interface Lab, the ATR lab in Kansai (see below about ATR), as well as Matsushita and Fujitsu. These companies are ready to jump in once the market solidifies I was told. At University of Tokyo's "suburban" campus, Professors Fujimasa and Tachi also have a VR lab associated with the University's Research Center for Advanced Science and Technology (RCAST). There is also a committee composed of more than a dozen university researchers who coordinate, informally, activity in VR. Participants are from Tokyo, Keio, RCAST, Kyoto, Tokyo Inst of Tech, Tsukuba, etc., and include computer scientists, engineers, medical doctors, and others. There is also a journal, Human Interface News and Report, published several times each year, that contains the proceedings of the annual Human Interface Conference, as well as other papers and lists of meetings. The titles and authors of Vol 6 No 2 are attached below. Japanese are very active in the development of computer games and there is already at least one product incorporating a data-glove, although Hirose told me that the company that manufactured the glove has gone under and that large quantities of them are now available very cheaply. Projects at Hirose's laboratory are described briefly below. (1a) See-through helmet mounted display. The idea is to optically superimpose a virtual 3D object onto a real environment. The system consists of Sony view finders, a lens system and half mirrors; the image displayed on the view finder screen is focused about 1m before the eye using the lens system and half mirror, and has a view of about 20 degrees. In other head mounted displays, narrow fields of view can cause loss of spatial direction, but in this system the user has a "real" world to orient with. (1b) Light weight helmet mounted display (HMD). Of course the lighter the better--ideally no more than eyeglass weight is desired. Hirose claims they have developed the world's smallest HMD, with 5.0 by 7.5cm LCDs having 200 by 300 line resolution and weighing 170g including cables. I tried this system. It works, but the display is not illuminated, and so is only visible in bright background light; nevertheless its size and weight make it an impressive step forward. (1c) Virtual holography. The idea is to avoid using either a helmet or a data glove. A key application here is to CAD systems which require higher resolution than available with current HMDs. Hirose has substituted a conventional stereo CRT, but uses a head-tracker to allow the kind of interaction usually associated with a helmet. This system cannot generate the sense of an all encompassing universe sometimes associated with HMDs, or very large screens which are very exciting visually, but they can generate sophisticated and high resolution displays in small regions of space, and these might be perfect for detailed CAD/CAM applications. In addition to high resolution, this system also adds a mechanism for providing tactile feedback. Data-gloves can move around freely in the air even when the computer generated hand hits a solid object. Hirose's system requires the user's finger to be placed in a magnetic ring which is free to move within a guide. Four magnetic sensors located on the the ring measure the location of the finger and move so as not to touch the finger. However, once the finger intersects the computer generated object, motion to the object's surface is disabled by locking the unit. My sense is that force feedback is an essential element in providing effective applications of VR technology; I am not aware of too much work in this direction in the US; the Japanese are doing advanced work in tactile feedback. (3a) Software visualization. Hirose's idea here is to use VR to generalize flow charts and block diagrams by adding a third dimension. His applications are to large, complex software systems such as network control software. (Hirose's example is the regional power system around Tokyo.) There are many other attempts to simplify programming of such systems, and a graphical programming environment is common. He wants to fuse both block diagrams and time into a three dimensional representation. The idea would be that users could directly manipulate the blocks, move around the 3-D representation, etc. Several tools are being developed. (i) Virtual editor to define, modify, and delete processes and message passing among processes. (ii) Virtual Measure/Ruler to measure the exact synchronization of processes. (iii) Virtual path finder to locate and display the critical path to determine the total network throughput for a given network. (iv) Network simulator. A virtual 3D object is necessary for this kind of system, and thus this represents a very nice application of object-oriented programming. A prototype is being developed using an Ikegami 80inch 120Hz stereo projector with CrystalEyes LC glasses which generates a realistic 3D work space. Also a VPL DataGlove through which the user can handle virtual objects. This runs under an Iris 4D 210 VGX workstation for graphics and a Sun Sparc station for text. Hirose is planning to add his virtual holography techniques in order to enhance the sensation of handling processes in 3D. Personally, I have a "show-me" attitude about this approach. But having said that I hasten to add that understanding distributed computing is so difficult, and there are so many look-alike efforts, that a really new idea like this one is definitely worth cheering for. (3b) Virtual physical space simulator. This is more game-like. Hirose is thinking about altering some physical parameters (gravitational constant, air viscosity, light velocity, etc.) interactively while viewing a virtual world. His audience for this seems to be mostly students. Hirose is planning to describe some of his latest work at Human Interface '91, (Nov 23-25, 1991, Tokyo) and has a preprint that will be available at that time. (A symposium on VR is also planned for 31 Oct-1 Nov.) Many of the papers from Hirose and Tsukio's lab are written in English (this is not the case with most of the other Japanese research in this subject) and Hirose can be contacted via electronic mail. Work at the ATR lab in Kansai has focused on developing an effective video conferencing system as part of a larger project "Fundamental Research on Intelligent Communications". In their system, a user sits in front of two large lenticular screens that form a "V" facing him, with images of other people on the screens generated by liquid crystal projectors. ATR wants to enhance the sense that the people are really in the same place by improving their sense of "being there". They do this by monitoring eye movement of the participants and adjusting the images synchronously. At the moment their system reflects light off a user's pupil to follow movement of the cornea (non-contact eye movement detection). For related experiments, there is also a contact-type detector that is mounted on a pair of eyeglasses. This is one of the more basic research projects in the VR field, although its application is very specific. At the recent Computer World '91 in Osaka we were treated to a description of the future of VR by Robert Jacobson of University of Washington. Jackson commented that fundamental breakthroughs are not needed and that research capabilities were well distributed, but that industrial work was undercapitalized. He felt that by 1995 the industry will sort itself out and by 1999 various high end markets will become evident. I do not think that anyone in the audience disputed the directions that VR will take. However, I doubt very much that today's computer graphics leaders are simply going to sit around and wait for the VR guys who are now working in their basements or university labs to gobble up their users. What is more likely is that the graphics vendors with resources to commit to R&D will incorporate more and more VR into their own products by internal development, joint ventures, acquisitions, etc. 1991 Human Interface News and Report, Vol. 6, No. 2 Performance Evaluation of Recognition and Manipulation of Virtual Objects by Force Display H. Iwata, H. Noma, T. Nakashima (Institute of Engineering Mechanics, University of Tsukuba) Virtual Block World T. Mizuguchi, Y. Hirata, M. Sato, H. Kawarada ((Research Laboratory of Precision Machinery and Electronics, Tokyo Institute of Technology) Robot's Teaching by Operator's Movement in Virtual Reality T. Takahashi (NTT Human Interface Laboratories) The Dependence of Tactile Characteristics on the Skin Surface Temperature Obtained by Mechanical Stimuli Applied on a Human Finger T. Izumi, S. Ino, M. Takahashi, T. Ifukube (Res. Inst. App. Elec., Hokkaido Univ) H. Kimura (NTT Appl. Elec. Lab.) "Virtual Scienc" of Accuracy in Generated Environments - Focussing on the Effect of Time Delayt in Virtual Space - R. Kijima, M. Hirose (Faculty of Engineering, the University of Tokyo) Reality on Binocular Head - Mounted Display - T. Maeda, E. Ohyama (Mechanical Engineering Laborlatory) S. Tachi (RCAST, the University of Tokyo) Artificial Reality with Virtual Creature K. Hayashi, T. Fujita, K. Hirota, C. Matsumoto, S. Hishiyama, K. Murakami (Fujitsu Laboratories Ltd. Human Interface Laboratory) Musical Virtual Space S. Ohteru, S. Hashimoto, A. Sato (Department of Applied Physics, Waseda University) The Application of Virtual Reality of Mechanical Design K. Kameyama, K. Ohtomi (Toshiba R&D Center Mechanical Engineering Laboratory) A Discussion about Some Applications of Artificial Reality T. Onitsuka Analyzing Body Shape of Japanese Women - Can Computers Take the Place of Human Eyes? - T. Kurokawa (Faculty of Engineering and Design, Kyoto Institute of Technology) T. Kishimoto, A. Shinozaki (Wacoal Corp.) Sensibility for Liquor Making S. Imai (Research Laboratories of Distilled Spirits and Liqueur, Suntory Ltd.) Report on Tutorial for Human Interface '90 M. Kurosu (Design Center, Hitachi Ltd.) Report on Human Interface '90 Workshops M. Suwa (Electrotechnical Laboratory) --------------------------END OF REPORT----------------------------------