From: John Costella Subject: REV-APPS: Visit to Monash University Computer Science Date: Wed, 6 Dec 1995 16:19:45 +1100 (EETDT) From: John Costella I got some favourable feedback to my report on my visit to Integra in Adelaide, so I thought I might follow it up with another on my visit to Dr Ronald Pose at Monash University. Unfortunately, familiarity prevents me from bringing you a newcomer's view of my travel experience to Monash, since I lived about a kilometre away from it for several years. I am talking here about the original (Clayton) campus of Monash (they have since swallowed up an innumerable number of other tertiary institutions in the amalgamation frenzy of the late '80s). It is located about 25 kilometres south-east of the centre of Melbourne. This is still well within the suburban sprawl of the city (I myself live just about on the suburban fringe, another 20 kilometres further out, in the same direction). The process of amalgamation seems to have created a considerable overlap of computer-oriented departments at Monash: some eight departments spread over three campuses. Ronald Pose is a Senior Lecturer in the Computer Science department (which, in my brief days as a student at Monash, was at the time the *only* computer department!). So why was I visiting Dr Pose at all? Well, my visit to Integra also brought me information about some remarkable research and development done by Dr Pose, and his PhD student Matthew Regan. Many of you here will be familiar with their work, from demonstrations around the U.S. and a presentation at SIGGRAPH '94, but unfortunately my knowledge of the latest work in VR has a few holes in this period. Remarkably, Regan and Pose had been working on improving the "dumb frame buffer" of conventional display hardware architectures, for much the same reasons as given in my own "Galilean Antialiasing" papers. But their approach is different; and, interestly, our two methods are to a large extent complementary. The big difference is that they have a working prototype! Let me explain what their idea is (their patent is pending, but I believe all information I give here is detailed in their published papers): Their basic observation will sound familiar to you: Whilst the slightest movement or rotation of a VR participant's point of view invalidates the entire image in the frame buffer, the new image required is to a large extent just a simple transformation of the old. Hence, a "smart frame buffer", that can handle as much of this transformation in hardware as possible, will reduce by a significant factor the load on the rendering engine. Their second observation is that, by and large, the most important effect is that of *rotation* of the user's viewpoint. So essentially they designed and built hardware that tracks the head orientation automatically. How do they do this? Well, pretend for the moment that the participant's eye is not translating at all, but that it can rotate arbitrarily (three degrees of freedom). If you surrounded this eye by a sphere, and somehow mapped a pixmap onto the surface of this sphere, you could render your virtual world onto this spherical pixmap. You could then build hardware that did the matrix multiplication of the orientation degrees of freedom, fetched the pixels, and mapped them into a display frame buffer that was scanned out to your head-mounted display, for the actual viewport at any particular instant in time. This was their original idea. Let's now make it more practical. Instead of choosing a sphere to render onto (yuk! just look at my long Galilean Antialiasing paper to see how bad the transformations get---polygons are no longer flat, and so on), they instead chose a *cube*. The invariable question then is, "What about the corners?" Well, topologically speaking, a cube is not really more than a deformation of a sphere. As long as you build your hardware to reverse this transformation, it doesn't matter at all. (And let me say in advance that you CANNOT see any remnant of the "corner" at all ... it is mapped away perfectly.) The cube of course lets you use the huge investment in standard "planar" viewport hardware and software to render your worlds. (Although it may seem that a cube increases your work six-fold, in practice the situation is not as bad as this, and you can parallelise the process to a large extent. You also gain far more from having instantaneous rotation than you lose by having to keep track of stuff behind the participant, and can minimise the overhead further by rendering this extra stuff at lower quality, changing the "important" faces as the orientation changes.) Next, Pose and Regan figured out how to organise their memory in such a way that they can access it video speeds. This is not trivial, since the normal tricks do not work when you are not simply fetching scan lines at a time. I'll let you read their papers to examine their solutions. Then they designed the hardware to do the mapping of the viewport via the matrix multiplication. They also have some antialiasing hardware (although the results are already so amazing that they have not yet fired up the antialiaser). They had the good sense to also include the ability to correct for arbitrary optical distortion for particular head-mounted displays. The correction values are stored in look-up tables, which can be downloaded from the computer. Hence the corrections carry zero performance penalty, yet do not lock you into a single display device. A nice touch. And then Matthew Regan built the prototype. It is quite an amazing set of boards --- practically all hand-wired! The cube has 512 x 512 resolution on each face, and so in the configuration they require, it's quite an intricate wiring job. They tell me the budget for the project was about A$30,000 ... and most of that went on the static RAM chips. And as anyone who has seen the system can testify, it actually works! Using an input device (for simplicity they have a regular mouse set up; clicking one of the buttons changes from two Euler angles to a different two, so you can verify full 3 d.o.f. functionality), you can change the orientation of the scene in the frame buffer *instantaneously*. There is essentially no latency (something like microseconds), and the frame rate whumps along at 60 Hz. You definitely *cannot* tell that the scene has been mapped onto a cube. It is an amazing experience! So why isn't their system yet built into graphics workstations or PC add-on cards? I believe that this is in the pipeline. But there are a couple of aspects of their prototype that have still led a number of important people to seek more "proof". The biggest problem that they seem to have is that the scene they are showing has been pre-rendered on a high-end SGI workstation. Anyone who understands what they have done realises that the method works. But for maximum "punch" you really need to interface the hardware to a sturdy virtual world engine to show it working "in real life". But this leads to the more fundamental question: how do they handle *translations* of the participant's point of view. Hardware-wise, they don't. (This is where my own "smart pixmaps" may help, but that's a different story.) But they have also come up with an idea they term "prioritised rendering", in which they compute how "out-of-date" an object's rendering is, and put the most urgent ones on a "fast" cycle, and less urgent ones on "slower" cycles (say 5 Hz). Now if you think that you already read about this sort of idea in my long paper of October 1992, you may be right. And I don't claim to have invented it either; although I'm not sure of earlier references. But nevertheless Pose and Regan have formulated concrete algorithms for performing this philosophy, which are a little different from my own, simply because they do not need to worry about the main effect of rotations - --- their hardware takes care of that; and conversely because *I* do not need to worry about uniform or accelerated motion --- my pixmaps take care of that! The problem is that for Regan and Pose to build a complete virtual world engine on these principles, and interface it to their hardware, would be to largely reinvent the wheel. But conversely, the big players in graphics workstations and accelerator cards would want to see the system working in a real environment. It is a difficult stage to be at. But I think you will see their hardware being incorporated into real machines in the not-too-distant future ... So let me encourage anyone in relevant manufacturing companies, who has not seen the Monash system, to arrange a visit to Melbourne some time soon! Ronald is also looking for anyone to lend them a head-mounted display with a low-lag orientation tracker, to try out the system on a real person. Finally, their contact information. By email, try: rdp@cs.monash.edu.au regan@cs.monash.edu.au and for snailmail try Department of Computer Science, Monash University, Clayton, Victoria 3168, Australia. Tel: +61 3 9905-5203 Fax: +61 3 9905-5146 And the usual disclaimer that I am not affiliated with Monash University, and so on. ===================================================================== Dr John P. Costella School of Physics, The University of Melbourne mailto:jpc@physics.unimelb.edu.au http://www.ph.unimelb.edu.au/~jpc Phone: +61 3 9344-5435 Home: +61 3 9768-9268 Fax: +61 3 9347-4783 ===================================================================== -----BEGIN PGP SIGNATURE----- Version: 2.6.2i iQCVAgUBMMVEOWrlpFIGn9BdAQHU0wQAnBymqYWNe5K8D/utbK79ztN9vqdCWDNc 4G2cANFMGCV1o80MQByMW9RaF9+6EzhrAbUrpS+axqRUYPT3iv76XvmRJe3Wi9lv wEBM+uDVhrMF6+mGIOSr30q0KX0H/sna59kCB1hIjwfDKkt+Z4kG5ACr3LvhuCyq Bj/8ByAthxA= =2lBK -----END PGP SIGNATURE-----