From: brucec@phoebus.labs.tek.com (Bruce Cohen) Subject: Re: half silvered lenses (was Re: Direct Neural Input (Was Re: Date: 13 Nov 91 21:51:52 GMT Organization: Computer Research Lab, Tektronix Inc. In article <1991Nov12.221654.8194@milton.u.washington.edu> craig@utcs. utoronto.ca (Craig Hubley) writes: > So far there is no information that couldn't be provided to the interface > via proximity or bend sensing, of the magnetic, sonar, radar, or video > varieties... In other words, the computer could track the same things > albeit less directly and more crudely... Well, that's true, but I don't see what that buys you, unless you want the computer to move the object to match what it thinks the user's sensory-motor system expects. This could be done to some crude level, I guess, by estimating the parameters of the user's control system and anticipating. I don't know just how variable those parameters are from task to task, over time, or from user to user, but I suspect that anticipating effectively might be very hard. I'm also unsure of the effect on users of having objects come to meet them. There's a marvelous story about an experiment done by the British neurosurgeon W. Gray Walter in which he used signals captured from motor neurons to signal a slide projector to advance. The signals were correlated with the early development of the impulse to move the subject's finger on the advance control of the projector. The subjects were rather disconcerted by the way that the slide projector "anticpated" their decision to advance the slide, and frequently tried to hit the button again, thinking they had made a mistake. > So the key, as with all user interfaces, is fast response time? If I > understand this correctly you need a frame rate of only about 10 fps... > hardly insurmountable. For many tasks, I suspect that less than 10 frames per second will do, possibly as little as 6-7. The positive feedback and loss of coordination effect seems to have a sharp onset somewhere in this range. But the onset does seem to depend on the task. Someone want to provide some hard numbers? All I have are some informal experiments I did to prove to some doubting Thomases that less than 10 frames was good enough sometimes (they wanted me to rip apart a year's worth of work because they were convinced that less than 20 would be unacceptable). > For one thing, if we are mixing virtual objects > with the real world, and need not render a background, Actually, a background (and floor) might be beneficial since you can pattern or shade it to give depth cues. > we need not > necessarily have very high-res anyway. Personally I think NTSC would do. > For anything more detailed than this, you will probably want to switch > over to all-virtual mode so that you can concentrate on the details > without the distractions of transparency, clutter of physical objects, etc. Personally, I'd rather have the virtual world overlay the real one; I'd be more confident of not tripping over my chair (no smilies AT ALL! I did that once while trying to videotape an event with a minicam. Restricting or eliminating your view of the real-world while wandering around in it is likely to be very rough on the ankles if nothing else.) > The "right" answer is variable-resolution objects and prefixed allotments > of rendering time. At the focus of the users' attention and action (which > is easy to tell given that you know what they're doing / looking at) you > render furiously, starting with gross shapes and going to more detail as > you get background objects roughed in... when he moves his head you > rapidly do it again... anyone else get angry when you want to flip a view of > some complex object around 60 degrees and the stupid draw/CAD-CAD/anything > software insists on rendering the entire thing in full detail at each 5 degree > increment ? Agreed. In fact, I'd go a little farther and advocate variable time allotments with maximums. Some sort of scene object then allots time to the objects based on what's in the foveal area, what's moving, etc. > Clearly, there are many situations in reality where the quality of our view > of something is degraded - dark, fog, smoke, lost my glasses, etc. Our > brains deal well with this and are good at assuming detail. However, there > are NO situations in nature where the frame rate of reality is > degraded, Luckily, when things move we're lots worse at seeing things like fine detail, texture, and even the total number of vertices of a polygon. Putting blurred blobs of approximately the right size in approximately the right place every 50 milliseconds is not only good enough, it actually can eliminate artifacts like strobing. This lowering of requirements for detail also happens when we move our eyes, so saccadic motions give the graphics system time to flush the current graphic computations, make a guess at the next hi-res area, and start some new computations. >>the brain's response is to assume that the correction didn't occur, and >>to increase the correction. Out on the arm, which really did correct, >>this will cause overshoot and oscillation. If you manage to get close > > OK, I get it. The brain takes the law into its own hands...! Yep, the brain's the boss. You'll remember that the next time you take that top step that your brain thought was there :<), ------------------------------------------------------------------------ Speaker-to-managers, aka Bruce Cohen, Computer Research Lab email: brucec@crl.labs.tek.com Tektronix Laboratories, Tektronix, Inc. phone: (503)627-5241 M/S 50-662, P.O. Box 500, Beaverton, OR 97077