From: Craig Hubley Subject: Re: half silvered lenses (was Re: Direct Neural Input (Was Re: Date: Mon, 11 Nov 1991 21:15:47 -0500 Organization: UTCS Public Access In article <1991Nov10.195633.19784@milton.u.washington.edu> brucec@phoebus. labs.tek.com (Bruce Cohen) writes: >In article <1991Nov9.033143.7224@milton.u.washington.edu> craig@utcs.utoronto. >ca (Craig Hubley) writes: > >> In article <1991Nov5.033024.17522@watserv1.waterloo.edu> dstamp@watserv1. >> waterloo.edu (Dave Stampe-Psy+Eng) writes: >> >>>Yes, exactly. What I was proposing involved being able to interact with >>>virtual *objects*, while letting the user see his hands, the computer >>>keyboard, etc. This solves a LOT of the psychophysical and kinesthetic >>>problems. But the time delay problem kills it, for now at least. (sigh). > > [me misunderstanding the issue omitted] > >As an example, suppose that you're wearing a glove and goggles, and you >decide to pick up the bottle labelled "Drink Me" on the table in front >of you. As you move your hand towards the bottle, your brain tracks the >relative position of hand and bottle visually against the kinesthetic >information from your arm that tells how the arm and its joints are So far there is no information that couldn't be provided to the interface via proximity or bend sensing, of the magnetic, sonar, radar, or video varieties... In other words, the computer could track the same things albeit less directly and more crudely... >oriented. As you make a change in the motion of some muscle, to correct >for an error in the approach tracjectory, the brain assumes some maximum >amount of time between the instigation of the movement and detection of >the effect. If the display takes substantially longer than this maximum >(which is on the order of 100-200 milliseconds) to update the display, So the key, as with all user interfaces, is fast response time? If I understand this correctly you need a frame rate of only about 10 fps... hardly insurmountable. For one thing, if we are mixing virtual objects with the real world, and need not render a background, we need not necessarily have very high-res anyway. Personally I think NTSC would do. For anything more detailed than this, you will probably want to switch over to all-virtual mode so that you can concentrate on the details without the distractions of transparency, clutter of physical objects, etc. I am thinking about the difference between CAD/CAM, with high-res and highly-detailed objects vs. navigation beacons or short text messages... in the former you are taking advantage of space-multiplexing (i.e. lots of space filled with mostly static objects that you can concentrate on) and time-multiplexing (i.e. a smaller space filled with rapidly changing objects that require less concentrated attention). This pops up in hypermedia environments too, with things like Intermedia tending towards the former and HyperCard towards the latter... The "right" answer is variable-resolution objects and prefixed allotments of rendering time. At the focus of the users' attention and action (which is easy to tell given that you know what they're doing / looking at) you render furiously, starting with gross shapes and going to more detail as you get background objects roughed in... when he moves his head you rapidly do it again... anyone else get angry when you want to flip a view of some complex object around 60 degrees and the stupid draw/CAD-CAD/anything software insists on rendering the entire thing in full detail at each 5 degree increment ? Clearly, there are many situations in reality where the quality of our view of something is degraded - dark, fog, smoke, lost my glasses, etc. Our brains deal well with this and are good at assuming detail. However, there are NO situations in nature where the frame rate of reality is degraded, (despite Haight-Ashbury, The Six Million Dollar Man, etc. - I said *nature* and even in slow-mo the frame rate is reduced in strict proportion...) so our eyes/brains/sanity rightfully go haywire when this happens. At least mine do, and I know there is research backing this up - "make it MOVE real, no matter how sorry it looks" - Fred Brooks, plenary address, CHI'88... Graphics toolkit developers take note - the first usable toolkit that supports variable-resolution, fixed-frame-rate opportunistic rendering will WIPE THE REST OF YOU DESERVEDLY OUT OF EXISTENCE. I will laugh and spit on your poor paupers' graves, you technology-driven scum... 1/2 :) <- I'm afraid people would assume a full smiley if I didn't add this, but I definitely half mean it. >the brain's response is to assume that the correction didn't occur, and >to increase the correction. Out on the arm, which really did correct, >this will cause overshoot and oscillation. If you manage to get close OK, I get it. The brain takes the law into its own hands...! >enough, you'll probably knock the bottle off the table. If it was real, that is..., if it was virtual, you could have it blow up in a cloud of gore and smoke. Wow, man... Craig Hubley Craig Hubley -- Consultants in object-oriented technology & techniques, -- Craig Hubley & Associates -- user interface design & user productivity -- craig@gpu.utcs.Utoronto.CA UUNET!utai!utgpu!craig craig@utorgpu.BITNET craig@gpu.utcs.toronto.EDU {allegra,bnr-vpa,decvax}!utcsri!utgpu!craig 61 Tiverton Avenue, Toronto, Ontario M4M 2N1 Canada (US TC) call 416-362-7632