From: Joseph Michael Subject: Re: ONLINE: Holodeck System Date: Mon, 07 Oct 96 00:23:25 GMT Message-ID: <844647805snz@stellar.demon.co.uk> Organization: Stellar Drive In article <3250BEDE.41C6@nlr.nl> Hilbrink@nlr.nl "Niels Hilbrink" writes: Firstly appologies for posting a typo in the URL - it should read http://www.stellar.demon.co.uk/holodeck.htm >Looks very nice, little chape changing robots :), but how do you keep >your subjects from walking of the platform ? That is just a diagram. Obviously, the subject is in a controlled environment like a room or a maze. I have added some extra info to the pages like the concept of teleporting and enclose them below:- ------- http://www.stellar.demon.co.uk/holodeck.htm -------- Introduction A proposal is enclosed here to build the world's first holodeck system. Shape changing robots are ideally suited to provide a first approximation synthesis of a holodeck system. Holodeck is short for holographic projection deck - a concept made popular in Star Trek series where you can enter a chamber and interact with holograms as through they were real. One cannot build such holodecks at present but in the not too distant future a mass entertainment system that offers all the delights of a real system could be built. So what is a holodeck? Its the addion of Hardware Reality (HR) to a Virtual Realit (VR) helmet. If you could wear a virtual reality helmet, walk into a room (built of shape changing robotic cubes), command a chair to appear, walk up to it and sit down then as in figure 1 above, then we have the makings of a very good approximation of the holodeck system!! Can we do this without the virtual reality helmet? At present this will be not possible. For instance, if we look at the sky, the birds will not fly in the sky just because we command it! Neither will it turn into a sunny day. If two people were in the same vicinity of each other, then the spaces occupied would immediately start to clash because of the huge size of the spaces involved. We will always end up needing a virtual reality helmet or something equivalent to feed our eyes with information that fools us into believing that we are somewhere we are not. We need to be sure that we can feed the eye with sufficient information so that when we are looking at a bird flying in a clear blue sky the eye is fooled without having to actually construct such a huge environment around us. The virtual reality helmet allows the physical environment size to be reduced. Having built the environment, we must now add substance to all the elements we could interact with. Shape changing robots and their tools linked to the virtual reality helmet is certainly good enough to create objects such as chairs, tables, floor boards, steps, lifts, windows, doors etc. as illustrated in figure 1 that would allow the sensory system to be fooled into making us believe that we are interacting in an environment that necessarily does not physically exist in its entirety. Every time we get close to an object that we could interact with physically, the shape changing robots change shape to approximate that object so that we can interact with it properly. Only the objects that we are close to need to be synthesised to save energy while the other objects can be masked out using the virtual reality headset. Shape changing robots can provide the tactile feedback we need to sustain the illusion. Shape changing robots can be made to react fast enough so that you could walk up to a chair and find it exactly as you see on your visor. The resolution of the robotic cubes determine the finest features resolved to the human touch. The sensors within the robotic cubes gives the robot reactive behaviour so that if you think you were looking at a mattress and it was built of shape changing robots with sensors then it can be made to sag to much the same degree as you would expect a normal mattress to sag. In well defined environments, where the options to view and interact have been pre-limited, tooling such as real mattresses can be brought in at the time of use to provide the realism that may not be available straight away using finer resolution robotic cubes. Body Suits of The Future It should be pointed out here that if the shape changing robots are around 0.1mm to 1mm in resolution and provided they have adequate reaction speed, then they can be worn as a suit and all the additional shape changing robots can be dispensed with(!) Most of the time the suit does not make contact with the body except where it is attached as a garment. It is held within a millimetre reach of the skin. When you reach out and touch something in the virtual world, the suit deforms allowing you to make contact with itself. This gives for example a hand the sensation that it is touching something without the real object having to be present. Providing the suit can react fast enough to deformation requests, you could stroke the virtual object and the suit would react against the skin as through it were in contact with the actual object. Adding Extra Realism Regardless of which way you implement the system, a fundamental problem with these systems are the missing sensations. You need heat, wind, sound and vibration feedback which can all be present in the environment. Fortunately, shape changing robots can tool up with actuators that generate these stimuli which enhances the interaction with the environment to a greater extent providing its not a body wide suit. Body wide suites need to carry tools to generate these which then makes the suit exceptionally bulky. Adding Extra Levels of Interaction Even after all this, there is still something missing and that is the interaction of the user with the objects that are found in the virtual world. For example, you could get wet but that is not a problem if an actuator pumped fluid at you to give that wet feeling. Free flowing fluids become a problem if it spills into the environment and you had to re-image it and present it back to the user so that he is aware of how much is spilt and where its boundaries are if its lying in a puddle. To overcome these problems, cameras must be mounted on the user and around the user to determine how the interaction between the user and the fluid object result set looks like. If the user is carrying an object and drops the object, it is necessary to re-image the object so that when he walks away from the object, he can look back and view what he has dropped. You have to install weight detection equipment in the shoes of the user so that cameras mounted on the user and around the person can detect weight gain / loss and focus in on the objects that have been picked up or dropped. You should for example be able to pick up a piece of paper, read its contents because camera has become aware of weight gain and allow you to focus in on what you have picked up. Afterwards you could crunch it and drop it and view the crumpled piece of paper as it should look through the VR helmet. Nano Technology Nano technologists dream about creating a holodeck that could fulfil our desires to interact with environments not possible in conventional reality. Whatever they do, something close to a VR helmet must be provided in order to complement the synthesised reality with that of our expectations such as huge expanses of space. You can't go around creating birds and deep blue skies except in the VR helmet. If shape changing robots can provide this functionality with technology that is infinitely cheaper than nano technology, then that begs the question why do we need nano technology to accomplish the same goals?? Herein probably lies another nail in the coffin that may put back the development of nanotechnology as we envision today by a few more decades. A holodeck is a principle target for implementation using utility fog and/or conventional nano technology. But that can never happen while cheaper alternative in the form of shape changing robots exist. Telepresence & Teleporting The kind of telepresence offered by this technology is quite different from conventional robots. With this system, you can 1) Teleport objects 2) You can teleport yourself 3) You can grow and shrink but you can do all this and interact with the environment that you create. For example you can appear inside a car in a car factory, remove a defective radio and fit a new one, and then teleport into another vehicle to fix its exhaust, and then shrink to get inside the engine through the exhaust and wrench the spark plug gaps apart on a defective spark plug all without ever leaving the comfort of your home. You could do some of this using conventional robots but you are limited with such technologies in what you can do and how you interact with your environment. With this new technology, you won't face those limitations. With spark plug example, one millimetre robots carrying cameras can enter the engine through the exhaust and present a digital image of what is on view and provide the necessary robotic tooling to repair fault. You can miniaturise yourself and once inside touch the walls of the engine cylinder and you will find it exactly to scale. Carrying this analogy further, if you knew that you needed a special tool to affect repair, you can look for it in the factory floor and 'teleport it' to where you are. The shape changing robots will deform and fetch the object to where you are. Once a teleport command is executed, the object disappears in flash from your visor and reappears to where you are. It will take some time of course while it jumps from one point to another because the object has to be physically transported across the factory floor but the concept of teleporting yourself and/or objects around you would become a familiar concept when these machines are finally rolled out. Virtual Design Studio The possibility of telepresence gives us the ability to be present anywhere we want to be without having to be actually being present. The marriage of HR and VR allows the design of machines that can cater to every kind of whim we may have about manufacturing. We could build a system whereby we design a product and fully test & assemble the product without ever having to be at the manufacturing site. One such facility may be a machine shop. This machine shop of the future as a facility has a wire through which you log in, design your product, make it, assemble it and package it ready for shipping. After you have paid, the product is released and is shipped to your premises exactly as you packaged it. How much time you log on into each piece of machinery and how much materials you use is billed to you. The greatest advantage is that you can get things make extremly quickly. For low volume products, the design process and the manufacturing process is one and the same. If you did the operation once, then you merely download the project file into the manufacturing site and wait the for the manufactured product to be delivered. It may be that products from several different producers have to be manfactured, assembled and shipped to your facility for final assembly and integration. All this could also be automated. All you would be expected to do is to spend a lot of time designing and putting together the components as design engineer. If you have a team and other people at the manufacturing facility who can log into your program, then a great deal of 'devolved' experience could be brought into play to manufacture your product more efficiently with better design decisions. Parallel manufacturing activity could take place such as test engineers going on line to test your product as it gets manufactured, for tolerance checks, interference etc On top of all this, it is possible that over a period of time, other parameters such as materials selection, clearance parameters, etc. could be added to the design suite to avoid making serious design mistakes. It also implies you need less participation by other engineers. As an example, if you wanted to make a plastic bearing, a materials menu option could walk you through the advantages and disadvantages of using common materials. It means you won't need an engineer to run you through the material selection process. .--------------------. .--------------------------. | Joe Michael \______________________/ joe@stellar.demon.co.uk | : \__________________________/: | Futuristic . Shocking . Mind Blowing . Shape Changing Robots | :-------. : | \ http://www.stellar.demon.co.uk/ | `---------+--------------------------------------------------------------'