Olfaction for Virtual Reality Martin Zybura and Gunnar A. Eskeland Quarter Project Industrial Engineering 543 University of Washington Winter 1999 for Professor Tom Furness Table of Contents
Abstract *
Introduction *
Physiological Aspects of Smell *
Sensing Technologies *
Uses of Olfactory in VR *
Research in Olfactory displays and Olfaction for VR *
Proposed Prototype and Areas of Future Research *
Conclusion *
Bibliography *
Abstract
The inclusion of visual and tactile senses into virtual reality is widespread, but the sense of smell has been largely ignored. Hardware to produce olfactory stimulation in virtual reality is currently limited. Sensory information available from the olfactory system will increase the user's sense of presence in virtual reality. This paper reviews uses for olfactory interfaces in virtual reality, the olfactory sense, current research in olfactory interfaces, and discusses further research. In addition to standard virtual reality applications, telepresence is discussed in the context of sensing technologies. Currently available commercial products are discussed, including the FiVe FiRe training system.
Virtual reality has, since the onset several decades ago, been dominated by visual stimuli, with tactile and auditory information researched and added to the scene in the later years. "Olfactory information has been mainly ignored as input to the virtual environment participant, in spite of the fact that olfactory receptors provide such a rich source of information to the human" (Barfield & Danas, 1995). However, some research has taken place, including practical uses of olfactory stimulation in virtual environments for training in the handling of hazardous materials, firefighter training, medical diagnosis, entertainment, and the visualization of processes such as chemical reactions. A part of this is the work that has been done in studying the propagation of and the requirements for odors in the virtual environment, leading to the definition of an olfactometer and the very first olfactory displays.
Cater (1992) talks about the importance of the ambient smell of the physical environment to create a sense of presence in the virtual environment. The participant might be in a virtual world, such as a forest, but have ambient clues telling him he is in a laboratory, thus reducing the virtual world experience. Before investigating virtual olfaction and virtual olfactory displays, it is import to have some understanding of how the human olfaction occurs.
Physiological Aspects of Smell
Olfaction is defined as the act of smelling, "whereas to smell is to perceive the scent of something by means of the olfactory nerves." Odorants are, as stated by Kromer et. al (1994), substances whose characteristics can be determined by chemical analysis (Barfield and Danas, 1995). A persons olfactory system operates in a fashion similar to other sensing processes in the body. Airborne molecules of volatile substances come in contact with the membranes of receptor cells located in the upper part of the nostrils. The olfactory epithelium, the smell organ, covers a 4-10 cm^2 area and consist of 6-10 million olfactory hairs, cilia, that detect different smells of compounds. Exited receptors send pulses to the olfactory bulb, a part of the cortex, with a pattern of receptor activity indicating a particular scent. Because the airways are bent and thus the airflow past the receptors normally is low, we sniff something to get a better sensation (see figure 1).
In addition to the cilia, the fifth cranial nerve (trigeminal) has free nerve endings distributed throughout the nasal cavity. These nerve endings serve as chemoreceptors and react to irritating and burning sensations. The trigeminal nerve connects to different regions of the brain and provides the pathway for initiation of protective reflexes such as sneezing and interruption of inhalation.
Figure 1 Olfactory System (Collins Advanced Science, 1999)
If the concentration is high enough both the olfactory and the trigeminal sensors will be triggered by most odorants (Barfield and Danas, 1995).
"Basic units of smell are caprylic, fragrant, acid, and burnt" (Cater, 1992). Other terms that have been used are fruity, spicy, floral, and green. In general the description of odors is limited to adjectives, in contrast to the rich vocabulary used to report visual stimuli (Carosso & Ridout, 1993, in Barfield and Danas, 1995). "Olfaction is also similar to the visual and auditory modalities with regard to age-related changes, in that olfactory sensitivity deteriorates with age. Peak performance in olfactory identification is reported to occur between the third and fourth decades of life" (Barfield and Danas, 1995).
Humans fare pretty well with odor detection when other senses, like vision, are used alongside. Without cues from the other senses subjects in a study failed for two out of three odors to make the correct identification (Zellner, Bartoli, and Eckhard, 1991, in Youngblut et. Al, 1996). Furthermore, it is not uncommon to have a high rate of false detection and report the existence of smells that in fact are not present (Richardson and Zucco, 1989, in Barfield and Danas, 1995).
According to Youngblut et. al "odors can be used to manipulate mood, increase vigilance, decrease stress, and improve retention and recall of learned material" (1996). Various scents have been shown to improve tasks performed by subjects (Krueger, 1995, in Youngblut et. al, 1996). Even the suggestion of a smell can lead to reactions as if the odor was present.
Technologies for sensing smell electronically have been developed. This next section will look at some of these technologies and uses that may enhance the sense of olfaction in virtual reality. Like with any other engineering tasks models are used to understand how individual odors are identified. The most common model is a simulation of the neuro-biological information processing system. There are about 1 million sensory neurons in a mouse, and approximately 10 million in a human. Electronic sensors of smell (electronic noses) are much simpler than any of these true biological systems. For virtual reality applications this does not pose a serious problem. In a telepresent virtual reality system the sensing technologies would be limited to a predetermined set of odors (Keller et. al, 1995).
Electronic noses have been developed as automatic detection systems for various odors, vapors, and gasses. Detectors for odors such as natural gas and gasses such as carbon monoxide are used with safety alarms. In a virtual reality telepresence experience you in addition to sense and determine the smell, want to transport a signal of the smell to a distant site and reproduce it. To accomplish this the electronic nose consists of a sensing unit and a pattern recognition system. The sensing element is built either as an array of several different elements measuring specific characteristics of the smell, or as one sensing device that outputs an array of measurements for each odor. This signature pattern is then compared to a pattern in a database to determine the odor. A unique mapping is produced for each odor. Another method to determine odors is to build an electronic nose with a unique sensor for every odor the nose must detect. This becomes difficult when there are numerous odors to be detected, and the highly selective chemical sensors required are expensive and not easily built (Keller et. al, 1995).
Artificial Neural Networks (ANN) have long been used to analyze data for pattern recognition. Experiments have found that when an ANN is combined with a sensor array the number of detectable odors is greater than for the sensor array electronic nose alone. It is also shown that less sensitive sensors can be linked to an ANN and produce good results, cutting down on the need for expensive sensors.
An ANN is inspired by the way biological nervous systems such as the human brain process information. An ANN, like a human, learns from experience and examples, with each ANN configured for a specific application such as data classification or pattern recognition. In an ANN information is processed in parallel and the speed and performance is substantial once the ANN is trained by repetitive presentations of data from a known source. This is known as the back propagation technique, and during the training process known odors is compared to the ANN output. Initial weightings are given and modified as the training process progresses. The difference between the targeted output and the actual output is sent back through the network and the magnitudes of the weightings are changed. This is continuous until the difference between the ANN output and the actual value is minimal (Gardner and Bartlett, 1992). Identification time for an odor in a trained ANN is simply the time it takes for the movement of data through the system (Keller et. al, 1995).
An ANN normally has three layers – input, hidden, and output. The training is done repeatedly so that the desired output is obtained on the output unit. There is one output unit per odor required to be identified (Gardner and Bartlett, 1992). Figure 2 illustrates the ANN and the flow through it. The output from the sensor (S) is fed into the input layer, which feeds the signal into the artificial neurons in the hidden layer. The processed signals from the hidden layer are sent into the outer layer where the output for the odor identification (T) is produced.
Figure 2 Artificial Neural Network (ANN)
Various types of sensor arrays may be used in an artificial nose, with five categories identified as electrochemical, mechanical, radiant, thermal, and magnetic. The use and design of the electronic nose will determine the type of sensor needed, with electro-chemical sensors most commonly used (Gardner and Bartlett, 1992).
In an actual working model the sensor array smells the odor, and the signal is digitized. This signal is fed into a computer where the ANN is residing in software. The ANN identifies the odor as a chemical compound, and this information is sent through to the odor regeneration system and transmitted to the user (Keller et. al, 1995). ANNs have proved successful in detecting odors in many applications; analyzing fuel mixtures, detecting oil leaks, identifying odors from household chemicals, quantifying components in gas mixtures, and monitoring the odors of food and beverages (Keller et. al, 1995). Continued research into ANNs and sensors will expand the detectable odors and decrease the time it takes the ANNs to learn the odor pattern. A model of the chemical vapor sensing system is presented in figure 3. The smell is sensed by the chemical sensor array, transmitted through the array to the ANN and the odor is identified.
Figure 3 Chemical Vapor Sensing System
In their paper, Barfield and Danas state that a person often will have a reduced feeling of presence while immersed in a virtual environment. The reason for this is the conflicting olfactory inputs experienced because:
most current virtual environments are inappropriately scented, even though no effort is made to provide olfactory information to the virtual environment participant. This is because ambient cues are always present in the environment the human physically occupies (1995).
People react differently to the same smell and concentration, and according to Cater "odor is identifiable by humans over a 100dB range (10 dB greater than our visual dynamic range!)"(1992). These facts magnify the requirement that for olfactory displays it is necessary to determine the threshold needed for the individual to perceive a particular odor from a mixture of several odors. According to Barfield and Danas more research is required to determine the necessary concentration of a smell to distinguish it from the ambient smells. Further, the ability to detect an odor is affected by previous exposure to other olfactory inputs, mostly in the way that a person is impaired from smelling a new scent. In some instances a person "may require 20 to 60 sec intervals for resolution of differing odors" (Cater, 1992). However, under some circumstances the sensitivity to a smell may be enhanced by a previous stimulus (Engen and Bosak, 1969, in Barfield and Danas, 1995).
Research in Olfactory displays and Olfaction for VR
Barfield and Danas define "a virtual olfactory display as a collection of hardware, software, and chemicals that can be used to present olfactory information to the virtual environment participant" (1995) (see figure 4). Further, they discuss needed characteristics of an olfactometer, a device used to define and manipulate the concentration, duration, and flow rate of an odorant for presentation to a user via the olfactory display.
Figure 4 Virtual Olfactory Display Conceptual Schematic
They conclude that no standard method for the description of an olfactometer exists, and offer a set of characteristics they believe can be used. The basis for these characteristics is McKenna’s and Zeltzer’s study, published in 1992, of visual displays and their formation of a basis for comparing display technologies.
Richardson and Zucco stipulated that about 400000 odorous substances exist. This is suggested as a "true-odor" limit, analogous to the 16.7 million colors needed for a " true-color" display.
Further, studies by Von Bekesy (1964) indicates the ability of humans to localize an odor to about 10 degrees in the horizontal plane, as well as having a field of smell subtending about 130 degrees.
The intensity of an odor in an olfactory display is determined by the concentration of the odorant in the smelling volume. Dravnieks stated in 1975 that humans have a concentration range spanning three orders of magnitude.
To have a believable delivery of odors to a subject the smelling volume delivered must be continually refreshed as well as being updated, maybe by purging, to reflect changes in the environment.
Suitable units for information rate, the data rate needed to accomplish a smelling volume condition, and bandwidth, the capability of an olfactometer to create different odors and concentrations, must be established. The suggestions are ppm/sec for information rate and Hz for bandwidth (Barfield and Danas, 1995).
Several technologies exist for storage of odors, including liquids, gels, and solid waxes, some of which have been used for early olfactory displays. For release of the odors several of these storage methods require that the compound is vaporized using heat or electrostatic methods. Other considerations to make concern the actual method of delivery to the user. There has to be air flowing to transport the smell and some sort of local presentation just in front of his or her nose (Youngblut et. al, 1996). Further, an olfactory display needs to clean the air input and be able to evacuate and clean exhaled air.
One viable method at this time utilize microencapsulated odorants. At the Southwest Research Institute, in San Antonio, Texas, John Cater and others at the Deep Immersion Virtual Environment Laboratory (DIVE) developed the first known display, a system called DIVEpak, capable of delivering eight odors (Youngblut, et. al, 1996).
[A] proprietary micro-encapsulation technique is used to encapsulate and occlude the odorants when not desired. On need for activation and delivery of any selected odor from a cartridge of pre-selected and encapsulated odors, the system performs somewhat like an automatic scratch-n-sniff player, providing computer-selected variable-intensity odors to the nose with a 1/4 sec response time (Cater, 1992).
This system requires that original odors are analyzed with a gas chromatograph, synthesized, and encapsulated, however, it is repeatable and brings an exact smell to the virtually immersed person. The main problem with this system is that only a set of predetermined odors can be presented at a given time.
Other delivery methods include the use of micro-valves or nozzles similar to those used on ink-jet printers, however, these need considerable work before they can become practical for use in virtual environments.
Two commercially available products were identified by Youngblut et. al. The first was produced by the BOC Group plc, of Britain, and transports odorants in a high pressure solvent, like carbon dioxide, to the user. The other was developed by Ferris Productions, Inc. and uses a low pressure air stream into which controlled amounts of liquid odors are released. Both of the systems can be used with or without a head-mounted display. Further, Youngblut et. al discusses four research groups (the University of Pisa, Dowling College, Artificial Reality Corporation, and Marketing Aromatics, Ltd.) identified as investigating olfactory display systems at the time. The research covers the determination of odors for reproduction in a display, as well as the benefits of olfactory stimuli and technical aspects of delivery systems (1996). Further literature concerning the research by these groups was not located.
5, Inc., in collaboration with the US Air Force Fire Research Group, has developed the FiVe FiRe Training system for use with virtual reality equipment (see figures 5 and 6).
The odor system is back mountable, except for the pressure tank, of course, and supplies up to seven simultaneous odors to the trainee's facemask under forced air flow. Odors range from burning wood, grease and rubber to sulfur, oil and diesel exhaust. Lifetime of the odor cartridges is 6 months to a year without refilling. Olfactory output is …completely proportional from a hint of odor to a stench that makes you want to rip the mask off. …It's all quite impressive and adds about $5,000 to the cost of a VR system (Cater, 1999).
Figure 5 FiVe FiRe Training System (Cater, 1999)
Figure 6 FiVe FiRe in Use (Cater, 1999)
Proposed Prototype and Areas of Future Research
Any olfactory interface in a virtual environment must have specific considerations for that environment. Some common considerations for olfactory interfaces are directionality, portability, operating capacity, and environment modeling.
Directionality refers to the process of locating an odor at a specific location in space. Similar to a stereoscopic head mounted display, an olfactory interface would have to use head orientation and position tracking to present a localized smell. Further, a separate display for each nostril is required to provide inter nasal time and intensity differences.
Another issue is that an olfactory display must be portable to facilitate unencumbered motion. Since any olfactory interface developed will have to handle physical substances to produce the smell, miniaturization will not be as easy as with purely electronic interfaces. Microencapsulation technologies for the storage of odors have been successfully employed, and literature searches have not provided evidence that better methods are developed. There are various ways to release a microencapsulated odor, including heat release, micro valves, and ink-jet printer type nozzles. All these methods, though small, will take up considerable space compared to an electronic interface. CAVE type technologies would help in the presentation of ambient odors, however, they have disadvantages. One is that localization of a smell will be more difficult to accomplish.
Unlike a visual or auditory display that can continually access a specific file to reproduce a certain scene or sound, there is an operating capacity consideration for olfactory influences. Like a printer cartridge, olfactory storage technologies have a finite capacity until the odorant must be replenished. Operating capacity is directly linked to the portability issue. CAVE technologies also have the capability of storing a larger capacity of odorants than a head mounted display unit.
Certain other aspects of the environment must be modeled and implemented in an olfactory interface. The time spent in areas where odors are present will determine the need to refresh the odors. Habituation is the desensitization to a particular stimulus after continual exposure to that stimulus. Odors appear strong at first, but a person becomes desensitized after some time. E.g. after a long day of work at a fish market, a person will not notice the strong smell of fish on their clothes and hands. The interface must determine the amount of time exposed to an odor and adjust the concentration accordingly (ASTM, 1967).
The ability to mix a predetermined set of chemicals in order to produce all odors is not yet feasible. We propose that a prototype for olfactory interface can be designed with interchangeable odor packets. Odor packets containing specific odors can be used to propagate a predetermined set of odors for a specific virtual reality application. The prototype must have position tracking and microencapsulation of the odors may be used. Ink-jet type nozzles or micro valves are used to control the presentation of the odors. The system physically would be set up with a belt pack and tubes running to and from a head mounted display and a mask. This allows the user to be untethered. If the goal is to leave the user unencumbered from a belt or backpack loaded with odors, there are other possible methods. For instance, the use of a sealed room or pod in which the user only inhales treated air and air exhaled is evacuated from the system could be used.
Research in olfactory interfaces can be broken down into five general subjects. The first is position tracking. For a person moving through a virtual world and smelling objects as he or she comes upon them, the integration of the smell technologies and the user’s position is essential. Odor storage and delivery are the next areas. Providing an efficient method of storage is important to producing a non-obtrusive interface as is the method of delivery. Concentration and duration regulation require further study to ensure the strength of the smell is proper and that the smell persists only as long as it would in the natural environment. The last and most important area is how to produce the odors. An ideal olfactory interface will be able to produce any smell from a limited number of chemicals or odors stored, much like colors are mixed in a printer.
It has taken years for visual and tactile interface designs to develop in virtual reality and they are still not at the level where the common consumer would use or purchase them. Ivan Sutherland’s obtrusive head mounted displays of the 1960’s were only able to produce visual images of simple shapes. This is very primitive as compared to the head mounted displays available currently. Olfactory interfaces will follow the same path, but we believe at a quicker pace. ARPA sponsored programs and the defense sciences office are conducting current research into olfactory interfaces. Breakthroughs in these technologies will come from well funded government programs similar to some of the visual VR technologies. The economic incentive for the private sector to develop these technologies is limited.
The potential use for olfactory interfaces is great. Humans are accustomed to using all five senses and duplicating this in virtual reality is crucial to the success of the medium. To arrive at the full potential of virtual reality, olfactory interfaces must be incorporated.