Michael Tidwell, Richard S. Johnston, David Melville, and Thomas A. Furness III, Ph.D.
Human Interface Technology Laboratory
University of Washington, Seattle, WA 98195
The Virtual Retinal Display (VRD) is a personal display device under development at the University of Washington's Human Interface Technology Laboratory in Seattle, Washington USA. The VRD scans light directly onto the viewer's retina. The viewer perceives a wide field of view image. Because the VRD scans light directly on the retina, the VRD is not a screen based technology. There are no liquid crystal displays or cathode ray tubes (CRTs) in the system. The purpose of this paper is to explain basic concepts of VRD operation, present some VRD performance achievements, and to discuss applications for the VRD. The paper also addresses issues related to the viability of the VRD as a commercial product.
The Human Interface Technology Laboratory (HITL) of the Washington Technology Center at the University of Washington is developing a novel display device in which a coherent light source is utilized to scan an image directly on the retina of the viewer's eye. A prototype of this device, the Virtual Retinal Display (VRD), has been developed and is being perfected under a four-year project. The work is funded by Micro Vision, Inc., Seattle, which holds an exclusive license to manufacture and distribute the VRD.
Using the VRD technology, the long-range project goal is to build a display with the following characteristics:
* Very small and lightweight, mountable on eye glasses
* High resolution, approaching that of human vision
* Large field of view, greater than 100 degrees per eye
* Full color with superior color resolution as compared to standard displays
* Capable of fully inclusive or see-through display modes
* Brightness sufficient for outdoor use
* Very low power consumption
* True stereo display with depth modulation
A brief review of how the eye forms an image will aid in understanding the VRD.
A point source emits waves of light which radiate in ever-expanding circles about the point. The pupil of an eye, looking at the source, will see a small portion of the wavefront. The curvature of the wavefront as it enters the pupil is determined by the distance of the eye from the source. As the source moves farther away, less curvature is exhibited by the wavefronts. It is the wavefront curvature which determines where the eye must focus in order to create a sharp image (see Figure 1).
If the eye is an infinite distance from the source, plane waves enter the pupil. The lens of the eye images the plane waves to a spot on the retina. The spot size is limited by the aberrations in the lens of the eye and by the diffraction of the light through the pupil. It is the angle at which the plane wave enters the eye that determines where on the retina the spot is formed. Two points focus to different spots on the retina because the wavefronts from the points are intersecting the pupil at different angles (see Figure 2).
Neglecting the aberrations in the lens of the eye, one can determine the limit of the eye's resolution based on diffraction through the pupil. Using Rayleigh's criteria  the minimum angular resolution is computed as follows:
angular resolution = 1.22 lambda / D
lambda = wavelength of light
D = diameter of the pupil
If we assume a 2 mm pupil diameter (the size in a bright light situation) and light near the center of the visible spectrum at 550 nm, the minimum angular separation required to resolve two points is 1.15 arc minutes. Thus, to approach the resolution of the eye, the VRD must be capable of scanning with angular resolution of less than 2 arc minutes.
The following sections describe the operational concepts and features of the VRD.
3.1 The Basic System
In a conventional display a real image is produced. The real image is either viewed directly or, as in the case with most head-mounted displays, projected through an optical system and the resulting virtual image is viewed. The projection moves the virtual image to a distance that allows the eye to focus comfortably. No real image is ever produced with the VRD. Rather, an image is formed directly on the retina of the user's eye. A block diagram of the VRD is shown in Figure 3.
To create an image with the VRD a photon source (or three sources in the case of a color display) is used to generate a coherent beam of light. The use of a coherent source (such as a laser diode) allows the system to draw a diffraction limited spot on the retina. The light beam is intensity modulated to match the intensity of the image being rendered. The modulation can be accomplished after the beam is generated. If the source has enough modulation bandwidth, as in the case of a laser diode, the source can be modulated directly.
The resulting modulated beam is then scanned to place each image point, or pixel, at the proper position on the retina. A variety of scan patterns are possible. The scanner could be used in a calligraphic mode, in which the lines that form the image are drawn directly, or in a raster mode, much like standard computer monitors or television. Our development focuses on the raster method of image scanning and allows the VRD to be driven by standard video sources. To draw the raster, a horizontal scanner moves the beam to draw a row of pixels. The vertical scanner then moves the beam to the next line where another row of pixels is drawn.
After scanning, the optical beam must be properly projected into the eye. The goal is for the exit pupil of the VRD to be coplanar with the entrance pupil of the eye. The lens and cornea of the eye will then focus the beam on the retina, forming a spot. The position on the retina where the eye focuses the spot is determined by the angle at which light enters the eye. This angle is determined by the scanners and is continually varying in a raster pattern. The brightness of the focused spot is determined by the intensity modulation of the light beam. The intensity modulated moving spot, focused through the eye, draws an image on the retina. The eye's persistence allows the image to appear continuous and stable.
Finally, the drive electronics synchronize the scanners and intensity modulator with the incoming video signal in such a manner that a stable image is formed.
A good understanding of different methods of scanning a light source on the retina can be found in the papers of Webb, et al [2,3,4]. In his scanning laser ophthalmoscope development, Webb scanned a laser into the eye forming an NTSC resolution raster on the retina. The reflection of the light off the retina was then captured. Using this technique an image of the retina could be formed.
3.2 VRD Features
The following sections detail some of the advantages of using the VRD as a personal display.
3.2.1 Size and Weight
The VRD does not require an intermediate image on a screen as do systems using LCD or CRT technology. The only required components are the photon source (preferably one that is directly modulatable), the scanners, and the optical projection system. Small photon sources such as a laser diode can be used. As described below the scanning can be accomplished with a small mechanical resonant device developed in the HITL. The projection optics could be incorporated as the front, reflecting, surface of a pair of glasses in a head mount configuration or as a simple lens in a hand held configuration. HITL engineers have experimented with single piece Fresnel lenses with encouraging results. The small number of components and lack of an intermediate screen will yield a system that can be comfortably head mounted or hand held.
Resolution of the current generation of head mounted and hand held display devices is limited by the physical parameters associated with manufacturing the LCDs or CRTs used to create the image. No such limit exists in the VRD. The limiting factors in the VRD are diffraction and optical aberrations from the optical components of the system, limits in scanning frequency, and the modulation bandwidth of the photon source.
A photon source such as a laser diode has a sufficient modulation bandwidth to handle displays with well over a million pixels. If greater resolution is required multiple sources can be used.
Currently developed scanners will allow displays over 1000 lines allowing for the HDTV resolution systems. If higher resolutions are desired multiple sources, each striking the scanning surface at a different angle, can be used.
If care is taken in the optical system design then the primary cause of diffraction will be the primary scanning aperture. The aperture with the current scanner, developed in the HITL, is a mirror. The scan angle from the mirror must be magnified for large field of view systems yielding a smaller effective aperture. The current mirror size of 3 millimeters will limit resolution in a 50 degree field of view system to better than two arc minutes. Further refinement in scanner design should improve this figure.
3.2.3 Field of View
The field of view of the VRD is controlled by the scan angle of the primary scanner and the power of the optical system. Initial inclusive systems with greater than 60 degree horizontal fields of view have been demonstrated. Inclusive systems with 100 degree fields of view are feasible. See through systems will have somewhat smaller fields of view. Current see through systems with over 40 degree horizontal fields of view have been demonstrated.
3.2.4 Color and Intensity Resolution
Color will be generated in a VRD by using three photon sources, a red, a green, and a blue. The three colors will be combined such that they overlap in space. This will yield a single spot color pixel, as compared to the traditional method of closely spacing a triad, improving spatial resolution.
The intensity seen by the viewer of the VRD is directly related to the intensity emitted by the photon source. Intensity of a photon source such as a laser diode is controlled by the current driving the device. Proper control of the current will allow greater than ten bits of intensity resolution per color.
Brightness may be the biggest advantage of the VRD concept. The current generation of personal displays do not perform well in high illumination environments. This can cause significant problems when the system is to be used by a soldier outdoors or by a doctor in a well lit operating room. The common solution is to block out as much ambient light as possible. Unfortunately, this does not work well when a see through mode is required.
The VRD creates an image by scanning a light source directly on the retina. The perceived brightness is only limited by the power of the light source. Through experimentation it has been determined that a bright image can be created with under one microwatt of laser light. Laser diodes in the several milliwatt range are common. As a result, systems created with laser diode sources will operate at low laser output levels or with significant beam attenuation.
3.2.6 Power Consumption
The VRD delivers light to the retina efficiently. The exit pupil of the system can be made relatively small allowing most of the generated light to enter the eye. In addition, the scanning is done with a resonant device which is operating with a high figure of merit, or Q, and is also very efficient. The result is a system that needs very little power to operate.
3.2.7 A True Stereoscopic Display
The traditional head-mounted display used for creating three dimensional views projects different images into each of the viewer's eyes. Each image is created from a slightly different view point creating a stereo pair. This method allows one important depth cue to be used, but also creates a conflict. The human uses many different cues to perceive depth. In addition to stereo vision, accommodation is an important element in judging depth. Accommodation refers to the distance at which the eye is focused to see a clear image. The virtual imaging optics used in current head-mounted displays place the image at a comfortable, and fixed, focal distance. As the image originates from a flat screen, everything in the virtual image, in terms of accommodation, is located at the same focal distance. Therefore, while the stereo cues tell the viewer an object is positioned at one distance, the accommodation cue indicates it is positioned at a different distance.
With the VRD it is theoretically (this is currently in the development stage) possible to generate a more natural three dimensional image. The VRD has an individual wavefront generated for each pixel. It is possible to vary the curvature of the wavefronts. Note that it is the wavefront curvature which determines the focus depth. This variation of the image focus distance on a pixel by pixel basis, combined with the projection of stereo images, allows for the creation of a more natural three-dimensional environment.
3.2.8 Inclusive and See Through
Systems have been produced that operate in both an inclusive and a see through mode. The see through mode is generally a more difficult system to build as most displays are not bright enough to work in a see through mode when used in a medium to high illumination environment where the luminance can reach ten thousand candela per meter squared. As discussed above, this is not a problem with the VRD.
Using seed funds from the Washington Technology Center the first VRD prototype was developed in the HITL by Dr. Tom Furness, Joel Kollin, and Bob Burstein . The project's initial goal was to prove the viability of forming an image on the retina using a scanned laser. As a result of the work, a patent application was filed and the technology licensed to a Seattle based start up company, Micro Vision, Inc. Under terms of the agreement, Micro Vision is funding a four-year effort in the HITL to develop the technologies that will lead to a commercially viable VRD product. This development work began in November 1993.
4.1 Prototype #1
The original prototype had very low effective resolution, a small field of view, limited gray scale, and was difficult to align with the eye. One objective of the current development effort was to quickly produce a bench-mounted system with improved performance. This new system, Prototype #1, performed to specifications as shown in Table 1.
Prototype #1 uses a directly modulated red laser diode at a wave length of 635 nanometers as the light source. The required horizontal scanning rate of 73,728 Hertz could not be accomplished with a simple galvanometer or similar commercially available moving mirror scanner. The use of a rotating polygon was deemed impractical because of the polygon size and rotational velocity required. It was thus decided to perform the horizontal scan with an acousto-optical scanner. The vertical scanning rate of 72 Hertz is within the range of commercially available moving mirrors and is accomplished with a galvanometer.
The use of the acousto-optical scanner comes with a number of drawbacks:
* It requires optics to shape the input beam for deflection and then additional optics to reform the output beam to the desired shape. Figure 4 is a schematic of the optical path for Prototype #1. Total optical path length for this system is 45 centimeters.
* It requires complex drive electronics that operate at frequencies between 1.2 GHz and 1.8 GHz.
* Its total scan angle is 4 degrees. Thus, additional optics are needed to increase the angle to the desired field-of-view. Due to the optical invariant, this optical increase in angle comes with the penalty of decreased beam diameter which leads to a small exit pupil. The small exit pupil necessitates precise alignment with the eye for an image to be visible.
* It is expensive and will not, in the foreseeable future, allow us to reach our cost goals for a complete VRD system.
4.2 Prototype #2
To overcome the limitations of the acousto-optical scanner, HITL engineers have developed a miniature mechanical resonant scanner. This scanner, in conjunction with a conventional galvanometer, provides both horizontal and vertical scanning with large scan angles, in a compact package. The estimated recurring cost of this scanner will allow the VRD system to be priced competitively with other displays. Prototype #2 of the VRD uses the mechanical resonant scanner. Achieved performance specifications for this system are given in Table 2. The system was built and demonstrated during the summer of 1994. The VGA resolution images produced are sharp and spatially stable. A schematic of the optical path of Prototype #2 is shown in Figure 5. The total optical path length for this system is 8 centimeters.
4.3 Mechanical Resonant Scanner
The mechanical resonant scanner has many unique features. Foremost among these is the fact that the device has neither a moving magnet nor a moving coil. Instead, it uses a flux circuit whose only moving part is the torsional spring/mirror combination. Eliminating moving coils or magnets greatly lowers the rotational inertia of the device, thus raising the potential operating frequency. Figure 6 shows a drawing of the current version of the mechanical resonant scanner. Dimensions of the scanner are .9 centimeters high by 1.3 centimeters wide by 2.8 centimeters long.
The mechanical resonant scanner is used in conjunction with a conventional galvanometer in a combination which allows for an increase in the optical scan angle. When the mirrors of the two scanners are arranged in such a manner that a light beam undergoes multiple reflections off the mirrors, then the optical scan is multiplied by the number of reflections off that mirror. Optical scan multiplication factors of 2X, 3X and 4X have been realized. Prototype #2 uses a system with 2X scan multiplication in the horizontal axis.
4.4 Prototype #3
The third prototype system developed uses the same scanning hardware as Prototype #2 but uses three light sources to produce a full color image. In addition the eyepiece optics have been modified to allow for see through operation. In the see through mode the image produced by the VRD is overlaid on the external world. Performance specifications for this system are shown in Table 3.
4.5 Future Prototypes
Our plan calls for a new VRD prototype every six months. Each prototype will incrementally add features to the system. Future prototype work will concentrate on improving image resolution, increasing image field of view, expanding the exit pupil, generating full color images, and shrinking the system to head-mounted size.
4.6 Basic Research
While development results during the first year have been very positive, much remains to be accomplished. A number of technical challenges must be overcome in order to make the VRD a commercially viable product. Basic research is, therefore, continuing in a number of areas.
4.6.1 Color System Development
In order to produce full color images three sources must be used. The blue and green color sources used in Prototype #3 are gas lasers that are larger than desired for a portable system. Laser diodes can now be purchased for the red source, however, green and blue laser diodes are not currently available. A significant industrial effort is under way to develop these shorter wavelength devices , however, the devices are not likely to be available for a number of years. As an alternative, small green lasers are now being produced which use a crystal to frequency double a neodymium YAG laser. These devices are larger than desired and are not directly modulatable at the required frequency. They do however, offer a short term solution.
In the HITL we are investigating a number of alternatives to blue and green laser diodes. One frequency doubling technique being researched uses rare earth doped fibers as the doubling medium. A second technique uses wave guides placed in a lithium niobate substrate for the doubling.
The above methods all utilize a laser as the light source. Additional work is directed at using non-lazing, light-emitting diodes (LEDs) as the light source. In order for this to be successful two primary issues are being addressed. The first issue is how to focus the LED output to the desired spot size. The second issue is the development of fabrication techniques that will allow us to directly modulate the LEDs at the desired frequency.
4.6.2 Exit Pupil
The exit pupil in the current prototypes is still quite small. The exit pupil for Prototype #2, for example, is approximately 1.5 millimeters. Thus, the eye must be aligned with the exit pupil to view the image. This will not present an issue in a hand held unit but is not optimal for a head mounted unit. Methods of enlarging the exit pupil are therefore being developed.
4.6.3 Holographic Optical Elements
To minimize the weight of optical components in the system holographic optical elements are being developed. A complete holographic laboratory has been set up. Results to date include the development of an off-axis reflective lens that can be used in a see through system to combine the video image with the outside world view.
4.6.4 Advanced Scanning Methods
The mechanical resonant scanner has been demonstrated to work for displays up to 800 lines. Development of an even faster mechanical resonant scanner is underway. Based on this work it appears practical to build mechanical resonant scanners that operate at greater than 1000 lines at 72 Hertz.
Theoretically the VRD can produce images that approach the resolution of the eye. This will require scanners that can operate with over 3000 resolution lines. Methods including parallel scanning and ultra high speed scanners are being investigated to meet this requirement.
Application industries for the VRD range from medicine to manufacturing, from communications to traditional virtual reality helmet mounted displays (HMD's). The VRD provides high luminance and high resolution and can also be configured as see-through or inclusive (non-see-through), head mounted or hand held, making it adaptable to a number of applications. Some specific applications in the aforementioned industries are described in subsequent sections.
One examination performed by radiologists is the fluoroscopic examination. During a fluoroscopic examination, the radiologist observes the patient with real-time video x-rays. The radiologist must continually adjust the patient and the examination table until the patient is in a desired position. When the patient is in a desired position, the radiologist takes a film copy of the x-ray image. The positioning process can be difficult and cumbersome because the radiologist must visually keep track of a patient, a video monitor, and an examination table simultaneously. Because the VRD can operate in a see-through mode at high luminance levels, it is an ideal display to replace the bulky video monitor in a fluoroscopic examining room. The radiologist could see through the x-ray display and see the patient as well. Other features such as a display luminance control or on/off switch could easily be included for this application.
Surgery to remove a cancerous growth requires knowledge of the growth's location. Computed tomographic or magnetic resonant images can locate a tumor inside a patient. A high luminance see-through display, such as the VRD, in conjunction with head tracking, could indicate visually where a tumor lies in the body cavity. In the case that a tumor lies hidden behind, say, an organ, the tumor location and a depth indicator could be visually laid over the obstructing organ. An application in surgery for any display would clearly require accurate and reliable head tracking.
The same characteristics that make the VRD suitable for medical applications, high luminance and high resolution, make it also very suitable for a manufacturing environment. In similar fashion to a surgery, a factory worker can use a high luminance display, in conjunction with head tracking, to obtain visual information on part or placement locations. Drawings and blueprints could also be more easily brought to a factory floor if done electronically to a Virtual Retinal Display (with the option of see-through mode). Operator interface terminals on factory floors relay information about machines and processes to workers and engineers. Thermocouple temperatures, alarms, and valve positions are just a few examples of the kind of information displayed on operator interface terminals. Eyeglass type see-through Virtual Retinal Displays could replace operator interface terminals. A high luminance eyeglass display would make the factory workers and engineers more mobile on the factory floor as they could be independent of the interface terminal location.
The compact and light weight nature of the mechanical resonant scanner (MRS) make an MRS based VRD an excellent display for personal communication. A hand held monochrome VRD could serve as a personal video pager or as a video FAX device. The display could potentially couple to a telephone. The combination of telephone services and video capability would constitute a full service personal communication device.
5.5 Virtual Reality
The traditional helmet display is an integral part of virtual reality today. The VRD will be adapted for this application. It can then be used for educational and architectural applications in virtual reality as well as long distance virtual conference communications. Indeed it can be utilized in all applications of virtual reality. The theoretical limits of the display, which are essentially the limits of the eye, make it a promising technology for the future in virtual reality HMD's.
The ongoing VRD development project at the HITL has proven the viability of building displays which scan images directly on the viewer's retina. Such displays offer performance improvements when compared to currently available head-mounted displays in the following areas:
* Size and weight
* Power consumption
To date, three prototype systems have been constructed. The current prototype uses a proprietary mechanical resonant scanner to generate VGA resolution color images. The system's simple optical design yields a device that is small and easy to adjust when compared to earlier prototypes utilizing acousto-optic scanners for horizontal deflection.
Many challenges remain before the VRD reaches it's full potential. Chief among these is the development of the low cost blue and green light sources needed for a full color display.
Finally, the VRD is applicable to a wide variety of applications in a number of fields including medicine, manufacturing, communications, and virtual reality.
 E. Hecht and A. Zajac, Optics, 1979, Addison-Wesley, pp. 353-354.
 R. H. Webb, G. W. Hughes,, and O. Pomerantzeff, "Flying spot TV ophthalmoscope", Applied Optics, Vol. 19, pp. 2991, 1980.
 R. H. Webb, "Optics for laser rasters", Applied Optics, Vol. 23, pp. 3680, 1984.
 R. H. Webb, G. W. Hughes, F. C. Delori, "Confocal scanning laser ophthalmoscope", Applied Optics, Vol. 26, pp. 1492, 1987.
 J. Kollin, "A Retinal Display for Virtual-Environment Applications", Proceedings of the Society for Information Display, Vol. 24, pp. 827, 1993.
 G. F. Neumark, R. M. Park, and J. M. DePuydt, "Blue-Green Diode Lasers", Physics Today, June 1994, pp. 26.