Copyright 1995 Society of Photo-Optical Instrumentation Engineers.

This paper was published in Proceedings of Helmet-and Head-Mounted Displays and Symbology Design Requirements II and is made available as an electronic reprint with permission of SPIE. Single print or electronic copies for personal use only are allowed. Systematic or multiple reproduction, or distribution to multiple locations through an electronic listserver or other electronic means, or duplication of any material in this paper for a fee or for commercial purposes is prohibited. By choosing to view or print this document, you agree to all the provisions of the copyright law protecting it.

The Human Interface Technology Laboratory at the University of Washington is developing a new display device, the Virtual Retinal Display (VRD), in which a coherent light source is used to scan an image directly on the retina of the viewerís eye. Development work is funded by Micro Vision, Inc., Seattle, which holds an exclusive license to manufacture and distribute the VRD. Using the VRD technology it is possible to build a high resolution, wide field-of-view, full color personal display device that is light weight and will operate in a high brightness environment. Current work is aimed at developing the technologies that will make the VRD a commercially viable product from both a performance and cost standpoint. Prototypes produced to date include a full color, VGA resolution device based on a unique mechanical resonant scanner as the horizontal scanning element. This paper will briefly explain the VRD concept and discuss potential applications of the technology. It will also describe the current research and development efforts which are aimed at creating a high performance yet low cost display system.

The Human Interface Technology Laboratory (HITL) at the University of Washington, in partnership with Micro Vision, Inc., is developing the Virtual Retinal Display (VRD) 1,2, a novel display device that does not require the use of a cathode ray tube or flat panel display screen. Instead, a coherent light source is utilized to scan an image on the retina of the viewerís eye. The VRD approach has several advantages:

• The display's resolution is limited by diffraction and optical aberrations in the scanned beam and not by how small an individual pixel element in a large array of pixel elements can be made. It can therefore display very high resolution images.
  

• The display's brightness is controlled by the brightness of the scanned beam. With a laser as the light source the display is bright enough for use outdoors, on a bright day.
  

• The display can operate in either an inclusive or a see-through mode. The see-through mode is generally a more difficult system to build as most displays are not bright enough to work in this mode when used in a medium to high illumination environment.
  

• The display requires only simple and well understood manufacturing processes and thus can be low cost.
  

• The display projects most of its generated light into the eye and can thus be very energy efficient, resulting in low power consumption.
  

• All components in the VRD are small and light, ideally suited to a portable display.
  

To create an image with the VRD a photon source (or three sources in the case of a color display) is used to generate a beam of light. The use of a coherent source (such as a laser diode) allows the system to draw a diffraction limited spot on the retina. The light beam is intensity modulated to match the intensity of the image being rendered. The modulation can be accomplished after the beam is generated. If the source has enough modulation bandwidth, as in the case of a laser diode, the source can be modulated directly.

The resulting modulated beam is then scanned to place each image point, or pixel, at the proper position on the retina. Our development focuses on the raster method of image scanning and allows the VRD to be driven by standard video sources. To draw the raster, a horizontal scanner moves the beam to draw a row of pixels. The vertical scanner then moves the beam to the next line where another row of pixels is drawn.

After scanning, the optical beam must be properly projected onto the retina. The goal is for the exit pupil of the VRD to be coplanar with the entrance pupil of the eye. The lens and cornea of the eye will then focus the moving beam on the retina, forming an image. A simple block diagram of the VRD is shown in Figure 1.

The process of scanning a coherent light source on the retina was used by Webb 3,4 in the development of the scanning laser ophthalmoscope. In Webb's system the portion of the optical beam that reflected off the retina and passed back through the lens and cornea of the eye was captured. The captured signal was then used to modulate a synchronized video signal allowing an image of the retina to be displayed on a CRT. Webb noted that if the input optical beam were modulated, the patient perceived an image.

Many of the issues that need to be considered for a commercial display device were not addressed by Webb because they were not as important for a special purpose medical instrumentation system. These issues include cost, size, portability, high resolution, and color It is the goal of our current work to develop the technologies that will allow commercially viable products to be produced based on the retinal scanning concept. Specifically, our development targets are as follows:

• System manufacturing cost should be less than $100 for a monocular display and $200 for a stereo display in large quantities.
  

• A head mounted stereo display should weight less than 8 ounces.
  

• Systems should be capable of resolutions of 1024 lines at a 60 Hertz frame refresh rate.
  

• Full color and monochrome systems should be available.
  

• Systems should be bright enough to allow see-through operation outdoors.
  

A key issue in developing a cost effective and portable VRD is developing a method of scanning the optical beam in the horizontal direction. The ideal horizontal scanner would scan a large diameter optical beam over a wide scan angle at a high scan frequency. In addition, the scanner would not introduce optical or chromatic aberrations. Specific requirements of the horizontal scanner are discussed below.

To be compatible with existing video standards it is desired that the VRD scan in traditional raster formats. The scan rates for these formats can be determined by multiplying the number of lines in the display by the refresh rate of the display. On the low end is the RS-170 standard for interlaced video which contains 525 lines that are refreshed 30 times per second resulting in a horizontal scanning frequency of 15,750 Hertz. A typical high resolution computer monitor contains 1024 lines that are refreshed 72 times a second, for a horizontal scan rate of 73,728 Hertz.

The field-of-view or image size seen by the user is directly related to the angle through which the optical beam is scanned. The scan angle for the faster horizontal scan is not likely to match the total angular field-of-view desired for the display. An optical system must therefore be used to magnify the scan angle. Unfortunately, because of the optical invariant, as the scan angle is optically increased the optical beam diameter is decreased. This effect can be characterized by the equation

where

D(in) = diameter of the input optical beam (or it's limiting aperture)

f(in) = half angle of the input optical beam's deflection

D(out) = diameter of the output optical beam (the exit pupil)

f(out) = half angle of the output optical beam's deflection.

Angular resolution of the display is limited by aberrations in the optical system and by diffraction. The diffraction limiting aperture in a practical VRD is the system's exit pupil which, in most practical designs, is the projection of the aperture of the horizontal scanning device. Using Rayleigh's criteria for resolving two points, angular resolution can be computed as

where

q = angular resolution

l = wavelength of light

D(out) = diameter of the circular exit pupil.

Substituting we find that

where in a VRD

D(in) = aperture of the horizontal scanner

f(in) = half angle of the horizontal scanner's deflection

f(out) = half angle of the system field-of-view.

Thus, for best resolution we desire the widest possible scanning aperture and the largest possible scanner deflection.

In addition, a small exit pupil necessitates exact eye alignment for an image to be seen. A head mounted system with too small an exit pupil will not allow wearers to move their eyes to view details at the edges of the image.

Two basic classes of scanners were analyzed for the VRD application. The first group operate on the principle of diffraction through a variable grating. These scanners include both electro-optic (primarily experimental) and acousto-optic (commercially available) devices. A typical acousto-optic scanner can operate at a high scan frequency but has a small scan angle. One representative device (manufactured by Brimrose, Inc.) has a 13.6 mm aperture and a half angle of scan of approximately 1.75 degrees. A VRD constructed using this device with a 50 degree field-of-view would have a .9 mm exit pupil and an angular resolution of 3 arc minutes.

While it is possible to build a working retinal scanning display with an acousto-optic scanner, it is difficult to built a commercial product based on this technology. First, to achieve good resolution, the scanner requires optics to shape the input beam for deflection and additional optics to reform the output beam to the desired shape. This leads to a system that is much larger than desired and one in which multiple optical surfaces must be precisely aligned. Second, the drive frequencies needed for the scanner are in the 1 to 2 gigaHertz range. The drive electronics and cabling for a system operating at these frequencies adds considerable cost and complexity to the product. Third, the acousto-optic scanner will deflect light of different wavelengths at different angles. This can be corrected for in the electronics but would add considerable cost and size to the system. Finally, the devices are expensive and will not, in the foreseeable future, allow for a cost effective display.

The second class of scanners operate by reflecting the optical beam off a mirrored surface which is moving such that the beam's angle of incidence relative to the surface is changing. Devices in this category include piezoelectric deflectors, galvanometers, rotating polygons, and resonant scanners. Piezeo electric deflectors have very small deflection angles (<1 degree) which would require significant magnification to obtain a usable field-of-view. As discussed above this leads to a very small exit pupil and low resolution. Galvanometers are capable of scanning through wide angles (>60 degrees) with a large aperture but at frequencies that are much lower than desired for the horizontal scan. These devices will work well for the slower vertical scan operating at 60 to 100 Hertz. Rotating polygons are capable of performing the horizontal scan; however the rotational velocity required and the mechanical inertia generated 5 make this method unsuitable for a hand held or head mounted display. Resonant scanners appear to offer the best solution but the current generation of commercially available resonant scanners do not operate at the desired frequencies. Typical devices operate up to approximately 8 kHz.

To meet the horizontal scanning requirements HITL engineers have developed a mechanical resonant scanner (patent applied for) with many unique features. Foremost among these is the fact that the device has neither a moving magnet nor a moving coil. Instead, it uses a flux circuit whose only moving part is the torsional spring/mirror combination. Eliminating moving coils or magnets greatly lowers the rotational inertia of the device, thus raising the potential operating frequency. Devices have been built that will support over 800 display lines at a 60 Hertz refresh rate. Figure 2 shows the mechanical resonant scanner.

The majority of our recent work has centered around developing and perfecting the mechanical resonant scanner for use in a 640 by 525 line, 60 Hertz VGA display. This device has a mirror size of 3 mm by 6 mm. The mechanical deflection, when driven at resonance, is 8 degrees. This results in an optical beam deflection of 16 degrees. The mechanical resonant scanner can be used in conjunction with a second mirror (this mirror could be stationary or, as is the case with our prototype systems, the vertical scanning mirror) which allows for an increase in the optical scan angle. The two mirrors are arranged such that an optical beam undergoes multiple reflections off the scanning mirror. When this occurs the optical scan is multiplied by the number of reflections off the scanning mirror, see Figure 3. Optical scan multiplication factors of 2X and 3X (4X has been achieved with smaller mechanical deflection) have been realized resulting in total scan angles of 32 and 48 degrees.

In addition to the large scan angle and high scanning frequency the mechanical resonant scanner exhibits several other features that make it ideal for use in a commercial VRD. The device is small, measuring .9 centimeters high by 1.3 centimeters wide by 2.8 centimeters long. The drive signal is a low (±15 volts) voltage sinusoidal or square wave at the resonant frequency. The system has a large amount of stored energy resulting in a very uniform and repeatable scan. Being a reflective device, all colors are reflected at the same angle. Finally, the mechanical resonant scanner is made from common materials and requires no exotic manufacturing processes resulting in a volume manufacturing cost estimated to be under $3.

A bench mounted prototype, using the mechanical resonant scanner as the horizontal beam scanner, has been developed. The prototype delivers VGA resolution images in full color or monochrome. Performance specifications for this system are shown in Table 1.

In the prototype the mechanical resonant scanner is packaged with a galvanometer (used for vertical scanning) in a small scanning engine. The optical path is configured such that the optical beam reflects off a relay mirror onto the horizontal scanner. From here the beam reflects off the vertical scanner and back off the horizontal scanner before exiting the scanning engine through a window. With this configuration the horizontal scan angle is doubled as described above with a scan angle of 25 degrees obtained.

In the monochrome mode, the prototype uses a single directly modulated red laser diode. Two lenses are placed after the laser to produce the desired beam. The first is a cylindrical lens that produces equal divergence in both laser axes. The second produces a slightly converging beam. The lenses are mounted with the laser and this unit is plugged directly into the scanning engine. The beam deflects off the scanners and comes to focus at a point outside the scanning engine. The focus plane is positioned at the focal point of an eyepiece. Proper choice of the eyepiece focal length will then produce the desired system field-of-view.

In the color mode, the prototype uses three light sources, a directly modulated red laser diode and externally modulated green and blue gas lasers (helium neon for green and argon for blue). The colors are combined using dichroic beamsplitters producing a single color beam. Blue and green intensity modulation is achieved using two acousto-optic modulators. One problem encountered in this arrangement is the different phase delay between the blue and green signals and the red signal. A delay is introduced in the blue and green signals through the acousto-optic modulator. This delay corresponds to the time it takes the acoustic wave to travel across the crystal to the point where the laser beam is encountered. Adjusting this for the minimum distance yielded a delay of approximately 100 nanoseconds. To compensate for this a delay line was placed in the red signal path.

The eyepiece optics have been designed to be interchangeable, allowing for see-through or fully inclusive system operation. In the fully inclusive mode, a single lens is used. The viewer looks into the lens to view the image. A horizontal field-of-view of 50 degrees has been demonstrated with an exit pupil size of 1.5 millimeters. In the see-through mode the scanned beam passes through a beamsplitter and reflects off a curved mirror before reflecting off the same beamsplitter and into the eye. At the same time a view of the outside world is passed through the beamsplitter and into the eye. Figure 4 shows a diagram of the see-through optical system. This system has a field-of-view of 40 degrees.

Test results on the resolution of the VGA prototype have been very encouraging. Text down to five point is easily read. When the display is set up with an "on" pixel followed by an "off" pixel followed by an "on" pixel, two pixels are clearly visible when the test is run in either the horizontal or vertical direction (this is also true with the contrast reversed). The HITL is currently developing the instrumentation that will automatically perform modulation transfer function measurements at any point in the image.

While development work to date has been very encouraging a number of issues must be addressed before a complete line of VRD based products is commercially available. These issues include continued development related to the mechanical resonant scanner, exit pupil size, methods of generating color in small packages, increased resolution, and safety testing.

Two issues are currently being addressed relating to the design of the mechanical resonant scanner. The first is the fact that the scanned beam moves in a sinusoidal fashion. The beam moves faster at the center of the scan than near the edges. If uncorrected, this will lead to pixels that are wider in the center of the display and compressed near the edges and to a display that is brighter at the edges than in the center. The simplest method of correction is to vary the pixel display time 7 and the pixel intensity as the beam scans across the image. For example, in a 60 Hertz VGA resolution display the line display time is 31.75 microseconds and each pixel is active for 39.7 nanoseconds (800 effective pixel periods of which 640, or 80 percent, of the pixels are visible). In the VRD the display would be blanked for 3.17 microseconds at each end of the scan to remove the highly non-linear portion of the sweep. Pixel durations would then range from 95.8 nanoseconds at the screen edges to 30.0 nanoseconds at the center of the image. Intensities would also be varied such that the intensity is increased where the beam is moving faster and decreased where the beam is moving slower.

The second mechanical resonant scanner issue is the variation of the phase delay between scanner drive signal and actual scanner position with temperature variations. As the temperature changes the resonant frequency changes slightly. This change in resonant frequency will change both the scanner deflection and the phase delay. The resonant frequency changes are small over anticipated operating temperatures and the change in deflection will result in only a few percent change in image field-of-view. Moreover, this change will be slow and is not noticeable in most applications. The phase change is, however, much greater and will result in a significant alteration of the image. To compensate for these changes a feedback mechanism is being developed that measures the phase difference and compensates for it.

The system exit pupil is still quite small. While this small exit pupil has allowed some individuals with lens and cornea damage to see images clearly, it is not desirable in most applications. Work is being performed to solve this problem in a number of ways. The best solution is to enlarge the exit pupil. As described above, a larger the scanning aperture will result in a larger exit pupil. Thus it is important to have the largest possible scanning aperture. Methods of duplicating the exit pupil are also being investigated. Finally, methods of tracking the eyes movement and moving the exit pupil such that it lines up with the eye's pupil are being analyzed.

As described above, a full color VRD has been demonstrated. The disadvantages of this system are its size and cost. The blue and green sources are not small or inexpensive. Unfortunately neither blue or green laser diodes are currently available 8 as they are in the red. As laser alternatives, HITL staff are working on two frequency doubling methods, waveguide doubling and fiber doubling. The use of non lasing sources, particularly light emitting diodes, is also being researched. In fact, when the system is operated with the red laser diode below the lasing threshold an image is visible.

Moving the system to higher resolutions should present no significant problems. The current mechanical resonant scanner design should scale up to allow for a 1000 line display. Measured performance of the optical system will allow better than 2 arc minute resolution. If more than 1000 lines are desired a system consisting of the mechanical resonant scanner and a single light source will not be adequate. Other solutions being researched involve the use of very fast electro-optical scanning methods.

A second issue investigated is the minimum time the light source must illuminate a rod or cone for an image to be perceived. As the spot will be moving faster for higher resolutions this could present a limiting factor. With the current VGA resolution system and an earlier acousto-optical scanner based system displaying 1024 lines of data, this does not present a problem. Images are clearly visible and the viewer perceives high intensity resolution. Flicker frequencies have also been measured and match closely those found with other displays. As the eye's response to light occurs in under 200 femtoseconds 9, we believe this issue will not be a problem.

As with any display device the issue of user safety must be addressed. The use of coherent light sources in the VRD has highlighted this issue. Because of the low power levels required in the VRD we believe it does not present a safety hazard 5. Measured power levels into the eye typically are iunder 300 nanowatts. During the second half of 1995 we will begin an extensive series of safety tests on the VRD. These tests will be performed under the supervision of Dr. Eric Viirre, an ophthalmologist with a joint appointment to the HITL and the University of Washington School of Ophthalmology.

It is expected that the VRD will be an important market driver in the emerging personal display segment of the computer interface market as the VRD is light enough and small enough to render the category viable and the category's use pervasive. In addition, the ability to use the VRD in a see-through mode where generated text or images are superimposed over the user's normal view of the outside world opens a whole new range of applications.

For example, the VRD could be configured as a personalized medical terminal that interfaces to existing computing and communications infrastructure. Medical staff would wear a VRD personal display terminal much in the same way they presently carry a stethoscope. This display could be generic or personalized to the wearer's needs and wants (fitted with prescription lenses and worn as glasses). This terminal would interface to existing computing infrastructure as staff move throughout the workplace. Ultimately communication could be achieved via wireless cells and staff would receive relevant information as they achieve proximity to each active area (in a medical institution: at a patient's bedside, at the operating table, or in a training seminar).

Alternatively, the VRD may be part of a more extensive device - a wearable computer, comprised of processing, data storage, and human interface elements. Historically the inability to achieve a high resolution yet compact and low-power display has restricted use of such a system. Micro Vision will select corporate partners to provide remaining key subsystems.

The base technologies described in the previous sections are enabling in that they allow for many configurations of the finished product. The mechanical resonant scanner, for example, can be configured for different formats depending on the required resolution and field-of-view. One configuration of the VRD could display NTSC video and thus be compatible with standard North American television programming. A medical imaging application may demand SVGA resolution. The VRD is capable of both and more. It is this potential to scale and reconfigure the VRD that offers the opportunity to exactly match its characteristics to each market segment.

Potential applications of the VRD may be divided into industry types as summarized below:

Personal communications Fax pager

Cellular communicator

Wearable PDA

Video phone

Medical Radiology / ultrasound, fluoroscopy

Surgery / anesthesiology, endoscopy

Simulation and training

Prosthetic interface

Military, Security Portable command post

Communicator / hand-held, head-worn

Simulation and training

Manufacturing Process control monitor

Information assistant

Consumer Portable computer display

Location based entertainment

Virtual reality simulations, games

Scientific 3-D microscope

Scientific modeling and visualization

It is important to note that each product will not require a unique VRD configuration. Micro Vision will be developing generic design platforms, each of which will serve a range of products. In addition, Micro Vision is working closely with HITL engineers and scientists to ensure that research of the base VRD technologies is quickly translated into products. It is anticipated that products utilizing a monochrome, VGA resolution display could be available as early as 1996.

The VRD offers solutions to many of the problems that have plagued personal display devices. It will allow a display that is small, low cost, low power, high resolution, bright enough to operate in an outdoor environment, and functional in either an inclusive or see-through mode.

Development work at the HITL has centered around making the VRD technology commercially viable. The creation of the mechanical resonant scanner solves one of the toughest problem of commercializing the VRD, developing a low cost, high frequency, large deflection angle horizontal scanning device. Based on this scanner a full color, VGA resolution VRD prototype has been demonstrated. Future development will include refinement of the mechanical resonant scanner, enlarging the system's exit pupil, utilizing small blue and green color sources, increasing system resolution, and system safety testing.

The VRD appears to be an ideal display for a large number of commercial, industrial, consumer, and military applications. Micro Vision, Inc. will be matching product needs to VRD performance parameters and expects to have the first products available in 1996.

The authors wish to thank Dr. Tom Furness, Director of the HITL, for his vision in creating the VRD and continued support throughout development and Micro Vision, Inc. for supplying the funding that makes the development possible. In addition we wish to acknowledge the past and present members of the VRD team, without whose efforts this development would not be possible. These members include David Melville, Michael Tidwell, Robert Burstein, Joel Kollin, Heather Patrick, Steven White, Toni Emerson, Dr. Kellin Kuhn, Dr. Thomas Pearsall, Carrie Cornish, Mark DeFranza, Chris Doughty, Phillip Allison, and Dan Bertolet.

1. J. Kollin, "A Retinal Display for Virtual-Environment Applications", Proceedings of the Society for Information Display, Vol. 24, pp. 827, 1993.

2. M. Tidwell, R. S. Johnston, D. Melville, T. A. Furness, "The virtual retinal display - a retinal scanning imaging system", Virtual Reality World '95 Conference Documentation, IDG Conferences, pp. 325-333, 1995.

3. R. H. Webb, G. W. Hughes, O. Pomerantzeff, "Flying spot TV ophthalmoscope", Applied Optics, Vol. 19, No. 17, pp. 2991-2997, 1980.

4. R. H. Webb, G. W. Hughes, F. C. Delori, "Confocal scanning laser ophthalmoscope", Applied Optics, Vol. 26, No. 8, pp. 1492-1499, 1987.

5. D. E. Holmgren, W. Robinett, "Scanned laser displays for virtual reality: a feasibility study", Presence, Vol. 2, No. 3, pp. 171-184, 1993.

6. M. Tidwell, "Avirtual retinal display for augmenting ambient visual environments", unpublished Master's thesis, Department of Electrical Engineering, University of Washington, 1995.

7. G. F. Marshall, "Scanner refinements inspire new uses", Laser Focus World, Vol. 30, No. 6, pp. 57-63, 1994.

8. G. F. Neumark, R. M. Park, and J. M. DePuydt, "Blue-Green Diode Lasers", Physics Today, Vol. 47, No. 6, pp. 26-32, 1994.

9. Q. Wang, R. W. Schoenlein, L. A. Peteanu, R. A. Mathies, C. V. Shank, "Vibrationally coherent photochemistry in the femtosecond primary event of vision", Science, Vol. 266, pp. 422-424, 1994.