Joel S. Kollin and Michael Tidwell
Human Interface Technology Laboratory, Washington Technology Center
University of Washington, Box 352142
Seattle, WA 98195-2142
Copyright 1995 Society of Photo-Optical Instrumentation Engineers.
This paper was published in Proceedings of Novel Optical Systems Design
and Optimization and is made available as an electronic reprint with
permission of SPIE. Single print or electronic copies for personal use
only are allowed. Systematic or multiple reproduction, or distribution
to multiple locations through an electronic listserver or other
electronic means, or duplication of any material in this paper for a fee
or for commercial purposes is prohibited. By choosing to view or print
this document, you agree to all the provisions of the copyright law
protecting it.
Keywords: Retinal scanning, HMDs, portable displays, HOEs, virtual reality, augmented reality
1. Introduction
This paper presents an overview of the VRD project at the HIT lab from an optical systems designer's viewpoint, followed by details of some system components, design criteria, and specific subproject developments - primarily in optical imaging. Due to both breadth and patent considerations it is necessary to mention some aspects of the project in passing only.
The VRD has several advantages over CRTs, LCD, and other addressable-screen displays :
In late 1993 and 1994, Mike Tidwell redesigned the VRD to maximize the resolution possible with the A-O scanner while David Melville designed a new Mechanical Resonant Scanner (MRS) which would be capable of the high rates of horizontal scanning without the costs and other limitations of the A-O devices. The MRS was then utilized in full-color inclusive and "see-through" systems.
To achieve the desired resolution, all current VRD prototypes have used lasers for their superior spatial coherence characteristics. In order to use a point source such as an LED, the image of the source should be smaller than the diffraction limit of the scanner (Fig. 1). Using the lens magnification, one can determine the maximum source size that can be used before degrading the diffraction limited spot size at the image plane. The angular divergence of the source is effectively limited by treating the scanner as a stop. Light which does not hit the mirror does not contribute to the image plane spot size. From this geometric argument we can derive an equivalent point source size between 4 and 5 microns for a VGA resolution image in our current system. For a system where the scanner is illuminated with a collimated Gaussian beam, similar arguments can be made to determine the required divergence and beam waist from the equations for image plane spot size.
Fig. 1 Relationship between source size, objective focal length, scanner aperture & image spot size
Work is proceeding on the use of LEDs as the VRD light source. LEDs can be directly modulated, are inexpensive, and are available in a wide range of colors. The spatial coherence and modulation bandwidth of available visible LEDs is below the requirements for high-resolution displays. We are working with LED manufacturers to eliminate these issues.
3. Scanners
The horizontal scanning mechanism of the VRD must be capable of both relatively high scan rates (15 kHz-90+ kHz) and high resolution (500-2000+ pixels) for NTSC to HDTV formats, respectively. To date we have built SVGA format systems (80 kHz) in monochrome/greyscale using a A-O scanner and 30 kHz in full-color with a mechanical resonant one.
We are still researching other scanning technologies, especially those which might have advantages that outweigh the low cost and high efficiency of the MRS. Currently we are using a galvanometer for the vertical scanner while a less expensive solution is being developed.
h = 2.44 l (f/#)
where h is the resolvable spot size (diameter) and f/# (f-number) is the imaging element diameter divided by the distance to focus.
In a typical retinal scanning system, a telescope is used to form a reduced "image" of the scanner at the eye. This is not the image seen by the user, but an exit pupil or aperture stop of the system in image space. By forming an exit pupil smaller than the scanner aperture, the scan angle is effectively amplified by a proportional amount (Fig. 2). In other words, the ratio of the size of the aperture stop image (exit pupil) to the original scanner is inversely proportional to the magnification of the aerial image, in accordance with the Optical Invariant. However, this smaller exit pupil can lead to two problems: difficulty in making sure the light still enters the eye, and reduced resolution due to diffractive spreading as given by the above equation.
Fig. 2 Angular magnification in a pupil-forming system
Wide FOV polychromatic eyepiece design is non-trivial - adding light weight and form factor constraints makes it even more challenging. Here the small instantaneous ray bundle size helps considerably. For instance, with the fast (small f-number) eyepiece only the aberrations across one beam width will degrade the image, rather than over the entire aperture.
Tan Qeye / Tan Qscan = fobj / feye
where fobj and feye are the focal lengths of the objective and the eyepiece, respectively. The f/#'s of the lenses will be similarly scaled. The critical imaging component in this system would therefore be the eyepiece, whose f/# can limit the field-of-view of the system. Low f/# or "fast" lenses tend to be heavy and complex, especially for polychromatic systems. The first polychromatic system used a telescope eyepiece. The eyepiece was subsequently replaced with a mirror in a modified Schmidt camera configuration. Mirrors are simple, rugged, have no chromatic aberration and can be very light for their power. They can also be made partially reflective for a "see-through" system. The mirrorsÕ flaws and aberrations should be limited to approximately one-quarter wave across the any pupil-sized area on its surface, which is a much easier target to reach than quarter-wave accuracy over the entire aperture. Another advantage of mirrors used in a Schmidt configuration is that post-objective scanning can be used. Such systems are simpler and works well with the resulting curved aerial image field. One drawback of the classical Schmidt design is the field-of-view limitation due to the fold mirror keeping the viewer away from the mirror.
An important limit on field-of-view for any pupil-forming system is eye relief. The FOV can be defined by the (half-)angle formed by the chief or principle ray with the optical axis. This implies that its tangent is inversely proportional to eye relief, defined as the distance from the final element to the exit pupil, for an eyepiece of given focal length. Therefore a wider FOV system will less eye relief. Note that inadequate eye relief (less than 20 mm) may prohibit the use of eyeglasses, and even more eye relief may be needed for a wide FOV system. This is because the exit pupil must be closer to the center of the eye's rotation to permit viewing the edges of the image.
Pupil distortion is another concern. If the center of the exit pupil is farther from the eyepiece than the edges of the pupil, the eye must be moved closer to the eyepiece when viewing the periphery of the image. This is clearly unacceptable for a head-mounted display.
X = tan Q
where X is the displacement of the spot relative to the center as a function of scan angle. Normally this is solved by using a special 'f-Q' or scan lens that follows the condition X = Q instead. In addition, with the MRS we also must deal with the sinusoidal scan pattern:
Q = A sin (Kt)
where A is the maximum amplitude, K is a proportionality constant related to the frequency and t is time. If we combine the equations above, we have
X = tan [A sin (Kt)]
Where A is the maximum half-angle excursion of the sinusoidal scanner. If A = 1 radian, we find that X is a nearly linear function of t over most of its range. Unfortunately, with the +/- 12 degrees range of the current MRS scanner the function is dominated by the sin function. Currently we are implementing a frame buffer to remap the image to compensate for this distortion. This will also allow for more flexibility in the optical design by allowing us to electronically correct distortion while optimizing other constraints such as form factor and resolution.
5. Holographic Optical Elements
Fig. 3 Recording and Playing a HOE (Off-Axis Parabola Shown)
Fig. 4 Construction of a non-conformal reflection HOE using a spatial filter beam expander and concave mirror. Width of HOE and fringes are greatly exaggerated.
The HOE in Fig. 4 would be considered a non-conformal reflection HOE, meaning the fringes are not conformal or parallel to the surface. The playback beam will reflect off the HOE since the construction beams impinge from different sides, causing the interference fringes to be formed roughly perpendicular to the propagation directions. If the HOE is recorded on a curved substrate so that the fringes conform to the surface, it would act as a multi-layer coated mirror. A HOE of this type can reflect nearly all of the light whose propagation vector component k was twice that of the fringe spacing - this is known as the Bragg condition (Fig. 5). The Bragg condition is the combination of wavelength and angle that causes the impinging and diffracted light of the HOE to be in phase with each other. Not surprisingly, the wavelength and angle of the construction beams will determine the Bragg condition of the HOE fringes, although this can be modified after recording by chemical or physical distortion of the fringes. For a diffuse and/or polychromatic illumination source, those rays meeting the Bragg condition will be diffracted much more strongly - this is known as Bragg selection. We say that holograms with Bragg selection are "thick", meaning they have significant structure inside the plane of the hologram, which may be less than 7 microns but still contain 10 or more fringes to form a "multilayer stack". Reflection holograms are by definition thick, whereas transmission holograms can be thick or thin depending on the exposure geometry and choice of material. Embossed or surface relief holograms, such as those found on credit cards, are thin holograms and therefore have little or no Bragg selection. They are colorful because they are "rainbow" or Benton holograms, which are vertically dispersive transmission holograms mounted in front of a mirror. Thick reflection and transmission holograms can approach 100% diffraction efficiency (at least for a point-source hologram), whereas thin transmission holograms are limited to 33.9% maximum theoretical efficiency.
Fig. 5 Satisfying the Bragg condition for first-order diffraction off of a plane mirror
Besides a decrease in the bulk and weight, HOEs have several other advantages over conventional elements due to Bragg selection. For a see-through system, the wavelength used by the display can be selectively reflected from the HOE which also tends to reject outside light of the same color. This means higher contrast relative to a broadband beamsplitter for a given brightness and/or transmissivity for the system as a whole, although the outside light may be color shifted as a result. Also, the HOE can multiplex functions by acting differently for beams impinging from different directions or with different wavefronts. For example, light might pass through the HOE virtually undisturbed, be reflected from a beamsplitter, and then be magnified by the HOE. This is a system being tested by HITL as described below. A more complex system might act as a prism for one wavefront and a lens for another.
A reflection hologram for a full-color system can be accurately recorded by three separate exposures at chosen red, green, and blue laser wavelengths. Each set of fringes is Bragg selective for a narrow bandwidth - usually around the exposure wavelength, although this can be shifted and the bandwidth broadened. Since each color is independently exposed, it is possible to modify the exposure geometry to minimize chromatic aberrations or to combine spatially separate color images. Each exposure uses up some potential refractive index modulation, but this limitation can be circumvented by copying the HOE. This is often accomplished by placing the copy hologram in contact with the master and exposing it to the playback beam. The resulting image beam from the master interferes with the playback beam to form nearly identical fringes in the copy hologram. Computer generated diffractive optical elements can also be "contact copied" in this manner.
In general, HOEs also allow greater freedom in an optical system since diffractive optical elements cause chromatic dispersion opposite in sign to refractive elements. Diffractive power can be added to a refractive and reflective element by incorporating diffractive features on a lens or mirror surface. Thin holograms or other diffractive devices can be pressed onto the surface of molded or cast parts. Thick holograms can be placed on curved reflectors or lenses by coating the surface with photopolymer or gelatin and exposing. Conformal exposure increases expenses quite a bit - from perhaps $5 to $50, whereas complex lenses and reflectors can be molded for under $5 in large quantity.
Thick HOEs have some other disadvantages besides cost. Bragg selection can limit the field of the HOE, with efficiency and FOV decreasing away from the Bragg condition. At the same time, insufficient Bragg selection can lead to significant dispersion in a non-monochromatic source such as an LED, and even cross-talk between the different color exposures. Also, flare from bright ambient light could be diffracted into the eye by a reflection HOE acting as a transmission hologram. This is more of a problem for non-conformal holograms, where the fringes can be perpendicular to the plane of the hologram at the surface, allowing for significant diffractive transmission. Proper design and the use of baffles can help minimize flare. Also, non-conformal holograms can lead to distortion, since the HOE might intersect the scanned beam at a different point a curved mirror would.
Fig. 6 See-through VRD system incorporating HOE eyepiece (not to scale)
6. Experimental Results
Our first VRD system incorporating HOEs was designed using ZEMAX and exposed onto DuPont OmniDex 706 film. We started by constructing simple, nearly on-axis non-conformal 35mm focal length, f/0.8 mirrors by two different methods. At first we used a two-beam exposure at 532nm with back to back singlets to converge an expanded, spatially filtered beam to a point near the pinhole of another spatial filter. This method allows for more flexibility for future experiments with off-axis systems, but has a few drawbacks. First there is distortion introduced by the singlet pair - we cannot be measure this or predict it using ZEMAX directly but it is fairly obvious barrel distortion which accentuates rather than correcting the sinusoidal distortion. Also we noticed that there were several "false foci" recorded on the HOE - these were from stray surface reflections and manifested themselves as extra exit pupils with distorted images. These will presumably be reduced or eliminated by anti-reflective coatings on the optics and plate.
We then made a simple one-beam "Denisyuk" HOE by using a mirror to reconverge light to the spatial filter, as shown in Fig. 4. As in the previous setup, Bragg angle can be altered by varying the conjugates from the nominal 1:1 ration. Stray reflections were reduced, with only the specular highlight from the scanner being apparent. This system also makes it easier to construct multiple color HOEs by sequential exposure in different wavelengths. So far the results obtained for a two-color HOE recorded on DuPont 700HD 15-20 full-color reflection film have been disappointing, but we have just begun experimenting with this material. Exposure times are very long - 10-12 minutes for red (633nm He-Ne) followed by 3-4 minutes for the much stronger green (532nm doubled Nd-YAG). Once we have obtained better results and have optimized recording conditions, we plan to try three-color exposures with blue for a full-color VRD system.
A testing system now under construction will enable us to quantitatively measure system MTF, spot size, resolution and contrast. We can therefore only offer preliminary results based on test patterns which are observed visually. For the single color (green) HOEs we were able to resolve the full resolution of 640 by 480 pixels except for extreme left and right edges of the display, which are compressed by the sinusoidal distortion. This was determined by displaying gratings of alternating off and on pixels in the vertical and horizontal directions. Cosmetically, the holograms show surface defects incurred during lamination and striated variations in diffraction efficiency. The two-color HOEs were much dimmer and noisier than the green ones in the VRD setup, possibly due to errors in the Bragg selection, long exposure times necessary because of limited He-Ne power, and mechanical properties of the material.
Diffraction efficiency is probably also reduced by the current lasers used in the VRD - 543nm He-Ne for the green and 650nm (nominal) red laser diode. Baking the holograms seems to make them brighter in white light but not in the system, leading us to believe that efficiency is not improving much for our choice of off-peak wavelengths. Successful use of HOEs will require careful control over the peak Bragg wavelength and/or broad spectral bandwidths to match the source. LEDs are fairly well-matched to the bandwidth exhibited by DuPont photopolymer holograms if the peak wavelengths are close. In the next year we hope to add a wet darkroom and environmental controls to our optics facility which will make it much easier to try other materials such as those made by Polaroid. Recently we have also constructed a "VRD simulator" (Fig. 7) which allows us to examine optical subsystems (eyepieces) of the VRD without requiring access to the scanner and being limited in optical power or wavelength. The basic concept is to take light from a slide transparency and project it aerially as a real image in front of a apertured projection lens. The aperture of the projection lens corresponds to the stop in the system, simulating a scanner aperture and resulting pupil size. The distance from the projection lens to the image and the focal length determines the size and field angle of the aerial image, which is then re-imaged by the tested system which also forms an image of the projection aperture to the eye. The image can be made as bright as desired to test inefficient systems prior to optimization, and the chromatic response to a white-light source or various filtered colors can also be tested. This is especially useful for testing HOEs with Bragg condition problems. It is not currently capable of simulating distortion other than what is introduced by the projection optics.
Fig. 7 VRD Simulator
7. Summary
The Virtual Retinal Display has significant performance and ergonomic advantages, especially for portable displays. Subsystems and issues currently being addressed at the HIT Lab include: light source development, scanning technology, optical system engineering and compact eyepieces. In particular, holographic optical elements are a promising means of providing see-through eyepieces in a small form factor with unique advantages.
8. Acknowledgments
This work was performed under contract with MicroVision, Inc., 1420 Fifth Ave., Suite 2200, Seattle WA 98101.
The authors wish to acknowledge the critical contributions of the rest of the VRD team, who presently include David Melville, Rich Johnston, Prof. Thomas A. Furness, Bob Burstein, Heather Patrick, Steve White, Prof. Kelin Kuhn, Prof. Thomas Pearsall, Dan Bertolet, Carrie Cornish, Phillip Allison and Archie Gonzales. We also wish to thank Rich Rallison of the Ralcon Corportion for his helpful suggestions and Prof. Shawn Brixey of University of Washington Cross-Disciplinary Art program for his generous loan of equipment.
9. References
R. H. Webb, G. W. Hughes, and F. C. Delori, "Confocal scanning laser ophthalmoscope," Appl. Opt., Vol. 26, No. 8, pp. 1492-1499, 1987.
R. J. Jacobs, I. L. Bailey, and M.A. Bullimore, "Artificial pupils and Maxwellian view ," Appl. Opt., Vol. 31, No. 19, pp. 3668-3677, 1992.
R. Johnston and S. Willey, "Development of a commercial retinal scanning display," Helmet- and Head-Mounted Displays and Symbology Design Requirements, SPIE Vol. 2465, 1995.
J. S. Kollin, "The Virtual Retinal Display," Proc. 3Dmt '92, Montreal.
J. S. Kollin, "A retinal display for virtual environment applications," SID IntÕl Symp. Vol. XXIV, pg, 827, 1993.
M. Tidwell, R.S. Johnston, D. Melville, and T.A. Furness, "The virtual retinal display - a retinal scanning imaging system," Virtual Reality World '95 Conference Documentation, IDG Conferences. pp. 325-333, 1995.
M. Tidwell, "A virtual retinal display for augmenting ambient visual environments," unpublished Master's Thesis, Department of Electrical Engineering, University of Washington, 1995.
J. W. Goodman, Introduction to Fourier Optics, McGraw-Hill, New York, 1968.
W. J. Smith, Modern Optical Engineering, 2nd Ed.,McGraw-Hill, New York, 1990.
G. F. Marshall, Optical Scanning, Marcel Dekker, New York, 1991. Smith, 1990.
R.S. Johnston, unpublished HIT Lab Technical Report 1994. Tidwell, 1995.
R. Kinglake, Optical System Design, Academic Press, Orlando, 1983.
P. Hariharan, Optical Holography, Cambridge University Press, Cambridge, 1984.
W. Gambogi, K. Steijn et. al., "HOE imaging in DuPont holographic photopolymers." Diffractive and Holographic Optics Technology, SPIE Vol. 2152, 1994.
A.M. Weber, W.K. Smothers, T.J. Trout, and D.J. Mickish, "Hologram recording in DuPont's new photopolymer materials," Practical Holography IV, SPIE Vol. 2152, 1990.