A Virtual Retinal Display For Augmenting Ambient Visual Environments
by Michael Tidwell

Chapter 6: General Approach to the Problem

6.1 The Virtual Retinal Display

The general approach to this project is to apply technology of the Virtual Retinal Display [1]. The Virtual Retinal Display presents video information by scanning modulated light in a raster pattern directly onto the viewer's retina. As the light scans the eye, it is intensity modulated. On a basic level, as shown in Figure 6.1, the VRD consists of a light source, a modulator, vertical and horizontal scanners, and imaging optics (to focus the light beam and optically condition the scan).

Figure 6.1: Basic block diagram of the Virtual Retinal Display.

The resultant imaged formed on the retina is perceived as a wide field of view image originating from some viewing distance in space. Figure 6.2 illustrates the light raster on the retina and the resultant image perceived in space.

Figure 6.2: Illustration of light raster imaged onto the retina and the resultant perceived image.

In general, a scanner (with magnifying optics) scans a beam of collimated light through an angle. Each individual collimated beam is focused to a point on the retina. As the angle of the scan changes over time, the location of the corresponding focused spot moves across the retina. The collection of intensity modulated spots forms the raster image as shown in Figure 6.2.

6.2 Potential Advantages of the Virtual Retinal Display

6.2.1 Brightness

One problem with conventional helmet mounted display image sources is the low luminance levels they produce. Most liquid crystal array image sources have insufficient luminance levels for operation in a see-through display. The VRD, however, does not contain individual Lambertian (or nearly Lambertian) pixel emitters (liquid crystal cells or phosphors) as do most LCD arrays and CRT's. The only light losses in the VRD result from the optics (including the scanners and fiber coupling optics). There is no inherent tradeoff, however, between resolution and luminance as is true with individual pixel emitters. In individual pixel emitters, a smaller physical size increases resolution but decreases luminance. In the Virtual Retinal Display, intensity of the beam entering the eye and resolution are independent of each other. Consequently, the VRD represents a major step away from the traditional limitations on display brightness.

6.2.2 Resolution

As mentioned in the previous section there is a tradeoff between resolution and brightness in screen based displays. As resolution requirements increase, the number of picture elements must increase in a screen based display. These greater packing densities become increasingly difficult to manufacture successfully. The VRD overcomes this problem because the resolution of the display is limited only by the spot size on the retina. The spot size on the retina is determined primarily by the scanner speed, light modulation bandwidth, and imaging optics.

6.2.3 Yield

One limiting aspect in the manufacture of liquid crystal array image generators is the yield and reliability of the hundreds of thousands of individual liquid crystal cells present in these displays. For a liquid crystal array display to function properly at all times, each picture element must function properly. The Virtual Retinal Display requires only constant functionality from the light sources and the scanners. As resolution increases in virtual image displays, liquid crystal arrays will contain more and more individual liquid crystal cells. The Virtual Retinal Display will gain an increasing advantage over liquid crystal array image generators in terms of yield as resolution demands increase in the future.

6.2.4 Size

The theoretical size for horizontal and vertical scanners plus light sources for the VRD is smaller than the size of conventional liquid crystal array and CRT image sources. A typical size for a liquid crystal array image generator for helmet mounted display applications is one inch by one inch. The Mechanical Resonant Scanner used in this project was approximately 1 [cm] by 2 [cm]. Furthermore, the problem of scanner size has not been directly addressed. Further size reduction is certainly possible. It should be noted that light sources for a smaller, usable full color VRD must be much smaller than the sources used in this thesis. The potential size of light emitting diodes and diode lasers indicate that these sources show greatest promise for future systems in terms of size.

6.3 Components of the Virtual Retinal Display

6.3.1 Video Electronics

In its current form, the video electronics of the VRD controls the light intensity modulation, scanner deflection, and the synchronization between modulation and scanning. The horizontal and vertical synchronization signals in the video signal are used to determine scanner synchronization. A user selectable delay of up to one full line is incorporated into the video electronics to allow for phase difference between the horizontal scanner position and the modulation timing (discussed further in Section 9.3.2). Also, the respective drive levels for intensity modulation of each light source are output from the electronics [26].

6.3.2 Light Sources and Modulators

The light sources for the VRD generate the photons which eventually enter the eye and stimulate the photo receptors in the retina. The modulation of the light source determines the intensity of each picture element. The size of the scanning spot and the rate at which it can be modulated determine the effective size of each picture element on the retina. As the light is scanned across the retina, the intensity is synchronized with the instantaneous position of the spot thereby producing a two dimensional pattern of modulated light that is perceived as a picture.

6.3.3 Scanners

The scanners of the VRD scan the raster pattern (see Figure 6.2) on the retina. The angular deviation of the horizontal scanner combined with the angular magnification of the imaging optics determines the horizontal field of view. The angular deviation of the vertical scanner combined with the angular magnification of the imaging optics determines the vertical field of view. The horizontal scanner speed and the frame rate determine the number of horizontal lines in the display,

number of horizontal lines = horizontal scanner frequency / frame rate

where frame rate is the number of times per second the entire picture (or frame) is generated. The modulation rate and the horizontal scanner frequency determine the number of pixels per line in the display,

number of pixels per line = modulation frequency / horizontal scanner frequency

where the modulation frequency is the number of times per second the pixels are created (or modulated).

6.3.4 Imaging Optics

The imaging optics converge the diverging scan exiting the horizontal and vertical scanners and also angularly magnify the scan. In the current design, the imaging optics also collimate the individual ray bundles that eventually enter the eye. Care must be taken in the design of the imaging optics. The goal is to present an undistorted scan with good focus at the retina of the eye.

6.4 The Optical Invariant

6.4.1 Theory of the Optical Invariant

The optical invariant states that angular magnification of an object can only be achieved at the expense of image size [2,32]. For large angles,

d_itan[u_i] = d_ftan[u_f]

where d_i = initial aperture size, d_f = final aperture size, tan[u_i] = tangent of object angular extent, and tan[u_f] = tangent of image angular extent. The above equation is the Lagrange (or optical) invariant and it states that the product of the field and aperture are constant. As related to the VRD, the angular extent of the object relates to half the angular deviation of the scanner. The angular extent of the image corresponds to half the horizontal field of view. The initial aperture is the scanner aperture and the final aperture is the exit beam size (or exit pupil).

A relationship critical to understanding the VRD is the relationship between scanner aperture, exit pupil, scanner deviation, field of view, and resolution. To facilitate an understanding of this fundamental relationship, the following discussion will assume a diffraction limited optical system. In a diffraction limited optical system, only the beam size and not the aberrations determine the resolution. As the beam size increases it becomes more and more likely that system aberrations will be the limiting factor to the spot size (more accurately how small the spot can be). The relationship between the above variables is as follows:

Using a paraxial approximation, the diffraction limited spot size, D_s formed on the retina is,

D_s = 2.44 l f_eye / d_f

where l = the wavelength of light, f_eye = focal length of the eye, and d_f = exit pupil (or exit beam) diameter. D_s represents the diameter of the Airy disk of the spot formed on the retina. The angular resolution r in minutes of arc associated with a spot of extent D_s is,

r = 120 tan^-1[2.44 l f_eye / (2 d_f f_eye)]

= 120 tan^-1[1.22 l / d_f]

but,

d_f = d_i tan[u_i] / tan[u_f].

Now,

r = 120 tan^-1[1.22 l tan[u_f] / (d_i tan[u_i])].

Assuming a nominal wavelength off 555 [nm] and for d_i in [mm],

r = 120 tan^-1[6.77 10^-4 tan[u_f] / (d_itan[u_i])].

Specifically for this project,

u_i = 12.5 [deg.]

d_i = 3 [mm]

u_f = 20 [deg.]

where the half angle optical deviation from the horizontal scanner is 12.5[deg.], the scanner aperture is 3[mm], and the half angle horizontal field of view is 20[deg.]. Correspondingly,

r = 120 tan^-1[6.77 10^-4 tan[20^o] / (3 tan[12.5^o])]

= 2.55 [arcmin].

2.55 [arcmin] represents theoretically the smallest angular extent of a spot formed on the retina by the VRD of this project. The smallest angular extent of the spot formed on the retina is also the smallest angular extent of a picture element in the display if the picture element size corresponds to the diameter of the Airy disk spot size on the retina. In other words, consecutive pixels are considered to correspond to consecutive Airy disks arranged such that the first dark fringes of the Airy disks are coincident (i.e the Airy disk peaks are separated by an Airy disk diameter). Note this is different than Lord Rayleigh's criterion which would allow picture elements to correspond to consecutive Airy disks separated by the distance to the first fringe (i.e. the Airy disk peaks are separated by an Airy disk radius). The criterion used here is more conservative than Rayleigh's criterion (by a factor of two). However, it allows for "off" pixels to have more closely the same angular extent as "on" pixels than does Rayleigh's criterion.

6.4.2 The Mechanical Resonant Scanner and the Optical Invariant

The scanner for this version of the VRD is a mechanical resonant scanner (MRS). The MRS is a mechanically resonant device which has a moving reflective surface. The moving reflective surface deviates the light beam through an angle over time (i.e. it scans the light). The effective aperture of the MRS in this design is 3 [mm]. Figures 6.3-7 show the relationship between resolution, field of view, scanner mirror size, and scanner deviation for this type of VRD. Scanner mirror diameter and full angle horizontal field of view are independent variables in each plot and a third independent variable, scanner half angle optical deviation is represented by iterations of the two-variable plots. The scanner deviation for each plot is shown in Table VI.1. Please note in each of the Figures 6.3-7, the flat areas which appear to be constant with changes in mirror diameter and horizontal field of view are actually areas where the value of the function is above the range shown in the graph. For instance in Figure 6.3, the angular resolution in minutes of arc for a mirror diameter of four millimeters and a horizontal field of view of 80 degrees is not 10 minutes of arc but rather a value that is greater than 10 minutes of arc.

Table VI.1. Table of independent variable u_i for Figures 6.3-7.

Figure u_i [deg.]

6.3 4

6.4 6

6.5 8

6.6 10

6.7 12

Figure 6.3: Plot of relation between resolution, field of view, and mirror diameter for a mechanical resonant scanner (MRS) based VRD. This plot is for an MRS having a 4.0 [deg.] half-angle optical deviation.

Figure 6.4: Plot of relation between resolution, field of view, and mirror diameter for a mechanical resonant scanner (MRS) based VRD. This plot is for an MRS having a 6.0 [deg.] half-angle optical deviation.

Figure 6.5: Plot of relation between resolution, field of view, and mirror diameter for a mechanical resonant scanner (MRS) based VRD. This plot is for an MRS having 8.0 [deg.] half-angle optical deviation.

Figure 6.6: Plot of relation between resolution, field of view, and mirror diameter for a mechanical resonant scanner (MRS) based VRD. This plot is for an MRS having 10.0 [deg.] half-angle optical deviation.

Figure 6.7: Plot of relation between resolution, field of view, and mirror diameter for a mechanical resonant scanner (MRS) based VRD. This plot is for an MRS having 12.0 [deg.] half-angle optical deviation.

6.3.4 Discussion

Figures 6.3-7 illustrate the theoretical relationship between resolution, field of view, and Mechanical Resonant Scanner optical deviation. The graphs show that for constant resolution, the MRS mirror size must increase as field of view increases. The graphs also show that for constant field of view and resolution, an MRS with a large optical deviation can have a smaller mirror size than an MRS with a small optical deviation.

Human Interface Technology Laboratory

A Virtual Retinal Display For Augmenting Ambient Visual Environments by Michael Tidwell