Spatial Perception in Virtual Environments:
Evaluating an Architectural Application

CHAPTER 5. DESIGNING THE EXPERIMENT

The Experiment Conditions.

Three experimental categories are required in order to measure the extent to which virtual environments succeed in providing accurate perceptions of the basic characteristics of architectural spaces: an existing real-time computer walkthrough, a virtual environment and a real setting. By comparing both the category of virtual interfaces and the best existing method of spatial representation to the real setting, any differences between the two simulation categories can be "relativized" to the real one.

In the walkthrough category, the viewer looks at a television monitor to see the modeled space. The viewer can move their point of observation in the model by using the Spaceball movement interface described in Chapter 3.

There are at minimum two factors which differentiate the walkthrough from virtual interface: the stereoscopic eyephones and the head tracking. To isolate which one of the two factors will be responsible for potential differences between the two categories, a third intermediary condition has to be interceded. This third condition, which will be called the "Fixed" condition, includes the stereoscopic eyephones but not the head tracking device. To distinguish between the categories and this new condition, the computer walkthrough category will be referred to as the "Monitor" condition. The virtual interface will be called the "Tracked" condition, and the real setting will be called "Real" condition.

The control of the viewpoint is the same for both the Fixed and the Monitor conditions. However, whereas in the Monitor condition viewers look at a monitor, participants in the Fixed condition will be looking through the stereoscopic eyephones. Similarly, both the Fixed and the Tracked condition use the eyephones. However, in the Tracked condition, participants control their viewpoint with the head tracking device.

The four experimental conditions are then the (1) Monitor condition, (2) the Fixed condition, (3) the Tracked condition and (4) the Real condition.

Design of Experiment.

The study consists of evaluating three simulation conditions against the control group condition (the Real World Setting). The most common approach is a complete random design (C.R.D). All participants experience all conditions in a random order. The advantage of such an approach is that it does not require very many subjects to generate statistically significant results because the variation due to the subjects is taken out of the error term. Another advantage is that it compares the predictability of the simulations directly. Those participants who view the real setting after having experienced it in the simulation condition can immediately confirm the differences between the two environments.

Unfortunately, there are two major drawbacks to the complete random design. The first is one of inconvenience. The Real Setting test site, the Henry Art Gallery, is far from the laboratory where the simulation conditions are to be conducted, and, because it is a public space, the hours available for the experiment are limited to one afternoon.

The second problem is related to the tasks. Since the space that is being evaluated is supposed to be the same one, in the real and the simulation conditions, participants would benefit too much from the order effect. By the time they visited the site a second time, it is feared they would be already as good in their estimates as they could ever be. There would not be any variation to measure.

The alternative design is a random blocked design. Each participant group only experiences one of the four conditions. This design is much easier to administer. The disadvantage of this design however is that the variation due to subjects cannot be taken out of the error term. It makes it much more difficult to find significant differences.

One approach to increasing the chances of finding significant differences, is to have a large number of subjects. However, given the limited length of the experiment, this will not be a viable alternative. Another approach is to control the potential variation among the subjects by selecting a homogeneous subject group. Architects are a homogeneous subject group because they are all very familiar with the tasks required in this study. If the participants are limited to this group, the variation in the results can be attributed more to differences in the display conditions differences in the subject's ability to do the tasks. This does imply that the results of this study can, strictly speaking, only be applied to this group of people, and not to the general public.

It was suggested that, under normal conditions, each group needed an absolute minimum of 5 subjects, below which the variation in individual differences would hide any significant differences due to the display conditions. To strengthen the statistical value of the results, I calculated I would have time to do 7 participants for each condition (The limiting factor was the number of subjects that could be run in one afternoon at the Henry Art Gallery). As the study came to an end, it was clear that I would have to settle for 6 participants per condition, for a total of 24 subjects.

Recruiting Participants.

To qualify for participating in the experiment, subjects have to be professional space designers, or graduate students or professors in the field of architecture. Although graduate students might have less professional experience than the other members of the group, they were nonetheless included because of their more flexible schedule; they could more easily take part in the Real condition which could only be conducted during working hours.

To recruit participants for the simulation conditions, I sent invitations describing the experiment to a variety of architecture firms in the greater Seattle area. The intent was to invite as broad a range of participants as possible so that as many firms could get involved in the process of shaping this technology. Only those who responded that they were particularly open to new technologies were asked to participate. For the Real condition, flyers were also displayed in the architecture department on the campus.

Due to practical limitations, the groups were not completely similar across conditions. Half of the participants in the Real condition were graduate students who had potentially less experience in doing the estimating tasks than their professional counterparts who were in the simulation conditions. Also, the study could not be run in a completely random fashion. All the Real condition participants had to view the real space in one afternoon. The participation of the subjects in the simulation condition was spread out over 7 evenings, and the display conditions were picked at random between the Monitor, Fixed and Tracked conditions every time. It is hoped that the integrity of the experimenter and the way the experiment is conducted should suffice in alleviating concerns about the "evenness" under which the study is conducted.

Designing the Tasks.

Participants' estimates of room sizes vary as a function of the size and shape of the volumes, as well as from the effect of increased familiarity with the environment. This variation is averaged out by having participants estimate a number of different kinds of spaces for the task. Having participants estimate three of the seven spaces in the museum should be sufficient to generate reliable results. The estimated of length, width and height, are expressed as the percentage of the actual dimensions. This facilitates the statistical analysis. It is also a convenient way to show how people's estimates compare to the actual dimensions.

To have reliable results, participants were asked to do the pointing task three times. These values are expressed as the "error in degrees from actual direction".

After each of these tasks, participants were required to express the ease or difficulty of making each specific estimate as well as their level of confidence in their estimate. This information may support the idea that, if not better than existing forms of spatial representation, at least virtual environments are easier to interpret.

The description questionnaire was administered after the visit. It includes a list of 13 bi-polar adjectives which were selected out of a list of 66 (see Chapter 3). Participants select the more appropriate adjective which describes the main gallery space by circling one of the numbers on the semantic differential scale (see Appendix A: Questionnaire).

Choosing a Site.

Because the focus of this study is to evaluate people's perception of the most basic spatial characteristics, and because the details of spaces can play such a large role in our perception of them, it was important to select a space which was simple, free of associations, and which contained few architectural details. Furthermore, it was important to select a site in which the experimenter can control the elements as much as possible.

Museum spaces are ideal for conducting these kinds of studies because their architectural design is implicitly simple, so as not to conflict with the art on display. They are designed to be neutral, well lit public spaces with a combination of closed spaces for specific exhibits and circulation spaces.

After briefly reviewing locally available museums, several reasons compelled me to select the Henry Art Gallery on the campus of the University of Washington : its proximity to the HITLab, the relative simplicity of the space (which implied that it would not require too many polygons to model for the simulation), its lack of exterior views (which would not have to be contended with), and finally, its appeal as a very special and pleasant space.

The museum is a Beaux-Arts design built in the early 1900's. There are six gallery spaces. Each space has a vaulted ceiling and a large skylight which baths the space in cool white light. The gallery has a main space, by far the most pleasant of all, through which one must pass to visit the spaces at the end.

The study would have to be conducted at a time when there was no artwork on display because it was found that, as can be expected, the art hindered people's ability to focus on the tasks of the experiment. The museum would also have to be vacant, except for the experimenter and the participant. Environmental psychologists Proshansky states that "spaces, their properties, the people in them and the activities that involve these people represent significant systems for the individual participant and thereby influence his response to the physical setting" (Proshansky 1970). Under these constraints, the museum could only be made available one day, from 1 to 5 PM.

There are several drawbacks to the selection of this site for the study. The simplicity of the plan makes the orientation task easier. The little variation in the types of spaces simplifies the estimating task as well. All the galleries are explicitly enclosed rectilinear spaces. According to Thiel, these are all basically the same types of spaces (for a complete description of space types, refer to Thiel's work, Thiel 1964).

The Computer Model - Choosing an Appropriate Level of Detail.

The level of realism of the model depends on the kind of information that one wants to communicate. Since the purpose of the representation is to communicate the most basic characteristics of spaces, the model should only include the most basic spatial establishing elements (SEEs, see Thiel 1964). To determine what those elements should be, I held photographs of the Henry Art Gallery with an outstretched hand and I squinted. I assumed that whatever elements were still apparent must be significant space establishing elements. They included walls, door frames, skylight frames and ceiling and floor trims (Fig. 5.1 ).

(GIF is unavailable.)

Fig. 5.1 - Photograph of model - Trim detail.

The distinction in surface materials appeared as a color distinction. Details that could not be distinguished were the ventilation grills, light switches, the specific materials of the floors and walls, and a certain degree of detail in the wood trims. They would not be included in the model.

The Computer Model - Elements of Scale.

(GIF is unavailable.)

Fig. 5.2 - Photograph of model - Room with scale figure.

Scale is also very important in models. The architectural details alone in the model are not sufficient sources of scale. Traditionally, models include human figures and common household objects to set a sense of scale. In the virtual environment, the lateral movement of one's head is a small but insufficient indication of scale. The model has to include elements of scale as well. Three adult scale figures and one child scale figure, as well as numerous chairs were added to various spaces across the museum to help participants establish an accurate sense of scale (Fig. 5.2).

A little anecdote about scale figures: Before any scale figures were included, people perceived the 8 foot high doorways to be 6 feet and a half, as they often are in homes. When a normal chair was added for scale, is was interpreted to be a child's chair. To override the more pervasive sense of scale from the door frame, an adult scale figure was added. It was perceived to be a child. A small child holding the adult's hand was added to the scene and not until then was the door frame perceived to be larger that normal!