![[Previous Page]](/icons/button-back.gif){kind=icon}
![[Table of Contents]](/icons/button-contents.gif){kind=icon}
![[Next Page]](/icons/button-next.gif){kind=icon}
A pilot study was conducted for the VR treatment to see if the students could use the virtual chemistry world, if certain aspects of the world were confusing and needed to be modified, and if the tests were at the proper level for the student population. Several changes were made due to the results of the pilot study and included in the main study.
Students volunteered to be a part of this study by returning a parent permission form in compliance with the human subject review panel (HSR 23-510-E). These students came from chemistry classes in which atomic and molecular structure had already been taught. The students took pre-treatment written and oral chemistry tests on atomic and molecular structure. They will be discussed in more detail below. They then participated in one of the treatments discussed in the following sections. Finally, they took post-treatment written and oral tests.
For the VR treatment, thirty-eight high school students, 25 females and 13 males, who were in their second semester of junior level high school chemistry class participated. They had been taught atomic and molecular structure in their first semester of class. They were all novice users of VR. Students were encouraged by their chemistry teacher to volunteer for the study and received class extra credit for their participation. The study was conducted at the HITLab after school hours. All but four of the students were taught chemistry by the same teacher. Data from two of the 38 subjects were removed from the study because of failure to follow instructions.
The Video treatment consisted of twenty high school students, 10 females and 10 males, who were in their second semester of junior level high school chemistry class. They were from the same student population as the VR treatment. The study was conducted at the school during school hours. All of the students were taught chemistry by the same teacher.
The students for the Mac Interactive treatment were drawn from a similar, but different population than for the VR and video treatments and consisted of 14 students, 7 females and 7 males. The VR and Video treatments had exhausted the supply of possible participants and so a different high school was used. The students for this study had taken chemistry class the previous year from one of two teachers. Although these students had finished their chemistry course at least 4 months before and the VR and video students had still been in chemistry class at the time of their participation, the topic of atomic and molecular structure was taught early in the first semester for both groups. Therefore, an assumption of this study is that the subject was taught long enough ago for both groups that the groups consist of similar populations. A comparison of the pre-test scores found no significant difference between these students and the original student population used for the VR and Video treatments.
The Mac Run students were drawn from the same population as the Mac Interactive group There were 14 students, 5 females and 9 males.
The control group had 7 students, 5 females and 2 males drawn from the same population as the VR and video treatments.
The student population for the long term retention study was a subset of the students who participated in the VR, video, or control treatments. 18 students from the VR treatment, 17 students from the video treatment and 5 students from the control group participated.
A drawing of the physical set-up is shown in Figure 2. The helmet was the "VPL EyePhones" which has a resolution of 86,000 pixels and a field of view of 100 degrees. The helmet was connected to the ceiling with rope and a pulley system so as not to put any weight on the participant's head. The wires connecting the helmet to the computer were also attached through the pulley system.
Figure 2: VR Apparatus
The wand was a modified "joystick" for a standard video game. It was a hand-held device, as opposed to being attached to a base or table and it had three buttons and one trigger (see Figure 3). In software, the left button was programmed as "fly forward", the right button meant "fly backward", the trigger represented "grab object" and the middle button was programmed as "let go of object." Wires connecting the wand to the computer ran from the bottom of the wand and across the floor. The participants had to be careful to not trip on the wires as they turned around.
Figure 3: Wand
Polhemus position sensors were used for the position sensors in this experiment. Polhemus sensors are electro-magnetic sensors and require an emitting device and a receiving device. The emitting devices were placed on top of the helmet and inside of the wand and the receiving devices were placed next to the computer.
The computer was a Silicon Graphics Iris 320VGX. The software that ran the virtual world was Virtual Environment Operating System (VEOS) created by members of the HITLab (see Coco, 1993). The virtual chemistry world application was written in LISP using digital studio, a VEOS interface program written by Colin Bricken.
Students who participated in the VR treatment of the experiment were first told about the technology of VR. I explained that the graphical images that were displayed inside of the helmet were generated by the computer and would change as they moved their orientation and position. I then demonstrated the virtual chemistry world by creating a lithium atom and then bonding it to a hydrogen atom in the virtual world. The step by step process illustrated both the VR system as well as the rules of atomic structure that they had learned in chemistry class.
After the demonstration, each student put on the helmet, held the wand and built a virtual atomic structure. They were given the task of building a virtual water molecule, which was done by creating a virtual oxygen atom and then combining it with two hydrogen atoms. Figure 4 shows what the students would see in the virtual chemistry. The "plus" shape represented the proton, the "minus" shape represented the electron, and the sphere (a "zero" like object) represented the neutron. These were always available. For example, when one proton was grabbed, another proton appeared in its place. Since the students were taught that an electron has a spin and energy associated with it, a spin indicator represented by a triangle and an energy gauge represented by a bar also appeared in the world. A large wireframe cube was the "atom building area" where the students placed the atomic particles. A shape representing hydrogen and a box representing a "quick fill" function were also present. I will explain those two items in context of the procedure that the students used to build the water molecule.
Students began by grabbing a proton and placing it in the atom building area. They did this by moving their hand around until they saw their virtual hand intersecting the virtual proton. They then pushed the "grab" button on the wand and the virtual proton would stick to their virtual hand. They then moved their hand until the virtual proton intersected with the atom building area. They might have had to use the fly forward or backward buttons to reach the atom building area. They could then push another button to release the virtual proton from their virtual hand. This maneuvering took some practice.
Not available at this time.
Figure 4: Virtual Chemistry World
After the proton was placed, the students had to grab an electron. The electron object spun in a clockwise manner to indicate its spin status. The energy bar began at its default energy level of 1s. Therefore, the student did not have to adjust anything. When the first electron was placed, the 1s orbital would appear, represented by a wireframe sphere (see Figure 5). The electron would buzz around inside the sphere 90% of the time, appearing and disappearing randomly. The other 10% of the time, the electron would appear outside of the orbital object. This represented the chemistry concept that orbitals are merely probability areas where the electron can be found a certain percentage of the time. The 1s orbital appeared as a wireframe object because it was half full.
Before the student could place a second electron, its spin had to be changed. This was done by placing the virtual hand over the spin indicator and clicking the grab button. The spin indicator operated as a toggle switch, so clicking on this button changed the spin to whatever it had not been before. The student then grabbed the electron and placed it in the atom building area. When the second electron was placed in the atom building area, the 1s orbital turned solid to indicate that the orbital was full (see Figure 6). Inside of this orbital, the two electron continued to buzz around 90% of the time. If the student forgot to change the electron spin, the electron could still be placed inside of the
Figure 5: 1s Orbital
Figure 6: 1s2 2s1 Orbital
atom building area. However, instead of seeing the electron buzzing inside the solid orbital, the orbital would remain as a wireframe object, the electron would float back to its starting position, and a belching sound would be heard. This was counted as a spin error. If students attempted to place the same electron again without correcting for spin, I interrupted them and explained the error.
To place the third electron, the energy had to be increased to the 2s level. Students did this by repeatedly clicking on the energy gauge (shown in Figure 7). The gauge was similar to a thermometer and the "mercury" in the gauge would increase as the student clicked the icon. When the gauge was at the 2s level, the student could then grab the electron and place it in the atom building area. The 2s orbital then appeared, represented by a wireframe sphere that surrounded the solid 1s orbital (see Figure 7). Again, the electron would buzz around inside this area. If a student did not increase the energy to the correct level, when the electron was placed it would float back to its starting position with a belching sound. This was counted as an energy error.
Students did not have to keep the atom balanced by placing protons and neutrons as they went along (see Figures 8 to 10). I made this decision because the procedure for doing so would have been too tedious. Instead, after the students finished placing the
Figure 7: 1s2 2s2 2px1 Orbital
Figure 8: 1s2 2s2 2px1 2py1 Orbital
Figure 9: 1s2 2s2 2px1 2py1 2pz1 Orbital
Figure 10: 1s2 2s2 2px2 2py1 2pz1 Orbital
correct number of electrons for an oxygen atom, they clicked on a "quick fill" box and the correct number of protons and neutrons floated into the atom.
To help the atom building process, sound effects were included for feedback. Sounds existed for the completion of any sub-task such as grabbing a proton, placing a proton, changing the electron spin, or as previously mentioned, whenever an error was made. The students had to place eight electrons and "quick fill" the rest of the protons and neutrons to complete the oxygen atom. When the atom was complete, the students had to bond it with two hydrogen atoms. The object representing the hydrogen atom consisted of an electron buzzing around inside of a wireframe 1s orbital with a proton in the center. The students would grab the hydrogen atom and place it so it was touching one of the half filled orbitals of the oxygen atom (see Figure 11). When bonding occurred, the oxygen orbital and the hydrogen orbital would both turn solid to indicate that they should be considered to be full orbitals. Additionally, a laughing sound was heard to indicate that the orbitals "preferred" to be in that state. The students placed two hydrogen atoms to illustrate a stable water molecule with no available orbitals (see Figure 12).
Figure 11: H-O Molecule
Figure 12: H2O Molecule
The purpose of this treatment was to mimic the VR treatment, but without the immersion or interactivity aspects. A video tape was created by recording what was seen on the left eye display as I created virtual atoms within VR. While I built the atom, I also taped my voice narrating what was happening. For the tape, I built a lithium atom to imitate the VR demo I gave to the VR treatment students and then I built a water molecule to impersonate what the students did while in the virtual environment. I did not mimic any of the typical mistakes that the VR students made, such as not changing the spin or energy level. Full computer sound was included as well as my voice narrating the building process. This video tape was played on a VCR connected to a 19 inch television placed 6 feet away from the participant.
The video tape was played while the student watched silently. This situation allowed the student to view and hear the same symbolism that the VR participants experienced, but on a flat 2D television screen and without any chance of interactivity.
The purpose of this treatment was to mimic the VR treatment without the immersion element, while maintaining the interactivity aspect. I programmed a chemistry world to run on a Macintosh computer with a standard 2D display. Using MacroMedia's Authorware and Director software, I created the same virtual atomic objects for the Macintosh that I had programmed for the VR system. Students could grab the protons, electrons, and neutrons and drag them into the atom building area using a standard mouse input device. The spin indicator, energy level, "quick fill" box, and hydrogen atoms were all present and had the same functions as in the VR version. The Mac objects had a different look than the VR objects due to the lack of depth. Figure 13 shows the Mac view. There was no immersion effect with all of the objects in view at one time on the Mac.
A Macintosh Quadra with 8 MB of RAM, 40 MB of hard drive, and a 13 inch color screen was used for this treatment. The students sat one foot away from the screen while interacting with the program.
Figure 13: Macintosh Chemistry World
I demonstrated the Mac chemistry world by creating a lithium atom on the screen and then bonding it with a hydrogen atom. I explained the functionality of the system including the spin and energy indicators. All of the students had used a Macintosh before, so I did not need to explain how to use the mouse to click and drag objects. After the demonstration, the students created a water molecule on the screen.
The purpose of this treatment was to mimic the Mac Interactive treatment without the interactivity. In effect, this duplicates the no immersion and no interactivity state of the Video treatment. If interaction and immersion are the only significant factors, then the Mac Run results should be the same as the Video results. Changing the media can unfortunately result in changing more than the factors of interest, which in this case are immersion and interactivity. If the results of the Video and Mac Run treatments were not equal, then differences other than immersion and interactivity were involved. The reason this is important is that any unknown factors might also impact a fair comparison between the VR treatment and the Mac Interactive treatment.
I reprogrammed the Authorware software of the Mac Interactive to run through the building of a water molecule without a participant interacting with the computer. I narrated the water building process on an audio tape which I synchronized to the computer program. Again, a Macintosh Quadra with 8 MB of RAM, 40 MB of hard drive, and a 13 inch color screen was used for this treatment. The students sat one foot away from the screen while watching the program.
I demonstrated the process of building atoms and molecules in the same manner as I did for the Mac Interactive treatment. Afterwards, the students watched the computer as the water molecule was built on the screen. They also listened to the synchronized audio tape explain what was happening on the computer. As with the video treatment, there was no interactivity because I controlled the computer program.
The control treatment consisted of no multimedia presentation of atomic and molecular structure. The students were given the pre tests and then after a period of time, were given the post tests.
In addition to the main factors of immersion and interactivity, there were other factors worth exploring in the experiment. The prime one was student learning style, which has had a great deal of focus in the field of education. Howard Gardner in his book, Frames of Mind (Gardner, 1985) asserts that they are various natural styles of learning including linguistic, spatial, and logical-mathematical types. Different proponents of this theory have different ways of categorizing learning styles, but they all state that students will learn more easily if the style of instruction matches their style of learning. For example, a visual learner will do better if the instruction includes lots of pictures as opposed to all text. Unfortunately, there have been no conclusive studies proving that learning style is an important factor in education. In fact, many studies conclude that learning style is not an important factor.
Despite this lack of evidence, learning style is an interesting factor in studying VR, particularly the style of spatial learning. Spatial learners are characterized as people who can readily visualize and manipulate 3D forms in their mind. For example, the inventor Nikola Tesla claimed to be able to "project before his eyes a picture complete in every detail, of every part of the machine." (Gardner, 1985, pg. 187). Since chemistry is inherently composed of 3D particles, the ability to visualize and mentally manipulate these shapes is extremely helpful. A famous example is the chemist Friedrich Kekule's discovery of the structure of the benzene ring by dreaming of a snake eating its tail.
VR should exacerbate the difference between students who are identified as spatial learners and those who are not, since VR provides a 3D spatial environment for learning. For the instructional topic of atomic and molecular structure, this difference could be one of two opposite results. The first possibility is that VR will help spatial learners more since the material is being presented in a natural way for them. The other alternative is that the non-spatial learner will be helped more by VR. The problem for non-spatial learners in understanding spatial concepts might be due to a problem in creating a spatial mental model of the topic, not in manipulating the spatial mental model. If that is indeed the problem, then VR can present a spatial model for the student to use. If the problem is truly one of manipulating the mental model, then VR will not be very helpful.
To ascertain the students' spatial ability, they were given the spatial ability portion of the Differential Aptitude Test (DAT) battery designed by Psychology, Inc. The reason that this test was chosen was its relevance to spatial problem solving and its long history of use. Unfortunately due to experimental constraints, the DAT was only administered to the VR, Video, and control groups. The VR and Video treatments do offer the extreme differences in terms of spatial environment, so conclusions will still be able to be drawn with this limited testing structure. The test was given during the students' regular chemistry class, one week before the experimental treatments began. The 2 dimensional, paper test consisted of a series of 35 unfolded boxes of various shapes and shading with a multiple choice of folded boxes. With a time limit of 15 minutes, the students had to choose which one folded box corresponded to the unfolded box.
Pre-treatment and post-treatment chemistry tests were given to the students to gauge their acquisition of chemistry knowledge as a result of the treatments. This also allowed comparison among the various treatments. The challenge in creating these tests was to develop a metric that truly measured understanding of the subject instead of merely examining a student's ability to memorize the topic. Chemistry, along with many other topics, has two levels of knowledge associated with it. Chemistry happens to an excellent case study of a subject in which ability to follow the rules and understanding the rules can easily be confused. An example of the ability to memorize and follow the rules of chemistry, is being able to write that H2O is the formula for water, which is a molecule with no free valence electrons. Understanding what the rules mean and how they relate to each other is shown if the student cannot only complete the orbital fill diagram for H2O, but also understands the significance of that molecule having no free valence electrons.
After reviewing assessment options in the literature, I decided to use two tests, one written and one oral, for both the pre-treatment and post-treatment exams. Using two tests allowed me to acknowledge the range of assessment techniques and hopefully combine the strengths of the methods. These tools included: free flowing oral interviews, two-tiered multiple choice tests, "fill in the blank" tests, and multiple choice tests. The proponents of the oral interviews feel that asking open questions and having the students sketch some of the answers allows the interviewer to assess the students' deeper level of knowledge about the subject (Griffiths and Preston, 1992). They quantified their data by assessing the students according to a set of concepts. For example, they would ask questions about sizes of molecules so they could determine whether or not the student understood the concept of size.
The two-tiered multiple choice tests attempt to exploit the objectivity and ease of grading of the traditional multiple choice test with more information. So, a multiple choice question is asked about a certain topic and then a second multiple question is asked to elicit their reasoning in answering the first question (Treagust, 1988). The "fill in the blank" and multiple choice tests are commonly used in traditional textbooks and classrooms and need no elaborate explanation.
I chose to use the oral interview test and a written "fill in the blank" test. I agreed with the researchers who believe that the interview method gives the most complete information about the students' knowledge. However, in the context of a traditional high school chemistry curriculum there is a need to be able to relate the results here to the traditional method of assessment. Using both tests also allow me to compare the two methods and draw conclusions.
The written test was based on the textbook Chemistry (Herron et. al., 1993) and conversations with the students' teachers. The pre and post written tests had the same types of questions on them, but asked about different atoms and molecules. The pre test is shown in Appendix A and the post test is shown in Appendix B. The oral test, shown in Appendix C, was the same for both the pre and post test. The oral test was graded by an interviewer who did not know what treatment the participant experienced. The interviewer had a pre-med college background and was trained in the interview method during the pilot study. I graded the written test without knowing whose test I was grading. A numerical score was given for both the oral and written test. The grading of the oral test followed the method as outlined by Griffiths and Preston (1992). This method consisted of breaking down large concepts into very specific smaller concepts. For example, the concept "understanding a water molecule" was broken into smaller pieces such as "understanding that two electrons in the same orbital have different spins" and "a 2s orbital has higher energy than a 1s orbital." Throughout the test, the interviewer decided what smaller concepts the student had grasped. Each small concept was worth one point and the points were added for the final score.
Long term retention of knowledge was also assessed. For this experiment, I chose to test a sample of students 3 months after their participation in a treatment. Again, the oral test was the same, while the written test was altered slightly. Unfortunately, I was not able to include students from the Mac Interactive and Mac Run treatments in the long term retention testing. This was because these treatments were conducted at the end of the study.