Thomas A. DeFanti
Daniel J. Sandin
Robert V. Kenyon
Electronic Visualization Laboratory
University of Illinois at Chicago
Human performance experiments test and/or help understand: the hypothesis that Fitts Law can be used as a quantitative tool for identifying tasks that will result in a positive transfer of training; the effects of tactile and force feedback on human performance during complex tasks; and, the advantages of connecting two or more CAVEs to create an environment conducive to collaborative work (i.e., quantifying the effects of transmission delay and the precision of the placement of the objects from one CAVE to the other).
Hardware/Software performance experiments involve: reducing tracker delay; increasing tracker accuracy; and, improving the interconnection among CAVE systems.
Tele-Immersion. EVL's primary research focus is tele-immersion; i.e., enabling collaborative VR over high-speed networks. EVL is designing CAVE-to-CAVE experiments over the vBNS network to study latency and bandwidth issues. EVL is collaborating with the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and Argonne National Laboratory (ANL) in this effort, with EVL being the technical lead, as well as providing graduate students for assessing and improving latency issues, creating software tools, and working with companies.
Dual-user tracked stereo. EVL has made significant progress developing dual-user-tracked stereo (i.e., allowing two users to each see the VR display with correct perspective) by running the display and glasses at 160Hz. This has required hardware modifications to the glasses and new stereo format programming for the SGI Infinite Reality Engines.
CAVE library upgrades. EVL successfully ported CAVE libraries to the R10000 Infinite Reality systems and the Onyx2 (for the CAVE, ImmersaDesk, and Infinity Wall). The CAVE library was also ported to the SGI Indigo Maximum Impact and the O2/Octane systems for the ImmersaDesk.
New initiative: NASA collaborations for virtual prototyping. EVL is collaborating with Dr. Ahmed Noor of the University of Virginia/ NASA Langley on virtual prototyping for spacecraft design. The CAVE and ImmersaDesk provide designers with a visualization platform to view computational models created using supercomputers. EVL's goal is to provide designers with an interactive tool that not only displays the output of their simulations, but also allows them to interact with their simulations in real time; i.e., apply test conditions and change environmental states so that the effects of load, stress/strain, and the integrity of the components can be tested. Ultimately, through tele-immersion, several engineers located at various NASA industry centers could work on the same computational model using networked VR systems to test and validate spacecraft design before any hardware was built.
Tracker latency. EVL made significant progress minimizing tracker latency. Tracker delay dropped from its original 150 ms to a more reasonable value of 50-75 ms. EVL is trying to identify the source of these delays and understand why they occur. To achieve low latency, EVL used Scramnet reflective memory between the trackers and the CAVE computers. Not only was the latency reduced, but also the large variability in latency values. This translated into smoother, more responsive, interactive CAVE experiences. The lower latency also makes the CAVE and ImmersaDesk more comparable to other commercial VR systems, enabling better comparisons with data generated from human interaction experiments using Fitts Law. EVL is currently evaluating the use of local Ethernet rather than the more expensive Scramnet solution.
Line-of-Sight method for tracker calibration. Previous calibration methods required a very tedious procedure to gather dense correction tables. In the case of the CAVE, close to 1000 measurements were necessary to create such a table. EVL has been testing a new method, called Line-of-Sight, that produces a very dense correction table, corrects most erroneous areas, and requires very few measurements within the virtual environment (VE). The calibration procedure requires the user to visually align physical targets with virtual targets generated by the VE system. For example, calibration of the ImmersaDesk is performed with a grid of physical, spherical targets suspended above the screen. In the CAVE, similar physical targets are placed in front of the walls.
To calibrate the environment, the user walks in VR space and identifies areas where there is a discrepancy between the visual alignment of the real and virtual targets using "line of sight." A person uses the wand (a 6-degree-of-freedom mouse) to introduce a correction vector that aligns the two targets. Based on these vectors, a dense correction table is computed. In CAVE experiments, approximately 20 correction vectors are needed. Quantitative evaluation of the method is still in progress. Currently the method only corrects for positional errors; correction for rotational errors is being implemented.
Network and human behavior in CAVE-to-CAVE applications. EVL examined the round-trip network latency between performing an action using the CAVE/ImmersaDesk and getting the result of that action back through the network. The round-trip network latency for a 55-byte packet sent between two hosts at a constant rate of 30 packets/second was measured using Scramnet, Ethernet, and ISDN networks.
Scramnet latency averaged 310 microseconds with a standard deviation of 67 microseconds. Local (i.e., within EVL) Ethernet latency was 14 ms with a standard deviation of 5 ms. ISDN latency was 216 ms with a standard deviation of 175 ms.
This, however, does not tell the whole story. Figure 1 shows the time distribution of the packet delays during these experiments. This figure of ISDN delay shows that the delay, while small most of the time, can rise to very high values followed by a gradual reduction in the delay with packet numbers. This sawtooth pattern was seen in all networks using TCP/IP, but not for Scramnet or UDP. The size of the sawtooth was between 5-25 ms when using local Ethernet, and several seconds when using ISDN. The pattern of sawtooth delay is believed to be a consequence of the TCP/IP protocol requirement that packets are guaranteed to be delivered in the correct order. Episodes of long delay are believed to be initiated by lost packets or collisions that require the operating system kernel to retransmit damaged packets and all the packets after them. The consequence of this behavior is that the object controlled by the remote partner may freeze for some time and then resume its motion when the delayed, retransmitted data stream reaches the application, similar to playing back a videotape of the remote user's actions.
In addition to these tests, EVL just completed a set of experiments to quantify collaborative performance between CAVE and ImmersaDesk partners across various network types using delays of 10 ms and 200 ms with and without large variation (or jitter) in the delay distribution. Experiment results are currently being analyzed; however, the data clearly indicates that the jitter pattern of delay, as shown in Figure 1, has the most adverse impact on performance and that partners tolerate long but constant delay better than variable delay.
New initiative: Time Petri Net analysis of CAVE applications. EVL is exploring the use of time Petri Net analysis as a tool to enable CAVE application developers to better understand the relationship between CAVE characteristics and their application's performance. This work is in collaboration with Professor Buy of the UIC EECS department. A model of a generic CAVE application was created -- including the timing relationships between different software/hardware subsystems, interprocessor communication, timing characteristics and constraints on system functionality and performance -- and then analyzed for reachability, correctness, and safety using a Petri Net tool called CABERNET.
The next step is to model the asynchronous behavior among different CAVE application processes and identify the conditions that cause such anomalies. EVL, in collaboration with Dr. Buy, plans to develop a visual language for specifying VE applications and a set of translation tools that generate Petri Net models from high-level languages such as C++, ADA, or GUI-based descriptions.
Fitts measures of performance. The Fitts experiments, completed this Spring, yield some very interesting results that have provided quantitative understanding of how task difficulty impacts completion time for manual coordination tasks. These experiments used fixed targets with the stylus attached to the hand. The distance and precision of the targets used for these discrete movement experiments were systematically varied to obtain Fitts Law curves for these virtual world tasks. To generate Fitts Law curves, the target and amplitude sizes were varied to produce 5 different index of difficulty (ID) values. VE was tested using visual feedback, in addition to tactile, sound, and vibrations, to provide sensory cueing to the subject.
Figure 2 shows the relationship between Fitts performance in the CAVE (average of 6 subjects without feedback) to that taken in other environments. Sources for the published data are: Divers were taken from divers underwater (Kerr, 1978), Fitts from (Fitts and Peterson, 1964), and an immersive Head Mounted Display (HMD) from (Eggleston et.al., 1996). The ImmersaDesk (IDesk) shows preliminary data averaged from 2 subjects run at EVL. EVL is still in the process of analyzing this data so the conclusions drawn are preliminary and are subject to change.
However, from the analysis thus far, it can be hypothesized that the CAVE performance at low values of ID (2-4) are similar to that found by Eggleston et. al. (1996). As ID values increase, there is a sharp departure from the Eggleston data with the CAVE movement time values increasing rapidly. This indicates that the subjects become more deliberate in their movements to hit the target. Several hypotheses are under investigation to account for this rise in movement time. Among these are the experience of the subject with the CAVE, the visual characteristics of the CAVE, and the subject variability with respect to population averages.
One of the issues under examination is that these experiments relied on the CAVE floor to reflect the image from the projectors; however, the CAVE floor is the most difficult surface on which to produce a high quality image. Perhaps performance was poor since the image reflected from the CAVE floor lacked sufficient contrast/brightness and the distinct location of the target (particularly a small target) was difficult to see.
Interestingly, the data from last year's ImmersaDesk experiments show that with a bright, high-resolution, high-contrast image, such as that produced by the ImmersaDesk, performance is greatly improved and superior to HMD data. In fact, it appears very similar to that found in subjects working underwater (Kerr, 1978). It is tempting to speculate that the close relationship between these two environments may make the ImmersaDesk a useful tool for training divers to perform tasks underwater. The results of this work have been incorporated into a proposal to NASA to investigate the use of the CAVE/ImmersaDesk for astronaut training, in addition to neutral buoyancy tank training (Kenyon et. al., 1997).
T. DeFanti, D. Sandin, and M. Brown, "The Coming Defenestration: Immersive Environments Without Windows," IEEE Multimedia, Winter 1996, pp. 6-9.
T.A. DeFanti, M.D. Brown, and R. Stevens, "Virtual Reality Over High-Speed Networks," IEEE Computer Graphics & Applications, July 1996, pp. 42-43.
J. Leigh, A. Johnson, T. DeFanti, "CALVIN: an Immersimedia Design Environment Utilizing Heterogeneous Perspectives," Proceedings of IEEE International Conference on Multimedia Computing and Systems '96, Hiroshima, Japan, June 17 - 21, 1996.
J. Leigh, A.E. Johnson, C.A. Vasilakis, T.A. DeFanti, "Multi-Perspective Collaborative Design in Persistent Networked Virtual Environments," Proceedings of the IEEE Virtual Reality Annual International Symposium (VRAIS 96), Santa Clara, CA, March 1996, pp. 253-260 and 271-272.
T.A. DeFanti, D.J. Sandin, G. Lindahl, and M.D. Brown, "High-resolution and High-bandwidth Immersive Interactivity," Very High Resolution and Quality Imaging Conference, SPIE Proceedings, Vol. 2663, February 1996, pp. 28.
W.D. Reynolds and R.V. Kenyon, "The Wavelet Transform and the Suppression Theory of Binocular Vision for Stereo Image Compression," 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland, Sep. 16-19, 1996.
S.K. Isabelle, R.H. Gilkey, R.V. Kenyon, G. Valentino, J. Flach, C. Spenny, and T.R. Anderson, "Defense applications of the CAVE (CAVE Automatic Virtual Environment)," Proceedings of SPIE: 11th Annual Conference on Aerospace/sensing simulation and control, Orlando, Fl, April 20-25, 1997.
G.A. Gleason and R.V. Kenyon, "The Mandelbaum Effect may not be due to involuntary mis-Accommodation," The Association for Research in Vision and Ophthalmology (ARVO), Fort Lauderdale, Fl, May 11-16, 1997.
E.R. Boer and R.V. Kenyon, "Estimation of Time Varying Delay Time in Non-Stationary Linear Systems: An Approach to Monitor Human Operator Adaptation in Manual Tracking Tasks," IEEE Trans. Man, Systems and Cybern. Nov. 1997, to be published.
E.R. Boer and R.V. Kenyon, "Adaptation Hysteresis in Manual Tracking," IEEE International Conference Man Systems and Cybernetics, Orlando, FL, Oct 12-15, 1997.
The use of VR for training, followed by transfer-of-training from VR to the physical world, has broad applicability in the National Challenge areas, particularly in manufacturing and education. Using a simple sensory-motor task pioneered by Fitts [Fitts, 1954], human motor control and performance in virtual environments can be quantified. Results indicate that there are clear differences in performance in different VE systems that may influence how these devices are used.
The relationship between the technology and the training tasks that will result in a maximum transfer of training are of major interest -- particularly to industries where expensive, highly specialized equipment make training somewhat cost prohibitive. Early results indicate that latency is a severe problem and that restricting VR solely to the visual domain limits its training applicability. The quantitative assessment of VR as a training tool, including testing procedures and needed hardware/software improvements, is of major interest to industry today, and is the focus of this grant.
R.G. Eggleston, W. Janson and K. Aldrich, "Time delay and task factor effects on aimed movement performance in a 3D virtual environment," 1996 Image Conference, Scottsdale, AZ, 23-28, 1996.
R. Kerr, "Diving, adaptation, and Fitts' law," J. Motor Behavior, 10:255-260, 1978.
R. Kenyon, N. Ye, and D. Adkins, "Motor Performance in Virtual and Earth-based Environments and their Relationship to Work in Microgravity," Proposal to NASA NRA-HEDS-05, April 1997.
Connect to remote computations and data sources. This is inevitable and will be driven by every sector of computing and Web usage. The research concerns how to do this well, safely, and at a cost researchers can afford.
Provide enough anti-aliased image resolution to match human vision. CRT technology seems limited by market forces and development to 2048x2048 this decade. 20/20 vision is roughly 5000 pixels (at 90-degree angle of view), less is needed at the angle we normally view television or workstation screens, and more for wide-angle VR applications.