Multi-Modal Spatial Querying

Max J. Egenhofer
Scott Overmyer

National Center for Geographic Information and Analysis
Department of Spatial Information Science and Engineering
Department of Computer Science
University of Maine

CONTACT INFORMATION

NCGIA
Boardman Hall
University of Maine
Orono, ME, 04469-5711
Phone: (207) 581-2114
Fax : (207) 581-2206
Email: max@spatial.maine.edu

WWW PAGE

http://www.spatial.maine.edu/~max/max.html

PROGRAM AREA

Other Communication Modalities

KEYWORDS

Sketching, voice-sketch interaction, spatial queries, geographic information systems.

PROJECT SUMMARY

People who use multiple channels at the same time communicate more successfully about spatial problems than those who rely exclusively on either voice or pictures. This project investigates the use of two concurrent communication channels--graphics and speech--to achieve a similarly successful interaction between a person and a geographic information system (GIS). The objective is to construct a multi-modal spatial query language in which users interact with a geographic database by drawing sketches of the desired configuration, while simultaneously talking about the spatial objects and the spatial relations drawn. Through the combined use of graphics and sketch, more intuitive and more precise specifications of spatial queries are expected. This study will increase our understanding of multi-modal spatial interactions, and will lead to improved strategies for intelligent integration and processing of such multi-modal spatial queries in a GIS. The key to this interaction is the exploitation of complementary or redundant information present in both graphical and verbal descriptions of the same spatial scenes. A multiple-resolution model of spatial relations is used to capture the essential aspects of a sketch and its corresponding verbal description. The model stresses topological properties, such as containment and neighborhood, and considers metrical properties, such as distances and directions, as refinements where necessary. This model enables the retrieval of similar, not only exact, matches between a spatial query and a geographic database. Such new methods of multi-modal spatial querying and spatial similarity retrieval will empower experts as well as novice users to perform easier spatial searches, ultimately providing new user communities access to spatial databases.

PROJECT REFERENCES

M. Egenhofer, ``Query Processing in Spatial-Query-by-Sketch,'' Journal of Visual Languages and Computing, (in press).

M. Egenhofer, ``Spatial-Query-by-Sketch,'' VL'96: IEEE Symposium on Visual Languages, Boulder, CO, M. Burnett and W. Citrin (eds.), pp. 60-67, 1996.

M. Egenhofer, ``Multi-Modal Spatial Querying,'' Seventh International Symposium on Spatial Data Handling (SDH '96), Delft, The Netherlands, M.-J. Kraak and M. Molenaar (eds.), pp. 785-799, 1996.

M. Egenhofer and A. R. Shariff, ``Metric Details for Natural-Language Spatial Relations,'' ACM Transactions on Information Systems, (in press).

AREA BACKGROUND

The background for this work is the field of geographic information science, a newly developing area that cuts across multiple disciplines, including geography, computer science, and engineering, with significant roles for cognitive science, linguistics, and psychology, particularly for the design of interfaces for next-generation geographic information systems. Geographic information systems are widely used for the acquisition, storage, management, and visualization of geographic data and the extraction of information. Application domains include land parcel management at a local community level, vehicle navigation, natural resource exploitation (e.g., in the oil industry) and environmental hazards control. Lately such new exciting fields as GIS in business (e.g., in advertising or for retailing) have found their champions. Current GISs are huge monolithic systems, located on the desktops of GIS experts, and suffer from tedious user interfaces with thousands of commands. New interaction modalities, that are more natural and appropriate for dealing with spatial data, are necessary in order to cope with the next generation of GIS users who are expected to be primarily non-experts.

AREA REFERENCES

M. Egenhofer and D. Mark (1995) ``Naive Geography,'' in A. Frank and W. Kuhn (Ed.), Spatial Information Theory--A Theoretical Basis for GIS, International Conference COSIT '95, Semmering, Austria. Lecture Notes in Computer Science 988, pp. 1-15, Springer-Verlag, Berlin.

W. Kuhn (1992) ``Paradigms of GIS Use,'' in D. Cowen (Ed.), Fifth International Symposium on Spatial Data Handling, Charleston, SC, pp. 91-103.

D. Maguire, M. Goodchild, and D. Rhind, Ed. (1991) Geographical Information Systems. Longman, London.

D. Mark and A. Frank (1991) Cognitive and Linguistic Aspects of Geographic Space. Kluwer Academic Publishers, Dordrecht.

RELATED PROGRAM AREAS

Adaptive Human Interfaces.