Division of Engineering and Applied Sciences
Harvard University
When people communicate with a system, whether using some formal language (e.g., shell languages, query languages, menu-based systems) or a natural language, in so doing they execute actions. These user actions typically result in the system itself acting, and the resulting system actions are central to people achieving their goals. Thus, users participate in dialogues with systems about their goals and the task they want done. However, most current interfaces do not keep track of many features that are central to the success of dialogues in human collaborations; for instance, they do not use information about what a person is trying to achieve nor adequately model the dialogue context. As a result, they do little to support user-system collaboration on a task and most of the work of keeping track of the context of their communication is left to the user.
Our research is based on the results of two strands of previous research: (1) research towards the building of novel interfaces for particular problems in automated graphic design that exemplify and explore the view of communication with computers as a collaborative activity; and (2) the development of explicit theories of collaboration. It addresses a key missing element of prior research: an understanding of how theory can be integrated with practice so that our theoretical understanding of collaborative activity in the abstract can inform in a principled manner the design of concrete software interfaces, and thus greatly improve the dialogue capabilities, in multiple modalities, of current systems.
Our efforts center around three activities:
For one of our first concrete collaborative interfaces (developed before this project was started), we chose to address the problem of network diagram layout. Network diagrams are a common form of informational graphic, and constitute an area in which it may be difficult for a person to define the aesthetic qualities desired in the final design. Most research in network diagram layout has focused on automatic layout. We have created a collaborative, interactive constraint-based editor for network diagram layout, GLIDE, which improves upon general constraint-based editors by providing a small but powerful vocabulary of specialized constraints specifically designed for drawing network diagrams. GLIDE is the first system to support people in interactively specifying the visual organization of a diagram, and currently supports thirteen types of constraints, including: Alignment, Clustering, Zone, Even Spacing, Symmetry, T-Shape, and Hub-Shape. Using the collaborative approach, we take advantage of a user's expertise at globally designing the layout, and the computer's computational superiority; the user is responsible for an approximate layout of the nodes and for specifying any desired visual organization. The system then attempts to solve a constraint-satisfaction problem by calculating a local minimum using a mass-spring physical simulation. The process is intended both to be interactive and iterative. GLIDE provides simple interface mechanisms for the user to create, view, manipulate and remove constraints. Under this paradigm, the user has the flexibility to create interesting designs, without the burden (and tedium) of having to precisely place every object in the layout. It exemplifies the kind of system we will build under the first type of activity.
As an initial effort to use a general theory of collaboration in developing an interface (and in coordination with the Ubiquitous Information Project at Harvard), we are developing a collaborative interface testbed -- named the Distributed Information Access for Learning (DIAL) testbed -- to support distance learning and multi-media information access to students who are geographically distributed. Using DIAL, students will be able to access a variety of sources of course information over the WWW such as: electronic textbooks, lecture notes, assignments, exams, solutions, library reference materials, technical reports, videos of lectures, and video links to teaching assistants and professors. The current testbed system provides students with access to a web server which maintains a hierarchical context reflecting embeddings of subgoals from previous interactions with the student. The context is then used to interpret subsequent information queries: information access is thereby modelled in terms of a sequence of successively refined queries, instead of requiring the student to enter precisely what he is searching for as in, for example, conventional web searching tools. The interface is also collaborative in the sense that the student need not know all of the details regarding locations of pertinent information nor means of accessing them. A number of elements from the SharedPlan formalism (Grosz & Kraus, 1996, 1997) are explicitly represented within the system.
Barbara J. Grosz and Sarit Kraus, "Collaborative plans for complex group action." Artificial Intelligence. 86 (1996) 269-357.
Barbara J. Grosz and Sarit Kraus, "The Evolution of SharedPlans." Foundations and Theories of Rational Agencies. A. Rao and M. Wooldridge, eds. (1997) To appear.
Kathy Ryall, Joe Marks and Stuart Shieber, "An Interactive System for Drawing Graphs", Proceedings of Graph Drawing '96. Stephen North, ed., [Springer Verlag, Berkeley, CA Series: Lecture Notes on Computer Science] 1190 (1996) October.
Kathy Ryall, Joe Marks and Stuart Shieber, "An Interactive Constraint-Based System for Drawing Graphs." Proceedings of the 10th Annual Symposium on User Interface Software and Technology [UIST] (1997) To appear.
Barbara J. Grosz, "Collaborative Systems: 1994 AAAI Presidential Address." AI Magazine, 2 (1996) 17 Summer.
Jonathan Grudin, "Interface: An Evolving Concept." Communications of the Association for Computing Machinery, 36 (1993) 4 April [Special Issue on Graphical User-Interfaces] 110-119.
Jonathan Grudin, "The Computer Reaches Out: The Historical Continuity of Interface Design." Human Factors in Computer Systems (CHI), April (1990) 261-268.
Karen E. Lochbaum, "The Use of Knowledge Preconditions in Language Processing." Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-95) Chris S. Mellish, ed., [Morgan Kaufmann Publishers, Inc. San Mateo, CA] 2 (1995) 1260-1266.
C. Rich and C. Sidner, "COLLAGEN: When agents collaborate with people." In Proceedings: 1st International Conference on Autonomous Agents, Marina del Rey, CA (1997) February, 284-192.
Peter Wegner, "Why Interaction is More Powerful than Algorithms." Communications of the Association for Computing Machinery, 40 (1997) 5 May, 80-91.