CAREER: Prevention, Detection and Repair of Miscommunication in Spoken Natural
Language Dialog Processing Systems
Ronnie Smith
Mathematics
East Carolina University
Austin Building, Room 129.Greenville NC27858
Email: rws@cs.ecu.edu
Phone: 252-328-1906
Fax: 252-328-6414
WWW Page
www.cs.ecu.edu/~rws
Program Area
Speech and Natural Language Understanding
Keywords
Spoken Natural Language Dialog, Miscommunication
Project Summary
Improvements in speech recognition technology have made spoken natural language
interfaces a viable means of human-computer interaction. To fully exploit this mode of
communication, the speech recognition capabilities must be integrated within a dialog
processing mechanism.
An important unresolved issue in spoken natural language dialog processing is the handling
of miscommunication. In this project we have studied previously recorded human-human and
human-computer dialogs in order to investigate strategies for reducing miscommunication in
natural language dialog. The following steps have been or are being explored:
1. Categorizing initiative-dependent features of human-computer task-oriented dialogs as
users evolve from novice to expert.
2. Developing a context-based model for selective verification of user inputs whose
meaning is in question.
3. Developing strategies for permitting either computer or user-initiated subdialogs for
resolving miscommunications and potential miscommunications.
4. Implementing these theories and evaluating their performance under both simulated and
experimental conditions.
Project References
R.W. Smith, "An Evaluation of Strategies for Selectively Verifying Utterance
Meanings in Spoken Natural Language Dialog", International Journal of Human Computer
Studies, vol 48, pages 627-647, 1998.
R.W. Smith and S.A. Gordon, ``Effects of Variable Initiative on
Linguistic Behavior in Human-Computer Spoken Natural Language Dialogue,'' Computational
Linguistics, vol. 23, no. 1, pages 141-168, March 1997.
R.W. Smith, ``An Evaluation of Strategies for Selective Utterance
Verification for Spoken Natural Language Dialog,'' Proceedings of the Fifth Conference on
Applied Natural Language Processing,
pages 41-48, April 1997.
R.W. Smith, ``Practical Issues in Mixed-Initiative Natural Language
Dialog: An Experimental Perspective,'' Proceedings of the 1997 AAAI Spring Symposium on
Computational Models for Mixed Initiative Interaction, pages 158-162, March 1997.
R.W. Smith, D.R. Hipp, and A.W. Biermann, "An Architecture for Voice Dialog Systems
Based on Prolog-Style Theorem Proving," Computational Linguistics, vol. 21, no. 3,
pages 281-320, September 1995.
R.W. Smith and D.R. Hipp, Spoken Natural Language Dialog Systems: A Practical Approach,
Oxford University Press, 1994.
R.W. Smith, "Spoken Variable Initiative Dialog: An Adaptable Interface," IEEE
Expert, vol. 9, no. 1, pages 45-50, February 1994.
Area Background
My study of computational modeling of natural language dialog has focused on issues
concerning voice interfaces, real-time interaction, and integrated modeling of the
following behaviors: (1) collaborative problem solving, (2) subdialog completion and
movement, (3) contextual interpretation, (4) user-dependent response generation, and (5)
mixed-initiative interaction.
This wholistic modeling of natural language dialog requires an awareness of work on
various subproblems in dialog processing including quantification, presuppositions,
ellipsis, anaphoric reference, user modeling, expectation modeling, plan recognition, and
miscommunication handling.
An important methodlogy for validation of the computational model is system construction
and formal experimentation. The goal of empirically validating the model necessitates an
awareness of computational constraints and robust error handling techniques as well as
familiarity with past experimental studies on discourse behavior (usually of the
human-human or simulated
human-computer variety). Empirical study is beneficial in acquiring knowledge about how
human linguistic behavior during interaction with a computer may differ from what would
occur if the interaction was with another human.
Area References
S.W. McRoy and G. Hirst, "The Repair of Speech Act Misunderstandings by Abductive
Inference," Computational Linguistics. vol. 21, no. 4, pages 435-478, 1995.
S.E. Brennan and E.A. Hulteen, "Interaction and feedback in a spoken language system:
a theoretical framework," Knowledge-Based Systems. vol. 8, pages 143-151, 1995.
J.F. Allen, L.K. Schubert, G. Ferguson, P. Heeman, C.H. Hwang, T. Kato, M. Light, N.
Martin, B. Miller, M. Poesio, and D.R. Traum, "The TRAINS Project: A Case Study in
Building a Conversational Planning Agent." Journal of Experimental and Theoretical
Artificial Intelligence. vol. 7, pages 7-48, January, 1995.
S. Carberry, Plan Recognition in Natural Language Dialogue, MIT
Press, Cambridge, Mass., 1990.
S.R. Young, A.G. Hauptmann, W.H. Ward, E.T. Smith, and P. Werner, "High Level
Knowledge Sources in Usable Speech Recognition Systems," Communications of the ACM,
vol. 32, no. 2, pages 183-194, February 1989.
R. Wilensky, D.N. Chin, M. Luria, J. Martin, J. Mayfield, and D. Wu, "The Berkeley
UNIX Consultant Project", Computational Linguistics, vol. 14, no. 4, pages 35-84,
1988.
B.J. Grosz and C.L. Sidner, "Attentions, Intentions, and the Structure of
Discourse", Computational Linguistics, vol. 12, no. 3, pages 175-204,1986.
Related Program Areas
Adaptive Human Interfaces
Usability and User-Centered Design
Intelligent Interactive Systems for Persons with Disabilities
Potential Related Projects
Port the dialog model to other domains such as advisory dialogs or to other
environments such as telephone interactions. Achieve total integration of the model with a
speech recognition system to enhance robustness and efficiency. Conduct large-scale user
testing of the dialog model via a system that assists in completing a complex real-world
task (i.e., a task requiring a vocabulary of 500 words or more).