CAREER: Prevention, Detection and Repair of Miscommunication in Spoken Natural Language Dialog Processing Systems

Ronnie Smith
East Carolina University
Austin Building, Room 129.Greenville NC27858
Phone: 252-328-1906
Fax: 252-328-6414

WWW Page

Program Area

Speech and Natural Language Understanding


Spoken Natural Language Dialog, Miscommunication

Project Summary

Improvements in speech recognition technology have made spoken natural language interfaces a viable means of human-computer interaction. To fully exploit this mode of communication, the speech recognition capabilities must be integrated within a dialog processing mechanism.

An important unresolved issue in spoken natural language dialog processing is the handling of miscommunication. In this project we have studied previously recorded human-human and human-computer dialogs in order to investigate strategies for reducing miscommunication in natural language dialog. The following steps have been or are being explored:

1. Categorizing initiative-dependent features of human-computer task-oriented dialogs as users evolve from novice to expert.

2. Developing a context-based model for selective verification of user inputs whose meaning is in question.

3. Developing strategies for permitting either computer or user-initiated subdialogs for resolving miscommunications and potential miscommunications.

4. Implementing these theories and evaluating their performance under both simulated and experimental conditions.

Project References

R.W. Smith, "An Evaluation of Strategies for Selectively Verifying Utterance Meanings in Spoken Natural Language Dialog", International Journal of Human Computer Studies, vol 48, pages 627-647, 1998.

R.W. Smith and S.A. Gordon, ``Effects of Variable Initiative on
Linguistic Behavior in Human-Computer Spoken Natural Language Dialogue,'' Computational Linguistics, vol. 23, no. 1, pages 141-168, March 1997.

R.W. Smith, ``An Evaluation of Strategies for Selective Utterance
Verification for Spoken Natural Language Dialog,'' Proceedings of the Fifth Conference on Applied Natural Language Processing,
pages 41-48, April 1997.

R.W. Smith, ``Practical Issues in Mixed-Initiative Natural Language
Dialog: An Experimental Perspective,'' Proceedings of the 1997 AAAI Spring Symposium on Computational Models for Mixed Initiative Interaction, pages 158-162, March 1997.

R.W. Smith, D.R. Hipp, and A.W. Biermann, "An Architecture for Voice Dialog Systems Based on Prolog-Style Theorem Proving," Computational Linguistics, vol. 21, no. 3, pages 281-320, September 1995.

R.W. Smith and D.R. Hipp, Spoken Natural Language Dialog Systems: A Practical Approach, Oxford University Press, 1994.

R.W. Smith, "Spoken Variable Initiative Dialog: An Adaptable Interface," IEEE Expert, vol. 9, no. 1, pages 45-50, February 1994.

Area Background

My study of computational modeling of natural language dialog has focused on issues concerning voice interfaces, real-time interaction, and integrated modeling of the following behaviors: (1) collaborative problem solving, (2) subdialog completion and movement, (3) contextual interpretation, (4) user-dependent response generation, and (5) mixed-initiative interaction.
This wholistic modeling of natural language dialog requires an awareness of work on various subproblems in dialog processing including quantification, presuppositions, ellipsis, anaphoric reference, user modeling, expectation modeling, plan recognition, and miscommunication handling.

An important methodlogy for validation of the computational model is system construction and formal experimentation. The goal of empirically validating the model necessitates an awareness of computational constraints and robust error handling techniques as well as familiarity with past experimental studies on discourse behavior (usually of the human-human or simulated
human-computer variety). Empirical study is beneficial in acquiring knowledge about how human linguistic behavior during interaction with a computer may differ from what would occur if the interaction was with another human.

Area References

S.W. McRoy and G. Hirst, "The Repair of Speech Act Misunderstandings by Abductive Inference," Computational Linguistics. vol. 21, no. 4, pages 435-478, 1995.

S.E. Brennan and E.A. Hulteen, "Interaction and feedback in a spoken language system: a theoretical framework," Knowledge-Based Systems. vol. 8, pages 143-151, 1995.

J.F. Allen, L.K. Schubert, G. Ferguson, P. Heeman, C.H. Hwang, T. Kato, M. Light, N. Martin, B. Miller, M. Poesio, and D.R. Traum, "The TRAINS Project: A Case Study in Building a Conversational Planning Agent." Journal of Experimental and Theoretical Artificial Intelligence. vol. 7, pages 7-48, January, 1995.

S. Carberry, Plan Recognition in Natural Language Dialogue, MIT
Press, Cambridge, Mass., 1990.

S.R. Young, A.G. Hauptmann, W.H. Ward, E.T. Smith, and P. Werner, "High Level Knowledge Sources in Usable Speech Recognition Systems," Communications of the ACM, vol. 32, no. 2, pages 183-194, February 1989.

R. Wilensky, D.N. Chin, M. Luria, J. Martin, J. Mayfield, and D. Wu, "The Berkeley UNIX Consultant Project", Computational Linguistics, vol. 14, no. 4, pages 35-84, 1988.

B.J. Grosz and C.L. Sidner, "Attentions, Intentions, and the Structure of Discourse", Computational Linguistics, vol. 12, no. 3, pages 175-204,1986.

Related Program Areas

Adaptive Human Interfaces
Usability and User-Centered Design
Intelligent Interactive Systems for Persons with Disabilities

Potential Related Projects

Port the dialog model to other domains such as advisory dialogs or to other environments such as telephone interactions. Achieve total integration of the model with a speech recognition system to enhance robustness and efficiency. Conduct large-scale user testing of the dialog model via a system that assists in completing a complex real-world task (i.e., a task requiring a vocabulary of 500 words or more).


Send mail to the webmaster with questions or comments about this web site.
Last modified: January 29, 1999