Meriam Horchani
Laurence Nigay
ABSTRACT
In the context of natural multimodal dialogue systems, we address the challenging issue of the definition of cooperative answers in an appropriate multimodal form. Highlighting the intertwined relation of multimodal outputs with the content, we focus on the Dialogic strategy component, a component that defines from the set of possible contents to answer a user's request, the content to be presented to the user and its multimodal presentation. The content selection and the presentation allocation managed by the Dialogic strategy component are based on various constraints such as the availability of a modality and the user's preferences. Considering three generic types of dialogue strategies and their corresponding handled types of information as well as three generic types of presentation tasks, we present a first implementation of the Dialogic strategy component based on rules. By providing a graphical interface to configure the component by editing the rules, we show how the component can be easily modified by ergonomists at design time for exploring different solutions. In further work we envision letting the user modify the component at runtime.
- Berton, A., Bühler, D. and Minker, W. SmartKom-mobile car: User interaction with mobile services in a car environment. In Wahlster, W. (ed.), SmartKom: foundations of multimodal dialogue systems, 523--537, Springer, 2006.Google Scholar
- Bordegoni, M., Faconti, G., Feiner, S., Maybury, M. T., Rist, T., Ruggieri, S., Trahanias, P. and Wilson, M. A standard reference model for intelligent multimedia presentation systems. Computers standards and interfaces, 18, 6-7, 477--496, Elsevier, 1997. Google ScholarDigital Library
- Bunt, H., Kipp, M., Maybury, M. and Wahlster, W. Fusion and coordination for multimodal interactive information presentation. In Stock, O. and Zancanaro, M. (ed.) Multimodal intelligent information presentation, 325--339, Springer, 2005.Google Scholar
- Elting, C., Rapp, S., Möhler, G. and Strube M. Architecture and implementation of multimodal plug and play. In Proc. ICMI'03, 93--100, ACM Press, 2003. Google ScholarDigital Library
- Fréard, D., Jamet, E., Le Bohec, O., Poulain, G. and Botherel, V. Subjective measurement of workload related to a multimodal interaction task: Nasa-tlx vs. Workload profile. In Proc. HCII'07, Springer, 2007. Google ScholarDigital Library
- Horchani, M., Nigay, L. and Panaget, F. A platform for output dialogic strategies in natural multimodal dialogue systems. In Proc. IUI'07, 206--215, ACM Press, 2007. Google ScholarDigital Library
- Horchani, M., Fréard, D., Jamet, E., Nigay, L. and Panaget, F. A platform for designing multimodal dialogic and presentation strategies. In Proc. of the 2007 Workshop on the Semantics and Pragmatics of Dialogue DECALOG, 169--170, 2007.Google Scholar
- JADE, Java Agent DEvelopment Framework, jade.tilab.com.Google Scholar
- Louis, V. and Martinez T. JADE semantics framework. In Bellifemine, F., Caire, G. and Greenwood, D. (ed.), Developping multi-agent systems with JADE, John Wiley & Sons, 2007.Google Scholar
- Mansoux, B., Nigay, L., and Troccaz, J. Output Multimodal Interaction: The Case of Augmented Surgery. In Proc. of the 20th British Human Computer Interaction Group conference HCI'06, 177--192, BCS Conference Series, Springer-Verlag and ACM Press, 2006.Google Scholar
- Roth, F. and Hefley, W. E. Intelligent multimedia presentation systems: research and principles, In Maybury, M. (ed.), Intelligent multimedia interfaces, AAAI Press/The MIT Press, 1993. Google ScholarDigital Library
- Rousseau, C., Bellik, Y., Vernier, F. and Bazalgette, D. A framework for the intelligent multimodal presentation information, Signal Processing Journal, Special issue on Multimodal Interfaces, 86, 12, 3696--3713, Elsevier, 2006. Google ScholarDigital Library
- Sadek, D. Design considerations on dialogue systems: from theory to technology. In IDS'99, 173--187, 1999.Google Scholar
- Sinha, A. and Landay, J. Capturing user tests in a multimodal, multidevice informal prototing tool. In Proc. ICMI'03, 117--124, ACM Press, 2003. Google ScholarDigital Library
- UIMS. A metamodel for the runtime architecture of an interactive system. SIGCHI bulletin, 24, 32--37, 1992. Google ScholarDigital Library
- Wahlster, W. Dialogue systems go multimodal: the SmartKom experience. In Wahlster, W. (ed.), SmartKom: foundations of multimodal dialogue systems, 3--27, Springer, 2006.Google Scholar
- Zhou, M., Wen, Z., and Aggarwal, V. A graph-matching approach to dynamic media allocation in intelligent multimedia interfaces. In Proc. IUI'05, 114--121, ACM Press, 2005. Google ScholarDigital Library
Index Terms
- Natural multimodal dialogue systems: a configurable dialogue and presentation strategies component
Recommendations
Adaptations of multimodal content in dialog systems targeting heterogeneous devices
ICMI '04: Proceedings of the 6th international conference on Multimodal interfacesDialog systems that adapt to different user needs and preferences appropriately have been shown to achieve higher levels of user satisfaction [4]. However, it is also important that dialog systems be able to adapt to the user's computing environment, ...
From vocal to multimodal dialogue management
ICMI '06: Proceedings of the 8th international conference on Multimodal interfacesMultimodal, speech-enabled systems pose different research problems when compared to unimodal, voice-only dialogue systems. One of the important issues is the question of how a multimodal interface should look like in order to make the multimodal ...
A platform for output dialogic strategies in natural multimodal dialogue systems
IUI '07: Proceedings of the 12th international conference on Intelligent user interfacesThe development of natural multimodal dialogue systems remains a very difficult task. The flexibility and naturalness they offer result in an increased complexity that current software tools do not address appropriately. One challenging issue we address ...
Comments