| Intelligent computer systems often communicate with users through multiple modalities. Based on communication goals, presentation tasks, user profiles, environmental conditions, and resource limitations, computer systems make a best use of available modalities to construct multimodal messages for their users. Automatic multimodal presentation in existing HCI systems are mostly done in a goal-driven, plan-based, and context-aware manner. However, as multimodal presentations are created for human users to perceive and act upon, computer systems should understand not only how to convey information, but also how human minds are going to take in and process the information. With the knowledge of human cognition, multimodal presentations could be done in a cognitive-aware manner. Cognitive-aware multimodal messages make proper use of the human cognitive resources, thus they can be more efficiently perceived and processed. I work on encoding modality-related human cognitive principles into the automatic generation of multimodal presentations. It could be particularly beneficial for high-load information presentation systems which challenge the full capacity of the human cognitive capacity. |
