Duration: 2016--2020

Abstract: Natural human-computer interaction (HCI) is the future development trend of computer technology, as interactive technology has become the core competitiveness of terminals and application innovations. Since the traditional HCI theory was mainly developed for desktop computing, it is difficult to support natural interaction with diversified channels and terminal-cloud fusion. Thus It is urgent to establish a theoretical framework with clear optimization goals for natural interaction. In this project, we intend to address the following three key scientific problems: 1) the physiological and psychological mechanisms and the computing model of natural interaction; 2) the channel characteristics and interaction semantics of natural interaction modalities; 3) the principle of interaction path optimization based on context. This project consists of three sub-tasks: 1) the computing principles of the natural interaction. Researching people’s information processing ability of perception, cognition, and motion in interaction, through behavioral experiments, task analysis, and computing model; researching the generation, transformation, and regulation mechanism of interaction experience through conception grid analysis, physiological and psychological measurement; establishing user information processing model in an open environment through data mining and modeling technology; 2) Channel characteristics and interaction semantics of interfaces. Researching information fusion mechanisms (such as effective substitution, complementation, and interference) of multiple channel exchange information based on interaction semantics; 3) interactive context-awareness based on terminal-cloud fusion. Using distributed heterogeneous multi-sensing devices and scalable cloud computing resources, establishing multi-modality perceived data collection and fusion method, and context reasoning technology which combines rule-based reasoning and machine learning, achieving accurate recognition of multi-dimensional context. This project aims to make the followings achievements: 1) user information processing model for natural interaction; 2) discovery of mental models; 3) evaluation approach of natural interaction based on physiological data; 4) natural input channel and interaction semantics; 5) natural human-computer interaction paradigm of audiovisual tactile fusion feedback; 6) the theory and technology of context-awareness based on terminal-cloud fusion; 7) computing framework for natural HCI with joint optimization of interaction efficiency and naturalness.