CINACS International Graduate Research Group comprises researchers from different disciplines. The participating groups at the University of Hamburg are Systems Neuroscience, Neurophysiology and Pathophysiology, Biological Psychology and Neuropsychology, Natural Language Systems, Knowledge and Language Processing, and Technical Aspects of Multimodal Systems. Researchers at the Tsinghua University are associated with Computer Science, Biomedical Engineering and Cognitive Science. The common research topics underlying the backbone of interdisciplinary collaboration are different aspects of cross modal interaction, such as the interaction between sensory modalities, interaction between representational modalities, multimodal interaction in robotics, as well as recently flourishing disciplines such as BCI (Brain Computer Interaction).

Research in natural systems and modeling

The brain extracts information from the environment via the sensory systems and converts this information into perceptions, memories, and actions at more complex levels. Substantial knowledge is available on the nature of the initial processes within individual sensory channels, such as vision, auditory perception and touch, as well as the further processes of comprehension and memory. As well known, events and objects in the outside world are often jointly detected by two or more sensory systems, and the perceptual and behavioural consequences of multisensory stimulation are often not just the simple sum of percepts and actions evoked by either sensory component alone.Numerous multisensory behavioural and perceptual effects have been described, ranging from detection and orientation to cognitive development, the use of tools, memory and language processing. At the neural level, responses to multisensory stimuli result in the parallel activation of vast neural systems comprising numerous different cortical and subcortical structures.

The role of a dynamic "binding" of activity in different neural structures brought about by changes in the coherence of neural activity within sensory areas has been shown to be important in uni-sensory feature binding. Nevertheless these findings have not yet been investigated in the context of multisensory perception. The projects that focus on neurobiological aspects within CINACS aim at studying multisensory functions with respect to their cognitive/behavioural consequences, as well as with respect to their underlying common mechanisms. The overall goal is to derive algorithms and principles that allow for potential implementation and improvement of multisensory fusion in technical systems and architectures.

Research on artificial systems and engineering

The investigation of modality-specific and multisensory processing capabilities of the brain for the purpose of controlling or communicating with artificial systems has been receiving increased interest. The development of multisensory artificial systems is a recent advancement in robotics and computer science. Several research topics have recently emerged in this context. First, it is proposed that robots that are capable of processing different sensory energies (e.g., inputs from different "sensory systems") may be superior to those using only one sensory energy. Second, speech recognition capacities of artificial systems may be superior if they are able to integrate speech signals contributed by different modalities. This requires that the transformation steps that allow accessing common semantic representations (memory) be identified. Third, specifically from the perspective of Human-Computer Interaction and multimodal communicaton, designing complex artificial systems requires the human-system interface to be optimally tailored to the human processing capacities. For example, sensory substitution devices must fit to the processing capacities of the user both to provide as much complementary information as possible to that of the intact modalities and to allow an easy integration of this information with information gained by the intact senses. Finally, from the perspective of Brain-Computer Interaction (BCI) the use of modality-specific and multisensory processing capabilities of the brain to control or to communicate with artificial systems has recently found increased interest.