Presentation of the dissertation project and the research results to date:

Goal of the Project:

In my dissertation project I aim to investigate how people solve complex decision-making problems, in which choice uncertainty can be reduced by inferences from multi-modal sources, and how learning in such situations affects multi-modal attention in the human brain.

Theoretical Background:

In complex everyday decision-making problems, such as buying stocks, predicting the weather, or figuring out the shortest way to a certain destination, humans have to deal with great uncertainty concerning the correctness of their decisions and predictions. Basically, the environment provides a decision-maker with multi-modal information about the underlying problem that can be considered to reduce the amount of uncertainty. When buying stocks, for instance, experts in the field of business make suggestions about the most likely development of stock prices and this information can be used to increase the chance of purchasing good and rejecting poor stocks. However, such hints and advice introduce another form of uncertainty, as their validity (in predicting the stocks’ development) remains questionable.

It is well known that humans possess a repertoire of cognitive strategies to solve complex decision-making problems such as those described above (Payne et al., 1988). For instance, given a situation where multiple sources of information with different validities can be considered for making inferences, the most strenuous cognitive strategy is to integrate all information and weight each source by its validity. An easier strategy could be to integrate all available information without a differential weighting of the sources. This strategy should be of particular usefulness if the decision-maker is oblivious to the sources’ validities. A very strongly simplifying strategy is to consider only a single source of information.

Having this repertoire of strategies on hand, humans must decide which strategy is most appropriate in given situations. The Strategy Selection Learning Theory (SSL; Rieskamp & Otto, 2006) offers an explanation of how people use feedback from similar decision-making problems in the past to adapt their strategy selection in the presence and future. Basically, SSL resembles a traditional reinforcement-learning model: An agent chooses the cognitive strategy that he/she considers the most promising in a given environment. Having applied the strategy and having decided for an option (e.g., to buy a certain stock), the success/failure of this decision is used to update the strategy’s expectations.

Initial Questions and Hypotheses:

The first aim of the present dissertation project is to test the predictions of SSL behaviourally and to investigate the neural correlates of adaptive strategy selection using functional magnetic resonance imaging (fMRI). I use a multi-modal approach to further investigate implications of strategy selection on attention. During the experiment, subjects are asked to either buy or reject fictitious stocks to gain money. Information about the stocks is provided as ratings of (again fictitious) independent rating companies, which can be used to improve decision accuracy. Subjects face two different environments in which different cognitive strategies are appropriate: A single-cue strategy, in which an auditory cue is to be considered exclusively, and a multiple-cue strategy, in which visual cues should be taken into account as well.

Behaviourally, subjects should learn to choose the strategy that maximizes the decision accuracy in each environment. In parallel, several hypotheses concerning the neural representations can be tested: First, as the different strategies require different attentional foci (the single-cue strategy requires attention to the auditory domain, the multiple-cue strategy requires a distribution of attention to the auditory and the visual domain), cross-modal attentional modulation effects in the visual and auditory sensory cortices should change as a function of environment and learning. Second, neural representations of reward expectations in prefrontal areas (e.g., ventromedial prefrontal cortex) should be a function of cue information (i.e., the companies’ ratings) and of the applied strategy. Thus, the same stock could generate different expectations depending on the subject’s belief of the appropriate strategy. Similarly, expectation violations (i.e., prediction errors) – reflected as activity in the ventral striatum – should be a function of the deviation of the actual outcome from what was predicted by the applied strategy. Here, the use of model-based fMRI (O’Doherty et al., 2007) will serve as a precise tool to estimate the predictions of SSL at the neuronal level. Finally, an fMRI-based classification approach could support inferences about the likelihood of the application of a strategy in situations in which the behavioural response does not distinguish between the strategies (since the different strategies do not always suggest different decisions).

Fortunately, Prof. Christian Büchel and I collaborate directly with Prof. Jörg Rieskamp from the University of Basel, who initially proposed SSL in 2006 (Rieskamp & Otto, 2006).

Current State of the Project and Results:

So far, I conducted two behavioural experiments to test the predictions of SSL. In general, subjects indeed learned to choose the appropriate strategy in both environments as predicted by SSL. In the first experiment, I found a strong order effect in that subjects adapted faster to a multiple-cue environment after having faced a single-cue environment than vice versa. Presumably, this was due to the greater uncertainty of whether the multiple-cue strategy (compared to the single-cue strategy) was applied correctly or not. Accordingly, the multiple-cue strategy was simplified in the second experiment and the order effect was thus extinguished. Therefore, we decided to use the setting of the second experiment for the fMRI experiment. This fMRI experiment is currently being conducted.

Proceeding Steps of the Dissertation and Potential Collaborations:

Besides testing participants and acquiring fMRI data, I am currently working on the analysis, as well. Preliminary results of 16 participants look indeed very promising: In line with my hypotheses, the fMRI-BOLD response in the ventromedial prefrontal cortex correlates with the (strategy-dependent) expected value and response in the ventral striatum correlates with the (strategy-dependent) prediction error. Furthermore, I find a prediction-error-dependent re-activation in the primary visual and auditory cortices when subjects receive their rewards. This is very fascinating given that at this point of the task, there is no auditory stimulation at all. However, I still have to await the complete data set (of 24 participants) to make definite conclusions.

My dissertation project can be extended in several directions. It could be interesting to investigate how subjects integrate information over time when multiple cues are not provided simultaneously but sequentially. Basically, such a decision-making problem should resemble a diffusion-to-boundary process as often proposed for perceptual decision-making problems: Evidence in favour or against a certain option is accumulated over time until a threshold is reached and the decision is made. In light of its higher temporal resolution, the use of the event-related potential technique might come into play here. The group of Prof. Andreas Engel at the UKE would be a very adequate collaboration partner as similar research (in the perceptual domain) is conducted here (e.g., Donner et al., in press). Another extension would be to investigate the switch between cognitive strategies more in detail by changing the strategies’ appropriateness more frequently. In the past, Bayesian approaches to such reversal learning processes have proven to be particularly successful in predicting human behaviour (e.g., Hampton et al., 2006). Therefore, collaboration with the group of Prof. Wolfgang Menzel at the Department of Informatics could be fruitful as Bayesian learning methods (in linguistics) are used here as well. Of course, the expertise of Sabrina Boll and Andreas Marschner –from our research team at the UKE (Prof. Büchel) – with respect to the methods of (high-resolution) fMRI will help me to improve the accuracy of my own data analysis. A long-term goal of collaboration with other CINACS-students could be to implement the complex learning mechanisms that I study in humans in the field of robotics; potential partners for this co-operation are Dominik Off from the Department of Informatics as well as several students from Tsinghua University.

With respect to further potential collaborations with Tsinghua University in Beijing, I believe that my project will benefit from such a co-operation in the future. To give an example: The doctoral students and research groups at Tsinghua University are extremely experienced in the computational domain, especially with respect to classification methods such as support vector machines. Since one of my goals is to apply classification methods (and computational methods in general), support from Beijing should help me to improve the respective analyses. A particularly close collaboration could be established with the group of Prof. Guosong Liu, as their research on cellular mechanisms of learning and memory is of direct relevance for my own project. Having a psychological rather than a biological or chemical educational background, I am eager to learn more about and work on processes on the cellular level that underlie animals' and humans' mental capabilities.

References:

T. H. Donner, M. Siegel, P. Fries and A. K. Engel. Buildup of choice-predictive activity in human motor cortex during perceptual decision making. Current Biology

A. Hampton, P. Bossaerts and J. P. O’Doherty. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision-making in humans. Journal of Neuroscience, 26: 8360–8367, 2006.

J. P. O’Doherty, A. Hampton and H. Kim. Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104: 35–53, 2007.

J. W. Payne, J. R. Bettman and E. J. Johnson. Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, & Cognition, 14: 534-552, 1988.

J. Rieskamp and P. E. Otto. SSL: A theory of how people learn to select strategies. Journal of Experimental Psychology: General, 135: 207–236, 2006.