Member Login

JCal Pro Mini-calendar

September 2020 October 2020 November 2020
Mo Tu We Th Fr Sa Su
Week 40 1 2 3 4
Week 41 5 6 7 8 9 10 11
Week 42 12 13 14 15 16 17 18
Week 43 19 20 21 22 23 24 25
Week 44 26 27 28 29 30 31

Current Time

The time is 03:07:09.
ALUMNI Dr. Cengiz Acarturk Print E-mail

(Click here for a more recent webpage)

Cengiz Acartürk completed his Ph.D. study in February, 2010. The project has been conducted within the framework of the research project Interaction of Representational Modalities in Communication, supervised by Prof. Dr. Christopher Habel, in Knowledge and Language Processing Group (WSV), Department of Informatics, University of Hamburg. Cengiz was a member of CINACS International Research Training Group between July, 2006 and July, 2009.

The dissertation can be downloaded as a pdf file from this link.

The title and the abstract of the dissertation, as well as a list of published and presented studies are presented below.

Multimodal Comprehension of Graph-Text Constellations: An Information Processing Perspective


Documents containing information in different representational modalities (e.g., text, pictorial and diagrammatic illustrations, graphs) are widely used in personal and professional settings of daily life, both in printed media and in human-computer interaction environments, such as WWW. In particular, graph-text constellations are one of those cognitive means by which we accomplish high-level problem solving and communication tasks. This dissertation performs a systematic, scientific investigation of multimodal comprehension of graph-text constellations by humans—as natural cognitive systems—for providing empirical basis and scientific models for the design and development of artificial cognitive systems that have the capacity to support users during their design and production of graph-text constellations. First, an overview of the existing approaches in graph comprehension and language comprehension is presented. Second, a theoretical framework is developed by extending the reach of the existing approaches to embody comprehension of graphical and linguistic entities under the framework of the common conceptual basis for multimodal comprehension. Third, various aspects of the interaction between language comprehension and graph comprehension are empirically investigated by conducting experiments with human participants. The empirical investigations reveal how users integrate the information contributed by the representations in different modalities, how the changes in one modality lead to substantial influences in comprehension of the entities in the other modality, and how inappropriate design of graphs and graph-text combinations give rise to difficulties during the course of multimodal comprehension. Based on the analysis of those difficulties in comprehension from the perspective of HCI, this dissertation lays the foundations for appropriate design of graph-text constellations and for the development of software tools that would provide users with support for appropriate design during their course of designing and producing graph-text constellations.


Journal articles

Acarturk, C., Habel, C., Cagiltay, K., & Alacam, O. (2008). Multimodal comprehension of language and graphics: Graphs with and without annotations. Journal of Eye Movement Research, 1(3):2, 1-15. [URL]

Conference and workshop proceedings

Habel, C., & Acarturk, C. (2009). Eye-tracking evidence for multimodal language-graphics comprehension: The role of integrated conceptual representations. In C. Navarretta, P. Paggio, J. Allwood, E. Alsén & Y. Katagiri (Eds.), Proceedings of the NODALIDA 2009 Workshop. Multimodal Communication - from Human Behaviour to Computational Models (Vol. 6, pp. 9-14). Odense, Denmark: Northern European Association for Language Technology (NEALT). [URL]

Acarturk, C., Habel, C., & Cagiltay, K. (2008). Multimodal comprehension of graphics with textual annotations: The role of graphical means relating annotations and graph lines. In J. Howse, J. Lee & G. Stapleton (Eds.), Diagrammatic Representation and Inference. Lecture Notes in Computer Science 5223 (pp. 335-343): Springer Berlin / Heidelberg. [URL]

Acarturk, C., Habel, C., Cagiltay, K., & Alacam, O. (2008). Learning from text and graphs: The role of annotations and sensory modality. In A. Maes & S. Ainsworth (Eds.), Proceedings EARLI Special Interest Group Text and Graphics. Exploiting the Opportunities Learning with Textual, Graphical and Multimodal Representations (pp. 13-16). [URL]

Habel, C. and Acarturk, C. (2007). On reciprocal improvement in multimodal generation: Co-reference by text and information graphics. MOG 2007, Proceedings of the Workshop on Multimodal Output Generation. I. van der Sluis, M. Theune, E. Reiter, E. Krahmer (Eds.) University of Aberdeen, United Kingdom, 25-26 January 2007. [URL]

Talks and posters

July 28-29, 2009. (Collaborated study with Prof. Dr. Christopher Habel). Attributing causality in multimodal language-graphics comprehension. Poster presented at ESLP 2009, Embodied and Situated Language Processing, July 28-29, 2009. Rotterdam, Netherlands.

October 1, 2008. (Collaborated study with Prof. Dr. Christopher Habel). Causal Attribution induced by Text-Graphics Documents: The Role of Annotations. Presentation in KogWis 2008, September 28 - October 1, 2008. Dresden, Germany.

September 28, 2008. Multimodal comprehension of information graphics with linguistic annotations. Presentation in Doctoral Colloquium, KogWis 2008, September 28 - October 1, 2008. Dresden, Germany.

September 5, 2007. Multimodal Comprehension of Information Graphics and Text. Presentation in CINACS Summer School in Beijing, China.

August 19-23, 2007. Acarturk, C., Habel, C., Cagiltay, K. & Alacam, O. (2007). Multimodal Comprehension of Language and Graphics. Poster presented at ECEM 2007, 14th European Conference on Eye Movements, 19-23 2007, Potsdam, Germany.

April 19, 2007. Eye-tracking and Testing Learning Materials. Presentation in Tobii User Meeting. Frankfurt, Germany.

March 18, 2007. Cross-modal Interaction in Graph Comprehension and Language. Presentation in Doktoranden Workshop of KogWis2007, 8th Conference of the Society for Cognitive Science. Saarbrücken, Germany.

Contact Information

Cengiz Acarturk
acarturk (use the at sign here) gmail dot com
Last Updated ( Monday, 28 July 2014 )