IbPRIA 2017 Tutorial: “Structured Representations for Text and Images”
Frank Keller (iV&L Net committee member – WG1 Deputy Leader) will give a tutorial at IbPRIA 2017 in Faro, Portugal.
To achieve natural language understanding, the analysis of the syntactic and semantic structure of text is required, and a long tradition of research in this area exists. In this tutorial, I will argue that image understanding (working out who does what to whom in an image) also requires structured representations. I will discuss recent developments in methods for discovering image structure, including visual dependency representations, scene graphs, visual word senses, and visual semantic roles. I will also talk about how linguistic structure and image structure can be aligned for multimodal applications, including image description and visual question answering.
School of Informatics, University of Edinburgh, Edinburgh, UK
Frank Keller is professor of computational cognitive science in the School of Informatics at the University of Edinburgh. His background includes an undergraduate degree from Stuttgart University, a PhD from Edinburgh, and postdoctoral and visiting positions at Saarland University and MIT. His research focuses on how people solve complex tasks such as understanding language or processing visual information. His work combines experimental techniques with computational modeling to investigate reading, sentence comprehension, translation, and language generation, both in isolation and in the context of visual information such as photographs or diagrams. Prof. Keller serves on the management committee of the European Network on Vision and Language, is a member of governing board of the European Association for Computational Linguistics, and recently completed an ERC starting grant in the area of language and vision.