WG1 – Integrated Modelling of Vision and Language
WG1 focuses on core theoretical approaches to joint vision and language processing by developing vision and language models that are integrated
to various degrees. Important tasks will be to consolidate existing knowledge, define annotation standards, and draft a roadmap. The complexity of the area (due to its currently fragmented make-up and its emerging nature), calls for an innovative approach tightly focused on scientific/technological progress.
WG1 Leader: Katerina Pastra; Deputies: Frank Keller, Sergio Escalera
WG2 – Applications of Integrated Models
WG2 focuses on specific applications. This WG embraces both intra-Action collaboration and the involvement of new participants; collaboration between academic and industrial partners, both on academic projects and on real-world product development. The activity will stimulate ideas for novel cross-modal applications. It will include initiatives for bi-lateral exchanges, particularly for individuals who are not yet directly involved with funded projects located in the iV&L Net area, in this way addressing the wider community. Among the methods applied are intra-Action workshops and extra-network workshops that establish contacts with European and nationally funded projects.
WG2 Leader: Lucia Specia; Deputy: Nuno Pombo.
WG3 – Automatic Generation of Image and Video Descriptions
WG3 focuses on methods for annotating, labeling and describing visual data, including integration of language technologies in annotating visual data using suitable weakly supervised machine learning models, inference models that take into account language and visual constraints, latent class models for coping with variant low level features, alignment models, models that detect complementarity of the vision and text content, and text generation methods.
WG3 Leader: Luc van Gool; Deputy: Erkut Erdem.
WG4 – Semantic Image and Video Search
WG4 applies insights from integrated vision and language processing to the important application of video and image search, and focuses on suitable retrieval models that reason with and fuse the results of uncertain recognitions.
WG4 Leader: Tinne Tuytelaars; Deputies: Francois Bremond, Luisa Coheur.
WG5 – Industry and End-User Liaison
WG5 is responsible for developing links with industry and end users. The Action invites larger and smaller companies who are stakeholders in the areas addressed by iV&L Net to join an Industrial Advisory Board which has the dual function of (i) advising the Action’s MC and informing its activities, and (ii) fostering collaboration between industrial and non-industrial participants, including industry placements for ESRs. WG5 will also coordinate the recruitment of End-user Advisory Groups, including e.g. one of blind and partially sighted internet users, and one of video analysis stakeholders such as police forces and CCTV companies. Furthermore, WG5 will coordinate user requirement surveys and other methods for obtaining end-user input.
WG5 Leader: Stavri Nikolov; Deputies: Emiel Krahmer, Alex Jaimes.