Multimodal Machine Learning Lab

Description

Multimodal machine learning concerns the research and development of learning methods that can process data provided in different types (i.e., modalities) such as images, videos, text, and audio. Key challenges include:
  • learning vector representations shared across modalities (e.g., a text describing an image and the image itself should have similar representations),
  • inferring new knowledge from multimodal information (multimodal inference),
  • generating new texts, images, etc.,
  • transferring knowledge between modalities, and
  • the empirical analysis of multimodal models.
The project examines current research methods for problems related to these challenges.

General Information

Lecturer Prof. Dr. Martin Potthast
Teaching Assistants Niklas Deckers, Lukas Gienapp
Workload 4 SWS
Lab Wednesday, 10:00 - 14:00, Hörsaal 0315
Contact via email or office hours