Czech-French AI

Cordelia Schmid

INRIA, ELLIS Fellow INRIA Research Director, Head of the THOTH project-team

Bio & Research Focus

Cordelia Schmid holds a M.S. degree in Computer Science from the University of Karlsruhe and a Doctorate in Computer Science, from the Institut National Polytechnique de Grenoble (INPG). Her doctoral thesis received the best thesis award from INPG in 1996. She received the Habilitation degree in 2001 for her thesis. She was a post-doctoral research assistant in the Robotics Research Group of Oxford University from 1996-97. Since 1997 she has held a permanent research director position at Inria. She is a member of the German National Academy of Sciences, Leopoldina and a fellow of IEEE and the ELLIS society. She was awarded the Longuet-Higgins prize in 2006, 2014 and 2016 and the Koenderink prize in 2018 for fundamental contributions to computer vision that have withstood the test of time. She received an ERC advanced grant in 2013, the Humbolt research award in 2015, the Inria & French Academy of Science Grand Prix in 2016, the Royal Society Milner award in 2020 and the PAMI distinguished researcher award in 2021. She has been an Associate Editor for IEEE PAMI (2001-05) and for IJCV (2004-12), editor-in-chief for IJCV (2013-18), a program chair of IEEE CVPR 2005 and ECCV 2012 as well as a general chair of IEEE CVPR 2015, ECCV 2020 and ICCV 2023. Starting 2018 she holds a joint appointment with Google research.

Title & Abstract

Do you see what I see? Large-scale learning from multimodal videos

This talk presents recent progress on large-scale learning of multimodal video representations. We start by showing VideoBert, a joint model for video and language, repurposing the Bert model for multimodal data. This model achieves state-of-the-art results on zero-shot prediction and video captioning. Next, we present an approach for video question answering which relies on training from instruction videos and cross-modal supervision with a textual question-answer module. We show state-of-the-art results for video question answering without supervision (zero-shot VQA) and demonstrate that our approach obtains competitive results for pre-training and then fine-tuning on video question answering datasets. We conclude our talk by presenting the recent VideoCC dataset, which transfers image captions to video and allows obtaining state-of-the-art performance for zero-shot video and audio retrieval and video captioning.