PhD Position on Causal Multimodal Foundation Models

PhD Position on Causal Multimodal Foundation Models

Working at the UvA

Join our team!

Recent breakthroughs in Artificial Intelligence have led to the emergence of the first generation of foundation models capable of generalizing across tasks, domains, and modalities. These advances have opened up powerful new paradigms for solving complex, domain-specific problems through generalist models that can be efficiently fine-tuned for diverse applications. However, the promise of these models is limited by two key challenges: (i) their difficulty in robustly generalizing to new contexts and domains, and (ii) their limited capacity for reasoning and adapting over multimodal, spatio-temporal data streams.

This PhD project addresses these limitations by focusing on causal exploration, enabling agents to actively seek out informative interventions in order to learn the underlying structure of their environment. Rather than relying solely on passive data, the agent will experiment, hypothesize, and test causal relationships to construct more robust and transferable world models. This active, structured approach to learning is crucial for collecting fine-tuning data for flexible multimodal generalist foundation models (MGFMs) that can generalize to novel tasks and adapt to previously unseen settings autonomously through exploration.

Working at the UvA

Join our team!

Recent breakthroughs in Artificial Intelligence have led to the emergence of the first generation of foundation models capable of generalizing across tasks, domains, and modalities. These advances have opened up powerful new paradigms for solving complex, domain-specific problems through generalist models that can be efficiently fine-tuned for diverse applications. However, the promise of these models is limited by two key challenges: (i) their difficulty in robustly generalizing to new contexts and domains, and (ii) their limited capacity for reasoning and adapting over multimodal, spatio-temporal data streams.

This PhD project addresses these limitations by focusing on causal exploration, enabling agents to actively seek out informative interventions in order to learn the underlying structure of their environment. Rather than relying solely on passive data, the agent will experiment, hypothesize, and test causal relationships to construct more robust and transferable world models. This active, structured approach to learning is crucial for collecting fine-tuning data for flexible multimodal generalist foundation models (MGFMs) that can generalize to novel tasks and adapt to previously unseen settings autonomously through exploration.

All about this vacancy

What are you going to do?

You will conduct cutting-edge research at the intersection of deep reinforcement learning, causal representation learning, and multimodal foundation models. The aim is to develop “artificial scientist” agents capable of formulating and testing causal hypotheses through interaction, going beyond passive observation to active, grounded learning.

You will begin by designing reinforcement learning agents that explore complex environments starting from an incomplete or noisy causal graph extracted from a pretrained foundation model. These agents will need to reason, experiment, and adapt their behavior and world model by refining their causal understanding. As the project progresses, the insights and techniques developed will be used to inform new methods for fine-tuning multimodal generalist foundation models (MGFMs) to be causality consistent. Your work will contribute to the European Horizon ELLIOT project, which focuses on embedding spatial, temporal, object-level, and causal awareness into MGFMs.

This research is embedded in the Video & Image Sense lab at the University of Amsterdam, and you will be part of an interdisciplinary team contributing to the ELLIOT consortium, which includes 32 academic and industrial partners. ELLIOT aims to deliver open, reproducible foundation models and tools that benefit the wider European AI community.

Tasks and responsibilities:

  • Develop novel embodied learning agents capable of causal exploration and hypothesis testing in complex environments.
  • Contribute to the fine-tuning of multimodal foundation models for grounded, embodied tasks using causal structure and exploratory behavior.
  • Actively collaborate within the ELLIOT project and contribute to its use cases and shared objectives.
  • Present research progress in internal meetings and participate in knowledge exchange within the consortium.
  • Publish results at top-tier conferences and in international journals.
  • Contribute to teaching activities, including supervision of MSc/BSc students and assisting in labs.
  • Complete and defend a PhD thesis within the appointed duration of four years.

Your profile

  • An MSc degree in Artificial Intelligence, (Applied) Mathematics, Computer Science, or related field.
  • A strong background/knowledge in reinforcement learning, causal learning or multimodal modals is a big plus.
  • Excellent programming skills (Python and preferably PyTorch / Jax).
  • Experience with AI/HPC supercomputing and running software on scale.
  • You are highly motivated, independent, and creative.
  • Strong communication, presentation and writing skills and excellent command of English.
  • Prior publications in relevant machine learning conferences or journals are advantageous.

Our offer

 A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). The preferred starting date is September 1st 2025. This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 2,901 to € 3,707 (scale P). This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile PhD Candidate is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Besides the salary and a vibrant and challenging environment at Science Park we offer you multiple fringe benefits:

  • 232 holiday hours per year (based on fulltime) and extra holidays between Christmas and 1 January;
  • multiple courses to follow from our Teaching and Learning Centre;
  • a complete educational program for PhD students;
  • multiple courses on topics such as leadership for academic staff;
  • multiple courses on topics such as time management, handling stress and an online learning platform with 100+ different courses;
  • 7 weeks birth leave (partner leave) with 100% salary;
  • partly paid parental leave;
  • the possibility to set up a workplace at home;
  • a pension at ABP for which UvA pays two third part of the contribution;
  • the possibility to follow courses to learn Dutch;
  • help with housing for a studio or small apartment when you’re moving from abroad.

Are you curious to read more about our extensive package of secondary employment benefits, take a look here.

Where you will work

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute (IvI) is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.

The position is with dr. Efstratios Gavves at the University of Amsterdam within VIS lab, co-led by dr. Andrii Zadaianchuk and dr. Christian Gumbsch. VIS lab is a world-leading lab on Computer Vision and Machine Learning, and has over 30 PhD students, postdoctoral researchers and faculty members working on a broad variety of deep learning, computer vision, and foundation model subjects, like self-supervised learning, diffusion models, and test-time generalization for perception tasks like object detection, instance segmentation and activity recognition. The position is also embedded in the European ELLIS Network of Excellence in AI.

All about this vacancy

What are you going to do?

You will conduct cutting-edge research at the intersection of deep reinforcement learning, causal representation learning, and multimodal foundation models. The aim is to develop “artificial scientist” agents capable of formulating and testing causal hypotheses through interaction, going beyond passive observation to active, grounded learning.

You will begin by designing reinforcement learning agents that explore complex environments starting from an incomplete or noisy causal graph extracted from a pretrained foundation model. These agents will need to reason, experiment, and adapt their behavior and world model by refining their causal understanding. As the project progresses, the insights and techniques developed will be used to inform new methods for fine-tuning multimodal generalist foundation models (MGFMs) to be causality consistent. Your work will contribute to the European Horizon ELLIOT project, which focuses on embedding spatial, temporal, object-level, and causal awareness into MGFMs.

This research is embedded in the Video & Image Sense lab at the University of Amsterdam, and you will be part of an interdisciplinary team contributing to the ELLIOT consortium, which includes 32 academic and industrial partners. ELLIOT aims to deliver open, reproducible foundation models and tools that benefit the wider European AI community.

Tasks and responsibilities:

  • Develop novel embodied learning agents capable of causal exploration and hypothesis testing in complex environments.
  • Contribute to the fine-tuning of multimodal foundation models for grounded, embodied tasks using causal structure and exploratory behavior.
  • Actively collaborate within the ELLIOT project and contribute to its use cases and shared objectives.
  • Present research progress in internal meetings and participate in knowledge exchange within the consortium.
  • Publish results at top-tier conferences and in international journals.
  • Contribute to teaching activities, including supervision of MSc/BSc students and assisting in labs.
  • Complete and defend a PhD thesis within the appointed duration of four years.

Your profile

  • An MSc degree in Artificial Intelligence, (Applied) Mathematics, Computer Science, or related field.
  • A strong background/knowledge in reinforcement learning, causal learning or multimodal modals is a big plus.
  • Excellent programming skills (Python and preferably PyTorch / Jax).
  • Experience with AI/HPC supercomputing and running software on scale.
  • You are highly motivated, independent, and creative.
  • Strong communication, presentation and writing skills and excellent command of English.
  • Prior publications in relevant machine learning conferences or journals are advantageous.

Our offer

 A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). The preferred starting date is September 1st 2025. This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 2,901 to € 3,707 (scale P). This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile PhD Candidate is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Besides the salary and a vibrant and challenging environment at Science Park we offer you multiple fringe benefits:

  • 232 holiday hours per year (based on fulltime) and extra holidays between Christmas and 1 January;
  • multiple courses to follow from our Teaching and Learning Centre;
  • a complete educational program for PhD students;
  • multiple courses on topics such as leadership for academic staff;
  • multiple courses on topics such as time management, handling stress and an online learning platform with 100+ different courses;
  • 7 weeks birth leave (partner leave) with 100% salary;
  • partly paid parental leave;
  • the possibility to set up a workplace at home;
  • a pension at ABP for which UvA pays two third part of the contribution;
  • the possibility to follow courses to learn Dutch;
  • help with housing for a studio or small apartment when you’re moving from abroad.

Are you curious to read more about our extensive package of secondary employment benefits, take a look here.

Where you will work

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute (IvI) is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.

The position is with dr. Efstratios Gavves at the University of Amsterdam within VIS lab, co-led by dr. Andrii Zadaianchuk and dr. Christian Gumbsch. VIS lab is a world-leading lab on Computer Vision and Machine Learning, and has over 30 PhD students, postdoctoral researchers and faculty members working on a broad variety of deep learning, computer vision, and foundation model subjects, like self-supervised learning, diffusion models, and test-time generalization for perception tasks like object detection, instance segmentation and activity recognition. The position is also embedded in the European ELLIS Network of Excellence in AI.

Your place at the UvA

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Your place at the UvA

This is where you will be working

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the button below. We accept applications until and including 30 July 2025. Applications should include the following information (all files besides your CV should be submitted in one single pdf file):

  • A letter that motivates your choice for this position (max 1 page); 
  • Curriculum vitae, including your list of publications if applicable (max 2 pages); 
  • A research statement on how to approach the PhD project. Solid and creative ideas will be greatly appreciated. (max 2 pages). 
  • A link to your Master thesis – if online available, else include an abstract. 
  • A complete record of Bachelor and Master courses (including grades and explanation of grading system); 
  • A list of projects or publications you have worked on, with brief descriptions of your contributions, max 1 page; 
  • The names and contact addresses of at least two academic references (please do not include any recommendation letters). 

A knowledge security check can be part of the selection procedure (for details: national knowledge security guidelines).

Please use the CV field to upload your resume as a separate pdf document. Use the Cover Letter field to upload the other requested documents, including the motivation letter, as one single pdf file. Only complete applications received within the response period via the link below will be considered. The interviews for this position will be held in July and August. Do you have any questions or do you require additional information? Please contact: dr. Efstratios Gavves – [email protected]

Diversity, Equity & Inclusion

As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the button below. We accept applications until and including 30 July 2025. Applications should include the following information (all files besides your CV should be submitted in one single pdf file):

  • A letter that motivates your choice for this position (max 1 page); 
  • Curriculum vitae, including your list of publications if applicable (max 2 pages); 
  • A research statement on how to approach the PhD project. Solid and creative ideas will be greatly appreciated. (max 2 pages). 
  • A link to your Master thesis – if online available, else include an abstract. 
  • A complete record of Bachelor and Master courses (including grades and explanation of grading system); 
  • A list of projects or publications you have worked on, with brief descriptions of your contributions, max 1 page; 
  • The names and contact addresses of at least two academic references (please do not include any recommendation letters). 

A knowledge security check can be part of the selection procedure (for details: national knowledge security guidelines).

Please use the CV field to upload your resume as a separate pdf document. Use the Cover Letter field to upload the other requested documents, including the motivation letter, as one single pdf file. Only complete applications received within the response period via the link below will be considered. The interviews for this position will be held in July and August. Do you have any questions or do you require additional information? Please contact: dr. Efstratios Gavves – [email protected]

As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.