NL
EN

PhD Position on Multimodal Foundation Models

  • Faculty of Science
  • 14138
  • Master's
  • €2.901 - €3.707
  • 38 hours
  • Closes on
    30-06-2025
Apply

PhD Position on Multimodal Foundation Models

  • Faculty of Science
  • 14138
  • Master's
  • €2.901 - €3.707
  • 38 hours
  • Closes on
    30-06-2025
Apply

Working at the UvA

Join our team!

You will carry out research and development in the areas of multimodal foundation models, using advances in deep machine learning and computer vision. The goal is to invent, develop and evaluate novel generalist methods for pre-training and fine-tuning of multimodal foundation models, that align with human-values, expand their perceptual abilities, generalize at deployment, combine with any modality, and do so in an efficient and sustainable way. Importantly, their generalist nature should transfer to previously unknown and rather specialist multimodal use cases by balancing the generalist and specialist trade-off. The research is embedded in the Video & Image Sense lab at the University of Amsterdam, and you will actively collaborate within the European Horizon ELLIOT project, containing 32 partners. ELLIOT will deliver a family of open foundation models – both generalist and fine-tuned models of varying complexity – along with open datasets and methods to enable the AI community in Europe and worldwide to build, study, deploy, extend and evaluate foundation models and the whole pipeline necessary for their creation in a fully reproducible manner.

Working at the UvA

Join our team!

You will carry out research and development in the areas of multimodal foundation models, using advances in deep machine learning and computer vision. The goal is to invent, develop and evaluate novel generalist methods for pre-training and fine-tuning of multimodal foundation models, that align with human-values, expand their perceptual abilities, generalize at deployment, combine with any modality, and do so in an efficient and sustainable way. Importantly, their generalist nature should transfer to previously unknown and rather specialist multimodal use cases by balancing the generalist and specialist trade-off. The research is embedded in the Video & Image Sense lab at the University of Amsterdam, and you will actively collaborate within the European Horizon ELLIOT project, containing 32 partners. ELLIOT will deliver a family of open foundation models – both generalist and fine-tuned models of varying complexity – along with open datasets and methods to enable the AI community in Europe and worldwide to build, study, deploy, extend and evaluate foundation models and the whole pipeline necessary for their creation in a fully reproducible manner.

All about this vacancy

What are you going to do?

Tasks and responsibilities:

  • Develop new deep learning, computer vision and multimodal learning methods for pre-training and fine-tuning multimodal foundation models.
  • Actively collaborate within the ELLIOT project and contribute to its use cases.
  • Regularly present internally on your progress.
  • Regularly present intermediate research at international conferences and workshops, publish them in proceedings and journals.
  • Assist in teaching activities such as lab assistance and student supervision. 
  • Complete and defend a PhD thesis within the official appointment duration of four years.

Your profile

  • An MSc degree in Artificial Intelligence, (Applied) Mathematics/Physics, Computer Science, Engineering or related field.
  • A strong background/knowledge in machine learning and computer vision, natural language processing is a plus.
  • Solid mathematics foundations, especially statistics, calculus and linear algebra;
  • Excellent programming skills, preferably in Python.
  • Experience with AI/HPC supercomputing and running software on scale.
  • You are highly motivated, independent, and creative.
  • Strong communication, presentation and writing skills and excellent command of English.
  • Prior publications in relevant machine learning or computer vision conferences or journals are advantageous.

Our offer

A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). The preferred starting date is September 1st 2025. This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 2,901 to € 3,707 (scale P).This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile PhD Candidate is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Besides the salary and a vibrant and challenging environment at Science Park we offer you multiple fringe benefits:

  • 232 holiday hours per year (based on fulltime) and extra holidays between Christmas and 1 January;
  • multiple courses to follow from our Teaching and Learning Centre;
  • a complete educational program for PhD students;
  • multiple courses on topics such as leadership for academic staff;
  • multiple courses on topics such as time management, handling stress and an online learning platform with 100+ different courses;
  • 7 weeks birth leave (partner leave) with 100% salary;
  • partly paid parental leave;
  • the possibility to set up a workplace at home;
  • a pension at ABP for which UvA pays two third part of the contribution;
  • the possibility to follow courses to learn Dutch;
  • help with housing for a studio or small apartment when you’re moving from abroad.

Are you curious to read more about our extensive package of secondary employment benefits, take a look here.

All about this vacancy

What are you going to do?

Tasks and responsibilities:

  • Develop new deep learning, computer vision and multimodal learning methods for pre-training and fine-tuning multimodal foundation models.
  • Actively collaborate within the ELLIOT project and contribute to its use cases.
  • Regularly present internally on your progress.
  • Regularly present intermediate research at international conferences and workshops, publish them in proceedings and journals.
  • Assist in teaching activities such as lab assistance and student supervision. 
  • Complete and defend a PhD thesis within the official appointment duration of four years.

Your profile

  • An MSc degree in Artificial Intelligence, (Applied) Mathematics/Physics, Computer Science, Engineering or related field.
  • A strong background/knowledge in machine learning and computer vision, natural language processing is a plus.
  • Solid mathematics foundations, especially statistics, calculus and linear algebra;
  • Excellent programming skills, preferably in Python.
  • Experience with AI/HPC supercomputing and running software on scale.
  • You are highly motivated, independent, and creative.
  • Strong communication, presentation and writing skills and excellent command of English.
  • Prior publications in relevant machine learning or computer vision conferences or journals are advantageous.

Our offer

A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). The preferred starting date is September 1st 2025. This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 2,901 to € 3,707 (scale P).This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile PhD Candidate is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Besides the salary and a vibrant and challenging environment at Science Park we offer you multiple fringe benefits:

  • 232 holiday hours per year (based on fulltime) and extra holidays between Christmas and 1 January;
  • multiple courses to follow from our Teaching and Learning Centre;
  • a complete educational program for PhD students;
  • multiple courses on topics such as leadership for academic staff;
  • multiple courses on topics such as time management, handling stress and an online learning platform with 100+ different courses;
  • 7 weeks birth leave (partner leave) with 100% salary;
  • partly paid parental leave;
  • the possibility to set up a workplace at home;
  • a pension at ABP for which UvA pays two third part of the contribution;
  • the possibility to follow courses to learn Dutch;
  • help with housing for a studio or small apartment when you’re moving from abroad.

Are you curious to read more about our extensive package of secondary employment benefits, take a look here.

Your place at the UvA

Where you will work

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute (IvI) is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.

The position is with Prof. dr. Cees Snoek, Professor, head of the Video & Image Sense lab (VIS lab), at the University of Amsterdam. VIS lab is a world-leading lab on Computer Vision and Machine Learning, and has over 30 PhD students, postdoctoral researchers and faculty members working on a broad variety of deep learning, computer vision, and foundation model subjects, like self-supervised learning, diffusion models, and test-time generalization for perception tasks like object detection, instance segmentation and activity recognition. The position is also embedded in the European ELLIS Network of Excellence in AI.

Want to know more about our organisation? Read more about working at the University of Amsterdam.

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Your place at the UvA

This is where you will be working

Where you will work

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute (IvI) is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.

The position is with Prof. dr. Cees Snoek, Professor, head of the Video & Image Sense lab (VIS lab), at the University of Amsterdam. VIS lab is a world-leading lab on Computer Vision and Machine Learning, and has over 30 PhD students, postdoctoral researchers and faculty members working on a broad variety of deep learning, computer vision, and foundation model subjects, like self-supervised learning, diffusion models, and test-time generalization for perception tasks like object detection, instance segmentation and activity recognition. The position is also embedded in the European ELLIS Network of Excellence in AI.

Want to know more about our organisation? Read more about working at the University of Amsterdam.

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the button below. We accept applications until and including 30 June 2025. Applications should include the following information (all files besides your CV should be submitted in one single pdf file):

 

  • A letter that motivates your choice for this position; 
  • Curriculum vitae, including your list of publications if applicable (max 2 pages); 
  • A research statement on how to approach the PhD project. Solid and creative ideas will be greatly appreciated. (max 2 pages). 
  • A link to your Master thesis – if online available, else include an abstract. 
  • A complete record of Bachelor and Master courses (including grades and explanation of grading system); 
  • A list of projects or publications you have worked on, with brief descriptions of your contributions, max 1 page; 
  • The names and contact addresses of at least two academic references (please do not include any recommendation letters). 

 

A knowledge security check can be part of the selection procedure.
(for details: national knowledge security guidelines)

Please use the CV field to upload your resume as a separate pdf document. Use the Cover Letter field to upload the other requested documents, including the motivation letter, as one single pdf file. Only complete applications received within the response period via the link below will be considered. The interviews for this position will be held in July and August.

Do you have any questions or do you require additional information? Please contact:

Diversity, Equity & Inclusion

As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.
Read more

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the button below. We accept applications until and including 30 June 2025. Applications should include the following information (all files besides your CV should be submitted in one single pdf file):

 

  • A letter that motivates your choice for this position; 
  • Curriculum vitae, including your list of publications if applicable (max 2 pages); 
  • A research statement on how to approach the PhD project. Solid and creative ideas will be greatly appreciated. (max 2 pages). 
  • A link to your Master thesis – if online available, else include an abstract. 
  • A complete record of Bachelor and Master courses (including grades and explanation of grading system); 
  • A list of projects or publications you have worked on, with brief descriptions of your contributions, max 1 page; 
  • The names and contact addresses of at least two academic references (please do not include any recommendation letters). 

 

A knowledge security check can be part of the selection procedure.
(for details: national knowledge security guidelines)

Please use the CV field to upload your resume as a separate pdf document. Use the Cover Letter field to upload the other requested documents, including the motivation letter, as one single pdf file. Only complete applications received within the response period via the link below will be considered. The interviews for this position will be held in July and August.

Do you have any questions or do you require additional information? Please contact:

As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.

Other interesting vacancies for you

View all vacancies

Other interesting vacancies for you

View all vacancies

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.

This website uses cookies

We, and third parties, use cookies on our website. We use cookies to ensure that our website functions properly, to store your preferences, to gain insight into visitor behavior, but also for marketing and social media purposes (showing personalized advertisements). By clicking 'Accept', you agree to the use of all cookies. In our Cookie Statement. you can read more about the cookies we use and save or change your preferences. By clicking 'Refuse' you only agree to the use of functional cookies.