Postdoctoral Researcher in Evaluation of Agentic Conversational AI

Postdoctoral Researcher in Evaluation of Agentic Conversational AI

Working at the UvA

Join Us!

Are you excited to shape the next generation of agentic conversational AI systems? Join us as a postdoctoral researcher in the LESSEN consortium, where you will work on cutting-edge research at the intersection of conversational AI, large language models, multi-agent systems, and intelligent tool use.

Recent advances in AI are shifting toward agentic AI systems, where multiple LLM-based agents interact with tools, environments, and users to solve complex tasks. Studying and assessing such systems presents new challenges beyond traditional benchmarks. In this position, you will investigate methods for benchmarking and studying agentic conversational systems, while contributing to the development of shared resources and experimental frameworks for the LESSEN consortium.

You will become part of a collaborative and ambitious research environment, working closely with PhD researchers and academic and industrial partners in the consortium. This is an exciting opportunity for a researcher who wants to combine strong scientific contributions with real-world relevance in a rapidly evolving area of AI.

Working at the UvA

Join Us!

Are you excited to shape the next generation of agentic conversational AI systems? Join us as a postdoctoral researcher in the LESSEN consortium, where you will work on cutting-edge research at the intersection of conversational AI, large language models, multi-agent systems, and intelligent tool use.

Recent advances in AI are shifting toward agentic AI systems, where multiple LLM-based agents interact with tools, environments, and users to solve complex tasks. Studying and assessing such systems presents new challenges beyond traditional benchmarks. In this position, you will investigate methods for benchmarking and studying agentic conversational systems, while contributing to the development of shared resources and experimental frameworks for the LESSEN consortium.

You will become part of a collaborative and ambitious research environment, working closely with PhD researchers and academic and industrial partners in the consortium. This is an exciting opportunity for a researcher who wants to combine strong scientific contributions with real-world relevance in a rapidly evolving area of AI.

All about this vacancy

This is what you will do

In this role, you will investigate research questions related to agentic conversational systems, including benchmarking LLM-based conversational agents, studying multi-agent collaboration and coordination, designing metrics for long-horizon reasoning and planning, and studying tool-augmented conversational systems.

Given the 12-month duration of the position, your focus will be on delivering high-impact research outputs and helping establish reusable research infrastructure for the consortium. You will have the opportunity to develop new ideas, work across disciplines, and contribute to both methodological advances and practical applications in real-world settings.

Tasks and responsibilities

  • You will develop benchmarks and datasets for conversational AI.
  • You will design metrics and methodologies for studying agentic AI systems.
  • You will collaborate with the PhD researchers in the consortium.
  • You will conduct research and publish results at leading AI, NLP, and IR conferences.
  • You will contribute to the development of shared resources and experimental frameworks for the LESSEN consortium.

What we ask of you

You are excited by recent developments in agentic AI and want to help shape this emerging area through strong scientific research. You see careful evaluation not as a side activity, but as a way to push the field forward and generate insights that matter. You enjoy working with real-world data and settings, and you appreciate both the richness and the messiness that come with them. You like turning complex, dynamic behaviour into research questions, experimental setups, and findings that others can build on. You work well independently, but you also value collaboration with colleagues and external partners.

Your experience and profile 

  • A PhD in computer science, artificial intelligence, computational linguistics, information retrieval, natural language processing, or a closely related field.
  • Research experience in one or more of the following areas: conversational AI, large language models, NLP, IR, multi-agent systems, or AI evaluation.
  • Experience with conducting and publishing academic research.
  • Professional command of English, both spoken and written.

This is what we offer you

We offer a temporary employment contract for 38 hours per week for a period of 12 months. The preferred starting date is as soon as possible.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 3,546 to € 5,538 (scale 10). This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile Researcher 4 is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Curious about our extensive secondary benefits package? You can read more about it here.

All about this vacancy

This is what you will do

In this role, you will investigate research questions related to agentic conversational systems, including benchmarking LLM-based conversational agents, studying multi-agent collaboration and coordination, designing metrics for long-horizon reasoning and planning, and studying tool-augmented conversational systems.

Given the 12-month duration of the position, your focus will be on delivering high-impact research outputs and helping establish reusable research infrastructure for the consortium. You will have the opportunity to develop new ideas, work across disciplines, and contribute to both methodological advances and practical applications in real-world settings.

Tasks and responsibilities

  • You will develop benchmarks and datasets for conversational AI.
  • You will design metrics and methodologies for studying agentic AI systems.
  • You will collaborate with the PhD researchers in the consortium.
  • You will conduct research and publish results at leading AI, NLP, and IR conferences.
  • You will contribute to the development of shared resources and experimental frameworks for the LESSEN consortium.

What we ask of you

You are excited by recent developments in agentic AI and want to help shape this emerging area through strong scientific research. You see careful evaluation not as a side activity, but as a way to push the field forward and generate insights that matter. You enjoy working with real-world data and settings, and you appreciate both the richness and the messiness that come with them. You like turning complex, dynamic behaviour into research questions, experimental setups, and findings that others can build on. You work well independently, but you also value collaboration with colleagues and external partners.

Your experience and profile 

  • A PhD in computer science, artificial intelligence, computational linguistics, information retrieval, natural language processing, or a closely related field.
  • Research experience in one or more of the following areas: conversational AI, large language models, NLP, IR, multi-agent systems, or AI evaluation.
  • Experience with conducting and publishing academic research.
  • Professional command of English, both spoken and written.

This is what we offer you

We offer a temporary employment contract for 38 hours per week for a period of 12 months. The preferred starting date is as soon as possible.

The gross monthly salary, based on 38 hours per week and dependent on relevant experience, ranges between € 3,546 to € 5,538 (scale 10). This does not include 8% holiday allowance and 8,3% year-end allowance. The UFO profile Researcher 4 is applicable. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Universities of the Netherlands is applicable.

Curious about our extensive secondary benefits package? You can read more about it here.

Your place at the UvA

You will work in this team

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The Informatics Institute (IvI) at the University of Amsterdam is one of Europe’s leading research institutes in computer science and AI. Its mission is to conduct curiosity-driven and use-inspired fundamental research.

You will be embedded in the Information Retrieval Lab (IRLab), one of the research groups within IvI. IRLab works on search and information access from a modern artificial intelligence perspective, combining machine intelligence with human intelligence to help people locate and act on the information they need. The lab offers an active and collaborative research environment with expertise in information retrieval, natural language processing, large language models, conversational AI, recommender systems, human-AI interaction, and evaluation. This makes IRLab a strong home base for research on agentic conversational systems and their study in realistic settings.
More information about IRLab can be found
here.

Want to know more about our organisation? Read more about working at the University of Amsterdam.

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Your place at the UvA

This is where you will be working

You will work in this team

The Faculty of Science has a student body of around 8,000, as well as 1,800 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The Informatics Institute (IvI) at the University of Amsterdam is one of Europe’s leading research institutes in computer science and AI. Its mission is to conduct curiosity-driven and use-inspired fundamental research.

You will be embedded in the Information Retrieval Lab (IRLab), one of the research groups within IvI. IRLab works on search and information access from a modern artificial intelligence perspective, combining machine intelligence with human intelligence to help people locate and act on the information they need. The lab offers an active and collaborative research environment with expertise in information retrieval, natural language processing, large language models, conversational AI, recommender systems, human-AI interaction, and evaluation. This makes IRLab a strong home base for research on agentic conversational systems and their study in realistic settings.
More information about IRLab can be found
here.

Want to know more about our organisation? Read more about working at the University of Amsterdam.

More about the UvA

The University of Amsterdam is ambitious, creative and committed. An inspiration to students since 1632, a vanguard player in international science and a partner in innovation.
The University of Amsterdam is the largest university in the Netherlands, with the broadest range of courses on offer. An intellectual hub with 42,000 students, 6,000 staff and 3,000 PhD students. Connected by a culture of curiosity.

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the red button. We accept applications until and including 30 April 2026.

Applications should include the following information (all files besides your cv should be submitted in one single pdf file):

  • a detailed CV including the months (not just years) when referring to your education and work experience;
  • a letter of motivation;
  • a list of publications;
  • the names and email addresses of two references who can provide letters of recommendation.

A knowledge security check can be part of the selection procedure.
(for details: 
national knowledge security guidelines)

If you have any questions or do you require additional information? Please contact:

Acquisition in response to this vacancy is not appreciated.

Diversity, Equity & Inclusion

As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.
Studies show that women and members of underrepresented groups only apply for jobs if they meet 100% of the qualifications. Do you meet the educational requirements but not yet all of the requested experience? The UvA encourages you to apply anyway.

Important to know

Your application & contact

If you feel the profile fits you, and you are interested in the job, we look forward to receiving your application. You can apply online via the red button. We accept applications until and including 30 April 2026.

Applications should include the following information (all files besides your cv should be submitted in one single pdf file):

  • a detailed CV including the months (not just years) when referring to your education and work experience;
  • a letter of motivation;
  • a list of publications;
  • the names and email addresses of two references who can provide letters of recommendation.

A knowledge security check can be part of the selection procedure.
(for details: 
national knowledge security guidelines)

If you have any questions or do you require additional information? Please contact:

Acquisition in response to this vacancy is not appreciated.
As an employer, the UvA maintains an equal opportunities policy. We value diversity and are fully committed to being a place where everyone feels at home. We nurture inquisitive minds and perseverance and allow room for persistent questioning. With us, curiosity and creativity are the prevailing culture.
Studies show that women and members of underrepresented groups only apply for jobs if they meet 100% of the qualifications. Do you meet the educational requirements but not yet all of the requested experience? The UvA encourages you to apply anyway.

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.

Don't miss out on your dream job!

Sign up for a job alert and you'll receive automatic updates about new and relevant vacancies.