AI and machine learning models keep getting better, but how they make their decisions often remains unclear, because these depend on many incomprehensible model parameters that have been learned from data. For instance, why does a machine learning model predict that it is unsafe to discharge a certain patient from the intensive care? Or which characteristics make a machine learning model flag a certain bank transfer as potential money laundering? The aim of this project is to develop new mathematical techniques that can explain AI's internal mechanisms by generating understandable explanations from machine learning models. We will achieve this together by creating the first mathematical framework for explainable AI and developing new explanation methods. This will involve using tools from mathematical machine learning theory to prove mathematical guarantees about the performance of such new explanation methods, as well as programming to test out the methods and mathematical theory in experiments. (See example references [1,2,3] below for initial results.) During the PhD, you will initially work on a pre-specified subproject of your choosing, which allows you to develop your knowledge of the topic and research skills. As you gain experience, you will increasingly shape your own research directions. You will collaborate closely with dr. Tim van Erven and other team members, and connect with new (inter)national partners.

• Conduct research within the specified project, as described above.

• Disseminate your research findings through publications in academic journals or peer-reviewed international conference proceedings, and through presentations at international conferences.

• Participate in relevant seminars and engage in research collaborations locally and/or internationally.

• Participate in the Faculty of Science PhD training program.

• Participate in the department’s educational programs, assisting in teaching and supervision of undergraduate or Master students.

• Complete and defend a PhD thesis within the appointment duration of four years.

Example references:

For the application, you are not expected to study the following references in depth, but you may look into them for a high-level impression of the style of research.

1. H. Fokkema, R. de Heide and T. van Erven. Attribution-based Explanations that Provide Recourse Cannot be Robust. Journal of Machine Learning Research, vol. 24, no. 360, pp. 1-37, 2023.

2. H. Fokkema, D. Garreau and T. van Erven. The Risks of Recourse in Binary Classification. Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 238, pp. 550-558, 2024.

3. H. Fokkema, T. van Erven and S. Magliacane. Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions. ArXiv:2502.06536 Preprint, 2025.

We are looking for an enthusiastic and driven candidate who meets the following requirements:

Your experience and profile:

• Educational background: You should have a Master’s degree in mathematics or AI/computer science (with a strong mathematical component, including writing formal mathematical proofs), to be completed before the start of the position. Other degrees with a strong mathematical component will also be considered.

• Interests: You should have a strong interest in mathematical machine learning theory and explainable AI. (This point is about your motivation, not about the courses you have taken.)

• Programming skills: You should have very good programming skills in Python or similar, which allow you to effectively test out new explanation methods in experiments.

• Communication skills: You should have excellent written and verbal communication skills in English.

• Teamwork and independence: You should be highly motivated and committed to your research. While you should be able to work independently, you should also have a cooperative attitude and be willing to collaborate with the team members or other researchers.

Knowledge of mathematical machine learning theory or mathematical statistics is a plus, as are extra-curricular activities in, for instance, student associations, sports, music, etc. Prior knowledge of explainable AI is beneficial, but not required.