Current vacancies in the Algorithmics group

Automated decision-making is a key capability of any intelligent system.

Intelligent systems (e.g. for traffic planning, energy systems, robots, automated negotiation) must plan ahead, which is particularly challenging when either the future is uncertain, or they interact with many other agents.

To tackle such problems, the Algorithmics group focuses on two lines of research.

We develop new algorithms and solvers for automated planning and scheduling, improve upon existing techniques such as reinforcement learning, Markov decision processes, and mathematical programming, and integrate our results in working prototypes and toolboxes to be used by other researchers and industry.

Furthermore, we develop game-theoretical mechanisms which govern the interaction between multiple self-interested systems or users, and provide computationally efficient algorithms to compute equilibria in these interactions.

By tackling these fundamental challenges at the intersection of sequential decision-making, machine learning, and algorithmic mechanism design, we contribute to more efficient energy use, reducing highway congestion, and reducing the cost of asset maintenance in various industries. Our results are published in the internationally leading conferences and journals in planning and scheduling, multi-agent systems, and artificial intelligence.

We offer the following 3 PhD positions. To apply, please use the application form. Deadline: Feb 28.

PhD position 1 (LearnSDM)

This PhD position focuses on combining machine learning and model checking to learn and evaluate models for sequential decision making. The position is within the NWO TOP project LearnSDM, see below for details.

Main supervisor: Dr. M.T.J. Spaan.

PhD position 2 (FlexI)

This PhD position concerns developing algorithms for planning under uncertainty for optimizing flexible energy use in industry. The position is within the NWO project FlexI, see below for details.

Main supervisor: Dr. M.T.J. Spaan.

PhD position 3 (FlexI)

This PhD position focuses on developing algorithms for strategic decision-making regarding longer-term investments and trading. This position is also within FlexI, see below for details.

Supervisors: Dr. M.M. de Weerdt, Dr. R.A. Verzijlbergh.

LearnSDM project: Model Learning for Sequential Decision Making

AI’s tangible impact on our society is strongly growing, think of spam filters, smart thermostats, intelligent personal assistants and autonomous vehicles in the near future.

Although current systems may appear intelligent, at their core they rely extensively on human engineering. Intelligent decision making, i.e., optimizing rational behavior given large amounts of uncertain data, is a key challenge faced by designers of such intelligent systems.

Algorithms for sequential decision making (SDM, often modeled as a Markov Decision Process or an extension thereof) allow an agent to plan its actions given uncertain information, but require models of how the system interacts with the environment. Currently, these models are typically assumed to be given, for instance provided by an expert or discovered by trial-and-error.

In this curiosity-driven project, the goal is to develop algorithms for machine learning as well as model checking to learn and evaluate models for SDM in a principled way. Possible concrete case studies could be traffic flow optimization or smart charging of electric vehicles.

FlexI project: Optimising Flexible Energy Use in Industry

A large part of renewably generated power is intermittent, uncertain, and uncontrollable. As balance between demand and generation is required at all times, flexibility in electricity demand is needed to prevent having significant costly flexible controllable generation (from fossil energy sources) on stand-by. Research and even some first pilot studies have been performed to use flexibility of heating, cooling, and (electrical vehicle) charging in households. However, industry in the Netherlands, using about three times more energy than households (CBS, 2013), offers far more promising opportunities by considering flexibility from all used carriers of energy.

The problem is that unleashing this potential of energy flexibility in industry (switching between energy carriers, using and sharing buffers for heat, steam, intermediate products, varying production, etc.) requires optimising the daily operations not just regarding throughput, but also to include making cost-effective energy trading decisions. We will develop algorithmic techniques to support both these decision problems under uncertainty.

In close cooperation, the two PhD students will investigate questions such as:

  • Which existing approach using multi-stage stochastic optimisation and algorithms for Markov decision processes works best under which conditions for these problems?
  • How to deal with the scalability of the operational scheduling problem under uncertainty?
  • What (combination of) time resolutions, planning horizon, models and algorithms give the best results in time?
  • How to deal with the uncertainty regarding strategic decisions on investments and on trading in bilateral contracts and future markets?
  • What model for daily benefits of trading flexibility leads to profitable decisions?
  • How can the operational planning interact with the strategic decision making?

Stakeholders are operators and business analysts of large industrial plants such as the chemical industry in the Port of Rotterdam (in particular given our cooperation with Smart Port), but also policy makers, utility companies, and port authorities to explore and enable better infrastructures and new business opportunities.


We are looking for excellent candidates who meet the following requirements:

  • a MSc degree in Computer Science, Artificial Intelligence, Mathematics, Operations Research, or a closely related field;

  • affinity with interdisciplinary work;

  • good programming skills;

  • good English speaking and writing skills.


  • perform scientific research;

  • implement and evaluate research prototypes;

  • present results at international conferences;

  • publish results in scientific journals;

  • participate in activities of the research group.

Related PhD projects in the Algorithmics group

Current and previous PhD students in the Algorithmics group have been working on several innovative projects related to algorithm design, multi-agent systems, machine learning and artificial intelligence in general. A few examples of such projects are discussed below:

  • We designed faster algorithmic methods to solve the all-pairs shortest paths problem, which significantly advances the state of the art.
  • Together with Jedlix and Eneco we contribute to a real-world pilot project in which scalable planning and scheduling algorithms for electric vehicles are put into practice.
  • We designed novel algorithms to assign resources or budget (e.g., money) to multiple interacting agents. These algorithms can be applied in the context of planning within resource constraints imposed by an electricity grid. Another application domain is assigning a limited amount of money to projects, such that revenue is maximized.
  • We participated in a real-world pilot project in which reinforcement learning is used to automatically learn policies dictating the driving speed on highways.
  • We developed advanced optimization techniques to accelerate existing exact algorithms for Partially Observable Markov Decision Processes, a widely used model in the field of artificial intelligence and robotics.
  • In collaboration with NedTrain we developed sophisticated scheduling technology to enhance the flexibility of schedules for train maintenance in the Netherlands.
Wordcloud of Algorithmics publications 2012-2016

How to apply

To apply, please complete the application form.The deadline is February 28, 2017.

The application should consist of the following parts:

  • an explanation of your interest in the proposed research field;

  • a Curriculum Vitae;

  • copies of diplomas and other relevant certificates;

  • a complete list of attended courses and corresponding grades;

  • names and contact details of two referees;

  • proof of language skills (if applicable).