Claas A. Voelcker

W1140-108 College Street

SR Innovation Campus

Toronto, ON

M5G 0C6

I am a PhD student in Reinforcement and Machine Learning at the University of Toronto and the Vector Institute, supervised by Profs. Amir-massoud Farahmand and Igor Gilitschenski. In November, I will be starting a postdoc at UT Austin with Profs. Peter Stone and Amy Zhang.

My research focuses on model based reinforcement learning and closing the gap between learning acurate models for future predictions and learning high performing models for planning. I am interested in using techniques for representation and world model learning to stablize notoriously brittle Deep Reinforcement Learning approaches. Finally, I like thinking about how we can do better science in RL by thinking about what problems we should be benchmarking our exciting advances on.

Originally from Germany, I received a Bachelor and Master degree from the University of Darmstadt with Honors. There, I had the great pleasure to be supervised and mentored by Profs. Kristian Kersting and Jan Peters.

I am proud to serve as a core organizer for Queer in AI, where I help promote the interests of queer researchers and practitioners at AI /ML conferences and in the wider community.

news

Jul 01, 2025	Our paper Calibrated Value-Aware Model Learning with Probabilistic Environment Models will be presented at ICML 2025 in Vancouver next week! Let me know if you want to meet up for a coffee.
Jun 13, 2025	I accepted a postdoc position with Peter Stone and Amy Zhang at UT Austin. Looking forward to work with so many amazing students and faculty in the Texas Robotics Ecosystem on RL that matters for real-world robotics.
Mar 13, 2025	Our paper MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL was awarded a spotlight award at ICLR 2025! See you in Singapore.
Oct 13, 2024	I made a new website!

latest posts

Mar 15, 2025	loss functions and calibration
Mar 15, 2025	reward design and termination

selected publications

2025

MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL

Claas A. Voelcker, Marcel Hussing, Eric Eaton, Amir-massoud Farahmand, and Igor Gilitschenski

International Conference on Learning Representations, Apr 2025

Spotlight Bib PDF

spotlight

@article{voelcker2024mad,
  title = {MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL},
  author = {Voelcker, Claas A. and Hussing, Marcel and Eaton, Eric and Farahmand, Amir-massoud and Gilitschenski, Igor},
  journal = {International Conference on Learning Representations},
  year = {2025},
  month = apr,
}

Calibrated Value-Aware Model Learning with Probabilistic Environment Models

Claas A Voelcker, Anastasiia Pedan, Arash Ahmadian, Romina Abachi, Igor Gilitschenski, and 1 more author

International Conference on Machine Learning, Jul 2025

Bib PDF

@article{voelcker2025calibrated,
  title = {Calibrated Value-Aware Model Learning with Probabilistic Environment Models},
  author = {Voelcker, Claas A and Pedan, Anastasiia and Ahmadian, Arash and Abachi, Romina and Gilitschenski, Igor and Farahmand, Amir-massoud},
  journal = {International Conference on Machine Learning},
  year = {2025},
  month = jul,
  url = {https://openreview.net/forum?id=fgO1R1iVEi}
}

2024

Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence

Marcel Hussing, Claas A. Voelcker, Igor Gilitschenski, Amir-massoud Farahmand, and Eric Eaton

Reinforcement Learning Conference, Aug 2024

Bib PDF

@article{hussing2024dissecting,
  title = {Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence},
  author = {Hussing, Marcel and Voelcker, Claas A. and Gilitschenski, Igor and Farahmand, Amir-massoud and Eaton, Eric},
  journal = {Reinforcement Learning Conference},
  year = {2024},
  month = aug
}

When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning

Claas A. Voelcker, Tyler Kastner, Igor Gilitschenski, and Amir-massoud Farahmand

Reinforcement Learning Conference, Aug 2024

Bib PDF

@article{voelcker2024does,
  title = {When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning},
  author = {Voelcker, Claas A. and Kastner, Tyler and Gilitschenski, Igor and Farahmand, Amir-massoud},
  journal = {Reinforcement Learning Conference},
  year = {2024},
  month = aug
}

2023

λ-AC: Learning latent decision-aware models for reinforcement learning in continuous state-spaces

Claas A. Voelcker, Arash Ahmadian, Romina Abachi, Igor Gilitschenski, and Amir-massoud Farahmand

arXiv preprint arXiv:2306.17366, Nov 2023

Bib PDF

@article{voelcker2023lambda,
  title = {$\lambda$-AC: Learning latent decision-aware models for reinforcement learning in continuous state-spaces},
  author = {Voelcker, Claas A. and Ahmadian, Arash and Abachi, Romina and Gilitschenski, Igor and Farahmand, Amir-massoud},
  journal = {arXiv preprint arXiv:2306.17366},
  year = {2023},
  month = nov
}

2022

Value Gradient weighted Model-Based Reinforcement Learning

Claas A. Voelcker, Victor Liao, Animesh Garg, and Amir-massoud Farahmand

In International Conference on Learning Representations, Apr 2022

Spotlight Bib PDF

spotlight

@inproceedings{voelcker2022value,
  title = {Value Gradient weighted Model-Based Reinforcement Learning},
  author = {Voelcker, Claas A. and Liao, Victor and Garg, Animesh and Farahmand, Amir-massoud},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=4-D6CZkRXxI},
  month = apr
}