Claas A. Voelcker

W1140-108 College Street

SR Innovation Campus

Toronto, ON

M5G 0C6

I am currently looking for post-doc opportunities!

I am a PhD student in Reinforcement and Machine Learning at the University of Toronto and the Vector Institute, supervised by Profs. Amir-massoud Farahmand and Igor Gilitschenski.

My research focuses on model based reinforcement learning and closing the gap between learning acurate models for future predictions and learning high performing models for planning. I am interested in using techniques for representation and world model learning to stablize notoriously brittle Deep Reinforcement Learning approaches. Finally, I like thinking about how we can do better science in RL by thinking about what problems we should be benchmarking our exciting advances on.

Originally from Germany, I received a Bachelor and Master degree from the University of Darmstadt with Honors. There, I had the great pleasure to be supervised and mentored by Profs. Kristian Kersting and Jan Peters.

I am proud to serve as a core organizer for Queer in AI, where I help promote the interests of queer researchers and practitioners at AI /ML conferences and in the wider community.

news

Oct 13, 2024	I made a new website!

selected publications

2024

Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence

Marcel Hussing, Claas A. Voelcker, Igor Gilitschenski, Amir-massoud Farahmand, and Eric Eaton

Reinforcement Learning Conference, Aug 2024

Bib PDF

@article{hussing2024dissecting,
  title = {Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence},
  author = {Hussing, Marcel and Voelcker, Claas A. and Gilitschenski, Igor and Farahmand, Amir-massoud and Eaton, Eric},
  journal = {Reinforcement Learning Conference},
  year = {2024},
  month = aug
}

When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning

Claas A. Voelcker, Tyler Kastner, Igor Gilitschenski, and Amir-massoud Farahmand

Reinforcement Learning Conference, Aug 2024

Bib PDF

@article{voelcker2024does,
  title = {When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning},
  author = {Voelcker, Claas A. and Kastner, Tyler and Gilitschenski, Igor and Farahmand, Amir-massoud},
  journal = {Reinforcement Learning Conference},
  year = {2024},
  month = aug
}

MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL

Claas A. Voelcker, Marcel Hussing, Eric Eaton, Amir-massoud Farahmand, and Igor Gilitschenski

under review, Oct 2024

Bib PDF

@article{voelcker2024mad,
  title = {MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL},
  author = {Voelcker, Claas A. and Hussing, Marcel and Eaton, Eric and Farahmand, Amir-massoud and Gilitschenski, Igor},
  journal = {under review},
  year = {2024},
  month = oct,
}

2023

λ-AC: Learning latent decision-aware models for reinforcement learning in continuous state-spaces

Claas A. Voelcker, Arash Ahmadian, Romina Abachi, Igor Gilitschenski, and Amir-massoud Farahmand

arXiv preprint arXiv:2306.17366, Nov 2023

Bib PDF

@article{voelcker2023lambda,
  title = {$\lambda$-AC: Learning latent decision-aware models for reinforcement learning in continuous state-spaces},
  author = {Voelcker, Claas A. and Ahmadian, Arash and Abachi, Romina and Gilitschenski, Igor and Farahmand, Amir-massoud},
  journal = {arXiv preprint arXiv:2306.17366},
  year = {2023},
  month = nov
}

2022

Value Gradient weighted Model-Based Reinforcement Learning

Claas A. Voelcker, Victor Liao, Animesh Garg, and Amir-massoud Farahmand

In International Conference on Learning Representations, Apr 2022

Spotlight Bib PDF

spotlight

@inproceedings{voelcker2022value,
  title = {Value Gradient weighted Model-Based Reinforcement Learning},
  author = {Voelcker, Claas A. and Liao, Victor and Garg, Animesh and Farahmand, Amir-massoud},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=4-D6CZkRXxI},
  month = apr
}