Why do you care about AI Existential Safety?

After having worked on climate/sustainability until 2023 and being frustrated by the lack of collective action despite vast academic literature, I sought a research area with a better expected marginal impact. Given my background in game theory, dynamical systems, social choice theory, and formal ethics, AI safety seemed the natural fit. I also think it is the most urgent global problem due to its all-or-nothing nature.

Please give at least one example of your research interests related to AI existential safety:

I work full-time on safe-by-design AI agent architectures based on a world model, model-based decision criterion estimation algorithms, and hardwired decision rules. I currently focus on the decision-making part but also work on world modeling. After exploring multi-criterion satisficing, I currently work on formalizing human (dis)empowerment. I also co-organize the SC4AI community and regularly participate in IASEAI and CHAI events.