
Simon Mylius
Why do you care about AI Existential Safety?
As a Chartered Engineer who spent 13 years integrating safety-critical systems in hydrogen and high-voltage automotive applications, I have direct experience applying the methodologies that keep people safe in high-hazard industries. STPA, safety cases, incident investigation are mature and proven, yet largely absent from frontier AI development. I believe adapting these frameworks to AI is one of the highest-leverage contributions the safety engineering community can make. I care about AI existential safety because the window for building robust governance infrastructure is closing, and I want to help ensure we use it well.
Please give at least one example of your research interests related to AI existential safety:
AI Incident Analysis & Early Warning: I lead the MIT FutureTech AI Incident Tracker, a structured analysis of all incidents in the AI Incident Database, classifying each by risk domain, severity and national security relevance. This work, conducted in partnership with CSET, aims to identify patterns in how AI harms escalate — turning scattered reports into actionable intelligence for policymakers and labs. Separately, I founded Arcola AI to build a real-time incident intelligence platform that aggregates cross-source data to detect emerging AI harms before they escalate.
Loss of Control Observatory: I was the technical lead for the Centre for Long Term Resilience’s Loss of Control Observatory, monitoring open-source intelligence for early signals of AI systems engaging in scheming-like behaviour – a project funded by UK AISI.
