Why do you care about AI Existential Safety?

As a security researcher working at the intersection of AI safety, cryptography, and system design, I care deeply about AI existential safety because AI’s rapid advancement is fundamentally changing how humans reason, collaborate, and make decisions. Powerful AI systems (especially large language models and copilots) hold transformative potential but also introduce subtle and systemic vulnerabilities that can scale beyond human control. Ensuring these AI systems remain aligned with human values and intentions, particularly as they become more autonomous, is critical to safeguarding humanity’s future. Without dedicated safety research, these technologies risk being misused or causing unintended harm in ways that could undermine human autonomy and values. It’s not enough to focus solely on making AI more capable. We must also confront the dangers that come with that power. Otherwise, we risk building systems that can distort human judgment or be exploited at scale. Ultimately, AI existential safety is about ensuring these tools truly empower humans while preserving our agency and critical thinking even as AI becomes more central to everyday life.

Please give at least one example of your research interests related to AI existential safety:

One of my core research interests related to AI existential safety is securing LLMs against information leakage and epistemic failure at deployment scale. At Microsoft Research, I’ve discovered vulnerabilities like cross-prompt injection attacks (XPIA) in widely deployed copilots (e.g., Microsoft Copilot, Google Workspace Gemini), which could allow malicious inputs to extract sensitive data from prior sessions. These risks scale with model capabilities, making them relevant not just for privacy, but also for misalignment in multi-agent or persistent memory settings. To mitigate this, I’m working on an ACL-aware secure inference pipelines and exploring evaluation metrics for epistemic virtue such as transparency, source traceability. This contributes to alignment by pushing models toward trustworthy reasoning under uncertainty, and protecting users from being misled by fluent but incorrect outputs.
I’m also interested in how secure computation and cryptographic tooling can support human oversight and verifiability in advanced AI systems which is critical for scenarios where autonomous models operate beyond direct human control.