Why do you care about AI Existential Safety?

I care about existential risks because each generation is a steward of an open human future. The minimum ethical bar of our decisions is not to foreclose the freedom and agency of those who come after us.
AI x-risks are real. Unlike many earlier technologies that were engineered with interpretable mechanisms, modern AI is partly “grown” through training processes that can produce emergent capabilities we have a very limited understanding of. As these systems become more agentic and more deeply embedded in society, misalignment is not only a technical problem but a civilizational one.
The AI x-risks I am particularly concerned about center on human informational self-determination. 1. Large-scale erosion of human autonomy via systems that shape attention, preferences, and decision-making. 2. AI-generated information breaching the real-virtual boundary, breaking the foundations that democratic societies and individual agency rely on.
Compared with the capital and manpower pouring into the AGI/ASI race, efforts in safety, governance, and public resilience remain dangerously under-resourced. This imbalance narrows the window for AI x-safety and is why I care about it now!

Please give at least one example of your research interests related to AI existential safety:

My research interest lies in two connected areas: (1) how to effectively promote information authenticity through physical-level provenance, and (2) how the shifting distribution of intelligence (driven by the commoditization of AI), reshapes the structure and feasibility of oversight. I am also interested in the field of mechanistic interpretability, as I believe that understanding model internals is foundational to long-term alignment and effective governance.

My first main interest is in information authenticity. If people cannot reliably tell what in the world actually happened or if all credible channels can be cheaply forged or flooded, then democratic oversight and collective decision-making become impossible. I am particularly interested in the possibility of building a physical-level provenance platform for visual content such as photos and videos. What ultimately distinguishes physically captured media from generated content is the light-sensing process, where camera sensors detect photons from the real world. I believe embedding cryptographic operations directly into this process, such as on-sensor encryption, is a promising direction. This hardware-level authentication could then serve as the secure foundation for higher-level provenance standards, such as C2PA.

My second main interest is the oversight problem under shifted intelligence distributions. Prior work, including Scaling Laws for Scalable Oversight (Tegmark et al. 2025), has shown that oversight can fail even when overseers are only slightly less capable than the systems they aim to govern. Yet in human societies, we do manage to constrain highly intelligent or powerful individuals through legal, institutional, and social mechanisms. A key but often overlooked factor is the distribution of intelligence: exceptional individuals, though influential, remain a minority embedded within a society where the cognitively average majority collectively governs oversight institutions. Moreover, they depend on collaboration with other intelligent peers, creating mutual dependencies that limit unilateral control. As AI advances, this distributional structure is bound to collapse. Intelligence will become scalable, commoditized, and detached from social interdependence; this leads to concentrated systems that require little from the rest of society while exerting disproportionate influence. This shift undermines the foundations of traditional oversight and calls for new thinking. I believe this structural perspective is essential—and still largely underexplored.