Why do you care about AI Existential Safety?

We are developing a technology that may turn out to be more intelligent and way more efficient than humans. I truly believe such a technology can have immense impact to make everyone’s lives better in many different ways. However, controlling this intelligence could be one of the most challenging problems we will face in the coming decades. In the rapid race to develop artificial intelligence, it’s easy to overlook the safety and security of these models. We will not only face technical, but also societal challenges. As AI becomes more integrated into society, there’s a risk of gradually disempowering ourselves by relying too heavily on AI systems. Addressing the challenges posed by artificial general intelligence requires the collective effort of as many people as possible to ensure we navigate this path responsibly.

Please give at least one example of your research interests related to AI existential safety:

One of my primary research topics in the past has been red-teaming large language models (LLMs). By systematically testing models for vulnerabilities, we can identify and address potential risks before they manifest in real-world applications. When it comes to AI existential safety, it is important to adaptively evaluate for worst-case behaviors as those are the ones we expect may have a largest impact in the world. As AI capabilities advance, the stakes for properly evaluating safety measures will only increase. We must maintain rigorous standards for testing and validating our technical mitigations.