Skip to content

Michael Osborne

Position
Associate Professor
Organisation
University of Oxford
Biography

Why do you care about AI Existential Safety?

I believe that AI presents a real existential threat, and one to which I, as an AI researcher, have a duty to address. Nor is the threat from AI limited to a distant future. As AI algorithms are deployed more widely, within ever more sensitive applications, from healthcare to defence, the need for AI systems to be safer is with us today. In answer to these challenges, I believe that my particular interests – Bayesian models and numeric algorithms – offer a framework for AI that is transparent, performant and safe.

Please give one or more examples of research interests relevant to AI existential safety:

In control engineering for safety-critical areas like aerospace and automotive domains, it has long been a requirement that computer code is verifiably safe: the designers must guarantee that the code will never reach a state in which it might take a catastrophic decision. AI methods, however, are vastly more complex and adaptive than classic control algorithms, meaning that similar guarantees have not yet been achieved. As AI systems begin to have increasing influence on our lives, they must become better monitored and controlled.
I am interested in new, verifiably safe, algorithms for the most elementary computational steps that make up AI systems: numerical methods. Numerical methods, particularly optimisation methods, are well-known to be critical to both the performance and reliability of AI systems. State-of-the-art numerical methods aim to create minimal computational error through conservative assumptions. Unfortunately, in practice, these assumptions are often invalid, leading to unexpectedly high error.

Instead, I aim to develop novel numerical algorithms that explicitly estimate their own error, incorporating all possible error sources, as well as adaptively assigning computation so as to reduce overall risk. Probabilistic numerics is a new, rigorous, framework for the quantification of computational error in numerical tasks. Probabilistic Numerics was born of recent developments in the interpretation of numerical methods, providing new tools for ensuring AI safety. Numerical algorithms estimate latent (non-analytic) quantities from the result of tractable (“observable”) computations. Their task can thus be described as inference in the statistical sense, and numerical algorithms cast as learning machines that actively collect (compute) data to infer a non-analytic quantity. Importantly, this notion applies even if the quantity in question is entirely of a deterministic nature—uncertainty can be assigned to quantities that are not stochastic, just unknown. Probabilistic Numerics is the treatment of numerical computation as inference, yielding algorithms that take in probability distributions over input variables, and return probability distributions over their output, such that the output distribution reflects uncertainty caused both by the uncertain inputs and the imperfect internal computation. Moreover, Probabilistic Numerics, through its estimates of how uncertain and hence how valuable is a computation, allows the allocation of computation to itself be optimised. As a result, probabilistic numeric algorithms have been shown to offer significantly lower computational costs than alternatives. Intelligent allocation of computation can also improve safety, by forcing computation to explore troublesome edge cases that might otherwise be neglected.

I aim to apply the probabilistic numeric framework to the identification and communication of computational errors within composite AI systems. Probabilistic numerical methods offer the promise of monitoring assumptions in running computations, yielding a monitoring regime that can safely interrupt algorithms overwhelmed by their task’s complexity. This approach will allow AI systems to monitor the extent to which their own internal model matches external data, and to respond appropriately cautiously.

Sign up for the Future of Life Institute newsletter

Join 40,000+ others receiving periodic updates on our work and cause areas.
cloudmagnifiercrossarrow-up linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram