An Open Letter

Research Priorities for Robust and Beneficial Artificial Intelligence

Artificial intelligence (AI) research has explored a variety of problems and approaches since its inception, but for the last 20 years or so has been focused on the problems surrounding the construction of intelligent agents – systems that perceive and act in some environment. In this context, “intelligence” is related to statistical and economic notions of rationality – colloquially, the ability to make good decisions, plans, or inferences. The adoption of probabilistic and decision-theoretic representations and statistical learning methods has led to a large degree of integration and cross-fertilization among AI, machine learning, statistics, control theory, neuroscience, and other fields. The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems.

As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research. There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. The potential benefits are huge, since everything that civilization has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools AI may provide, but the eradication of disease and poverty are not unfathomable. Because of the great potential of AI, it is important to research how to reap its benefits while avoiding potential pitfalls.

The progress in AI research makes it timely to focus research not only on making AI more capable, but also on maximizing the societal benefit of AI. Such considerations motivated the AAAI 2008-09 Presidential Panel on Long-Term AI Futures and other projects on AI impacts, and constitute a significant expansion of the field of AI itself, which up to now has focused largely on techniques that are neutral with respect to purpose. We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial: our AI systems must do what we want them to do. The attached research priorities document gives many examples of such research directions that can help maximize the societal benefit of AI. This research is by necessity interdisciplinary, because it involves both society and AI. It ranges from economics, law and philosophy to computer security, formal methods and, of course, various branches of AI itself.

In summary, we believe that research on how to make AI systems robust and beneficial is both important and timely, and that there are concrete research directions that can be pursued today.


If you have questions about this letter, please contact Max Tegmark.

To date, the open letter has been signed by over 8,600 people. The list of signatories includes:

Stuart Russell, Berkeley, Professor of Computer Science, director of the Center for Intelligent Systems, and co-author of the standard textbook Artificial Intelligence: a Modern Approach.
Tom Dietterich, Oregon State, President of AAAI, Professor and Director of Intelligent Systems
Eric Horvitz, Microsoft research director, ex AAAI president, co-chair of the AAAI presidential panel on long-term AI futures
Bart Selman, Cornell, Professor of Computer Science, co-chair of the AAAI presidential panel on long-term AI futures
Francesca Rossi, Padova & Harvard, Professor of Computer Science, IJCAI President and Co-chair of AAAI committee on impact of AI and Ethical Issues
Demis Hassabis, co-founder of DeepMind
Shane Legg, co-founder of DeepMind
Mustafa Suleyman, co-founder of DeepMind
Dileep George, co-founder of Vicarious
Scott Phoenix, co-founder of Vicarious
Yann LeCun, head of Facebook’s Artificial Intelligence Laboratory
Geoffrey Hinton, University of Toronto and Google Inc.
Yoshua Bengio, Université de Montréal
Peter Norvig, Director of research at Google and co-author of the standard textbook Artificial Intelligence: a Modern Approach
Oren Etzioni, CEO of Allen Inst. for AI
Guruduth Banavar, VP, Cognitive Computing, IBM Research
Michael Wooldridge, Oxford, Head of Dept. of Computer Science, Chair of European Coordinating Committee for Artificial Intelligence
Leslie Pack Kaelbling, MIT, Professor of Computer Science and Engineering, founder of the Journal of Machine Learning Research
Tom Mitchell, CMU, former President of AAAI, chair of Machine Learning Department
Toby Walsh, Univ. of New South Wales & NICTA, Professor of AI and President of the AI Access Foundation
Murray Shanahan, Imperial College, Professor of Cognitive Robotics
Michael Osborne, Oxford, Associate Professor of Machine Learning
David Parkes, Harvard, Professor of Computer Science
Laurent Orseau, Google DeepMind
Ilya Sutskever, Google, AI researcher
Blaise Aguera y Arcas, Google, AI researcher
Joscha Bach, MIT, AI researcher
Bill Hibbard, Madison, AI researcher
Steve Omohundro, AI researcher
Ben Goertzel, OpenCog Foundation
Richard Mallah, Cambridge Semantics, Director of Advanced Analytics, AI researcher
Alexander Wissner-Gross, Harvard, Fellow at the Institute for Applied Computational Science
Adrian Weller, Cambridge, AI researcher
Jacob Steinhardt, Stanford, AI Ph.D. student
Nick Hay, Berkeley, AI Ph.D. student
Jaan Tallinn, co-founder of Skype, CSER and FLI
Elon Musk, SpaceX, Tesla Motors
Steve Wozniak, co-founder of Apple
Luke Nosek, Founders Fund
Aaron VanDevender, Founders Fund
Erik Brynjolfsson, MIT, Professor at and director of MIT Initiative on the Digital Economy
Margaret Boden, U. Sussex, Professor of Cognitive Science
Martin Rees, Cambridge, Professor Emeritus of Cosmology and Astrophysics, Gruber & Crafoord laureate
Huw Price, Cambridge, Bertrand Russell Professor of Philosophy
Nick Bostrom, Oxford, Professor of Philosophy, Director of Future of Humanity Institute (Oxford Martin School)
Stephen Hawking, Director of research at the Department of Applied Mathematics and Theoretical Physics at Cambridge, 2012 Fundamental Physics Prize laureate for his work on quantum gravity
Luke Muehlhauser, Executive Director of Machine Intelligence Research Institute (MIRI)
Eliezer Yudkowsky, MIRI researcher, co-founder of MIRI (then known as SIAI)
Katja Grace, MIRI researcher
Benja Fallenstein, MIRI researcher
Nate Soares, MIRI researcher
Paul Christiano, Berkeley, Computer Science graduate student
Anders Sandberg, Oxford, Future of Humanity Institute researcher (Oxford Martin School)
Daniel Dewey, Oxford, Future of Humanity Institute researcher (Oxford Martin School)
Stuart Armstrong, Oxford, Future of Humanity Institute researcher (Oxford Martin School)
Toby Ord, Oxford, Future of Humanity Institute researcher (Oxford Martin School), Founder of Giving What We Can
Neil Jacobstein, Singularity University
Dominik Grewe, Google DeepMind
Roman V. Yampolskiy, University of Louisville
Vincent C. Müller, ACT/Anatolia College
Amnon H Eden, University Essex
Henry Kautz, University of Rochester
Boris Debic, Google, Chief History Officer
Kevin Leyton-Brown, University of British Columbia, Professor of Computer Science
Trevor Back, Google DeepMind
Moshe Vardi, Rice University, editor-in-chief of Communications of the ACM
Peter Sincak, prof. TU Kosice, Slovakia
Tom Schaul, Google DeepMind
Grady Booch, IBM Fellow
Alan Mackworth, Professor of Computer Science, University of British Columbia. Ex AAAI President
Andrew Davison, Professor of Robot Vision, Director of the Dyson Robotics Lab at Imperial College London
Daniel Weld, WRF / TJ Cable Professor of Computer Science & Engineering, University of Washington
Michael Witbrock, Cycorp Inc &
Stephen L. Reed,
Thomas Stone, Co-founder of PredictionIO
Dan Roth, University of Illinois, Editor in Chief of The Journal of AI Research (JAIR)
Babak Hodjat, Sentient Technologies
Vincent Vanhoucke, Google, AI researcher
Itamar Arel, Stanford University, Prof. of Computer Science
Ramon Lopez de Mantaras, Director of the Artificial Intelligence Research Institute, Spanish National Research Council
Antoine Blondeau, Sentient Technologies
George Dvorsky, Contributing Editor, io9; Chair of the Board, Institute for Ethics and Emerging Technologies
George Church, Harvard & MIT
Klaus-Dieter Althoff, University of Hildesheim, Professor of Artificial Intelligence; Head of Competence Center Case-Based Reasoning, German Research Center for Artificial Intelligence, Kaiserslautern; Editor-in-Chief German Journal on Artificial Intelligence
Christopher Bishop, Distinguished Scientist, Microsoft Research
Jen-Hsun Huang, NVIDIA CEO
John Schulman, UC Berkeley & OpenAI
Koichi Takahashi, PI at RIKEN, Co-chair of Whole Brain Architecture Initiative, CIO of Robotic Biology Institute
JT Turner, Knexus Research Corp
Vernor Vinge, San Diego, Professor Emeritus of Computer Science
Steve Crossan, Google
Charina Choi, Google
Matthew Putman, CEO of Nanotronics Imaging
Owain Evans, MIT, Ph.D. student in probabilistic computing
Viktoriya Krakovna, Harvard, Statistics Ph.D. student, FLI co-founder
Janos Kramar, FLI researcher
Ryan Calo, U. Washington, Assistant Professor of Law
Heather Roff Perkins, U. Denver, visiting professor
Tomaso Poggio, Director, Center for Brains, Minds and Machines
Joshua Greene, Harvard, Associate Professor of Psychology
Anthony Aguirre, Santa Cruz, Professor of Physics, co-founder of FLI
Frank Wilczek, MIT, Professor of Physics, Nobel Laureate for his work on the strong nuclear force
Marin Soljacic, MIT, Professor of Physics, McArthur Fellow, Founder of WiTricity
Max Tegmark, MIT, Professor of Physics, co-founder of FLI and FQXi
Meia Chita-Tegmark, Boston University, co-founder of FLI
Michael Vassar, founder of MetaMed and ex-president of MIRI (then known as SIAI)
Seán Ó Héigeartaigh, University of Cambridge, Executive Director, CSER
Andrew Snyder-Beattie, Oxford, Future of Humanity Institute Project Manager (Oxford Martin School)
Cecilia Tilli, Oxford, Future of Humanity Institute researcher (Oxford Martin School)
Geoff Anders, founder of Leverage Research
JB Straubel, co-founder of Tesla
Sam Harris, Project Reason
Ajay Agrawal, U. Toronto
James Manyika, McKinsey
James Moor, Dartmouth
Wendell Wallach, Yale
Sean Legassick, MobGeo
Shamil Chandaria, London U, Institute of Philosophy
Michele Reilly, Turing Inc.
Michael Andregg, Fathom Computing
Ulrich Junker, IBM
Miroslaw Truszczynski, University of Kentucky
Christian Steinruecken, University of Cambridge, graduate student in AI
Mark Waser, Digital Wisdom Institute
Douglas Clark, CEO, Métier
Steven Schmatz, University of Michigan
Corey Henderson, Computer Security Researcher
Jeffrey D. Rupp
Amit Kumar, VP & GM, Yahoo Small Business
Jesus Cepeda, PhD in Robotics and AI, Monterrey, Mexico
Rodolfo Rosini, CEO, Storybricks
CD Athuraliya, Machine learning student, USJP, WSO2
Kathryn McElroy, UX Designer for IBM Watson
Massimo Di Pierro, DePaul University
Anirban Bhattacharya, Computer Science Researcher
Lan Laucirica, SpaceX
Jesse Brown, UC San Francisco, Neuroscience postdoctoral scholar
Barun K Saha, PhD student at IIT Kharagpur
Jonathan Yates, IBM Watson Group EMEA
Sam Richard, UI Architect, IBM Watson
James Miller, Smith College, Author Singularity Rising
Joel Pitt, Independent Researcher (ex-OpenCog)
Achu Wilson, C.T.O Sastra Robotics
Jiří Tulach, CTO, Position s.r.o.
Alexandru Litoiu, Yale University
Mark Watson, Author and consultant specializing in artificial intelligence
Michael Kuhlmann, Colony Networks
George Kachergis, Postdoctoral researcher at New York University
Brian Driscoll, Sr. Systems Engineer, Osprey Software Development
Louis Choquel, Entrepreneur, Software Engineer
Roberto Paura, Italian Institute for the Future
Soheil Yasrebi, Loverino Inc.
David Duvenaud, Harvard University
James Babcock, Praxamed
Peter Marshall,, CEO
Marc Bejarano
Igor Trajkovski,
Appu Shaji, Head, R&D, EyeEm
Tim Daly, CMU LTI Senior Research Programmer
Stefan Schubert, LSE Philosophy
Colin Lewis, RobotEnomics
Jared Peters, co-founder of Origamir Robotics
Darryl McAdams, Language Engine
Mike Slinn, Micronautics Research
Tsvi Benson-Tilsen, University of Chicago, MIRI associate
Nathaniel Thomas, Stanford University, PhD student in quantum computing
Kyle Lussier, Founder / CEO of and Countervaillance
Marek Rosa, CEO at Keen Software House
Diana Hu, Data Scientist, OnCue TV
Alejandro Machado, Carnegie Mellon University, graduate student
Max Kesin, Palantir, ML developer
Alexandros Marinos, CEO,
Patrick LaVictoire, MIRI research associate
Michael Warner, AI researcher
John Hering, Lookout
Ronnie Vuine, Micropsi industries
Chris Nicholson, Skymind
Rene Verheij, AI programmer
Rudy Krol, Amazon Web Services
Simon Hughes, PhD Candidate Machine Learning, DePaul
Aneesh Subramanian, University of Oxford
Jon Baer, AI researcher
James McDermott, University College Dublin
Zavain Dar, VC and Lecturer
Derek Brown, LinkedIn, Addepar
Gabriel Synnaeve, Ecole Normale Supérieure / EHESS
Denny Vrandecic, Google, Founder of Wikidata
Robert W. Williams, Univ Tenn & Human Brain Project
Peteris Erins, Consultant at McKinsey & Company
Anubhav Ashok, University of Texas at Austin, Student and Apple Intern 2014
Naomi Moneypenny, AI Researcher & Chief Technology Officer, ManyWorlds, Inc
David Cieslak, Aunalytics
Stephan Zuchner, U of Miami, Professor and Chair for Human Genetics; Co-founder The Genesis Project and ViaGenetics Inc
Evan Goldschmidt, Google
Anna Salamon, Center for Applied Rationality
Mark Koltko-Rivera, The Ontos Companies
John Hammersley, co-founder of Overleaf / WriteLaTeX
Malcolm Greaves, CMU
Rob Bensinger, MIRI researcher
Marcello Herreshoff, MIRI research associate, Google
Paul Pallaghy, Neo AI Systems P/L, Melbourne, Australia
Percy Liang, Stanford, AI researcher
Theresa Carbonneau, STG
Gert de Cooman, Ghent University
Nicholas Kong, Google
Jeff Nelson, Founder, Chromebook project @ Google
Christian Kaiser, Order of Magnitude Labs
Gabriel Garrett, Artificial Consciousness Engineer
Miles Brundage, Arizona State University
Matthew Luciw, Boston University, Neurala, AI researcher
Vijay Saraswat, IBM TJ Watson Research Center
Ben Hamner, Chief Science Officer, Kaggle
William Eden, Vice President, Thiel Capital
Dan Von Kohorn, v2 Ratings
Nicholas Haan, Singularity University
Kristian Rönn, CEO and co-founder of Meta Mind AB, previously Projects Officer at the Future of Humanity Institute

To see the full list, click here.