Entries by

MIRI’s February 2018 Newsletter

Updates New at IAFF: An Untrollable Mathematician New at AI Impacts: 2015 FLOPS Prices We presented “Incorrigibility in the CIRL Framework” at the AAAI/ACM Conference on AI, Ethics, and Society. From MIRI researcher Scott Garrabrant: Sources of Intuitions and Data on AGI News and links In “Adversarial Spheres,” Gilmer et al. investigate the tradeoff between test

MIRI’s January 2018 Newsletter

Our 2017 fundraiser was a huge success, with 341 donors contributing a total of $2.5 million! Some of the largest donations came from Ethereum inventor Vitalik Buterin, bitcoin investors Christian Calderon and Marius van Voorden, poker players Dan Smith and Tom and Martin Crowley (as part of a matching challenge), and the Berkeley Existential Risk

MIRI’s December 2017 Newsletter and Annual Fundraiser

Our annual fundraiser is live. Discussed in the fundraiser post: News  — What MIRI’s researchers have been working on lately, and more. Goals — We plan to grow our research team 2x in 2018–2019. If we raise $850k this month, we think we can do that without dipping below a 1.5-year runway. Actual goals — A bigger-picture outline of what we

MIRI’s November 2017 Newsletter

Eliezer Yudkowsky has written a new book on civilizational dysfunction and outperformance: Inadequate Equilibria: Where and How Civilizations Get Stuck. The full book will be available in print and electronic formats November 16. To preorder the ebook or sign up for updates, visit equilibriabook.com. We’re posting the full contents online in stages over the next

MIRI’s July 2017 Newsletter

The following was originally posted here. A number of major mid-year MIRI updates: we received our largest donation to date, $1.01 million from an Ethereum investor! Our research priorities have also shifted somewhat, reflecting the addition of four new full-time researchers (Marcello Herreshoff, Sam Eisenstat, Tsvi Benson-Tilsen, and Abram Demski) and the departure of Patrick LaVictoire and Jessica

MIRI’s June 2017 Newsletter

Research updates A new AI Impacts paper: “When Will AI Exceed Human Performance?” News coverage at Digital Trends and MIT Technology Review. New at IAFF: Cooperative Oracles; Jessica Taylor on the AAMLS Agenda; An Approach to Logically Updateless Decisions Our 2014 technical agenda, “Agent Foundations for Aligning Machine Intelligence with Human Interests,” is now available as a

MIRI’s May 2017 Newsletter

Research updates New at IAFF: The Ubiquitous Converse Lawvere Problem; Two Major Obstacles for Logical Inductor Decision Theory; Generalizing Foundations of Decision Theory II. New at AI Impacts: Guide to Pages on AI Timeline Predictions “Decisions Are For Making Bad Outcomes Inconsistent“: Nate Soares dialogues on some of the deeper issues raised by our “Cheating Death in

MIRI’s April 2017 Newsletter

The following post was originally published here. Our newest publication, “Cheating Death in Damascus,” makes the case for functional decision theory, our general framework for thinking about rational choice and counterfactual reasoning.In other news, our research team is expanding! Sam Eisenstat and Marcello Herreshoff, both previously at Google, join MIRI this month. Research updates New at

MIRI March 2017 Newsletter

Research updates New at IAFF: Some Problems with Making Induction Benign; Entangled Equilibria and the Twin Prisoners’ Dilemma; Generalizing Foundations of Decision Theory New at AI Impacts: Changes in Funding in the AI Safety Field; Funding of AI Research MIRI Research Fellow Andrew Critch has started a two-year stint at UC Berkeley’s Center for Human-Compatible

MIRI December 2016 Newsletter

We’re in the final weeks of our push to cover our funding shortfall, and we’re now halfway to our $160,000 goal. For potential donors who are interested in an outside perspective, Future of Humanity Institute (FHI) researcher Owen Cotton-Barratt has written up why he’s donating to MIRI this year. (Donation page.)Research updates New at IAFF: postCDT:

MIRI’S November 2016 Newsletter

Post-fundraiser update: Donors rallied late last month to get us most of the way to our first fundraiser goal, but we ultimately fell short. This means that we’ll need to make up the remaining $160k gap over the next month if we’re going to move forward on our 2017 plans. We’re in a good position

MIRI October 2016 Newsletter

The following newsletter was originally posted on MIRI’s website. Our big announcement this month is our paper “Logical Induction,” introducing an algorithm that learns to assign reasonable probabilities to mathematical, empirical, and self-referential claims in a way that outpaces deduction. MIRI’s 2016 fundraiser is also live, and runs through the end of October. Research updates

MIRI September 2016 Newsletter

Research updates New at IAFF: Modeling the Capabilities of Advanced AI Systems as Episodic Reinforcement Learning; Simplified Explanation of Stratification New at AI Impacts: Friendly AI as a Global Public Good We ran two research workshops this month: a veterans’ workshop on decision theory for long-time collaborators and staff, and a machine learning workshop focusing on generalizable environmental goals, impact

MIRI August 2016 Newsletter

Research updates A new paper: “Alignment for Advanced Machine Learning Systems.” Half of our research team will be focusing on this research agenda going forward, while the other half continues to focus on the agent foundations agenda. New at AI Impacts: Returns to Scale in Research Evan Lloyd represented MIRIxLosAngeles at AGI-16 this month, presenting

MIRI’s June 2016 Newsletter

Research updates New paper: “Safely Interruptible Agents.” The paper will be presented at UAI-16, and is a collaboration between Laurent Orseau of Google DeepMind and Stuart Armstrong of the Future of Humanity Institute (FHI) and MIRI; see FHI’s press release. The paper has received (often hyperbolic) coverage from a number of press outlets, including Business

MIRI May 2016 Newsletter

Research updates Two new papers split logical uncertainty into two distinct subproblems: “Uniform Coherence” and “Asymptotic Convergence in Online Learning with Unbounded Delays.” New at IAFF: An Approach to the Agent Simulates Predictor Problem; Games for Factoring Out Variables; Time Hierarchy Theorems for Distributional Estimation Problems We will be presenting “The Value Learning Problem” at

MIRI’s April AI News and Updates

Research updates A new paper: “Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents” Abstract: Löb’s theorem and Gödel’s theorem make predictions about the behavior of systems capable of self-reference with unbounded computational resources with which to write and evaluate proofs. However, in the real world, systems capable of self-reference will have limited memory

MIRI March Newsletter

Research updates A new paper: “Defining Human Values for Value Learners“ New at IAFF: Analysis of Algorithms and Partial Algorithms; Naturalistic Logical Updates;Notes from a Conversation on Act-Based and Goal-Directed Systems; Toy Model: Convergent Instrumental Goals New at AI Impacts: Global Computing Capacity A revised version of “The Value Learning Problem” (pdf) has been accepted

MIRI’s February 2016 Newsletter

This post originally comes from MIRI’s website. Research updates New at IAFF: Thoughts on Logical Dutch Book Arguments; Another View of Quantilizers: Avoiding Goodhart’s Law; Another Concise Open Problem General updates Fundraiser and grant successes: MIRI will be working with AI pioneer Stuart Russell and a to-be-determined postdoctoral researcher on the problem of corrigibility, thanks

MIRI’s December Newsletter Is Live!

Research updates New papers: “Formalizing Convergent Instrumental Goals” and “Quantilizers: A Safer Alternative to Maximizers for Limited Optimization.” These papers have been accepted to the AAAI-16 workshop on AI, Ethics and Society. New at AI Impacts: Recently at AI Impacts New at IAFF: A First Look at the Hard Problem of Corrigibility; Superrationality in Arbitrary