Entries by Rob Bensinger

MIRI March 2017 Newsletter

Research updates New at IAFF: Some Problems with Making Induction Benign; Entangled Equilibria and the Twin Prisoners’ Dilemma; Generalizing Foundations of Decision Theory New at AI Impacts: Changes in Funding in the AI Safety Field; Funding of AI Research MIRI Research Fellow Andrew Critch has started a two-year stint at UC Berkeley’s Center for Human-Compatible […]

MIRI December 2016 Newsletter

We’re in the final weeks of our push to cover our funding shortfall, and we’re now halfway to our $160,000 goal. For potential donors who are interested in an outside perspective, Future of Humanity Institute (FHI) researcher Owen Cotton-Barratt has written up why he’s donating to MIRI this year. (Donation page.)Research updates New at IAFF: postCDT: […]

MIRI’S November 2016 Newsletter

Post-fundraiser update: Donors rallied late last month to get us most of the way to our first fundraiser goal, but we ultimately fell short. This means that we’ll need to make up the remaining $160k gap over the next month if we’re going to move forward on our 2017 plans. We’re in a good position […]

MIRI October 2016 Newsletter

The following newsletter was originally posted on MIRI’s website. Our big announcement this month is our paper “Logical Induction,” introducing an algorithm that learns to assign reasonable probabilities to mathematical, empirical, and self-referential claims in a way that outpaces deduction. MIRI’s 2016 fundraiser is also live, and runs through the end of October. Research updates […]

MIRI September 2016 Newsletter

Research updates New at IAFF: Modeling the Capabilities of Advanced AI Systems as Episodic Reinforcement Learning; Simplified Explanation of Stratification New at AI Impacts: Friendly AI as a Global Public Good We ran two research workshops this month: a veterans’ workshop on decision theory for long-time collaborators and staff, and a machine learning workshop focusing on generalizable environmental goals, impact […]

MIRI August 2016 Newsletter

Research updates A new paper: “Alignment for Advanced Machine Learning Systems.” Half of our research team will be focusing on this research agenda going forward, while the other half continues to focus on the agent foundations agenda. New at AI Impacts: Returns to Scale in Research Evan Lloyd represented MIRIxLosAngeles at AGI-16 this month, presenting […]

MIRI July 2016 Newsletter

Research updates A new paper: “A Formal Solution to the Grain of Truth Problem.” The paper was presented at UAI-16, and describes the first general reduction of game-theoretic reasoning to expected utility maximization. Participants in MIRI’s recently-concluded Colloquium Series on Robust and Beneficial AI (CSRBAI) have put together AI safety environments for the OpenAI Reinforcement Learning Gym. Help is welcome creating more […]

MIRI’s June 2016 Newsletter

Research updates New paper: “Safely Interruptible Agents.” The paper will be presented at UAI-16, and is a collaboration between Laurent Orseau of Google DeepMind and Stuart Armstrong of the Future of Humanity Institute (FHI) and MIRI; see FHI’s press release. The paper has received (often hyperbolic) coverage from a number of press outlets, including Business […]

MIRI May 2016 Newsletter

Research updates Two new papers split logical uncertainty into two distinct subproblems: “Uniform Coherence” and “Asymptotic Convergence in Online Learning with Unbounded Delays.” New at IAFF: An Approach to the Agent Simulates Predictor Problem; Games for Factoring Out Variables; Time Hierarchy Theorems for Distributional Estimation Problems We will be presenting “The Value Learning Problem” at […]

MIRI’s April AI News and Updates

Research updates A new paper: “Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents” Abstract: Löb’s theorem and Gödel’s theorem make predictions about the behavior of systems capable of self-reference with unbounded computational resources with which to write and evaluate proofs. However, in the real world, systems capable of self-reference will have limited memory […]

MIRI March Newsletter

Research updates A new paper: “Defining Human Values for Value Learners“ New at IAFF: Analysis of Algorithms and Partial Algorithms; Naturalistic Logical Updates;Notes from a Conversation on Act-Based and Goal-Directed Systems; Toy Model: Convergent Instrumental Goals New at AI Impacts: Global Computing Capacity A revised version of “The Value Learning Problem” (pdf) has been accepted […]

MIRI’s February 2016 Newsletter

This post originally comes from MIRI’s website. Research updates New at IAFF: Thoughts on Logical Dutch Book Arguments; Another View of Quantilizers: Avoiding Goodhart’s Law; Another Concise Open Problem General updates Fundraiser and grant successes: MIRI will be working with AI pioneer Stuart Russell and a to-be-determined postdoctoral researcher on the problem of corrigibility, thanks […]

MIRI’s December Newsletter Is Live!

Research updates New papers: “Formalizing Convergent Instrumental Goals” and “Quantilizers: A Safer Alternative to Maximizers for Limited Optimization.” These papers have been accepted to the AAAI-16 workshop on AI, Ethics and Society. New at AI Impacts: Recently at AI Impacts New at IAFF: A First Look at the Hard Problem of Corrigibility; Superrationality in Arbitrary […]

From the MIRI Blog: “Formalizing Convergent Instrumental Goals”

Tsvi Benson-Tilsen, a MIRI associate and UC Berkeley PhD candidate, has written a paper with contributions from MIRI Executive Director Nate Soares on strategies that will tend to be useful for most possible ends: “Formalizing convergent instrumental goals.” The paper will be presented as a poster at the AAAI-16 AI, Ethics and Society workshop. Steve […]

What to think about machines that think

From MIRI: In January, nearly 200 public intellectuals submitted essays in response to the 2015 Edge.org question, “What Do You Think About Machines That Think?” (available online). The essay prompt began: In recent years, the 1980s-era philosophical discussions about artificial intelligence (AI)—whether computers can “really” think, refer, be conscious, and so on—have led to new […]

MIRI News: October 2015

MIRI’s October Newsletter collects recent news and links related to the long-term impact of artificial intelligence. Highlights: — New introductory material on MIRI can be found on our information page. — An Open Philanthropy Project update discusses investigations into global catastrophic risk and U.S. policy reform. — “Research Suggests Human Brain Is 30 Times As […]