Research updates

• A new paper: “Parametric Bounded Löb’s Theorem and Robust Cooperation of Bounded Agents”

  Abstract: Löb’s theorem and Gödel’s theorem make predictions about the behavior of systems capable of self-reference with unbounded computational resources with which to write and evaluate proofs. However, in the real world, systems capable of self-reference will have limited memory and processing speed, so in this paper we introduce an effective version of Löb’s theorem which is applicable given such bounded resources. These results have powerful implications for the game theory of bounded agents who are able to write proofs about themselves and one another, including the capacity to out-perform classical Nash equilibria and correlated equilibria, attaining mutually cooperative program equilibrium in the Prisoner’s Dilemma. Previous cooperative program equilibria studied by Tennenholtz and Fortnow have depended on tests for program equality, a fragile condition, whereas “Löbian” cooperation is much more robust and agnostic of the opponent’s implementation.

• New at the Intelligent Agents Foundations Forum (IAFF):
  What Does it Mean for Correct Operation to Rely on Transfer Learning?
  Virtual Models of Virtual AIs in Virtual Worlds

General updates

• We’re currently accepting applicants to two programs we’re running in June: our 2016 Summer Fellows program (details), and a new Colloquium Series on Robust and Beneficial AI (details).
• MIRI has a new second-in-command: Malo Bourgon.
• We’re hiring! Apply here for our new research position in type theory.
• AI Impacts is asking for examples of concrete tasks AI systems can’t yet achieve. You can also submit these tasks to Phil Tetlock, who is making the same request for Good Judgment Open.
• MIRI senior researcher Eliezer Yudkowsky discusses his core AI concerns with Bryan Caplan. (See Caplan’s response and Yudkowsky’s follow-up.)
• Yudkowsky surveys lessons from game-playing AI.