“To make AI systems broadly useful for challenging real-world tasks, we need them to learn complex human goals and preferences. One approach to specifying complex goals asks humans to judge during training which agent behaviors are safe and useful, but this approach can fail if the task is too complicated for a human to directly judge. To […]
“How do we combine concepts to form thoughts? How can the same thought be represented in terms of words versus things that you can see or hear in your mind’s eyes and ears? How does your brain distinguish what it’s thinking about from what it actually believes? If I tell you a made up story, yesterday […]
Three generals are voting on whether to attack or retreat from their siege of a castle. One of the generals is corrupt and two of them are not. What happens when the corrupted general sends different answers to the other two generals? A Byzantine fault is “a condition of a computer system, particularly distributed computing systems, where components […]
What motivates cooperative inverse reinforcement learning? What can we gain from recontextualizing our safety efforts from the CIRL point of view? What possible role can pre-AGI systems play in amplifying normative processes? Cooperative Inverse Reinforcement Learning with Dylan Hadfield-Menell is the eighth podcast in the AI Alignment Podcast series, hosted by Lucas Perry and […]
What role does inverse reinforcement learning (IRL) have to play in AI alignment? What issues complicate IRL and how does this affect the usefulness of this preference learning methodology? What sort of paradigm of AI alignment ought we to take up given such concerns? Inverse Reinforcement Learning and the State of AI Alignment with Rohin […]
Are there such things as moral facts? If so, how might we be able to access them? Peter Singer started his career as a preference utilitarian and a moral anti-realist, and then over time became a hedonic utilitarian and a moral realist. How does such a transition occur, and which positions are more defensible? […]
How are we to make progress on AI alignment given moral uncertainty? What are the ideal ways of resolving conflicting value systems and views of morality among persons? How ought we to go about AI alignment given that we are unsure about our normative and metaethical theories? How should preferences be aggregated and persons idealized […]
What role does metaethics play in AI alignment and safety? How might paths to AI alignment change given different metaethical views? How do issues in moral epistemology, motivation, and justification affect value alignment? What might be the metaphysical status of suffering and pleasure? What’s the difference between moral realism and anti-realism and how is each […]
What role does cyber security play in AI alignment and safety? What is AI completeness? What is the space of mind design and what does it tell us about AI safety? How does the possibility of machine qualia fit into this space? Can we leak proof the singularity to ensure we are able to test […]
In a classic taxonomy of risks developed by Nick Bostrom (seen below), existential risks are characterized as risks which are both terminal in severity and transgenerational in scope. If we were to maintain the scope of a risk as transgenerational and increase its severity past terminal, what would such a risk look like? What would […]
Inverse Reinforcement Learning and Inferring Human Preferences is the first podcast in the new AI Alignment series, hosted by Lucas Perry. This series will be covering and exploring the AI alignment problem across a large variety of domains, reflecting the fundamentally interdisciplinary nature of AI alignment. Broadly, we will be having discussions with technical and […]
Value alignment has thus far been primarily conceived as an area of AI safety research that focuses on aligning AI’s actions and goals with human values or what is deemed to be “the good”. It is the study of creating complex machine learning processes which inquire about and converge on better and better ethics and […]
“Didn’t the threat of nuclear holocaust end with the cold war?” “Actually no, the threat is even greater now than ever before.“ There are approximately 15,000 nuclear weapons on Earth, about 1,800 of which are on hair-trigger alert, ready to be launched within minutes – which has nearly happened by mistake during a large number of close […]
In the shadow left by the attacks on Lebanon, Paris, and Iraq, hundreds met this past Saturday for a Massachusetts Peace Action conference to discuss building sustainable security. Various panels, which included speakers such as Noam Chomsky, Chung-Wha Hong, and Jamie Eldrige, explored the current socio-economic and political landscape in search of meaning and feasible goals […]
In an article titled “The New Utopians,” Jeet Heer reflects upon humanity’s dark predictions of dystopia and bright dreams of utopia. While human beings have always longed for a more perfect world, Heer notes that contemporary culture’s imagination has lost its optimism of utopia and is now dominated by dystopian conceptions of the future. This shift […]
There is new evidence that the concentration of greenhouse gases in the atmosphere has passed another milestone. The United Nations weather agency recently released a report that finds the atmospheric concentrations of carbon dioxide have reached 397.7 parts per million (ppm) in 2014, which is substantially higher than the 350ppm level deemed safe by scientists. […]
Photo from the original article. Frank Hoffman has recently coined the term “Pink Flamingo,” which refers to fully visible events and situations that are ignored and then yield catastrophic results. In a recent article about the India/Pakistan situation, David Barno and Nora Bensahel argue that “the current nuclear standoff between India and Pakistan may be […]