Lucas: Hey everyone. Welcome back to the AI Alignment Podcast. I'm Lucas Perry, and today we'll be speaking with Joshua Greene about his research on human cognition as well as John Rawls' veil of ignorance and social choice. Studying the human cognitive engine can help us better understand the principles of intelligence, and thereby aid us in arriving at beneficial AGI. It can also inform group choice and how to modulate persons' dispositions to certain norms or values, and thus affect policy development in observed choice. Given this, we discussed Josh's ongoing projects and research regarding the structure, relations, and kinds of thought that make up human cognition, key features of intelligence such as it being combinatorial and multimodal, and finally how a particular thought experiment can change how impartial a person is, and thus what policies they support.

And as always, if you enjoy this podcast, please give it a like, share it with your friends, and follow us on your preferred listening platform. As a bit of announcement, the AI Alignment Podcast will be releasing every other Wednesday instead of once a month, so there are a lot more great conversations on the way. Josh Greene is a professor of psychology at Harvard, who focuses on moral judgment and decision making. His recent work focuses on cognition, and his broader interests include philosophy, psychology and neuroscience. And without further ado, I give you Josh Greene.

What sort of thinking has been predominantly occupying the mind of Josh Greene?

Joshua: My lab has two different main research areas that are related, but on a day to day basis are pretty separate. You can think of them as focused on key aspects of individual intelligence versus collective intelligence. On the individual intelligence side, what we're trying to do is understand how our brains are capable of high level cognition. In technical terms, you can think of that as compositional semantics, or multimodal compositional semantics. What that means in more plain English is how does the brain take concepts and put them together to form a thought, so you can read a sentence like the dog chased the cat, and you understand that it means something different from the cat chased the dog. The same concepts are involved, dog and cat and chasing, but your brain can put things together in different ways in order to produce a different meaning.

Lucas: The black box for human thinking and AGI thinking is really sort of this implicit reasoning that is behind the explicit reasoning, that it seems to be the most deeply mysterious, difficult part to understand.

Joshua: Yeah. A lot of where machine learning has been very successful has been on the side of perception, recognizing objects, or when it comes to going from say vision to language, simple labeling of scenes that are already familiar, so you can show an image of a dog chasing a cat and maybe it'll say something like dog chasing cat, or at least we get that there's a dog running and a cat chasing.

Lucas: Right. And the caveat is that it takes a massive amount of training, where it's not one shot learning, it's you need to be shown a cat chasing a dog a ton of times just because of how inefficient the algorithms are.

Joshua: Right. And the algorithms don't generalize very well. So if I show you some crazy picture that you've never seen before where it's a goat and a dog and Winston Churchill all wearing roller skates in a rowboat on a purple ocean, a human can look at that and go, that's weird, and give a description like the one I just said. Whereas today's algorithms are going to be relying on brute statistical associations, and that's not going to cut it for getting a precise, immediate reasoning. So humans have this ability to have thoughts, which we can express in words, but we also can imagine in something like pictures.

And the tricky thing is that it seems like a thought is not just an image, right? So to take an example that I think comes from Daniel Dennett, if you hear the words yesterday my uncle fired his lawyer, you might imagine that in a certain way, maybe you picture a guy in a suit pointing his finger and looking stern at another guy in a suit, but you understand that what you imagined doesn't have to be the way that that thing actually happened. The lawyer could be a woman rather than a man. The firing could have taken place by phone. The firing could have taken place by phone while the person making the call was floating in a swimming pool and talking on a cell phone, right?

The meaning of the sentence is not what you imagined. But at the same time we have the symbol grounding problem, that is it seems like meaning is not just a matter of symbols chasing each other around. You wouldn't really understand something if you couldn't take those words and attach them meaningfully to things that you can see or touch or experience in a more sensory and motor kind of way. So thinking is something in between images and in between words. Maybe it's just the translation mechanism for those sorts of things, or maybe there's a deeper language of thought to use, Jerry Fodor's famous phrase. But in any case, what part of my lab is trying to do is understand how does this central really poorly understood aspect of human intelligence work? How do we combine concepts to form thoughts? How can the same thought be represented in terms of words versus things that you can see or hear in your mind's eyes and ears?

How does your brain distinguish what it's thinking about from what it actually believes? If I tell you a made up story, yesterday I played basketball with LeBron James, maybe you'd believe me, and then I say, oh I was just kidding, didn't really happen. You still have the idea in your head, but in one case you're representing it as something true, in another case you're representing it as something false, or maybe you're representing it as something that might be true and you're not sure. For most animals, the ideas that get into its head come in through perception, and the default is just that they are beliefs. But humans have the ability to entertain all kinds of ideas without believing them. You can believe that they're false or you could just be agnostic, and that's essential not just for idle speculation, but it's essential for planning. You have to be able to imagine possibilities that aren't yet actual.

So these are all things we're trying to understand. And then I think the project of understanding how humans do it is really quite parallel to the project of trying to build artificial general intelligence.

Lucas: Right. So what's deeply mysterious here is the kinetics that underlie thought, which is sort of like meta-learning or meta-awareness, or how it is that we're able to have this deep and complicated implicit reasoning behind all of these things. And what that actually looks like seems deeply puzzling in sort of the core and the gem of intelligence, really.

Joshua: Yeah, that's my view. I think we really don't understand the human case yet, and my guess is that obviously it's all neurons that are doing this, but these capacities are not well captured by current neural network models.

Lucas: So also just two points of question or clarification. The first is this sort of hypothesis that you proposed, that human thoughts seem to require some sort of empirical engagement. And then what was your claim about animals, sorry?

Joshua: Well animals certainly show some signs of thinking, especially some animals like elephants and dolphins and chimps engage in some pretty sophisticated thinking, but they don't have anything like human language. So it seems very unlikely that all of thought, even human thought, is just a matter of moving symbols around in the head.

Lucas: Yeah, it's definitely not just linguistic symbols, but it still feels like conceptual symbols that have structure.

Joshua: Right. So this is the mystery, human thought, you could make a pretty good case that symbolic thinking is an important part of it, but you could make a case that symbolic thinking can't be all it is. And a lot of people in AI, most notably DeepMind, have taken the strong view and I think it's right, that if you're really going to build artificial general intelligence, you have to start with grounded cognition, and not just trying to build something that can, for example, read sentences and deduce things from those sentences.

Lucas: Right. Do you want to unpack what grounded cognition is?

Joshua: Grounded cognition refers to a representational system where the representations are derived, at least initially, from perception and from physical interaction. There's perhaps a relationship with empiricism in the broader philosophy of science, but you could imagine trying to build an intelligent system by giving it lots and lots and lots of words, giving it lots of true descriptions of reality, and giving it inference rules for going from some descriptions to other descriptions. That just doesn't seem like it's going to work. You don't really understand what apple means unless you have some sense of what an apple looks like, what it feels like, what it tastes like, doesn't have to be all of those things. You can know what an apple is without ever eaten one, or I could describe some fruit to you that you've never seen, but you have experience with other fruits or other physical objects. Words don't just exist in a symbol storm vacuum. They're related to things that we see and touch and interact with.

Lucas: I think for me, just going most foundationally, the question is before I know what an apple is, do I need to understand spatial extension and object permanence? I have to know time, I have to have some very basic ontological understanding and world model of the universe.

Joshua: Right. So we have some clues from human developmental psychology about what kinds of representations, understandings, capabilities humans acquire, and in what order. To state things that are obvious, but nevertheless revealing, you don't meet any humans who understand democratic politics before they understand objects.

Lucas: Yes.

Joshua: Right?

Lucas: Yeah.

Joshua: Which sounds obvious and it is in a sense obvious, right? But it tells you something about what it takes to build up abstract and sophisticated understandings of the world and possibilities for the world.

Lucas: Right. So for me it seems that the place where grounded cognition is most fundamentally is in between when like the genetic code that seeds the baby and when the baby comes out, the epistemics and whatever is in there, has the capacity to one day potentially become Einstein. So like what is that grounded cognition in the baby that underlies this potential to be a quantum physicist or a scientist-

Joshua: Or even just a functioning human. 

Lucas: Yeah.

Joshua: I mean even people with mental disabilities walk around and speak and manipulate objects. I think that in some ways the harder question is not how do we get from normal human to Einstein, but how do we get from a newborn to a toddler? And the analogous or almost analogous question for artificial intelligence is how do you go from a neural network that has some kind of structure, have some that's favorable for acquiring useful cognitive capabilities, and how do you figure out what the starting structure is, which is kind of analogous to the question of how does the brain get wired up in utero?

And it gets connected to these sensors that we call eyes and ears, and it gets connected to these effectors that we call hands and feet. And it's not just a random blob of connectoplasm, the brain has a structure. So one challenge for AI is what's the right structure for acquiring sophisticated intelligence, or what are some of the right structures? And then what kind of data, what kind of training, what kind of training process do you need to get there?

Lucas: Pivoting back into the relevance of this with AGI, there is like you said, this fundamental issue of grounded cognition that babies and toddlers have that sort of lead them to become full human level intelligences eventually. How does one work to isolate the features of grounded cognition that enable babies to grow and become adults?

Joshua: Well, I don't work with babies, but I can tell you what we're doing with adults, for example.

Lucas: Sure.

Joshua: In the one paper in this line of research we already have published, this is work led by Steven Franklin, we have people reading sentences like the dog chased the cat, the cat chased the dog, or the dog was chased by the cat and the cat was chased by the dog. And what we're doing is looking for parts of the brain where the pattern is different depending on whether the dog is chasing the cat and the cat is chasing the dog. So it has to be something that's not just involved in representing dog or cat or chasing, but of representing that composition of those three concepts where they're composed in one way rather than another way. And we found is that their region in the temporal lobe where the pattern is different for those things.

And more specifically, what we've found is that in one little spot in this broader region in the temporal lobe, you can better than chance decode who the agent is. So if it's the dog chased the cat, then in this spot you can better than chance tell that it's dog that's doing the chasing. If it's the cat was chased by the dog, same thing. So it's not just about the order of the words, and then you can decode better than chance that it's cat being chased for a sentence like that. So the idea is that these spots in the temporal lobe are functioning like data registers, and representing variables rather than specific values. That is this one region is representing the agent who did something and the other region is representing the patient, as they say in linguistics, who had something done to it. And this is starting to look more like a computer program where the way classical programs work is they have variables and values.

Like if you were going to write a program that translates Celsius into Fahrenheit, what you could do is construct a giant table telling you what Fahrenheit value corresponds to what Celsius value. But the more elegant way to do it is to have a formula where the formula has variables, right? You put in the Celsius value, you multiply it by the right thing and you get the Fahrenheit value. And then what that means is that you're taking advantage of that recurring structure. Well, the something does something to something else is a recurring structure in the world and in our thought. And so if you have something in your brain that has that structure already, then you can quickly slot in dog as agent, chasing as the action, cat as patient, and that way you can very efficiently and quickly combine new ideas. So the upshot of that first work is that it seems like when we're representing the meaning of a sentence, we're actually doing it in a more classical computer-ish way than a lot of neuroscientists might have thought.

Lucas: It's Combinatorial.

Joshua: Yes, exactly. So what we're trying to get at is modes of composition. In that experiment, we did it with sentences. In an experiment we're now doing, this is being led by my grad student Dylan Plunkett, and Steven Franklin is also working on this, we're doing it with words and with images. We actually took a bunch of photos of different people doing different things. Specifically we have a chef which we also call a cook, and we have a child which we also call a kid. We have a prisoner, which we also call an inmate, and we have male and female versions of each of those. And sometimes one is chasing the other and sometimes one is pushing the other. In the images, we have all possible combinations of the cook pushes the child, the inmate chases the chef-

Lucas: Right, but it's also gendered.

Joshua: We have male and female versions for each. And then we have all the possible descriptions. And in the task what people have to do is you put two things on the screen and you say, do these things match? So sometimes you'll have two different images and you have to say, do those images have the same meaning? So it could be a different chef chasing a different kid, but if it's a chef chasing a kid in both cases, then you would say that they mesh. Whereas if it's a chef chasing an inmate, then you'd say that they don't. And then in other cases you would have two sentences, like the chef chased the kid, or it could be the child was chased by the cook, or was pursued by the cook, and even though those are all different words in different orders, you've recognized that they have the same meaning or close enough.

And then in the most interesting case, we have an image and a set of words, which you can think of it as as a description, and the question is, does it match? So if you see a picture of a chef chasing a kid, and then the words are chef chases kid or cook pursues child, then you'd say, okay, that one's a match. And what we're trying to understand is, is there something distinctive that goes on in that translation process when you have to take a complex thought, not complex in the sense of very sophisticated by human standards, but complex in the sense that it has parts, that it's composite, and translate it from a verbal representation to a visual representation, and is that different or is the base representation visual? So for example, one possibility is when you get two images, if you're doing something that's fairly complicated, you have to translate them both into words. It's possible that you could see language areas activated when people have to look at two images and decide if they match. Or maybe not. Maybe you can do that in a purely visual kind of way-

Lucas: And maybe it depends on the person. Like some meditators will report that after long periods of meditation, certain kinds of mental events happen much less or just cease, like images or like linguistic language or things like that.

Joshua: So that's possible. Our working assumption is that basic things like understanding the meaning of the chef chased the kid, and being able to point to a picture of that and say that's the thing, the sentence described, that our brains do this all more or less than the same way. That could be wrong, but our goal is to get at basic features of high level cognition that all of us share.

Lucas: And so one of these again is this combinatorial nature of thinking.

Joshua: Yes. That I think is central to it. That it is combinatorial or compositional, and that it's multimodal, that you're not just combining words with other words, you're not just combining images with other images, you're combining concepts that are either not tied to a particular modality or connected to different modalities.

Lucas: They're like different dimensions of human experience. You can integrate it with if you can feel it, or some people are synesthetic, or like see it or it could be a concept, or it could be language, or it could be heard, or it could be subtle intuition, and all of that seems to sort of come together. Right?

Joshua: It's related to all those things.

Lucas: Yeah. Okay. And so sorry, just to help me get a better picture here of how this is done. So this is an MRI, right?

Joshua: Yeah.

Lucas: So for me, I'm not in this field and I see generally the brain is so complex that our resolution is just different areas of the brain light up, and so we understand what these areas are generally tasked for, and so we can sort of see how they relate when people undergo different tasks. Right?

Joshua: No, we can do better than that. So that was kind of brain imaging 1.0, and brain imaging 2.0 is not everything we want from a brain imaging technology, but it does take us a level deeper, which is to say instead of just saying this brain region is involved, or it ramps up when people are doing this kind of thing, region function relationships, we can look at the actual encoding of content, I can train a pattern classifier. So let's say you're showing people pictures of dog or the word dog versus other things. You can train a pattern classifier to recognize the difference between someone looking at a dog versus looking at a cat, or reading the word dog or reading the word cat. There are patterns of activity that are more subtle than just this region is active or more or less active.

Lucas: Right. So the activity is distinct in a way that when you train the thing on when it looks like people are recognizing cats, then it can recognize that in the future.

Joshua: Yeah.

Lucas: So is there anything besides this multimodal and combinatorial features that you guys have isolated, or that you're looking into, or that you suppose are like essential features of grounded cognition?

Joshua: Well, this is what we're trying to study, and we have the ones that have result that's kind of done and published that I described about representing the meaning of a sentence in terms of representing the agent here and the patient there for that kind of sentence, and we have some other stuff in the pipeline that's getting at the kinds of representations that the brain uses to combine concepts and also to distinguish concepts that are playing different roles. In another set of studies we have people thinking about different objects.

Sometimes they'll think about an object where it's a case where they'd actually get money if it turns out that that object is the one that's going to appear later. It looks like when you think about, say dog, and if it turns out that it's dog under the card, then you'll get five bucks. You see that you were able to decode the dog representation in part of our motivational circuitry, whereas you don't see that if you're just thinking about it. So that's another example, is that things are represented in different places in the brain depending on what function that representation is serving at that time.

Lucas: So with this pattern recognition training that you can do based on how people recognize certain things, you're able to see sort of the sequence and kinetics of the thought.

Joshua: MRI is not great for temporal resolution. So what we're not seeing is how on the order of milliseconds a thought gets put together.

Lucas: Okay. I see.

Joshua: What MRI is better for, it has better spatial resolution and is better able to identify spatial patterns of activity that correspond to representing different ideas or parts of ideas.

Lucas: And so in the future, as our resolution begins to increase in terms of temporal imaging or being able to isolate more specific structures, I'm just trying to get a better understanding of what your hopes are for increased ability of resolution and imaging in the future, and how that might also help to disclose grounded cognition.

Joshua: One strategy for getting a better understanding is to combine different methods. fMRI can give you some indication of where you're representing the fact that it's a dog that you're thinking about as opposed to a cat. But other neuroimaging techniques have better temporal resolution but not as good spatial resolution. So EEG which measures electrical activity from the scalp has millisecond temporal resolution, but it's very blurry spatially. The hope is that you combine those two things and you get a better idea. Now both of these things have been around for more than 20 years, and there hasn't been as much progress as I would have hoped combining those things. Another approach is more sophisticated models. What I'm hoping we can do is say, all right, so we have humans doing this task where they are deciding whether or not these images match these descriptions, and we know that humans do this in a way that enables them to generalize, so that if they see some combination of things they've never seen before.

Joshua: Like this is a giraffe chasing a Komodo Dragon. You've never seen that image before, but you could look at that image for the first time and say, okay, that's a giraffe chasing a Komodo Dragon, at least if you know what those animals look like, right?

Lucas: Yeah.

Joshua: So then you can say, well, what does it take to train a neural network to be able to do that task? And what does it take to train a neural network to be able to do it in such a way that it can generalize to new examples? So if you teach it to recognize Komodo Dragon, can it then generalize such that, well, it learned how to recognize giraffe chases lion, or lion chases giraffe, and so it understands chasing, and it understands lion, and it understands giraffe. Now if you teach it what a Komodo dragon looks like, can it automatically slot that into a complex relational structure?

And so then let's say we have a neural network that we trained, is able to do that. It's not all of human cognition. We assume it's not conscious, but it may capture key features of that cognitive process. And then we look at the model and say, okay, well in real time, what is that model doing and how is it doing it? And then we have a more specific hypothesis that we can go back to the brain and say, well, does the brain do it, something like the way this artificial neural network does it? And so the hope is that by building artificial neural models of these certain aspects of high level cognition, we can better understand human high level cognition, and the hope is that also it will feed back the other way. Where if we look and say, oh, this seems to be how the brain does it, well maybe if you wired up a network like this, what if we mimic that kind of architecture in a neural network and an artificial neural network, does that enable it to solve the problem in a way that it otherwise wouldn't?

Lucas: Right. I mean we already have AGIs, they just have to be created by humans and they live about 80 years, and then they die, and so we already have an existence proof, and the problem really is the brain is so complicated that there are difficulties replicating it on machines. And so I guess the key is how much can our study of the human brain inform our creation of AGI through machine learning or deep learning or like other methodologies.

Joshua: And it's not just that the human brain is complicated, it's that the general intelligence that we're trying to replicate in machines only exists in humans. You could debate the ethics of animal research and sticking electrodes in monkey brains and things like that, but within ethical frameworks that are widely accepted, you can do things to monkeys or rats that help you really understand in a detailed way what the different parts of their brain are doing, right?

But for good reason, we don't do those sorts of studies with humans, and we would understand much, much, much, much more about how human cognition works if we were--

Lucas: A bit more unethical.

Joshua: If we were a lot more unethical, if we were willing to cut people's brains open and say, what happens if you lesion this part of the brain? What happens if you then have people do these 20 tasks? No sane person is suggesting we do this. What I'm saying is that part of the reason why we don't understand it is because it's complicated, but another part of the reason why we don't understand is that we are very much rightly placing ethical limits on what we can do in order to understand it.

Lucas: Last thing here that I just wanted to touch on on this is when I've got this multimodal combinatorial thing going on in my head, when I'm thinking about how like a Komodo dragon is chasing a giraffe, how deep does that combinatorialness need to go for me to be able to see the Komodo Dragon chasing the giraffe? Your earlier example was like a purple ocean with a Komodo Dragon wearing like a sombrero hat, like smoking a cigarette. I guess I'm just wondering, well, what is the dimensionality and how much do I need to know about the world in order to really capture a Komodo Dragon chasing a giraffe in a way that is actually general and important, rather than some kind of brittle, heavily trained ML algorithm that doesn't really know what a Komodo Dragon chasing a giraffe is.

Joshua: It depends on what you mean by really know. Right? But at the very least you might say it doesn't really know it if it can't both recognize it in an image and output a verbal label. That's the minimum, right?

Lucas: Or generalize the new context-

Joshua: And generalize the new cases, right. And I think generalization is key, right. What enables you to understand the crazy scene you described is it's not that you've seen so many scenes that one of them is a pretty close match, but instead you have this compositional engine, you understand the relations, and you understand the objects, and that gives you the power to construct this effectively infinite set of possibility. So what we're trying to understand is what is the cognitive engine that interprets and generates those infinite possibilities?

Lucas: Excellent. So do you want to sort of pivot here into how Rawls' veil of justice fits in here?

Joshua: Yeah. So on the other side of the lab, one side is focused more on this sort of key aspect of individual intelligence. On the more moral and social side of the lab, we're trying to understand our collective intelligence and our social decision making, and we'd like to do research that can help us make better decisions. Of course, what counts is better is always contentious, especially when it comes to morality, but these influences that one could plausibly interpret as better. Right? One of the most famous ideas in moral and political philosophy is John Rawls's idea of the veil of ignorance, where what Rawls essentially said is you want to know what a just society looks like? Well, the essence of justice is impartiality. It's not favoring yourself over other people. Everybody has to play this side by the same rules. It doesn't mean necessarily everybody gets exactly the same outcome, but you can't get special privileges just because you're you.

And so what he said was, well, a just society is one that you would choose if you didn't know who in that society you would be. Even if you are choosing selfishly, but you are constrained to be impartial because of your ignorance. You don't know where you're going to land in that society. And so what Rawls says very plausibly is would you rather be randomly slotted into a society where a small number of people are extremely rich and most people are desperately poor? Or would you rather be slotted into a society where most people aren't rich but are doing pretty well? The answer pretty clearly is you'd rather be slotted randomly into a society where most people are doing pretty well instead of a society where you could be astronomically well off, but most likely would be destitute. Right? So this is all background that Rawls applied this idea of the veil of ignorance to the structure of society overall, and said a just society is one that you would choose if you didn't know who in it you were going to be.

And this sort of captures the idea of impartiality as sort of the core of justice. So what we've been doing recently, and we as this is a project led by Karen Huang and Max Bazerman along with myself, is applying the veil of ignorance idea to more specific dilemmas. So one of the places where we have applied this is with ethical dilemmas surrounding self driving cars. We took a case that was most famously recently discussed by Bonnefon, Sharrif, and Rahwan in their 2016 science paper, The Social Dilemma of Autonomous Vehicles, and the canonical version goes something like you've got an autonomous vehicle, and AV, that is headed towards nine people and nothing is done. It's going to run those nine people over. But it can swerve out of the way and save those nine people, but if it does that, it's going to drive into a concrete wall and kill the passenger inside.

So the question is should the car swerve or should it go straight? Now, you can just ask people. So what do you think the car should do, or would you approve a policy that says that in a situation like this, the car should minimize the loss of life and therefore swerve? What we did is, some people we just had answer the question just the way I posed it, but other people, we had them do a veil of ignorance exercise first. So we say, suppose you're going to be one of these 10 people, the nine on the road or the one in the car, but you don't know who you're going to be.

From a purely selfish point of view, would you want the car to swerve or not, and almost everybody says, I'd rather have the car swerve. I'd rather have a nine out of 10 chance of living instead of a one out of 10 chance of living. And then we asked people, okay, that was a question about what you would want selfishly, if you didn't know who you were going to be. Would you approve of a policy that said that cars in situations like this should swerve to minimize the loss of life.

The people who've gone through the veil of ignorance exercise, they are more likely to approve of the utilitarian policy, the one that aims to minimize the loss of life, if they've gone through that veil of ignorance, exercise first, than if they just answered the question. And we have control conditions where we have them do a version of the veil of ignorance exercise, but where the probabilities are mixed up. So there's no relationship between the probability and the number of people, and that's sort of the tightest control condition, and you still see the effect. The idea is that the veil of ignorance is a cognitive device for thinking about a dilemma in a kind of more impartial kind of way.

And then what's interesting is that people recognize, they do a bit of kind of philosophizing. They say, huh, if I said that what I would want is to have the car swerve, and I didn't know who I was going to be, that's an impartial judgment in some sense. And that means that even if I feel sort of uncomfortable about the idea of a car swerving and killing its passenger in a way that is foreseen, if not intended in the most ordinary sense, even if I feel kind of bad about that, I can justify it because I say, look, it's what I would want if I didn't know who I was going to be. So we've done this with self driving cars, we've done it with the classics of the trolley dilemma, we've done it with a bioethical case involving taking oxygen away from one patient and giving it to nine others, and we've done it with a charity where we have people making a real decision involving real money between a more versus less effective charity.

And across all of these cases, what we find is that when you have people go through the veil of ignorance exercise, they're more likely to make decisions that promote the greater good. It's an interesting bit of psychology, but it's also perhaps a useful tool, that is we're going to be facing policy questions where we have gut reactions that might tell us that we shouldn't do what favors the greater good, but if we think about it from behind a veil of ignorance and come to the conclusion that actually we're in favor of what promotes the greater good at least in that situation, then that can change the way we think. Is that a good thing? If you have consequentialist inclinations like me, you'll think it's a good thing, or if you just believe in the procedure, that is I like whatever decisions come out of a veil of ignorance procedure, then you'll think it's a good thing. I think it's interesting that it affects the way people make the choice.

Lucas: It's got me thinking about a lot of things. I guess a few things are that I feel like if most people on earth had a philosophy education or at least had some time to think about ethics and other things, they'd probably update their morality in really good ways.

Joshua: I would hope so. But I don't know how much of our moral dispositions come from explicit education versus our broader personal and cultural experiences, but certainly I think it's worth trying. Certainly believe in the possibility that, understand, this is why I do research on and I come to that with some humility about how much that by itself can accomplish. I don't know.

Lucas: Yeah, it would be cool to see like the effect size of Rawls's veil of ignorance across different societies and persons, and then other things you can do are also like the child drowning in the shallow pool argument, and there's just tons of different thought experiments, it would be interesting to see how it updates people's ethics and morality. The other thing I just sort of wanted to inject here, the difference between naive consequentialism and sophisticated consequentialism. Sophisticated consequentialism would also take into account not only the direct effect of saving more people, but also how like human beings have arbitrary partialities to what I would call a fiction, like rights or duties or other things. A lot of people share these, and I think within our sort of consequentialist understanding and framework of the world, people just don't like the idea of their car smashing into walls. Whereas yeah, we should save more people.

Joshua: Right. And as Bonnefon and all point out, and I completely agree, if making cars narrow the utilitarian in the sense that they always try to minimize the loss of life, makes people not want to ride in them, and that means that there are more accidents that lead to human fatalities because people are driving instead of being driven, then that is bad from a consequentialist perspective, right? So you can call it sophisticated versus naive consequentialism, but really there's no question that utilitarianism or consequentialism in its original form favors the more sophisticated readings. So it's kind of more-

Lucas: Yeah, I just feel that people often don't do the sophisticated reasoning, and then they come to conclusions.

Joshua: And this is why I've attempted with not much success, at least in the short term, to rebrand utilitarianism as what I call deep pragmatism. Because I think when people hear utilitarianism, what they imagine is everybody walking around with their spreadsheets and deciding what should be done based on their lousy estimates of the greater good. Whereas I think the phrase deep pragmatism gives you a much clearer idea of what it looks like to be utilitarian in practice. That is you have to take into account humans as they actually are, with all of their biases and all of their prejudices and all of their cognitive limitations.

When you do that, it's obviously a lot more subtle and flexible and cautious than-

Lucas: Than people initially imagine.

Joshua: Yes, that's right. And I think utilitarian has a terrible PR problem, and my hope is that we can either stop talking about the U philosophy and talk instead about deep pragmatism, see if that ever happens, or at the very least, learn to avoid those mistakes when we're making serious decisions.

Lucas: The other very interesting thing that this brings up is that if I do the veil of ignorance thought exercise, and then I'm more partial towards saving more people and partial towards policies, which will reduce the loss of life. And then I sort of realize that I actually do have this strange arbitrary partiality, like my car I bought not crash me into a wall, from sort of a third person point of view, I think maybe it seems kind of irrational because the utilitarian thing initially seems most rational. But then we have the chance to reflect as persons, well maybe I shouldn't have these arbitrary beliefs. Like maybe we should start updating our culture in ways that gets rid of these biases so that the utilitarian calculations aren't so corrupted by scary primate thoughts.

Joshua: Well, so I think the best way to think about it is how do we make progress? Not how do we radically transform ourselves into alien beings who are completely impartial, right. And I don't think it's the most useful thing to do. Take the special case of charitable giving, that you can turn yourself into a happiness pump, that is devote all of your resources to providing money for the world's most effective charities.

And you may do a lot of good as an individual compared to other individuals if you do that, but most people are going to look at you and just say, well that's admirable, but it's super extreme. That's not for me, right? Whereas if you say, I give 10% of my money, that's an idea that can spread, that instead of my kids hating me because I deprived them of all the things that their friends had, they say, okay, I was brought up in a house where we give 10% and I'm happy to keep doing that. Maybe I'll even make it 15. You want norms that are scalable, and that means that your norms have to feel livable. They have to feel human.

Lucas: Yeah, that's right. We should be spreading more deeply pragmatic approaches and norms.

Joshua: Yeah. We should be spreading the best norms that are spreadable.

Lucas: Yeah. There you go. So thanks so much for joining me, Joshua.

Joshua: Thanks for having me.

Lucas: Yeah, I really enjoyed it and see you again soon.

Joshua: Okay, thanks.

Lucas: If you enjoyed this podcast, please subscribe, give it a like, or share it on your preferred social media platform. We'll be back again soon with another episode of the AI Alignment Series.