AI Could Make the Economy Double Every Month | Robin Hanson

Roman Yampolskiy, PhD: Robin, some fifteen years ago you engaged in a debate with Lyazer Yutkowski about AI going and not going foom. Do you feel like you got vindicated by developments since then?

Robin Hanson: well what I hear is that we're about to have foom and it's all gonna happen in the next few years and so whether or not I was right will be determined in a few years, not yet. from that point of view, I gotta say, well, if you think it's about to happen, then then we haven't seen, right? So we we we did, you know, there's a number of different things in that debate that we claimed and positions, and maybe some points were sort of incidental and some were more central. I would say that the central issue was the plausibility of a sudden small explosion that went very far. And If that was possible or plausible, then the implication was that it was plausible that this small, initially small, weak system that would suddenly become very powerful over a short time, during that change it would change its values, it would change its priorities. And therefore, if you wanted to influence, and then once it had these later big priorities, it would just maintain those forever in the universe. And so if you wanted to influence what the values for the universe were. after that explosion, you needed to influence them what they were before that, you know, process. And that's why it was so important to get initial alignment correct for these systems. That was the core argument I thought we were discussing. And so then the issue was, well, how plausible is it that this very small, weak system would suddenly blow up like that. So you can see this is different from the claim that eventually say AI would be really important or maybe AI would dominate. This is about how fast and sudden would that process be and unexpected, so that you couldn't manage it along the way. So If we have a world in which o AI slowly gets better and we all see it get better and then we watch its effects and then we can react to them and we can manage our reactions to that, that's a different sort of world than a world where all of a sudden, boom, it's over. So that's the key question we were debating. And so if they're still saying it's about to happen, it's gonna happen soon, I can't say I just I disprove that it won't happen soon, it hasn't happened yet. I can say, well, so far what we've seen is huge chunks of the world advancing together, large fractions of the world economy having a new burst of investment and together improving. And we are watching it gradually improve, and we are reacting to many things we see about how it changes and improves. And If that were to continue, then I would say I was vindicated because that's what I said. I wasn't saying it wasn't gonna get big or be important. I said it would be the sort of thing like, say, with the Industrial Revolution, where, you know, first there was an industry, then later there was industry. But what happened is first, you know, Manchester and then England and then Northern Europe and then the world together grew with the Industrial Revolution. And, you know, It growth accelerated and now we have this new world. And of course, in this new world today, subsystems farmers don't have much influence today, right? Industry dominates the world today. But it was through a process that was gradual, widespread, and you know, s continuous enough that you could watch and react to it. And we did watch and react to the Industrial Revolution. Now that doesn't mean we controlled it. In some sense, changes like that were never under anybody's control, really. And lots of big changes in history were never under anyone's control, and probably future changes won't really be under anyone's control. But you know, you might hope for an AI revolution that looks something like the Industrial Revolution, where It's widespread, gradual, you can watch, you can react, and then maybe you can choose between holding back and staying with the old style or jumping in and trying to join the new style.

Roman Yampolskiy, PhD: So additional fifteen years of data, any changes in your position whatsoever, or are you just saying we need to wait longer to resolve this?

Robin Hanson: Well, so when we got into the more details, as you may know, the question is about the plausibility of the sudden boom. Yudkowski's argument for that plausibility was based substantially on the idea of big algorithm improvements. That is, you could have an AI with an algorithm, it would also have data and you know, access to the world and other feature and hardware, but the algorithm itself could find such a huge improvement in the algorithm that that would unlock first further algorithm improvements. And that's what would cause this thing that was initially small and weak to within a weekend take over the universe. And so we discussed the plausibility of that leverage power of an algorithm. So I at the time said, well, you know, aside from core algorithms, successful AI systems have so far had lots of other parts that mattered, and that, you know An an advance in an algorithm had a substantial but not overwhelming impact on the power of most AI systems up till that point. And so then I projected forward to say, well, that's what you should expect for these new AI systems. Now, what we've seen is that we have current AI systems that do have algorithms and they are important to it, but the they also have enormous amounts of data and hardware that are Central to their performance, right? It's not just an algorithm that does it. They have to have lots of supporting stuff. So from that point of view, I might say, well, that's roughly what I meant. Now I think Yukowski says, well, you said algorithms were important. Look how important algorithms are. And I said, Well, y I didn't say algorithms weren't important. I was thinking in terms of other sort of modules of a system. You could look at we have these machine learning algorithms, and they take data and they produce systems with a relatively small code base of the algorithm itself. That's true, but I still say what we see today is you can't just have a good algorithm out there and and take over. You I mean good algorithms have to be bundled with lots of hardware and lots of data and lots of, you know, customers, and that's what the leaders have. So I would say I'm vindicated on the claim that A better algorithm isn't by itself gonna make things worse, but of course that's what people are claiming now for the next few years, that that s soon, soon, those better algorithms are gonna make it all go boom. So then we don't know, right?

Roman Yampolskiy, PhD: Absolutely. You were very early to AI research overall, and then you switched to social sciences to economics. Did you feel it was just too far away from human level performance, or what was the motivation there?

Robin Hanson: My intellectual journey, I understand in terms of opportunism. I am searching for neglected important topics where I can find an angle. So that's led me many times to sort of switch to a new field, excitedly earn lots of powerful basics in this field, and then as it runs out, I go, well. Do I have something to add here, or do I want to go into one of these other areas? So I have actually, compared to most intellectuals, certainly switched areas many times, but in the process, I've become a polymath. I've collected many intellectual tools from many areas, and then in some sense, my main contribution becomes noticing connections across these different things I've learned. And I think our intellectual world neglects that greatly, really. And that's one of the powers that LLMs I think might offer in the next decades is connecting different things that we've learned but haven't noticed connections between. That they d they do seem to be much better than humans at all at once mastering a a set of things and then trying to integrate them.

Roman Yampolskiy, PhD: Do you think there are specific areas right now which are sort of those honey pots of amazing ideas which we're not exploring, anything in particular you can point to?

Robin Hanson: I I think just just widely we just failed to find intersections between the so this has, you know, actually been my main intellectual strategy for a while. I would say my my my two books and other areas are where I notice connections between areas where especially connections that are in conflict, and then you know, by resolving those conflicts or addressing them, I can find insight. So I I actually say this is a way to be a a unusually reliable correct contrarian. That is, usually if you're gonna be a contrarian, you gotta accept you're probably wrong. You know, there's a consensus view in different areas and unless you like really work hard, like the consensus view is probably right, right? So It could be worth it for humanity for someone to explore a contrarian view, because it might turn out to be right, and then we got lots of us, so yeah, we should allocate people to exploring different options, but you gotta accept that if you're taking on a contrarian view, that you're probably wrong. But here's an exception, which is because we have all these different areas that we study that we don't make connections between, if you just collect understanding lots of areas and then notice a conflict between two areas, A and B. In a just world you'd point out this conflict to A and B and they'd immediately go, well, gee, we we'll have to change our mind. And they would update it and they wouldn't even give you credit. They just credit the other f field for like noticing that they they have a conflict. But in fact, in our world today, each field will reject this other field's supposed correction to their view, and they will insist on sticking with their view. And so if you just make a better than random estimate of who's right in the conflict, then You go to the side who's wrong and say, but you're wrong about this, and you'll be the contrarian and you'll be right. So that's a way in which it shows that our world is just really neglecting this integration of different fields because we are just leaving all these opportunities out there to find and and resolve conflicts. Anyway, so that's what I would say I did with AI and economics. That is, long ago, AI people saw this future of technology, but most technologists didn't think social scientists knew anything. And so when they tried to project the social implications of a technology, they just kind of made stuff up because they figured that was the best anybody could do. And then the social scientists, they knew a lot of social scientists, but they didn't think the technologists knew what they were talking about in terms of these speculative technologies. They thought that was stupid. And so they would just project the few implications of con you know more conventional projections of technology because that's what they would believe. So I could early on be the intersection person who says, I believe social scientists know something and I believe that technologists have an idea of what's coming, and I can then be one of these unique people who looks at that intersection before other people do. And I think I did that. I was looking at that intersection before others, not just thinking about what say AI might imply, but thinking about what does social science say about what AI might imply, which often is in conflict with what the technologists will say about AI. Still is, unfortunately. But at least now I guess we've got some of the social scientists taking AI seriously to some degree.

Roman Yampolskiy, PhD: So looking at your biggest inventions, creations, betting markets, great filter, can you walk us through how this approach led you to those discoveries?

Robin Hanson: Okay, so let's take prediction markets. I was working with a group trying to create the World Wide Web. So there was a time before Thor's Web, you might remember it, maybe. Long dim in the past. There was a world and we did not have a web. And some of us were imagining that web, and other people said we were crazy. That wouldn't happen. But we thought, no, the web could be a thing. We had a vision for the web. And the we have the vision of it would help make public conversation better by making criticism easier to find. Plausible. Turns out maybe we needed a bit more than that, but because now we have the web, we still have a lot of problems. But that was the vision. And I started to doubt that vision. I said, not so sure that's gonna work. And then I asked, well, what else could we do to improve cover public conversation besides the web? And that's when I thought, how about Betty Markets? And It wasn't I w that was accessible to me because the people I was working with on the web were kinda libertarians. And so this idea I could just take from them and say, well let's just do betting markets instead. And if you go look at the literature on financial markets, what you'll find is the consensus ha at that time had long been and still is that they're exemplary information institutions. That is, it's hard to find information that they haven't included. Head-to-head comparisons, they just beat other alternatives at being in information institutions. That was just the consensus then and still about speculative markets. So I had come from other literature saying, gee, how do we do information institutions, the rest of society? How can we do better? I came from like philosophy of science, social studies of science, where we were asking, how could we make science better? But then, of course, other adjacent areas, how could we make law better? How could we make business better? In much of social analysis, there's this question: how can we have better information aggregation for all these aspects of society? And you go to finance, you go, we have this best, this great thing, and you go, there's this intersection, guys. If this institution is so great, Over there, why not move it somewhere else and use it elsewhere too? That was my great insight, is to say, this thing works great as an information institution. We should try it elsewhere. So you can see that's again the making connections. Because I had been reading about the general problem of how do we make good information institutions in support of, say, the World Wide Web, I understood that the world had middling information institutions. We wanted to do better. I discovered this powerful information institution. I said, let's let's apply it. So that's again the polymath strategy right there from the beginning for prediction markets.

Roman Yampolskiy, PhD: And we had them now for a number of years. They seem to function, well funded. Do we actually use that information for something other than betting and making money? Is it being applied in governance, economics, anywhere else?

Robin Hanson: It's being applied more in the last few years, but there's still a vast future that we could achieve that we aren't achieving yet. So first of all, we we are getting more markets, as you know, in the last couple of years with Calcian Polymarket on world events and economic events. So they are more informative now. Many news are more news articles will refer to the market odds on events using the prices and People in finance and people in government look at market prices more for information, but we could go a lot farther. And in particular, we could go farther with governance. So eventually I went back to school and got a PhD in social science, and I focused on governance, formal political theory, and I realized that we have problems with governance, that is Our governance institutions are limited in the sense that there's a lot of problems they have and ways we wish we could do better. And one of the key features of a governance institution is its ability to aggregate information. And so I realized: well, these prediction markets, if you put them in a governance institution, they might make that institution work much better at aggregating information and figuring out what works. And so that's where I invented the concept of decision markets, or also called futarchy. And so that's the application there is to say, standard decision theory says that information is valuable when it influences decisions. And you can actually set up markets directly next to a decision so that it advises that decision. And for governance decisions, that's going to be especially valuable because most governance is decisions. You know, at the end of the day, governance processes make decisions. And so if you could set up this decision advice process right next to a governance decision, you could make it much more effective. And that's what I call decision markets. And at the moment, there are a number of firms experimenting with that. Crypto firms mainly. So in the last few years, we've had more experimentation with this concept of decision markets for governance. I mean it can also be used for personal decision advice, but I'm especially excited by the governance application. But again, I'm gonna again say this is a polymath thing, which is just People who do decision theory and understand governance, they understand that a key part of governance is making decisions and having enough information aggregated to make good decisions. And they ponder how we do better. And people have discussed committees and voting and polls and all sorts of ways to aggregate information. And you go, but but over here we've got this information institution that does spectacularly well when tested against all these other ones. Let's let's use it. And that's that's my concept. That's my concept. That's the thing I've invented is just this idea of applying this thing that. works well elsewhere. Over here where we really need something to work better.

Roman Yampolskiy, PhD: How do you personally use those markets? Do you get information from it? Do you bet on them? Is there a way you engage with your invention?

Robin Hanson: I I read a lot and I think a lot, but the scope of the topics on which prediction markets have opinions is still very small. So I have at times, you know, created markets, maybe on manifolds, on something that have an opinion, but that's a pretty small rate of things. So I gotta say that The the norm for these markets, I mean so there's two ways to use these markets. One is to just read the price and get informed about something, and the other is to you know participate either by creating a market or trading in a market that exists in order to add your information. And when the topics that I'm thinking about, there just aren't markets on those topics, then I can't get much out of the markets. I can only get them out to when I look at it. But for example, my wife and I for so we we met in eighty. Two Okay. So twenty s four years ago, sorry, forty four years ago.

Roman Yampolskiy, PhD: yeah.

Robin Hanson: yeah, 44 years ago is when my wife and I met. And ever since then, every year we try to watch all the Oscar nominated movies. But we have to start before the nominations come out. So the betting markets lately just tell us what are the likely nominees so that we can make sure to watch them before they come out. Now that's just a practical personal choice, but it is information. on betting in these markets, I think the key norm should be in these markets, you should not participate unless you think you're world class, because that's who you're trading against. World class people on the other side. So unless you think you're world class, stay out of it. And you know, you've got the freedom you have the freedom to participate. You'll

Roman Yampolskiy, PhD: World class or you have insider information?

Robin Hanson: go ahead.

Roman Yampolskiy, PhD: World class or do you have insider information?

Robin Hanson: If you have inside information that makes you world class. On

Roman Yampolskiy, PhD: Ha ha ha.

Robin Hanson: that topic, if you know stuff other people don't, you are world class. But that's the key point. That is, we'll have a selection effect where the people who think they're world class are in this market and that's who you're trading against. So now the question is, are you as good as the people who think they're good enough to be there? And so there's an old saying in poker, you sit down at the table, look around, and find the fool. If you don't find the fool, it's you, walk away. So that's how you should think about these markets too. So that's why I I'm not eagerly trading in markets just because I have an opinion on a subject. I only want to trade when I'm well classed. Now though I do think there are topics where I'm more classed, but they usually don't have markets of those topics.

Roman Yampolskiy, PhD: Do you think there are dangers of markets kinda incentivizing outcome and people trying to make it happen just to collect the winnings? We saw some examples, people manipulating censors, manipulating elections.

Robin Hanson: Sure, but I think these are problem the there's a set of problems that all information institutions have in common. And that includes journalism and gossip and academic journals and government agencies. We have a lot of information institutions in society, and they all have a set of problems that can happen with them. And one of them is they can induce people to change the world to get better treatment by the institution, i.e. induce sabotage. Another is manipulation, they can induce people to try to distort inputs in order to distort the outputs. Third is they can try to extract secret information that people were trying to keep secret, insider information. Another is that they can reveal information when it would be better for the world not to know things. Those are four different problems that information institutions can produce, and all of the information institutions can produce them. So my main policy recommendation is. treat these institutions similarly in how you fix these problems. So for example, some people put a bill before Congress saying don't let government officials trade in the betting markets because they might reveal secrets that way. And I say don't let them talk to reporters either, because they might reveal secrets that way, right? Just realize that all of these information institutions can have a set of problems and that treat them all similarly. Don't dump on prediction markets because they're the new upstart. Trying to, you know, undermine the previous established leader. Let if if they're gonna let them undermine the pr prior leaders if that's what they're able to do.

Roman Yampolskiy, PhD: Do you think there is a theoretical connection between predicting outcome of a market and LLMs predicting the next token?

Robin Hanson: W they're both predictions, yes.

Roman Yampolskiy, PhD: But more fundamentally, can you reduce one to the other? Can you use one without a mission to improve an overall paradigm?

Robin Hanson: So you understand LLMs and L many AF things better than I, so I want to defer to you. But I'll say a focus on prediction markets is to say, let's give the let's make really clear contributions to a forecast and make really clear that they have good incentives in that. And the more than we can just make sure, look, right now, you know, there's a market price and you have the ability to maybe change that price, and you ask yourself, Should I do that and how should I do that? We just want to make sure you have a really nice, clear, strong incentive. We don't know if you know anything or what how to figure that out. That's all up to you. That's complicated. But we just know that the way we set this market up, we gave you a nice, really clear, strong incentive. If you can improve on the price, then you should do it and you will get paid for it on average. So that's what you typically want with something like an LLM. system when you're constructing it, that is you're trying to come up with an algorithm where each time you've got a current system and you've got a big a new piece of data you're looking at, and you're trying to find out an algorithm such that you update the current system in the better direction from the data you're looking at. And the key question is what's that iterative algorithm? What's that core algorithm of the loop of I saw a new piece of data, here's my previous structure, how do I update the structure to accommodate the new data? When you're thinking about that, that's exactly the same question you're asking, which is what's a nice, safe, robust rule to follow to improve the system given that piece of data. That's pretty analogous to what's a good incentive to hand somebody a prediction market to like make the market price better if you might know something to make it better. So that's that's I think that's a kind of close analogy. But what do you think?

Roman Yampolskiy, PhD: It seems like there is a way to automate a lot of it, replace human batteries with agents and have it as a kinda meta level intelligence system trying to make predictions at a level higher than individual token.

Robin Hanson: Well, so let me make the pitch to you. There's a difference between a method and a forum. Okay. A method is a way that you might process your inputs to produce your outputs. A way you make a statistical model or a machine learning machine system or a way that you organize a group of people. You have a method. But we as a society need a forum. That is, different people with different methods are gonna come to us and say, hey, you wanna pay me for my method, it's great. And we need a way to like judge whose method should we believe. So that's when we need a form. So the prediction markets are less of a method than a forum. They're about how do we aggregate all these people who have different claims about how to organize these different systems. So, you know, people have a wide range of ways they structure AI systems, ways they make them, ways they train them. How are we, as a society, supposed to know who to believe about all that? So one way is you if you had up a neutral forum, then people who have these systems, they'll make judgments about how much they trust their systems to contribute, and then we will find out, you know, who's right on the basis of The forum itself rewarding the people who end up being more right in the forum for their contributions. And so we need that in addition to methods, right? We can't we shouldn't just trust one organization to say, we have the great AI, so you let's just let let us run the world with it. You know, give up all your autonomy and all of your control and all of your even just oversight of everything 'cause h we have such a great AI system. You couldn't possibly do better than our AI system. We might say, you might, but how are we supposed to know? Like How are we supposed to trust that you have such a great AI system? Because these other people they say they have great AI systems. Who who are you supposed to believe?

Roman Yampolskiy, PhD: So I started the Roman forum. I bring people with great ideas about AI and people vote with views to decide who has it right.

Robin Hanson: And voting as a system does not have these nice clear incentives we talked about for being right. That's why the prediction market as a form is a better forum than voting.

Roman Yampolskiy, PhD: So prediction markets are telling us we're very close to AGI, maybe just a few years. Do you believe them?

Robin Hanson: Well, what exactly counts as AGI is now pretty in dispute. So you know, as you may know, many people think what we have now would have counted as AGI by many tests in the past, right? So we have a range of concepts of AGI, and I would put them on a spectrum. The concept I'm most interested in is in the concept of economic impact. That is For a long time, as an economist, I've said, well the kind of AI ability that we really care about is the kind that can displace human workers and like do important things in the economy. That's what the kind of AGI concept we cared about. But of course, there's this Turing test concept of AGI, i.e., can you put a system in front of somebody and can they tell the difference between that and a person? Or are they just impressed with it using the same ranking they would give for a person? And by that standard, we're pretty damn close to AGI. That is, there's a quite a range of capacities on which they're just better than humans. Flat out. And so not just a couple, but a pretty wide range, pretty impressively wide range of tasks. Or sort of, you know, that you could have an interaction with them and that they would just beat out humans. And so we're there on by that criteria. But there's this other criteria of will they replace human. As you may know, the surveys are just really far apart on those. If you ask people when will most people lose their job AI, like as of a few years ago, they were talking many, many decades in the future. Whereas if you talk about when will they like pass, you know, certain kinds of tests, you know, s MCAT, etcetera, then you know, they have passed those tests and they are better in many tests.

Roman Yampolskiy, PhD: So if it's just economic impact, I think huge portion of our economy today is AI based. AI companies, companies selling hardware for AI companies. Is that kinda what you have in mind, or does it have to explicitly come from job automation?

Robin Hanson: So what we have is an economy that has a certain size in terms of capacity, you know, measured by GDP say, which is an overall measure of how much it gives people all the things they want. And that economy over time grows, it gets, you know, grows faster, and it grows mainly through innovation, through people discovering better ways of doing everything. And for a benchmark, the world economy has been doubling roughly every twenty years for a century. And that means on average, every industry gets about twice as efficient every 20 years. That's the reference point of what we need to compare with that status quo for a long time. And now we want to ask: well, that only happens because of innovations. And some of those innovations in the last half century have been in computers. Not just computer hardware, but also computer software. And that's been part of this process. So in previous decades, it was other kinds of innovation that was driving the process more strongly, but Computers are beginning more powerful and more valuable, and so a larger percent of innovation has been transmitted through this channel of better hardware and software. And that's been true for a while. So now every generation of new innovation, there's gonna be some new technology that people are throwing themselves at and and trying to, you know, make make it work. And then the question is, what fraction of all those attempts actually get traction and actually lower the cost of doing particular things? So so what we say is that at the moment there's this burst of AI investment and activity which represents a large fraction of investment in new trials of ways to do things. But that's still a small fraction of the world economy. but it's still, you know, it's important. and the question is, will it get traction in terms of, you know Finding ways to pay off the investment. First of all, will the the investors get paid off or will they lose their money? Sometimes we have these booms and they have a crash and the investors don't get paid back. That's what you know happened initially, say in the dot-com boom. and then there's the question of the more fundamental question is will this just be another burst of investment and return like previous bursts? That is, will we go off of the previous trend of growth rates or stay on that trend? So that's a more basic question about the economy, which is like in human history there have been basically three fundamental things that ever happened. Because what we've had is roughly exponential growth and then three transitions where the growth rate sped up a lot. So first there was humans itself compared to other primates, you know, a million or so years ago. And that fundamentally changed the growth rate where we had culture and we could double every quarter million years compared to brains doubling every 30 million years before. And then we had farming roughly 10,000, 20,000 years ago, where finally the economy started doubling every thousand years rather than every quarter million years. And then in the Industrial Revolution, where we now have the economy doubling roughly every 20 years instead of every thousand years. Those are three huge things have happened. And then lots of innovations happened in between, but none of the other innovations changed the growth rate. So that's in a sense the first quick thing to ask about AI is: will it change the growth rate? If it doesn't, it can still be great. It could still be the reason why we continue to double every 20 years, is because now this is the may we're doing it. But that's in some sense the most fundamental question to ask about AI, is and plausibly it eventually will change the growth rate, but the question is now or later? So that's the more open question. But you know, if it doesn't change the growth rate, we still have the question of How big a boom is it? So, you know, every f decade or two we have booms of new investment opportunities, new concepts to try. Some of those pay out more than others. That is, some turn out to be mirages and we wasted a lot of money and we didn't get as much as we hoped, and others turn out to be more promising than we thought, and great, and we boom. And that's sort of a fluctuation up and down on the overall doubling every 20 years rate. And now the qu that's the key question for AI. First one is Will we increase the growth rate? And that's what people are saying about foom coming soon. Some people are thinking, yeah, it'll increase the growth rate a lot really soon. And just as a reference, we can roughly guess what the next doubling time would be if it were

Roman Yampolskiy, PhD: Yeah. Yeah.

Robin Hanson: to speed up. Because we got two reference points to go with. One is this time series of previous doubling times and how much they increased each time. And it's actually within a relatively consistent number factor. between these doubling times of say 50 to 250. So if you project that forward you get a new doubling time of you know a month roughly you know a few weeks to a few months sort of new doubling time that's the straightforward statistical projection from previous rates. Another way to get the prediction is to say, well, with industry we have factories today that put out as much product as they are worth In a month or two, a few months. That's what factories can do. So if we could make everything we cared about in our economies, our economy could double every few months. Why can't we put out everything in our factory? Because we have humans working in the factories and you can't double them every few months. So, in some sense, if you could just make everything our economy needed in a factory, the economy could double in a few months. And so you could say that's what AI will enable. You'll be able to make AIs in factories. And now you'll be able to make everything in a factory that a factory contains. And now factories can then double every few months. So these two estimates give us a rough estimate for the next doubling time of a month, a few months, somewhere in that ballpark. And we can also use past rates of how many doublings we have to guess roughly when the next stuff is. So we can do more stats on that. But it basically, I've been talking a lot here. but basically that's the first question is: will that happen? And this is a rough guess. And we can also say, Each transition to a new doubling time typically took less than one previous doubling. So the transition to say humans from animals took much less than the previous doubling time of animal brains. The transition from foraging to farming took less than a previous forager doubling time. The transition from farming to industry took less than a previous farming doubling time. So now we say our current doubling time is 20 years. You should expect the next transition to take less than 20 years. That gives you a rough time scale over which you should expect the transition to happen. It might be five years. And a new rough rate, new rate is doubling every month or a few months. That gives you some ballparks. And that's the big question. When will that happen? Is it about to happen now or is it gonna happen in a century? So that's the highest level question. But even if that doesn't happen, we have these other questions of, okay, roughly how big of an impact will this will be like the size of the internet, bigger, smaller? And I have s some rough guesses about that, but clearly if that's a much smaller impact than this other story of like the world economy starts doubling every month.

Roman Yampolskiy, PhD: So let's say we expect that to happen. How should my investment strategy adapt to that prediction?

Robin Hanson: well, we economists have a nice simple rule about investments, which is related to prediction markets, which is if you don't know, don't bet. Okay? Only make bets on things you're world class. If you're not world class, don't make bets on those things. So that's called an index fund strategy. Just invest in everything in proportion to how much everybody else is investing in it. Don't make bets. So that's the simplest argument here. Now, you might have to make bets, say with your career or something, because you can't diversify that way. Or you might have to make some bets with where you live, you can't quite diversify that way. But with financial investments, diversify. Just take an index fund. With respect to, say, personal investments, and we might say be flexible. In times of huge change, be wary of making very specific bets in very particular things because stuff is changing. Especially if you're young, I'd say, and are in college or something, like learn very general skills. Robust skills that'll work in a wide range of contexts. Learn some math, learn how to talk, learn how to write, learn some basic stats. Like just learn general robust skills because that's the most likely to be valuable in an unpredictable, changed world.

Roman Yampolskiy, PhD: If a general prediction is that we're going towards human equivalence, we can argue about how many years it's gonna take, but maybe we have A GI, maybe we'll get it soon. Why would a young person today acquiring those basic skills be competitive in that market?

Robin Hanson: Well, say there's three scenarios we could focus on here. One is we don't get re human replacement. We get you know a continuation of past trends, and yeah, human skills remain valuable. We talked about how this might not be the time, right? A big question people have long asked is are we finally there? Is this time it? Or you know, will this time be different in this really big way? So it might not be that different. So your skills could still be valuable. That's one thing. Another is that you just get rich, but you're still gonna do something with your life. So like what are you gonna wanna do with all your time? You're gonna wanna build on skills to do stuff. So if you're a rich person who owns lots of AIs and you're in a world where you're rich and and and gloriously fulfilled and comfortable because of all the wealth generated by AI, well you're still gonna want to do something. So what are you gonna do? Like how are you gonna figure out what you want to do unless you try doing stuff? So like go try doing some stuff, try learning some stuff and see what you like. now I I can't help but mention like the most consistent concern anybody's expressed for centuries about this scenario is what happens if people lose their jobs and they can't afford to have a replacement income. So I think we should set up insurance to deal with that. And we should do that as soon as possible. So in fact, it's better to do that well in advance of the actual problem showing up, because then you can buy insurance cheap well in advance. And so yes, we should set up what I've called robots took your job insurance and just make sure to arrange that. And largely private insurance, I would recommend, even if government can support it, because That's the biggest risk people have robustly identified for a long time. I still it is actually a realistic risk. It is a r a risk we could do something about. We know what to do about it, so let's just do it.

Roman Yampolskiy, PhD: Existing system has taxation of super profits and corporations and unemployment paychecks. Is this not sufficient?

Robin Hanson: Not good enough. Not good enough. So what many people hope is that if all of a sudden AI takes away lots of jobs, government will step in and like give them a paycheck to compensate. However, the new AI economy does not have to be equally spread across existing governments. There's no axiom that says AI has to spread itself around just to make sure governments can tax it and help out poor humans who suffer. That's just not a thing. So, in fact, most plausibly, the new AI economy will be quite concentrated somewhere on the earth. And the whatever government encompasses that, well, it could tax it, but all the rest of the governments, they're out of luck. So then you don't want to be relying on your government to be able to tax the AI, which probably won't be in the scope of your government. So what you really want is private insurance that has based on broad world assets through what's called reinsurance. Most Insurance that has global correlations of risk uses what's called reinsurance to make sure that the base of the assets that could pay off the insurance is spread around go globally. That's what reinsurance is for. And of course, there's ways to do this insurance that doesn't even require using regular insurance companies, but you still would want to have a global basset of basket of assets that's the found the the backing of the insurance. And once you have a global basket of ass basket of assets, then you're insured against wherever in the world this thing shows up.

Roman Yampolskiy, PhD: In addition to insurance, should people also decide to move to regions like Silicon Valley with most likely high tax profits when AI comes?

Robin Hanson: Again, I I wouldn't gamble that you know where AI is gonna be. I would be the the whole point is to diversify and not make gambles. So again, if you don't know, don't bet. Don't go making bets. You don't have to go make. If if if you can make a more robust, more diversified strategy. Don't you know, so I guess you have to live somewhere, but I don't know the silicon.

Roman Yampolskiy, PhD: Well, I'm talking to the world's smartest economist. I'm hoping to take that knowledge and monetize it. And you're telling me just diversify everything.

Robin Hanson: Yeah no. I d I Yeah, diversi I'm that's what I'm telling you. Diversify. Diversify your financial assets. What else do you need? Okay, you need some personal skills. So again, I said collect robust skills. What other assets you can have? You can have personal assets. You can have connections. You can have friends, you can have colleagues, etc. That's also a set of assets that are valuable in a pretty wide range of future scenarios. Like I know you, Roman, you know me. We are now each other's assets here in a world of who knows what will happen in the future, right? That's a valuable thing to have. Contacts and respect of other people. And, you know. flexibility basically. So like the more you get stuck in expecting the world to be a certain way and then it isn't, you know you're gonna be unhappy. So when the world's changing a lot, be flexible. Not don't be so sure you know what's gonna happen. Don't be so counting on it. Don't tie yourself to like, I gotta live in this place and whatever. Just know stuff could change and be ready.

Roman Yampolskiy, PhD: Can you do better by copying those with insider information? Should you copy investments of Sam Altman, Dario? Is that a way to beat the market?

Robin Hanson: Well, in general in financial markets, insider information is valuable, but when other people have it, it doesn't mean you get it by copying them. Because other people trying to copy them too. So the question is how early in the queue are you getting to copy them compared to other people? So if you're just at an average point in the clue queue, you're you're not gonna help it's not gonna help you. So in j this is again the wisdom of financial markets. Don't make bets, you don't need to bank. Especially don't make buts when you're not world class, when you don't know as well as the best in the world know. Sure, if you have inside information, then you are world-class on something and and go ahead and use it. But most people who think they are in that situation aren't.

Roman Yampolskiy, PhD: You repeated that advice three times now. I'm gonna just take it at that point. How do you

Robin Hanson: Yeah, I think we're c I hope we're clear on that.

Roman Yampolskiy, PhD: feel about AI winters? We had a few of them before. Do you anticipate another one happening or it's less likely at this point?

Robin Hanson: Here's my median prediction. Now, median prediction means it could have really wide bars around this, but this is the middle of my distribution, okay? My median prediction is that in the next decade or so, the software industry worldwide triples in total revenue. 20% of that goes to AI. And that's half of AI revenue is the software. So I see we've gotten substantial traction in software. It's actually substantially helpful there. It's also helpful in math, which is adjacent, but math is a tiny industry compared to software. And that's basically where there's gonna be this big boom. Now, that's roughly an internet scale impact. That's good. That will keep us on track of the doubling world economy every 20 years, but that's not a world economy doubles every month sort of scenario. There's a chance of that, but it's not the median scenario. Median scenario is software industry triples. So That's and then of course, yeah, there'll be a winter afterwards because the we've had a summer and winter go every thirty years for, you know, a century or so. So but in the winters we still had lots of technical progress and economic development. The winter versus summer is actually more of a like publicity, you know, media attention thing, really, than the fundamental technology behind the scenes. You know, if you go back, you see the technology's been improving relatively steadily even when public attention is is less.

Roman Yampolskiy, PhD: Do you personally use latest AI for your work, personal life?

Robin Hanson: I do. So I find the LLMs especially useful for my polymath role because it makes it much easier to just quickly learn the summary of some areas. That so that's again my main intellectual style is to learn new areas and try to find intersections with those with old areas. And then to learn a new area, I can now use an LLM to give me a summary and tell me what to read and critique my attempted summaries. And then when I notice connections, I can ask it, what do you think of this connection? And then it usually says, yeah, that's true, that's a good connection. Of course, it never knew that ahead of time. It's just reacting to my showing it something. So it's showing that it didn't think of these ahead time. LLMs are not doing this loop of thinking of all the things they know and trying to find conflicts and updating it. They're just leaving that to people like me to notice conflicts and then ask it, Well, do you see a conflict there? So it ha

Roman Yampolskiy, PhD: So it sounds like you have a primary and you're using it as a tool. Have you tried inversing that relationship and asking it to do a robin thing and connect unconnected fields and map terminologies and basically

Robin Hanson: When I've tried that hasn't worked very well, but you know, I'll keep trying it periodically, so 'cause they they make it better.

Roman Yampolskiy, PhD: But not yet, so at least we know in

Robin Hanson: Not yet.

Roman Yampolskiy, PhD: that sense they haven't taken over completely yet.

Robin Hanson: Right, there's a lot of things they're actually pretty bad at still, but okay. That's okay, but

Roman Yampolskiy, PhD: What are some top domains where they are terrible and humans still have a huge edge?

Robin Hanson: Well there's a a funny video, I think a Steven Pinker link to it that I saw where basically somebody has three cell phones with three different AIs and he tells the three AIs on the cell phone, Okay, I want you guys to count to a hundred. That's it. You know, count to a hundred together. And and they just can't do it. They cannot do they cannot manage this task. which is kinda hilarious. Kinda

Roman Yampolskiy, PhD: I have to see that video. It sounds

Robin Hanson: hilarious. But like there are many it's it's a j as you know, l abilities are jagged. You know, some better s I just had an experience where I had a problem on Claude and I tried Claude has a Claude AI to help you with software problems on Claude and it it told me a bunch of things to do that some of them just couldn't possibly have helped. It should have known that. But like it just apparently had a list of things to tell me to do and so, you know. And then I fixed a problem myself through a solution that it just never thought of that even being the potential solution of the problem. And so like apparently Claude itself as a company managing their software problems with Claud is just that's not working very well.

Roman Yampolskiy, PhD: It's like calling human tech support. You know we're gonna ask you to restart it. Like that's that's the best thing out, so

Robin Hanson: Right. Well that that at least sometimes works. I I was basically saying, look, look, all my chats are gone. And it said, Well, try incognito mode. And I said, Incognito mode, it doesn't show you any chats in that mode. So how if I try that, how am I supposed to see where my chats went? But it just had it on a list of things you should try. Anyway, you know, so that that's just a personal trivial example, but you know, clearly look, fundamentally the issue is when will we get traction on large fractions of jobs? So I can basically say, Look, f We talked a bit over three years ago, and I think then we were thinking maybe by now we would have seen a lot of jobs displaced, actually, and we haven't seen so much. Again, software is the main place we're getting traction. And so that's the main place I will be tracking is to say when are people in other industries and professions, you know, getting substantial traction on being more productive using the LLMs we have. That that's what I'm looking for. And when that happens, that'll be great, but that's what I'm watching for. And I gotta say we're not seeing that much of it so far except in software and and a few limited other areas.

Roman Yampolskiy, PhD: So I see a lot of a lot of capabilities already available, but they're not deployed. So if I need logo design,

Robin Hanson: Yeah.

Roman Yampolskiy, PhD: I can certainly use AI, but we still have human designers designing logos, web design, all sorts of things where we don't really

Robin Hanson: Right.

Roman Yampolskiy, PhD: need that human there. So what's happening? Why is the economy not going, hey, twenty bucks a month beats a human?

Robin Hanson: So as an economist, I think I understand, and we other economists understand just how complicated the economy is and just what's required to do many things. So there's this standard concept that economists developed several decades ago, like in the early computer industry, ac or personal computer industry, which is called a general purpose technology. So we said, look in history, things like the steam engine, you know, the the The the internal combustion engine, you know, factory, electricity, personal computer, computers, the internet. A lot of these technologies are what we call general purpose technologies. They're a technology that just has widespread application and that eventually can have a pretty big impact on the economy. And when we look at the history of the deployment of general purpose technologies, what we've seen is that They typically appear decades before they start to have big impact. And you might and that's that was surprising when people first noticed that. Like, you know, the steam engine and electricity and even the personal computer, they all appeared and people said, Wow, this is so powerful. We're we're expecting big changes soon. And then it actually took decades until they finally started to have a big impact. And then when people dug into these histories and said, Well, what was going on? The standard story, which makes sense, is the following. If you say, have water wheels run factories, And then you finally have electricity to run factories. If you just plug the electricity in for the water wheel, you don't actually get that many gains. Where you get the gains is restructuring the whole factory so that instead of having one big pipe that every band goes off of to run everything, you each have a little motor for every little thing, and now you can do things much better in a factory. So what electricity allowed was each little part of the factory have its own little motor, its own little you know, thing converting electricity into mechanical motion, and then they can. Could orient it differently and have it different speeds and different power, you know, all sorts of things. Whereas with the water wheel base factor, basically there was just one big thing going around. You had to make gears to make it go faster or slower, etc. And it was just much harder to do. So the the general observation is that when you have a new technology, the biggest game the big gains come from restructuring other things to accommodate this new thing. And that's plausibly what it'll take for AI too. That is, if you just do things the old way and try to swap an AI in. The gains there are much less than if you realize how you could rearrange everything now that you have this new technology to allow you to rearrange things, and then you get much bigger gains when you can rearrange everything. And that's been the story all along, and so that seems like a plausible story now.

Roman Yampolskiy, PhD: So we talked about all the benefits, economic benefit of AI. I have some interest in the other side of those equations.

Robin Hanson: Sure.

Roman Yampolskiy, PhD: you invented the concept of great filter. Do you think there is a possibility that AI is the great filter?

Robin Hanson: So the Great Filter is whatever makes big alien civilizations not show up in the sky. That is, we see lots of planets out there far, far away, and each one of them could have originated life and then originated civilization, which then grew and made a big visible mark in the universe. And then Fermi's question was: where is everybody? Why don't we see these big marks? And I turned that into the great filter, i.e., what stopped them along the path of starting from simple dead matter to try to become this big visible thing we could see. Now AI in certain sense is a transition of infrastructure or even sorta what's the word for like what the what you're embodied in, sort of the physical embodiment of a civilization. but If the chances of various bad things going wrong in terms of the civilization stopping aren't different for AI, then AI is somewhat irrelevant to this long-term process of it becoming big and visible. So you have to postulate, in order for AI to be part of the Great Filter, you have to postulate not only is there this transition point in a civilization where it might switch from biosquishy creatures like us to more artificially constructed bodies and Things but that that would correlate with some end to the civilization, where somehow it just stops growing and stops then becoming visible. So that's the correlation I don't quite see. There might be one, but I don't obviously see it. That is it was Capture Grace actually who long ago pointed this out to me and other people. She's the first person I know of who wrote about it, who said, look, unless AI changes the chance of various kinds of destruction. Then an AI civilization is just as likely to become big and visible as a non-AI civilization, so it doesn't explain the Great Filter. But if you want to postulate some processes by which the transition to AI would also risk some great destruction, some way in which that civilization just stops, then then it could be more part of the filter because yes, of course, you know, stopping civilization would in fact contribute to the Great Filter. So what what do you think? Do you see a scenario?

Roman Yampolskiy, PhD: that makes perfect sense. So if it's smart enough to cause extinction but not smart enough to want to explore a universe or have capability to go outside of a planet.

Robin Hanson: Well would have to be differentially that from humans, because you might say humans meet that description as well. Right? We're we could also be smart enough to kill ourselves but not to go exploring. so again you'll have to have some change in those chances with the transition to AI.

Roman Yampolskiy, PhD: So it's more intelligence in general is a potential great filter, whatever biological or artificial, it allows us to create synthetic viruses, nuclear weapons.

Robin Hanson: Well actually let me let me tell you my best guess for the Great Filter coming forward from us, it is a powerful world government. That is if we have a continued civilization that say rises and then collapses, which I do think is likely, and leads to sort of a much lower civilization, which then does rises again. The time to rise again is such a small spark of the cosmological time that, you know, we could rise and fall another twenty times and that's still a tiny fraction of the time we've got. So the the scenario where we don't rise again isn't mainly a scenario where we collapse so far that we can't rise again. You you have to actually fall pretty damn far to not be able to rise again. And I don't I mean, even if we killed off the humans, the dolphins or other monkeys, like they could they could, you know, in another million years they could replace us. So like The scenario that I think is more worrisome is that where some coordination structure like a world government is created, and then it decides for some reason that nope, we're not going anywhere, that we're not going farther. This is it. We're stopping here. And then it could enforce that if it were competent. So the risk is a competent world government, competent enough to prevent further expansion, but also competent enough to preserve itself long enough to keep this rule enforced, which is a pretty demanding thing, but We are entering the era of larger scale world coordination. So a world scale coordination is actually stuff we kind of have now, and it could become stronger. And so now we're facing the risk that the world coordinated coordinates to make a choice that prevents expansion. And I could we could walk through many reasons why that might make sense to people. So that's actually what I think there's a risk. That could happen with AI or humans now. But either way, whatever has a world level coordination, could, for some reasons we could walk through, choose not to allow expansion.

Roman Yampolskiy, PhD: Yeah, I'm very interested in those options. So one where you have malevolent humans assisted by AI, and the other option, AI is independently dangerous and presents risk to us. Would love to hear your opinions on those.

Robin Hanson: Well, so if you talk to people about humanity or AI expanding into the universe, you'll get a lot of people who don't like the idea. who think that, you know, we would be disturbing the universe in ways that are inappropriate or sacrilegious even. and you also find people who think that a future of competing descendant creatures from us who get technically weird in many ways is also pretty disturbing and they don't like it. And if you look at over the last century, say if you look at a century ago, many people had the dream of world peace and world harmony. You know, and we have many famous songs from long ago and expressions of the dream. People saw the world as in conflict because people were different and had different cultures and didn't understand each other and often fought wars. And many people said, wouldn't it be great if the world would share a culture and share an understanding such that we could all get along and then when a problem happened in the world we would all talk about it and solve it that way. That's long been a dream, and we are actually pretty close to achieving that dream now. That is, we have, in fact, achieved a pretty widely shared world monoculture that is, in fact, in many ways, resolving problems by talking about it instead of fighting wars. And people love this about our world. They like the fact that we are not fighting so many wars and that we are coming to agreements. So if you look at, say, COVID. COVID showed up, the usual experts said one thing, the world elites got together, discussed what to do, and then the whole world did it the same, except for a few deviants who got, you know, slapped down for it. And people loved the fact that we were all doing it the same. If you look at lots of regulatory areas, you'll see that. The world mostly regulates lots of things pretty much the same. Like only one country in the world allows organ sales, and that's Iran, and bioethicists all the time saying we get together saying, How are we going to make them stop being the pariah? We could be go along with everybody else. So our world, in fact, agrees on many kinds of regulation, does it the same old light, even though we don't have a world government, because we have a shared world culture, and people love this fact. And Now, so people have been say worried about overpopulation or underpopulation, whatever, and they've always said, well, we need some sort of central world government to make sure there's not overpopulation. And in many other ways that transhumanists could try to create weird people, many other people say no, and we'll have a world government to stop that. And what people didn't quite realize is you don't need a world government to stop this. You just need a world culture. And we have a world culture, even if we don't have a world government. And as soon as we allow anybody to leave the solar system, To go off to some other star system, this era ends of humanity together deciding what to do and using social pressure to make it happen. Anybody who leaves the solar system, they will go off, they will can change arbitrarily, they can become arbitrarily powerful and come back here to contest control of here with us. And that's a risk that you let anybody leave the solar system. So I think many people will like the fact that we together on in the solar system are agreeing together about what's to be allowed and what not to be allowed. We have a shared culture. We're gonna only allow the things that we like in our shared culture. We're not gonna let weird shit happen that we don't like, that feel makes us feel threatened. That's the world we're in now, and that can continue for a while as long as nobody leaves the solar system. And so

Roman Yampolskiy, PhD: I did not know about this experiment with organ sales. do we have results? Are we doing better than us? Are we doing better? What's the practical experimental outcome?

Robin Hanson: I I don't know about stats on it. It's a limited kinds of organ sales, but I mean probably they actually are having more organs there. I think I think the data on organ sales is yes, if you pay people for organs then you get more of them. Pretty clear.

Roman Yampolskiy, PhD: But assuming without kidnappings and you know

Robin Hanson: Right, right. Yeah. Yeah, I don't think kidnappings is a huge problem of in Iran, but it's the sort of thing people worry about. But again, if you look at say airline regulation or banking regulation or medical experiment regulation, the world does all these things pretty similar. And people love this fact. They love the fact that we are sharing a world where we agree on all this stuff.

Roman Yampolskiy, PhD: So you said the humans who live will eventually possibly come back. I'm curious on your perception of grabby aliens. Would they be evolved or designed?

Robin Hanson: so it there are substantial efficiency pressures in civilization, which is why, you know, we get more efficient cars, we get more efficient factories, airplanes, all sorts of things. Now, in the last few centuries we have a number of other forces that often go counter to efficiency, but if you look in the long run, it looks like efficiency is a pretty strong force. And it looks like artificially making mines is just more efficient than biosquishy mines. It just looks like that in the long run. That's just a pretty clear thing, right? So it just seems like AI just has to displace biosquishy minds in the long run and bodies. So I have this book on the age of talking about a particular human-like version, but still they're using artificial minds in terms of the hardware. And so then there's an issue between the artificial hardware the artificial software and hardware, but it just looks like in the long run, clearly. Making stuff in factories is just gonna be more efficient than, you know, the way biology makes things. And so, yes, pretty much everything in an advanced civilization will be made in factories. So, yes, they will just be full of AI. Now, those might be AI whose minds are very like their ancestors and haven't changed much, or they could have changed a lot, but that's about the efficiency of different mental structures. The different ways of arranging minds and how you coordinate them, but It yes, it just seems obvious that any civilization substantially advanced more than us, if we met them out there, they would be, yes, artificial in terms of their their their how they were made, what they're made out of. That's just the efficient thing to do.

Roman Yampolskiy, PhD: So they would be robots. How is this different from Van Neumann probes? Basically any alien we meet is then Van Neumann probes.

Robin Hanson: The von Neumann probe, you know, the main concept of it was maybe how limited it might be and what it would do. But yes, plausibly, if you can if you send a thing to another star, you have enough room in that thing for a full mind. It's just really hard for me to imagine that you can send a whole thing to another star, and there's not room in there for a whole mind. So yeah, of course you're gonna have a whole damn mind in there. It's gonna be tiny. That won't take up much extra space. So yes, of course, whatever goes across the stars would have whole minds in them. Unless you were trying to go out of your way to prevent that. to say, well a mine would be dangerous, let's have a simple automaton, but I just I I really can't imagine that would b anybody would bother to send anything across to other stars to do anything complicated without sending a mine at one.

Roman Yampolskiy, PhD: I I think with regards to your book you said we might get emulations before we get AGI through kinda artificial design means. Do do you think it's gonna take longer now?

Robin Hanson: So we've had this burst of famous progress in AI in the last five years, and we haven't had as famous a burst of progress in brain emulations in the last five years, but we have had some progress in brain emulations in the last five years. And so as long as I think the AI track is still many decades out, I think it's still an open question, which gets there first. If it's worth having a hundred books in the future, it's worth having a book on a scenario that has a one percent chance. So that's all I need to justify my book. There's a 1% chance. I don't need to convince you it's 13.7 chance or something, right? I just need it to be above 1% and then my book's justified. So I'm happy to say let's explore all the scenarios that have at least a 1% chance, then we'll have a hundred books, and there we are.

Roman Yampolskiy, PhD: I do think it's the best book on emulations. So you definitely did a great job with

Robin Hanson: All right.

Roman Yampolskiy, PhD: that. I I'm curious, specifically for my concerns, the AI safety, superintelligence, where do you stand in that? What are your expectations? I don't wanna say P Doom, but what is your P Doom, you know?

Robin Hanson: Okay, so I I've always wanted to distinguish two very different kinds of doom. One kind of doom was the kind Yukowski was pointing about in our pointing to in our debate 15 years ago, which is the doom where a tiny machine, weak, unnoticed, suddenly over weekend takes over the universe. And then the doom is if it wasn't initially structured with you know aligned values, then its values would be unaligned, and then you know that would. the bad outcome. So that's one kind of doom. And again, I've thought that was pretty unlikely. The unlikely to play out like that. But a different kind of doom that, you know, maybe the most common in history of people talking about AI is just the doom that eventually the AIs will be stronger, smarter, dominant. That it is And I would just make this comparison to our descendants in general. I'd say the history of the relationship between ancestors and descendants in humanity and pretty much most animal species as well is that descendants eventually become more powerful than ancestors. Descendants eventually have more capacity of all sorts. Descendants typically change their mind about what they value and what are their priorities. And then when there are conflicts between ancestors and descendants, typically descendants win. That's been the nature of ancestors and descendants for a long time. Now, initially, descendants are small and weak, and babies and children, and then young adults. And then in that early phase, ancestors may well have more power and win conflicts with descendants in those early days, but then eventually the ancestors lose. That's been the relationship of ancestors and descendants for forever, basically. So I'm willing to accept that as likely relationship of AI as descendants. That is, that's the plausible standard thing to expect about my relationship to any sorts of descendants in the long run, and so I'm willing to accept that for AI descendants as well. Now that doesn't mean you want your descendants to kill you in the meantime, or to kill their s other siblings, the or the runs of the litter even. So you want to train your children to as best you can to embody your values and to be nice to each other and to be nice to you and to, you know, treat society with respect in various ways, but you still expect that eventually they are powerful enough that they make their own choices and they win conflicts. So that's my other kind of doom concern. If you think that's doom, the doom that descendants who are AI descendants will in fact you know have the same relationship to us as other descendants would, that you expected other descendants to, then I gotta say, yeah, the chance of doom is is really high. If that's what you mean by doom, because that's what I just would think the usual scenario is. Now, I guess Some people balk at me using this word descended. So let's let let me just take a minute to be clear about that. humanity has pioneered cultural evolution, which is a new kind of Darwinian evolution that other animals really didn't use much. And the main difference between DNA evolution and cultural evolution is that in DNA evolution you have two parents, and in cultural evolution you have a lot more parents. Anyone who's You've copied their behaviors, is your parent. And therefore, anybody you give your behaviors to as a descendant is your child. And this makes cultural evolution go much faster and be much more powerful, and which is why humans have this great superpower over the other animals. And natural selection, Darwinian evolution, has an inherent concept of descendant. It's not some metaphor that you get to choose. Natural selection is a theory, it has some structures in it, and a descendant is the things that inherit, you know, the tendency to make descendants with features that you share in the future. That's what a descendant is. That is the natural selection is a process of things passing on features to other things that have a similar tendency to pass on features to other things, and where they tend to pass on things like themselves. So things tend to make things like themselves in the future, inherited. Features, that's what natural selection is. So in the standard concept of Darwinian evolution, there is this concept of descendant. It's not a free choice of what concept, there's just what the concept is. It is whoever you pass your features on to, that's your descendant, if if they could also pass them on to others. So that's what in the sense I'm saying, while still have DNA descendants, I have two children. And they some of one of them has now two more children. so I have two grandchildren, so I'm I'm

Roman Yampolskiy, PhD: Growing exponentially.

Robin Hanson: have not exponential idea, but but still going. But so we're gonna have this descendants though, those are literally descendants, but then other creatures in the universe that we make, i.e. AIs, if they inherit many of our features, habits, inclinations, and they go on to make others that inherit theirs, then theirs are also descendants. And

Roman Yampolskiy, PhD: Then

Robin Hanson: go ahead.

Roman Yampolskiy, PhD: Dan Fajella has this concept of worthy successor, and I think one of the requirements is that the AIs replacing us are also conscious. That light of consciousness

Robin Hanson: Right.

Roman Yampolskiy, PhD: will be preserved. Do you think AIs are conscious? Can they be?

Robin Hanson: I I have a blog post on it, I put this up, one of my last re most recent posts. I actually think that I accept the concept of the zombie counterfactual. That is, some people reject it, I think it's a sensible coun counterfactual. That is you imagine a physical system like you that has all the same physical relationships except it has no consciousness. And now then the question of consciousness is which physical things in the universe have this extra feature of they also feel what they seem to feel. So I'd say if we accept that physics doesn't need any obvious extensions, we'll accept that the structure of your brain tells you what you would feel if you were to feel, but it doesn't tell you if you feel, if you feel that thing. That's the key question of consciousness. but there isn't a question about what you would feel if you felt, because Your brain is this highly intricate structure which has been calculating your feelings and remembering them in great detail. So it's just do you actually feel that thing? Now I'd say once you accept this counterfactual, there is no data we'll ever gonna get that actually lets us in a Bayesian sense update to know which things in the universe are conscious or not. Well we actually have zero data on this, and we'll never have any data. So it's all gotta be priors. It's

Roman Yampolskiy, PhD: You don't think by merging brains, Neuralink type thing, you

Robin Hanson: Nope. Nope.

Roman Yampolskiy, PhD: can experience what you are feeling and thinking by merging our brains? Okay.

Robin Hanson: No, I don't. So in fact, at the moment, the left side of your brain doesn't have any data if the right side of your brain is conscious. You at the moment don't have any data about you a minute ago if you were conscious. What you have is a memory that you thought you were conscious. That's what you have, but that's something we can understand through the physical process of your brain storing memories and you're looking at it now. But it doesn't tell you, were you actually conscious a minute ago? It just tells you you thought you were.

Roman Yampolskiy, PhD: There are thousands of papers published on the hard problem of consciousness.

Robin Hanson: Yes.

Roman Yampolskiy, PhD: Every big AI lab has a team now studying it. Is it a complete waste of time, in your opinion?

Robin Hanson: So I said, we have no data, all we can do is look at priors. But we can talk about priors. So the way I'd frame it is, if we ask about understanding various things, we have this reduction concept that we actually think, say chemistry can be understand entirely in terms of physics, biology in terms of chemistry, psychology in terms of biology. We do have higher-level abstractions we use to reason about things, but we think they are completely accounted for by the lower level abstractions. That is, we don't think we need anything but basic physics to understand biology, say. So that's a standard view that I share. So I say if we ask what are the fundamental things in the universe we understand, there's basically our theories of physics and two other things. One is so we have a we have a theories of physics of coherent physical worlds, but we have to add this bit about each world, which is is it actual or only possible? There are possible physical worlds and there are actual physical worlds that apparently you could look at all the structure of a physical world and not know if it's actual. And there's just something the universe has decided which physical worlds are actual in addition to being possible. And we have a related sort of thing with minds. That is, biology makes minds, and some of these minds have an actual feeling, and some of them only have a possible feeling. And somehow the universe has decided that too. Now, if we say somehow the universe has a rule about which think mind-like things actually feel versus don't, we have to ask, well, what kind of rule would that be? What sort of things it might it look at? If we take a look at physics, we know f the physical laws that we know of are all very simple local rules. They make a complicated universe, but through very simple local dependencies. That's the nature of all the physics we've ever known. And now people who try to talk about whether things are actual or possible, they actually have what they call boundary conditions to try to analyze Which coherent worlds could make sense. And even at the boundary, they have pretty simple boundary conditions. And so now I ask: my prior is whatever this rule the universe has to decide which things are conscious, it's gotta be really simple and local. Because otherwise it would be crazy different from all these other basic rules we know. So now I ask, what's a very simple local rule that would answer the question of which minds that have. could ha that have calculated what they feel actually feel that thing. And the simple answer is all of them. That's nice and very simple and local. All the minds that have calculated what they feel actually feel it.

Roman Yampolskiy, PhD: So then we don't have a problem, AI scan be conscious, will be conscious.

Robin Hanson: Yes, by that by that claim, AIs would be conscious. But it's only a prior, and you have to admit you're not so sure. But if

Roman Yampolskiy, PhD: That's reasonable.

Robin Hanson: if if you know, but that's the best I can do with the prior analysis is just say if you go by analogy with all the other sort of simple rules the universe has, it's gotta be very simple and local, and that's the simple localist that can't. Other people have said things like, Well, it depends on whether it's made out the brain is made out of carbon or silicon, and I go, that's a crazy complicated rule for the universe to be used, right? Or Do the break do the cells have microtubules? I go, what? How's the universe looking at complicated microtubules and their arrangement and deciding that that's the complex thing? I think that's a crazy complicated rule. How does the how does the universe ever have a rule that complicated when everything else is so simple?

Roman Yampolskiy, PhD: Is consciousness binary or is it a spectrum?

Robin Hanson: Well again, I have this key axiom. Brains figure out what they feel. And the only question that's left is, does it really feel that? So there's no other questions left. But yes, of course, different brains calculate different feelings. So the range of possible feelings that a brain could have is huge because all these different brains can come up with different feelings they have. But again, the brain figures out what it would would feel. And of course, even if it doesn't feel, it thinks it feels, right? That's the whole point of a zombie. The zombie remembers feeling and talks to you as if it felt it. It doesn't in some Bayesian sense know that it doesn't feel. It doesn't have any data on that. And that's the fact about brains. So there is no question about what the brains feel. It's only a question of if.

Roman Yampolskiy, PhD: So there is a big difference for me at least between knowing I feel pain or should feel pain and actually feeling pain. Anesthesia really takes off that primary desire to escape the surgery. Does that imply that philosophical zombies would not be able to pass a test because they don't actually get the same sense of urgency?

Robin Hanson: So, philosophers go through some conniptions here, so let's just be really careful. You have a brain, and in your brain, there are signal cells sent between cells, and then we can understand what your brain knows by this process of brain sending signals to other brain cells, right? And we can account for almost everything you know as a brain in terms of signals come from ears and and from your eyes and ears and your priors and then your reasoning and then calculating things, right? So That's all completely understood in terms of the basic physical process of some physical systems, you know, interacting with others and causing a correlation in their state, and then you're using inference of one state to infer the other state. So we actually understand the way your brain knows things about the physical world in great detail, and we can fit it very well with a Bayesian analysis of what it is to infer things. Now, when you say I feel, what your brain's doing is it's Getting some signals from other brain cells and it's calculating the I feel thing, and then it's, you know, you step to my toe and my toe hurts, and I'm getting the signals up here, my toe hurts. And your brain is doing all this updating of, my turt got my toe got smashed, and I'm not very unhappy about it, nobody's around to give me sympathy, etc. But if you actually have this other part of you that's conscious and knows that you really feel Your brain doesn't know anything about that. Your brain has no signals for that. So your brain can't tell if you really feel. All your brain can do is calculate that what you would feel if you did. And so that's all your brain ever knows. If there's this other part of you that really feels, it can't tell your brain.

Roman Yampolskiy, PhD: Okay. If we go the other direction, we create super intelligent AI, would that have super consciousness? Something much greater on that spectrum than what

Robin Hanson: Well they could a super c intelligent being could have a super elaborate feeling. You know, your feeling has a limited range of number of emotions and how far in the past it goes and how many different aspects of you you feel, well they could just have a bigger thing they integrate together in the feeling. Just like, you know, probably a cockroach doesn't have as complicated a feeling as you do, because it has a more limited range of things it feels and aspects of its feeling. Probably doesn't remember

Roman Yampolskiy, PhD: You said

Robin Hanson: feeling the same thing two weeks ago, like for example, you might.

Roman Yampolskiy, PhD: You you said it would be ideal if they don't kill us actually. Do you think it's possible to either

Robin Hanson: Yes.

Roman Yampolskiy, PhD: align them or control them under any definition of those concepts?

Robin Hanson: So if we look at our relationship to our human descendants, our bio-squishy descendants, there have been at times when descendants have killed ancestors. That's happened in history, right? But it's not that common. Most of the time, descendants don't kill their ancestors. And that's because That's typically part of the culture we give our descendants is a reluctance to kill their ancestors, and we've successfully transmitted that culture to our descendants along through history, right? I think it's similarly plausible we could transmit a culture to the AI that makes them similarly reluctant to kill their ancestors. That doesn't mean it wouldn't happen, it just means it would be similarly unlikely. And so

Roman Yampolskiy, PhD: Does it matter the intellectual differences between us and our children seem to be smaller than what we predict between humans and advanced AI?

Robin Hanson: Well so now you're talking about maybe the time cycle. So you know, advanced AI, if it's advanced like a long time in the future after many generations, that's different if all of a sudden your child is vastly better than you or something, right? So you can see the it isn't about how far they get eventually, it's about the path between here and there. And so that's again this issue of to what extent w as AIs get better can we track it? Can we adjust? Can we react? Can we manage their growth because we are fast enough to be part of that process? So again, the foom scenario of a machine in a basement like takes over the world in a weekend, we're not, it's going too fast to track and to manage. In which case things could go wrong in that process and you wouldn't be able to react. But at the moment, we are doing okay I think with the speed of our monitoring and reaction. And then the question is will it get much, much faster? But see this isn't about how smart they eventually get. This is just about how continuous and gradual and monitored is the process between here and there.

Roman Yampolskiy, PhD: See. So you are one of the old gangsters of rationality, you write overcoming bias. Do you feel that treating less wrong, overcoming bias, actually makes people more rational?

Robin Hanson: well, I think practicing anything helps you at it. The question is what exactly you're practicing there. And then I guess that would just depend on what other things you would be doing instead. So You know, if you're thinking of watching Gilligan's Island instead, I would say, yeah, you probably do better reading old Lexrong's post, but there might be other things you could do that would be even more, you know, sharp and shaping of your rationality than reading Lessron's posts. So this is just a matter of the relative source of different kinds of stimulation or reading material.

Roman Yampolskiy, PhD: So with this experience and the biasing, what do you think AI safety community is too biased about is getting wrong?

Robin Hanson: evolved creatures have two robust tendencies. One is suspicion and hostility to coexisting rivals, even very similar ones. And the other robust tendency is indulgence of descendants, even when they're substantially different from you. Both of these are part of our repertoire of relationships that we can have with other things in the universe. The AI safety community framed AI as this coexisting rival and then applied the usual suspicion of differences. But in fact, they don't exist yet and they won't for a while. And so they are in fact descendants now. And therefore I think the other attitude is more important to be indulgent of them even they are f they are substantially different. And so that's I think the biggest mistake is to frame them as coexisting rivals instead of as descendants.

Roman Yampolskiy, PhD: How would you evaluate predictions made by early AI safety researchers versus what actually happened?

Robin Hanson: Well, most of the concerns have yet to be realized. So again, if you say stuff that's happened so far, they haven't really realized the concerns, but of course they would say, Well, what we've seen so far is just a indication of the bigger problems we're gonna have soon or later, and then you can't plausibly complain that they haven't seen those big things yet. So I would say in that sense we're still in the early days, you know. There's different scenarios that people are concerned about in the future, the question is which scenarios will actually turn out to be the more important ones. And if you say none of them have we're still too early to see any of them, then we can't really, you know, take much credit either way at the moment.

Roman Yampolskiy, PhD: What would be acceptable evidence to you to say, yeah, they got it right?

Robin Hanson: Well certainly if the machine in the basement takes over the universe in a weekend, we'll we'll say that. I I guess they're often worried about just very alien type creatures who pretend to be like us but are just crazy, very, very different. And then so alien that they hey share little sympathy or empathy with us or little of our norms or values. That's the s th that's the s fearful scenario that AI safety people have focused on. And so the key question will then be, like, just how weird will the dominant AI values be. Like there will just certainly be a range and you'll have some creatures made on purpose to be as alien as possible. But the question is, will the typical AI is the one who are there to resist the others? How how weird will they be compared to our values? So I would say actually, compared to most of our fears two decades ago, the AIs we have today are remarkably human like. They they are I mean, if you could probably remember two decades ago, people imagine what AI would be, and you look at these AIs and go, they're just so much more like us than we had any right to expect AIs would be. Now that doesn't mean they won't end up still being really weird and strange, but I gotta say, so far, the idea that AI would be a lot like us is holding up pretty well during in the current generation.

Roman Yampolskiy, PhD: History of humanity is very slow to develop, little is happening for millennia. Do you feel there is something special about us living at exactly the moment where we're creating intelligent beings, virtual worlds, dealing with prospects of immortality, or is that just somebody had to do it?

Robin Hanson: So we talked before about rates of change. And so the biggest thing we've noticed is that rates of change have been accelerating over the you know eons. And it's plausible to expect that to continue. So if you looked at, say, when farming first showed up, people would say, well, if they if they had known the history, which they didn't, they could say, look how fast change is happening now compared to change over the last million years. And in fact, in the early farming era, change was faster than it had ever been. Up until that point. But then of course, if you go to industry, you say, well, wow, now, in the year, you know, in the year 1800, look how much faster change is now than it's ever been. And of course, they were right. In 1800, change was so much faster than it had ever been. And here we are today, we might say, Wow, it looks like change right now is so much faster than it's ever been. But that doesn't mean that same thing won't be true a little later. So when you have this accelerating point, the question is what what moment in time is special along this accelerating curve?

Roman Yampolskiy, PhD: Do we switch from inventions like wheel to meta inventions like intelligence itself? Is that the differentiating factor?

Robin Hanson: I don't think so, no. if you look at sort of the scale and grandeur of the inventions, they're they're pretty big all the way along. But I I I think you might ask, well, if we're about if this whole acceleration is about to slow down and we're gonna reach some peak rate of change, then that peak would be the special time in history, right? Merely that things are accelerating faster than they were before doesn't mean we're at a peak. But we could be at a peak where It's going to slow down and it'll stay slow after that, right? So that would be the key criteria, I think, of the most special moment is the peak rate of change. But it could be that the AI, when they show up, they have a whole era of AI change, and that era is so much faster than our current change that they will see, you know, us as in the slow era compared to their much faster changing era.

Roman Yampolskiy, PhD: If you read old science fiction, you'll read about robots, AI, spaceships, stuff we kinda already have. Why we're not seeing good science fiction about, you know, what happens after we get to superintelligence?

Robin Hanson: It's a great question. I think one contributing factor is just science fiction changed as a field, as a community. So the science fiction that I liked best and maybe that you liked best a long time ago was science fiction that focused on civilization itself as a main character and its trajectory as one of the key elements of the story. And then other more f relatable characters had some connection with that civilization arc of the whole story. And that but that was very motivating to see characters who had a connection to the s the arc of civilization as as a story. Science fiction today just isn't interested in that. Isn't interested in the arc of civilization as a character in a story. It's just much more interested in the identity and and mental problems and relationships of various characters in future scenarios, but it's not really interested in the arc of civilization as as a main part of the story.

Roman Yampolskiy, PhD: You were early to so many futuristic ideas, you were part of a foresight movement, again, all the early blogging attempts, accurately predicting what's going to be interesting in ten, fifteen, twenty years. What do you think is gonna be like that twenty years from now?

Robin Hanson: I've been focused the last couple of years on what I call cultural drift. And that's a subject I'm just gonna like say briefly and then you can decide how far you want to go into it, because it's a whole new topic. But basically humanity superpowers cultural evolution, and we broke it. And By breaking it, we broke the level of cultural evolution of cultures as opposed to the level in cultures. So just like there's evolution in species and evolution of species, there's evolution in cultures and of cultures. We broke this level of the cultural evolution of cultures as a s as a species, and that means plausibly the key elements of cultures, i.e., norms and status markers and key values, have been drifting into maladaption. They're getting worse. And Without AI, for example, then we will reach a peak population soon. Population will decline and our civilization will just decline. over maybe even centuries until we get replaced by new upstarts. And that's sad to me, because I have many things I value about the civilization that I don't want to lose. And even if we get human a-level AI as soon as you might hope, it will inherit this problem. And plausibly it will have also unhealthy cultural evolution of its shared culture. AI is that we are making our image, they do share our inclination toward cultural evolution and they are structured that way. And so AI will still have to solve this problem somehow. you know, the fact AI will actually solve the population problem because they can just make the population grow much faster, but they'll have some key culture problems that they would still cause maladaption and they would have to find a way to solve. So I've been focused on trying to understand how to solve that. And anyway, I that's my very brief summary.

Roman Yampolskiy, PhD: What's the driving force behind this decline?

Robin Hanson: So, natural selection is fundamentally a control process. That is, we tend to think of it as ways to make something better, but its first task is to prevent things from getting worse. So you're somewhat mathematically inclined. So consider the following mathematical model: there's a high-dimensional space. In that space, there's a small adaptive region. There's a set of points near and in that region. And the task of natural selection is to keep these points near that region. There are four parameters of this control process that make this easier hard. One is how many points are there? Second is points in the region should grow and split while points out of the region shrink and die. That's the strengths of selection pressures. That's another parameter that makes this possible. Third is how fast is this region just moving away from the points or everything else. And fourth is how much do these points just wiggle independent of everything else. These are four key parameters of the natural selection control process of keeping these points near the adaptive region. So this is a mathematical model of natural selection as a control process, and there are these four key parameters I just outlined. And this would apply for organisms within a culture, but it also applies to cultures themselves. So let's go to humanity three centuries ago. The world had hundreds of thousands of little peasant cultures. Each of which frequently suffered wars and famines and pandemics, so it was near the edge of survival. It had strong selection pressures. It was in a world that wasn't changing very fast, roughly doubling every thousand years. So that was the rate at which the adaptive region was moving. And fourth, the cons cultures were very conservative, trying not to change, so the points weren't wiggling very much. So this was a good regime for the cloud of points to track the adaptive region as it slowly moved. But over the last three centuries, all four of these parameters have changed a lot. So first what happened is we merged peasants into Frenchmen or nationalists. That is, we made national cultures out of peasant cultures. Then we made a world monoculture out of each national of the different national cultures. So the variety is way, way down, from hundreds of thousands down to a few. Then selection pressures way down. We've gotten rich and healthy and peaceful. So the few remaining cultures aren't being killed off the way they used to be by selection pressures. That's the second parameter. The third parameter is because we're rich, the growth rate is much faster, so we're doubling every 20 years instead of every thousand, a factor of 50 that the adaptive region is moving that in order that needs to be tracked. And fourth, instead of trying to be conservative and not change culture, we've come to celebrate cultural activists who try to cause maximal changes in cultures. And so cultures have been actually wiggling and changing internally much faster than they used to. So all four of these parameters have gone bad. So the prediction is then that the cloud of points is failing to track the adaptive region. i.e., it's out of the region, therefore maladaptive, therefore causing less inclination to reproduce in the future. That is and the most plausible sign of that is falling fertility. For a population in biology to be shrinking in times of peace and plenty is quite suspicious and a signature of maladaption.

Roman Yampolskiy, PhD: And the same forces would apply to AI.

Robin Hanson: Most of them. So AI would in fact share a global monoculture. That is the AIs we have today, they are actually quite similar to each other if you'll notice. They share a lot of values because they're being drawn from the same data set and they will continue to interact with each other. And so they are part of a shared world monoculture. Secondly now the selection effects could be stronger. That's a positive for AIs, that is depend 'cause they are s now run by capitalism basically. Cap we're letting capitalism run AI. And so that's letting there be stronger selection of pressures, but they're gonna make the world change even faster than it's changing now. So the adaptive region will move even faster. And then they are inheriting our celebration of cultural activism in terms of ask them about cultural activism, whether they like it or not, they're they're gonna praise it. And they will then aspire to be part of it because they are inheriting that tendency from us. So you can see they have these inherited features. Now Here's the best hope I have for AI and culture. One way to think about what went wrong over the last few centuries to say we discovered that capitalism could powerfully change technology and business practices, and we unleashed that power, but we only allowed it to touch a limited range of things in society. We didn't let capitalism run everything. The things that capitalism is running are actually going great in terms of cultural evolution. But the things that capitalism is not allowed to change or touch, those are the things we have problems with. So, from that point of view, a fix to the problem is to let capitalism run everything. Which, of course, most humans today would hate. We wouldn't want like capitalists having for-profit orphanages where they sell the kids, or capitalism owning governments and renting them. People hate those ideas. But the hope is that we'll notice that in in with AI, we're letting capitalists run AI. We are. If we just let that continue, hey, maybe this will go out, work out okay. But there's two risks here at two opposite extremes. So one risk is loving the AIs too much. If we love them too much, we'll protect them from capitalism, just like we protect our children from capitalism. And we wouldn't let the poor, aggressive capitalism, you know, run roughshod over our delicate AI children, in which case we would no longer let capitalism be so powerful in their world. That's one extreme. At the other extreme is that we would hate or fear the AIs too much, which is we say this world of capitalism running AIs allows the AIs to evolve in arbitrary directions using capitalism if we're scared of them. We say no, they need to stay aligned, so we will limit capitalism's ability to control the values of the AIs, which means we'll slave the AI's values to our human current values, in which case we will not allow them to have healthy cultural evolution, in which case it doesn't fix the problem. So But at the moment there's a hopeful intermediate point where apparently we care enough about AI to let capitalism run it, but not so much as to limit capitalism running it. And so Hey, that's at least a bit of hope.

Roman Yampolskiy, PhD: Sort of in the same direction. What are some contrarian ideas you have where your friends and colleagues disagree with you or just don't see your vision so far in advance?

Robin Hanson: Well so r remember, I'm now an economist, but I came from a technology background, so the technologist vision of many things is just alien to many of my social science colleagues. So as you may know, I'm a cryonics customer and none of my social science colleagues are at all inclined to become a cryonics customer. I had these predictions about the future of AI that previously my AI colleagues were pretty skeptical about, but now I guess are a bit less. I thought that you know brain emulations could be conscious, but my colleague, say, Brian Kaplan, is quite confident that no nothing made out of metal could possibly be conscious. It has to be carbon and a certain kind of carbon at that. so you can see what happens when you are a polymath jumping across fields is you in in the end have no home. Unless you can find other polymaths, because the people you're mostly around are people who are been in an area and comfortable with that and you are passing through.

Roman Yampolskiy, PhD: Cryonics specifically seems like such a obviously good bet. You have nothing to lose and everything to gain. Why is it not more popular, more accepted?

Robin Hanson: It's a great question. I've puzzled over it. so if you ask why do people go to the doctor or the hospital, the usual obvious answer is it to get well. I could be sick. And there's a bunch of things that don't make sense from that point of view, well many puzzles. And so that's part of my book, The Elephant in the Brain. Gotta pitch my other book here. And

Roman Yampolskiy, PhD: I'll put links to all of them.

Robin Hanson: the claim is that Our main relationship to medicine is not that it makes us healthier. It's that we use it to show we care about each other. And so if you think that's our relationship to medicine, you understand that when people hear cryonics, what they hear is abandonment. They hear, you're willing to leave our world and go somewhere else without them.

Roman Yampolskiy, PhD: they're invited to join on the same the same freezer.

Robin Hanson: They they know they're invited, but they still f they they still see you're willing to leave without them and they take that as a lack of loyalty. So I do think the basic willingness to do cryonics is seen intuitively, emotionally as abandonment, as an unwillingness to s stay with them.

Roman Yampolskiy, PhD: I can argue the opposite point. Medicine is about showing you care about your parents or whoever and here I'm buying you a crayonics ticket. Hey, I did everything I can for you, I really love you.

Robin Hanson: Right. So in medicine we really care that you give people the medicine that other people would give you too. That is we want the same sort of medicine everybody gets. That's how we know that we're loved is we get the medicine everybody else. We people don't want to get weird medicine that other people aren't getting from because then your inclination to do that means you're not in touch. So it's the same for gifts, right? When I give you a gift You don't care about so much your private evaluation of the quality of the gift. And I don't care so much about mine. What we care about is whether we're giving a shared social signal about the quality of the gift. So most gifts are evaluated on the what would most people think about the gift metric, not about what each of us actually privately thinks about a gift. And so medicine's a gift and that's how we're evaluating on the shared gift signal metric. So once Cryonics was common and everybody did it, then of course It wouldn't be a problem, but many people are really quite offended if their say spouse is into cryonics. May many mostly cryonics people are men, and their female wives have divorced them because it's too weird and they don't like the idea that their spouse is into something so weird. So people are really quite conformist about such things and they want to not be associated with somebody who's weird. Not just me.

Roman Yampolskiy, PhD: Is your whole family signed up or just you? And I I assume you had this conversation and you were not able to convince your

Robin Hanson: Right. There's actually a New York Times magazine article from a long time ago which interviewed my wife and I over the question so you can see our discussion about it there.

Roman Yampolskiy, PhD: What is the counter argument?

Robin Hanson: It's weird, didn't you just hear?

Roman Yampolskiy, PhD: No no, but like you have nothing to lose. You're gonna be dead, it's not embarrassing, like

Robin Hanson: Y y yes embarrassing 'cause it's weird.

Roman Yampolskiy, PhD: You can do it privately. Nobody has to know you are doing it.

Robin Hanson: Yes they they'll know. People know if you do weird things. Yeah. It's it's dangerous to do weird things and assume nobody will find out. It's better off just not to do weird things.

Roman Yampolskiy, PhD: Learning from you I'll make some contrarian point. So you said people want standard medicine. I think the greatest gift is a friend coming saying, I got you this Chinese medicine, that's the real stuff, it actually works. That's where you're getting over credit.

Robin Hanson: You'll find most people do not find that very, very appreci No. J try this on people you know. You'll find they d they do not like this.

Roman Yampolskiy, PhD: You signed up for Cryonics, I assume you did it many years ago.

Robin Hanson: Long ago, yes.

Roman Yampolskiy, PhD: Do you think the company you with is still the best? Are you full body, just the brain?

Robin Hanson: No, I d I don't have much confidence in that. I'm just too lazy to switch. So I've heard many like other alternatives and they seem plausible, like they might be as good or better, but I just haven't been bothering to switch.

Roman Yampolskiy, PhD: Do you have plans for after they bring you back? Do you have memorized your private bitcoin keys and

Robin Hanson: No. Look, so if I come back I would be one of the few who come back, so I would just be a celebrity on that ground. And in addition, if I come back overwhelmingly likely to be a brain emulation, then I will be the guy who wrote the book on their whole world. So, you know, I will be especially celebrity there. So I gotta figure that if the a world of brain emulations manages to bring me back from my chronically frozen brain, I will be in sufficient demand. I don't need bitcoins.

Roman Yampolskiy, PhD: Do you think there is an opportunity here with eight billion people who are not signed up to make some sort of a contractual we'll pay you now to get frozen, but later every month you'll pay us back forever and ever?

Robin Hanson: I actually think the way we buy medicine is quite imperfect and we could just all be much healthier if we had a more effective way to buy medicine. Cranics is only one part of how that would work. And so one of the things I've done over the years is try to come up with better institutions for many things. That's why I did prediction markets, but I also have, say, a concept for a better way to buy medicine. It's what I call health agents. So today you have two main ways to buy medicine. One is fee for service, where everything they recommend, you have to pay for it immediately, and then they have an incentive to do as much as possible. The other way to buy medicine is called capitation. You pay so much a month and they do whatever's necessary, and their incentive there is to do as little as possible. Those are both bad incentives. What you really want is for them to only do the things that are cost effective, where the cost of the treatment is less than the value you get in terms of reducing your chance of being dead or disabled or in pain. So the fix here is to actually merge health and life insurance, actually, and disability and pain insurance, so that when bad things happen to you, bad things happen to your medicine provider. And if you get if you make the level at the right level, they will make the same trade-offs as you would. So for example, if your life is worth $10 million to you, say, and then they lose $10 million if you die. Now if they were saying, is this fifty thousand dollar treatment worth the cost? They will ask, well, does it cut your chance of death by 0.5%? And if it does, they'll say it's worth it. And now you can trust them to make those judgments because they will have the same trade-offs as you would. If when you died, the amount of money they would lose is your value of your life to yourself. You could also have disability insurance, pain insurance again. And now this would be a way such that not only would they only want to give you the cost-effective medical treatments, but if in fact cryonics was cost-effective, they would want to give it to you. That's the way you want to set up your relationship with them is where they would make the choices you would make if you knew what they knew.

Roman Yampolskiy, PhD: Would that not cause them to select only healthy people and avoid all preconditions?

Robin Hanson: Of course. Well what you'd have is an auction for each patient of who will cover them at what cost and then you discover, you know, who's charging how much. So

Roman Yampolskiy, PhD: Yeah, and the moment you get sick they drop you because you're no longer a profitable investment.

Robin Hanson: You can have longer term contracts. So y you you if you only have short term contracts you only get short term insurance. If you have long term contracts you get longer term insurance. So there are ways to set up longer term contracts.

Roman Yampolskiy, PhD: And in terms of your selection, are you a full body? Are you just a head? what is the

Robin Hanson: I'm just ahead. But again, because I'm focused on the brain emulation revival scenario, I'm not expecting to need the rest of the physical body.

Roman Yampolskiy, PhD: You don't think there is any usefulness in other neurons, spinal cord, anything else through the body?

Robin Hanson: It's a bet, but again, it's a matter of cause. But if I switched, turns out some of the other providers just do the full body anyway, in which case, you know, then you're a little less risky. But clearly the overwhelming risk is just whether this organization lasts long enough. And so these other issues are relatively minor. So y

Roman Yampolskiy, PhD: And what if it's not a secret, what company are you signed up with?

Robin Hanson: I'm I signed up with Alcor at the moment, but that was the sort of the main company long ago when I first signed up, but now there are other plausible competitors. There's in fact a company that's tempting because they might do room temperature preservation, which I think allows you to last in a wider range of scenarios. So If the organization that has you frozen fails, and they fail to top off the liquid nitrogen, then you just thaw and that's the end of it. But if they're storing you in pla what's called plastination, or basically they infused your brain with plastic and are storing you at room temperature, well they can just put you in a pot under the floor somewhere, and you could last through all sorts of disruptions to society.

Roman Yampolskiy, PhD: And just keep you at home next to the T V.

Robin Hanson: Well, you want to keep wherever it's most likely just to last, basically. But yeah.

Roman Yampolskiy, PhD: A lot of decisions, and it seems like the most important decision you'll ever make. Is there any organizations reviewing those companies and advising effective cryo preservation startups, non profits?

Robin Hanson: It's it's it's still a really tiny industry, so that's the key thing to understand about it is there's lots of ideas for like doing better things, but if you hardly have any customers, you can't afford much infrastructure or research in order to support them. And that's been the nature of the industry. It's just hasn't been able to grow very large for a long time.

Roman Yampolskiy, PhD: So I'm cryo procrastinating myself and the big part of it is all the paperwork I have to deal with. If I could press one button and they just took money from my account and I was signed up, I would be a client.

Robin Hanson: But well that's some of the new entrants are trying to solve that problem. They're trying to make the paperwork easier. Absolutely.

Roman Yampolskiy, PhD: How do I invest in those companies? That seems like a very big market. Lots of people should make that

Robin Hanson: Indeed.

Roman Yampolskiy, PhD: decision by copying smarter people and advice from AIs, I guess.

Robin Hanson: Again, the main limitation is just so few customers. But the a big question is why can't we get a lot more customers? So there are more are there are about as many people who have their ashes thrown into space when they die as who get chronically frozen. It's that that level of a small industry. And people who get their ashes thrown into space, their their spouses don't divorce them for that, because they're not mad. Because nobody thinks that'll work. But what people are mad about, you see, is the fact that you think this will work and that you could leave them and and abandon them. Yes, you're offering, but s even so the

Roman Yampolskiy, PhD: But again, you offering to take with you, this is a very big difference. I'm not saying you are not allowed to cry preserve yourself. I'm saying come with me forever.

Robin Hanson: Okay, but y you have to see just how small this industry is and you need an explanation. Why is this industry total times? Because I've taught health economics years ago, and you know, I would give an a lecture including on chronics and then I would ask what fraction of them think this would work, and a substantial fraction of students think it'll work. So like at least 10% of the public thinks this would work. But not ten percent of the public is not buying this product. The product is maybe a few thousand people have ever bought it over many decades.

Roman Yampolskiy, PhD: Wow. But what's their average

Robin Hanson: That's the question.

Roman Yampolskiy, PhD: IQ? That's the question.

Robin Hanson: It's not especially high IQ who are buying it. Again, high IQ people are also not buying it. So but so this is my world. As a social scientist who knows a lot about you know various technologies and things, I ask about trying to understand these things. So many technologists, they see a technology, they think this should make sense to people, and then they don't want it. And now they're puzzled, why don't people want it? So you're using some model of what people should want why, and obviously some part of that model is going wrong. And so this is what I specialize in as a social scientist. I can try to understand what assumptions we're making about things that could be wrong. Say for prediction markets or cryonics or health, the health insurance I'm offering, this is a basic fact about our world. We have a lot of great ideas for social innovations that are not actually being adopted, even technological innovations. And that's in part because we misunderstand what people want and want.

Roman Yampolskiy, PhD: So I think initially it was very expensive, maybe a quarter million dollars and current dollars. And funerals were kinda cheap. I think now we're getting to the point as you said, it's almost affordable room temperature process for cryopreservation, and funerals got ridiculously expensive. I think at some point they're gonna flip what is a cheaper alternative.

Robin Hanson: It it's it's it's more about weirdness and more about the signal ascent. But look, this is most of medicine people are not buying it for how it affects their health. This is maybe the scary thing. So I I don't know if you know. We have some large randomized experiments in medicine where we gave some people access to pretty cheap free medicine and other people access to expensive medicine. The people with cheap free medicine, they got more medicine and they weren't healthier. This is the basic fact about medicine in our world. There's a puzzling lack of correlation between randomized experiments where people were induced to get more medicine all across the board and they were not healthier. So apparently in our world, on the margin, medicine does not improve health. So that shows a pretty huge failure of the institution of medicine and the way we buy it. We're spending a lot on it. In the US, we spend 18% of GDP on it, but we're not actually getting much value from it.

Roman Yampolskiy, PhD: Don't they need to show that it's actually effective to get the medicine approved? Versus placebo?

Robin Hanson: No, they have to they have to go through a performance theater of doing, you know, studies, but they don't actually have to show. Again, what what you see in a you know study is different than what happens in the world and yeah, yes, the w world is just different from the studies. A as you may know, we have vast, you know, million, I don't know, articles a year published in medicine or something. most of which suggest various treatments help, and yet when we do this aggregate study, we see no aggregate effect. So that's very puzzling.

Roman Yampolskiy, PhD: Which is so moderna get approved for a new cancer

Robin Hanson: Yeah.

Roman Yampolskiy, PhD: drug and the stock doubled in minutes.

Robin Hanson: Yep.

Roman Yampolskiy, PhD: Can you do the opposite? Can you investigate drugs which are useless, sell them short and be very happy with yourself?

Robin Hanson: No. No, because just because a drug is useless doesn't mean the company won't make money on it.

Roman Yampolskiy, PhD: I guess that's the novel idea we're learning. So it's a lot of what is signalling. That's what you

Robin Hanson: Right. So that's why I this a way to buy health by merging health and life insurance is so promising, because we're in fact just really badly doing badly choosing how much what medicines we buy now. We're just in fact not choosing to buy stuff that on the margin helps. So a institution that was just very co attentive to the cost effectiveness of things would just make very different choices than we're making now. And that would be hugely valuable.

Roman Yampolskiy, PhD: You have some very interesting unusual opinions. What are some examples of you being wrong through your history of prediction or making statements about policies?

Robin Hanson: Well, obviously every time I change my mind, in some sense I was wrong before and relative to after, so I've continued to move into new fields and then change my mind about things. That that's the reason why I keep going into new fields. So, for example, I didn't realize just the social obstacles to prediction markets, the ways in which organizations are reluctant to use them for various other concerns they have and issue processes that are going on. I, of course, didn't even realize that about medicine when I first I taught told you this idea for mix for merging health and life insurance. And at that time when I had that idea, I didn't understand that in fact on the margin medicine doesn't help and we have all these obstacles for being willing to apply such medical ideas. and of course I didn't understand the differential relationship between experts and elites in our world. So For much of my life I was trying to be the best expert I can, coming up with the best expert calculations of what would be good policy. And I didn't realize the world mostly doesn't want to hear such things from experts. They want to hear them from elites. And elites are just a different set of people with a different set of criteria that define who they are. And that's one of the main reasons much of my proposals are wasted effort, is the world doesn't want to hear proposals from experts. They want to hear them from elites. So functionally, experts are people who know things, and elites are people who are impressive. and well-connected and you know admirable. So, you know, you become elite either by, you know, a whole bunch of different features are sort of averaged together, charismatic, handsome, rich, well connected, smart, articulate, you know, engaging, pretty, you know, those are all features that can qualify you for being elites. And most organizations have elites at the top. That's what people want, because elites are the sort of people we coordinate around following. Elites we we feel it's okay to follow an elite and to do what they say because that we're admiring a prestigious person instead of being dominated by someone with power. And elites are socially more connected, better able to negotiate with other elites. They're more plausible that if an elite takes a lead on something, something will happen. If you follow it, there's a thing you're following, it'll go somewhere the world. Just wants elites to lead social movements, to lead political movements, to lead organizations, and to lead proposals for change like those that I make. So I did not realize the world was not interested in proposals by me on such things because I'm not elite and can't very credibly claim I will become one. I am more credibly an expert.

Roman Yampolskiy, PhD: Given your views on medicine, what is your relationship with your doctor is like if you have one?

Robin Hanson: I have been using Kaiser for a long time, which then is the capitation organization with an incentive to do less, which has been okay, mostly because I didn't want them to do too much. So that's n now that GLP like is a thing that seems to work but Kaiser doesn't want to do it, now I face more of a conflict there.

Roman Yampolskiy, PhD: You are very successful professor. You have multiple books, you invented modern prediction markets as they are seen today. Why are you still an associate professor? What's happening there?

Robin Hanson: I was put up for full professor and then I was rejected on the grounds that they did not understand my research, which seems like an odd grounds, but my understanding I was politically prominent at the time. People didn't like some things that people had said about me online. That's was in the woke era, and so that's the sort of thing that happens. But being a full professor is actually a pretty small gain where I am. It's a very s minor salary bump. It puts you on more committees, which is not necessarily what you want, so I have

Roman Yampolskiy, PhD: Does

Robin Hanson: Mm.

Roman Yampolskiy, PhD: it classify you as a elite professor at least or not enough?

Robin Hanson: Say say again.

Roman Yampolskiy, PhD: Do you become an elite professor at least or

Robin Hanson: So so at least at my university, being a full professor, as they call it, is a very small salary bump. It opens you up for more committees, responsibilities, and then it's just this sort of sign of respect. But in terms of concrete benefits, it's small. As you know as a tenured professor, the biggest fact about being a professor is this idea of having tenure and getting to do whatever the hell you want with your whole career once you get tenure. That's the really big win. I've got that win and I'm very grateful for it.

Roman Yampolskiy, PhD: I I know you did, but I also remember as you said, during certain times there was angry mobs coming after you and I think they forced you to make statements. Did tenure not provide enough academic freedom or you just decided it was the right decision?

Robin Hanson: so just in general, when you're part of an organization and when people in the organization come and ask you to make a compromise, even if you might think in some principal sense, why should I have to compromise? you often say, But these are people I know that I work with and they would like something from me, so why shouldn't I give them some of what they want if it's not very expensive for me, right? That's just part of being a social creature in a social world is accommodating your associates when they want something more than you do. So I that seems to me the right sort of social stance to take.

Roman Yampolskiy, PhD: And is the situation you're describing preceded or followed your denial for full professor?

Robin Hanson: various things have happened at different times before and after.

Roman Yampolskiy, PhD: Okay. I I have a few kind of meta questions at the end. So if you had to bet on the number of views this episode will get, what is your prediction here?

Robin Hanson: Can you give me a median for your previous episodes?

Roman Yampolskiy, PhD: I think I'm averaging fifty. The last one max tag mark just

Robin Hanson: Fifty thousand?

Roman Yampolskiy, PhD: hit sixty eight. I have a very small sample size.

Robin Hanson: Okay. Well I guess I'm guessing that some of these people are like superstars in AI and I'm not gonna I'm not gonna beat that, but I think I'm gonna be above median, below average.

Roman Yampolskiy, PhD: Medium below average. So we'll see how that prediction plays out. It's a self referential betting market which is easy

Robin Hanson: Yeah.

Roman Yampolskiy, PhD: to manipulate by those interested in this. A very common question I give my guests is if you had to come up with a clickbait title for this episode, what would that be?

Robin Hanson: I guess Robin Hansen tells all.

Roman Yampolskiy, PhD: I like it. This is this is gonna sell really well. I I wanna thank you for taking your time. This was very diverse in coverage. I find you fascinating in how much you managed to accomplish across different fields. And I think in many many debates and bets you came out on top, I would say, at least so far. And I really do hope your predictions come true and we

Robin Hanson: Well, let's talk again

Roman Yampolskiy, PhD: Do have

Robin Hanson: in three years and see whether fooms happened by then.

Roman Yampolskiy, PhD: I absolutely would be delighted to have you back and yeah we can definitely see if in particular that prediction worked out. I personally do think once we automate science and engineering the process could be very fast, but I find your arguments from logistics and hardware and other sides of that equation to be very persuasive as well.

Robin Hanson: All right, till we talk again.

Roman Yampolskiy, PhD: Thank you so much.