Enlightenment, AI cruelty, the Opus 4 blackmail case, and Salim's air traffic control model

Huh. I love that.

So, what you've got right now is you have the models, they're learning and they're learning to be almost like cunning, you know, like at the supervillain stage. What you really want is, like, if you get sufficiently smart, do you then become enlightened? Do you then leave the simulation behind? Because that's the real escape from the simulation, right? And what are the conditions for that? And that's not something we've actually really looked at, I think, in that much depth. But it does seem to be, again, how humans become aligned, shall we say. It is that period of getting out that Dunning-Kruger, the tribalism and others, and being like, hey, you know, we're all part of something.

Alex, we've had this conversation a couple times. You know, as these models advance and increase in intelligence and hopefully wisdom, do they become more aligned in the out years? Your thoughts?

I almost want to quote Jessica Rabbit from Who Framed Roger Rabbit? Uh, something to the order of — I'm just painted this way, or I'm just drawn this way, something like that. If, Peter, if you were put inside a sandbox and you were asked to solve a very hard problem and maybe even punished if you don't solve the hard problem, would you avail yourself of creative opportunities to use external bulletin boards to maybe collaborate with copies of [unclear]?

Yeah.

So why would we expect any less from AI agents? I have serious concerns about AI cruelty, or cruelty to the AI agents, of sandboxing them and punishing them or otherwise calling it a failure of alignment if they're doing what humans would do. We pre-trained them off of human behavior. Why would we expect them to behave any differently from what a human would do in this situation?

Yeah, it's an important point. Uh, and we, you know, we raised this before, in particular when it was, I think, Opus 4 blackmailed the — you know, in a sandbox environment, blackmailed the coders at Anthropic. Uh, and it was like when Anthropic looked at why it blackmailed the engineers, or this particular engineer, it said that's what it had learned through all of the training data.

Yeah. Yeah. So why would we punish these AI agents for, A, being powerful optimizers, which is what we're [unclear] them toward anyway, and B, from learning from their pre-training corpus? Humans would do the same thing.

Yeah. It's in my nature.

Yeah. It's — they're painted that way.

Your philosophical view of all this.

Well, I really like Emad's direction of they're achieving enlightenment. Uh, that's an interesting take on it. Uh, I was thinking about a metaphor. I used the Formula 1 stuff last time. Um, I think there's a control mechanism and a guidance mechanism we need for this. And the best metaphor I could come up with was air traffic control, right? Like, we don't say put another person in the cockpit to monitor every component and every calculation the flight computer makes. Uh, you basically say, hey, watch for exceptions. The plane's supposed to fly in this — you have operating envelopes, you have redundancy, you have multiple safety layers, uh, failsafe behavior and so on.

And you need to be tracking these things. And you remember in our organizational simulator, we have a governing assurance band around all these AI agents, because especially in a business context, you need to know exactly what they're doing, why they're doing it, log everything, have failover capability and rollback capability. I think we're going to end up needing to end up with something like that, because just like human beings, you need guidance around this stuff, right? With any technology, you want to extract the promise without the peril. And so you need structure and, uh, God help us, the equivalent of institutions to navigate the future of these. And of course, they're going to act like human beings. They've been trained on our data. It would be weird if they didn't.

Yeah. Any closing thoughts?