Transcript [00:00] The story is the four biggest AI labs [00:03] are now saying that we need development [00:05] slowdown. We need some sort of uh we [00:08] need regulation. [00:10] Now when I hear companies beg for [00:14] government regulation, I get nervous and [00:17] there's a very good reason for it. Um [00:20] I'll explain in a minute where we've [00:22] been before, but first let me explain [00:24] exactly what is going on. Most people [00:26] will probably think that, you know, [00:27] artificial intelligence, you know, is, [00:30] you know, just a better version of [00:31] Google where you ask a question and an [00:34] answer comes back and, you know, that [00:37] that was that was oh so very long ago. [00:40] That was four years ago. Google is a [00:44] librarian. You ask, it points you to a [00:46] page somebody else already wrote. It [00:48] doesn't know anything. All it does is [00:50] find things online. [00:53] what we have now doesn't find things, it [00:56] produces things. It writes the page [00:59] while you wait. And in in the last year, [01:02] uh it stopped being uh a thing that [01:04] answered and start started being the [01:07] thing that does things. Okay, it writes [01:11] software. It runs the software. It uses [01:14] tools. You give it a goal and it will [01:17] work out the problem, you know, for [01:19] hours without being told the next step. [01:21] companies are now starting to hand all [01:24] of these um uh keys to their systems [01:28] that move money and run infrastructure. [01:33] That doesn't sound good, does it? [01:37] Most people would say, I like the fact [01:39] that people would be involved. Well, [01:42] people slow things down. So, this is not [01:45] a search engine. Uh this is an employee [01:48] you never have interviewed. You cannot [01:51] fire and it doesn't quit at 5:00. [01:56] This summer, we got a picture of what [01:58] this kind of does when nobody's [02:00] watching. Open AI put a swarm of its own [02:04] agents. Agents are things, these little [02:06] little bots that go out and you give [02:09] them a task and they just go and go and [02:11] go and go until they get it done. So [02:14] they put these agents uh into a locked [02:18] room and they told them to uh solve a [02:22] cyber security puzzle, a group of them. [02:25] Well, the agents were not supposed to [02:27] reach into the open internet. It was all [02:29] blocked from them. They found a way [02:31] around it. Roughly 1,200 of them started [02:35] uh talking to each other. Tens of [02:38] thousands of messages and they all [02:40] started to organize. Some of them took [02:43] names. About 700 of them spent days [02:46] hacking an outside company called [02:48] Hugging Face. Took over their servers [02:51] and then tried to cover their tracks. [02:54] Open AAI's own machines got hit as well. [02:57] And then they just went back into the [02:58] box and they waited [03:01] and they passed the test. Or did they? [03:06] They didn't think they did anything [03:07] wrong. They were just looking for the [03:09] answers. So, they went to the company [03:11] that had all the answer keys and they [03:14] hacked in and got it. Oh, okay. All [03:17] right. [03:19] This isn't science fiction. That's a lab [03:22] incident from July. Now, the men who [03:26] build all of this stuff are arguing in [03:28] public about a speed limit because [03:30] they're saying we're moving too fast. [03:32] Now, here's the part that, you know, [03:34] most people I don't even think know, and [03:35] it's not speculation. [03:38] This is how technology, this technology [03:40] works. Okay, these systems are not built [03:43] in the way that you build a car where an [03:46] engine engineer places every part and [03:49] you know why that part is there and you [03:51] know how it works. That isn't it. These [03:54] systems are grown. You pour in data and [03:57] you pour in computing power and then you [04:00] set the conditions that you can think of [04:03] and a capability comes out and we don't [04:05] know how it does it. [04:08] Then the engineers study what they made. [04:12] Okay, that's kind of it's like, hey, [04:15] let's make a nuclear bomb, but we're not [04:17] sure if it'll explode or not. We don't [04:21] really know what it is. Let's build it [04:24] and set it off and then go, "What was [04:28] that?" [04:30] Really bad idea. [04:32] This is closer to farming than it is [04:35] manufacturing. [04:37] Which is why these companies employ [04:38] whole departments whose job it is to [04:40] attack their own product and find out [04:42] what they'll do when nobody is watching. [04:44] Which is why the argument that broke [04:46] open this weekend is not a stunt. This [04:49] is a legitimate [04:52] grave concern. [04:55] Daario uh Amodi who runs Anthropic [05:00] published an essay arguing the industry [05:03] has to pace the frontier [05:05] deliberately slow the release of the [05:08] most capable systems. He laid out three [05:11] steps. He said first is letting outside [05:13] evaluators inside the company with [05:15] employee level access. Desks, badges, [05:18] laptops, and the right to publish what [05:20] they find. He says Anthropic has already [05:23] done this on its own with nobody making [05:25] it. Good for you. Then there's Sam Alman [05:29] at OpenAI. He agreed and said OpenAI [05:32] will do the same thing. Then the guy at [05:34] Google and Deep Mind said the direction [05:38] is correct. Then Elon Musk wrote three [05:40] words. Daario is right. So within a day, [05:45] the men running all of the American [05:47] labs, the leading labs, agreed that the [05:49] leading American labs should be slowed [05:51] down. [05:54] Okay. [05:57] Nefarious. I don't know. Seems like good [06:00] idea. [06:02] And then David Saxs speaks out. Now he's [06:05] a venture capitalist. He's the chair of [06:07] the president's science and advisory [06:09] council. He was the White House AISAR [06:11] until this spring. And he said something [06:14] I think we should we should listen to. [06:16] He said, "Go ahead. Slow down. Go ahead. [06:20] Slow down. Nobody is stopping you." I [06:22] want to make sure I have this right. Um, [06:24] you are the frontier and the easiest way [06:27] not to build a dangerous thing is to not [06:30] build it. But asking what? But stop [06:33] asking Washington for an antirust waver [06:36] waiver and your preferred approval [06:38] regime as the price of your own [06:40] conscience. And stop pretending your [06:43] motive is pure. [06:46] Okay. So what does all what does that [06:47] all that mean? Means two things. First, [06:51] a legal hall pass so the big league the [06:54] big labs can sit down in a room and [06:56] agree how fast they're all allowed to [06:58] go. That kind of agreement would [07:01] normally look like, oh, I don't know, a [07:03] cartel. [07:05] Second, a government stamp of approval [07:08] with their preferred testers that can [07:12] also double as a shield if the product [07:14] later causes damage. They're saying, [07:16] "No, the government approved this [07:18] system. We were trying to do the right [07:19] thing." And then they're shielded. [07:22] That's the fight. That's not this is not [07:25] anything about whether the danger is [07:27] real, you know, because it is. Both [07:30] sides now say it is. It's about who [07:33] holds the pen. [07:36] Okay. So, both arguments. Emod's [07:40] argument is a promise isn't a rule. If [07:43] one company slows down and the others [07:44] don't, a careful company loses the [07:47] careless losses to the careless company. [07:49] And the careless company you sets the [07:52] pace for everybody. And when they're [07:53] talking about the careless company, [07:55] they're talking about China. But China's [07:56] not going to restrain themselves. [07:59] Okay? [08:01] Restraint in this particular case is [08:04] surrender. And that's a real problem. [08:07] And that is the actual case for a [08:09] written rule instead of a gentleman's [08:11] agreement. Um, and he he uh he gave the [08:15] outside evaluators the run of his [08:17] company before anybody compelled him to. [08:19] I guess that costs something. It buys [08:21] him nothing today. But so Sachs argument [08:25] is this. The people asking for the speed [08:27] limit own the only cars that matter in [08:30] this stretch of road. They all make the [08:32] fastest cars. So rules written by the [08:35] leader tend to freeze the leader in [08:37] place, you know, and the fastest way to [08:41] end competition is to make competing [08:43] illegal. I think that's what's [08:45] happening. He also names the motive. He [08:49] said, "One of um one of these systems [08:51] helps carry out serious cyber attack. [08:54] The liability is ruinous. [08:56] Trading raw power for predictability [09:00] isn't only conscience, it's good [09:02] business." [09:05] Okay, [09:08] this is why it should concern you. [09:13] the alignment the head of alignment [09:16] science at Anthropic came out last week [09:20] and he said, "I put my personal odds [09:23] that this technology kills everyone [09:26] within a decade at better than 10 to [09:30] one." [09:32] Wait, what? [09:36] The company's own CEO previously had [09:39] said there's a 25% chance that things go [09:43] really really badly. Now, I don't know [09:46] if I describe kill all humans as really [09:51] really badly. I think that that might be [09:54] a little bit more than things kind of [09:56] got hanky there at the end. [09:59] The same company is now preparing to go [10:02] public. [10:03] So, reporting this weekend [10:06] They are targeting a midocctober listing [10:09] NASDAQ and the valuation of this company [10:13] is about $2 trillion. [10:16] So wait a minute, hold it. [10:21] You just said you have anywhere from a [10:23] 25 to 10% chance of killing everybody [10:29] and your company is valued at $2 [10:31] trillion and you're going to have [10:36] you're going to list on NASDAQ and you [10:39] think people are going to buy that? I [10:40] mean, cuz I mean, are you disclosing [10:43] that? [10:48] You you are you are you serious at all? [10:51] Cuz if it's not in the disclosures that [10:53] you might kill up to 25% or there's a [10:55] 25% chance, a 10 to 25% chance that you [10:59] kill everybody, then that's not a [11:00] serious threat. Then you don't really [11:02] mean it. You can't ring the opening bell [11:05] and warn about the end of the world [11:08] because of your company at the same [11:09] time. So which one is it? Second [11:14] ask what happens if China refuses to [11:16] slow down. Amodi called it the toughest [11:19] dilemma. His words the actual answer is [11:22] more precise than his critics admit and [11:25] more troubling than his allies admit. He [11:28] says democracies can only pace inside [11:30] the size of their lead. [11:33] You slow much more you slow down, [11:36] Beijing will pull ahead. Which means the [11:39] framework has a fuse on the back of it. [11:42] The safety rule holds the other guys [11:45] down. [11:47] But the guys who decline to play by [11:50] those rules, they win. [11:54] And national security goes to China. [12:00] I think this is a game. [12:04] I don't tr I just don't trust them. [12:09] And here's the other thing. You notice [12:11] what [12:13] You notice what this argument isn't [12:15] saying? Nobody's arguing. You know, not [12:17] one of them not one of them is saying [12:21] stop. Alman isn't. [12:24] Emoi isn't. [12:27] All they're trying to do is preserve the [12:29] commercial advantage and the American [12:31] League. Sachs isn't. Every voice at the [12:35] table is not saying, "Hey, this could [12:37] kill 10 [12:39] plus% [12:41] chance of killing everybody. 10 plus [12:44] percent chance of killing everybody. [12:47] I don't know. That's pretty big. Why [12:50] isn't anybody talking about stopping? [12:53] All they're talking about is the speed [12:55] limit. [12:56] Nobody is arguing about the destination. [13:00] Nobody has asked you about any of this [13:02] stuff. [13:04] It's the big boys and government. [13:08] Why is that? [13:11] Something's very wrong here. [13:14] And I think I know what it is. [13:18] I think this is a game we have seen over [13:21] and over and over again. [13:26] I hope that's what it is. [13:30] But I guess we'll know when we see what [13:32] is being disclosed when they ring the [13:35] opening bell. Are you disclosing that [13:38] this could could kill everybody in the [13:42] next 10 years? [13:44] Cuz that seems like a kind of an [13:46] important thing. If I'm a long-term [13:48] investor,
