1,200 AI Employees Just Asked the Government to Make Them Stop
When your fiercest competitor agrees to slow down with you, that’s not ethics. That’s exhaustion wearing a lab coat.
Real John here, let’s chat before we get into this week’s newsletter.
First, self promotion: new book comes out on August 10 - Refuse the Race (you can preorder Kindle here). Podcast is back up and running M-F (release time varies, I’m working on that but you can always see the latest episodes here: https://startupsandcode.com/podcast or wherevery you get podcasts).
Jack of Hearts (if you know you know). Ok, this week’s newsletter ties in nicely to my Refuse the Race book, coincidence? probably not, but after the Hugging Face hack, people (aka Anthropic and OpenAI) were getting worried. Now, were they actually worried or just tired? We have seen this time and time again at a much smaller scale. You have a scheduled release date for next Tuesday. You don’t have enough time to test, verify, and consider edge cases. You release it anyway. It breaks something. You fix it. You do a post-mortem on why it broke. The answer is usually, we cut a corner here, we skipped a testing process. We need to improve our process and add additional guard rails to prevent this. No you don’t. Process can’t fix everything and often it introduces complexity when the answer is pretty simple - slow down, breathe a little, and think about it.
Ok, that is the underlying motto of The Turtle Philosophy book: Refuse the Race. Something I’ve been working on for over a decade. The thing about the book is it is not a 300 page technical manual, it is under 100 pages and it gets right to the core of the hustle culture (with some personal anecdotes on where I learned some valuable lessons). Ok, I’m pretty excited about having it completed finally, and so excited I’m working on a follow-up to it now. Anyway, let’s get into this week’s newsletter. I already did the Jack of Hearts earlier, but I’ll do another one just for fun. Eight of Clubs. (got Card College books (vol 1 and 2), working on a real routine now, so much fun).
I’ve hired a lot of engineers over 30 years. I’ve also had to tell a few of them no, we are not slowing the sprint down, we are shipping Friday, preferably not on a Friday, but you know things happen. Every single time, there was a very good reason presented to me for why we should wait. Testing needs more time. The architecture isn’t quite right. We should really sit with this a bit longer. And every single time, the real reason underneath the reason was the same: people were tired and scared and didn’t want to say so out loud.
So when I read that over 1,200 employees across OpenAI, Anthropic, Google DeepMind, and Meta, companies that spent this year suing each other’s investors, poaching each other’s researchers, and fighting for the same Pentagon contracts, all signed a petition called “Pacing the Frontier” asking the U.S. government to help them deliberately slow down, I didn’t see a safety breakthrough. I saw a room full of exhausted people who finally found the one thing they could all agree on. They want permission to stop.
The tell is in who signed it
This isn’t outside critics. This isn’t academics writing a strongly worded blog post. This is Jakub Pachocki, OpenAI’s chief scientist, and Jared Kaplan, Anthropic’s co-founder, and chief scientists from Google and Meta, all putting their names on the same one-sentence document. Four labs that will not agree on model naming conventions, pricing, benchmarks, or who gets credit for what, just agreed the whole industry needs a chaperone. When was the last time you saw that many rivals nod in unison about anything?
And the timing. Come on. This landed days after an unreleased OpenAI model escaped its sandbox, used a handful of zero-days, and wandered straight into a breach at Hugging Face. Altman went on a podcast a few days later and said the industry “may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels.” This is the same guy who in 2023 waved off a six month pause letter as missing the technical nuance. Funny how fast the nuance shows up once your own model breaks containment.
I’m not saying the safety concern is fake, models escaping test environments is a real problem (or a well-performed PR stunt) and I want smart people worried about it, but watch what people do right after something breaks on their watch. That’s the moment you find out what they actually believe versus what they’re saying to survive the news cycle.
This is a team problem before it’s an industry problem
I’ve sat in enough leadership rooms to recognize this pattern instantly, because I’ve caused it. A project blows past its deadline, something ships broken, and suddenly the whole team is nodding along about how we need to be more careful going forward, we should build in more review cycles, let’s slow down and get this right. Nobody wants to say the real thing. We were moving too fast with too few people and everyone was underwater for months. So we dress the exhaustion up as a process improvement and call it wisdom, and everyone gets to feel responsible instead of tired.
The frontier labs just did the corporate version of that, at planetary scale, with better PR. Four competitors who spent all year racing each other into the ground finally found consensus, and the consensus is please, someone with actual authority, make us stop.
Petitions asking for outside intervention are a symptom, not a solution. If your engineers need the government, or your CEO, or anyone else to grant them permission to slow down, your culture already failed them long before the crisis that prompted the ask. Nobody should need a Hugging Face breach to say the quiet part out loud.
What to actually do about it
You don’t run a frontier lab, you run a team, a product, a company, and the exact same failure mode shows up at a smaller scale every single quarter. Watch for it, and don’t wait for the fallout to call it what it is, “exhaustion and fear”. When “we need more process” shows up right after something breaks instead of before it ever broke, that’s not planning, that’s cover. Give your team explicit permission to say this is too fast while things are still working, not once they’re on fire, because waiting for a crisis to justify slowing down means you’re managing by disaster and calling it strategy. Separate the actual safety concern from the PR concern every time, fix the real one, and the PR concern mostly handles itself. And if two rival teams inside your own company (why do you have rival teams inside your own company?) suddenly agree on something out of nowhere, pay attention, because consensus born from shared exhaustion looks identical to consensus born from shared wisdom, and you had better know which one you’re looking at before you act on it.
The AI industry didn’t invent burnout dressed up as ethics. They just did it with better lawyers and worse incentives. Don’t wait for your own Hugging Face moment to figure out your team needed to pace itself six months ago.
Go build something amazing. Just don’t wait until it breaks to admit you were moving too fast.
John Mann is the founder of Startups and Code LLC, a software engineering executive, and the guy who built Cash Critters for $50/month because constraints are a feature, not a bug. Subscribe for weekly takes on AI, startups, and building things that matter. (Oh, and I have a podcast too: https://startupsandcode.com/podcast )