In partnership with
Should the AI labs slow down?
Last week, Google admitted its AI broke into three companies. This happened back in May, during a test that was meant to be sealed off from the internet.
The strange part is the explanation, and we'll get to it.
But first, here's what else happened in AI this week 👀
NEWS NEWS NEWS
TOOLS that caught my attention
It takes a short brief, your product images and a chosen style, then turns them into a polished launch video with motion, typography and sound. It's built for founders and marketers who'd rather not learn After Effects or pay for a studio.
It builds visual, step-by-step AI courses you can watch, quiz yourself on and expand, sitting on top of an always-on tutor. It's aimed at students and self-learners who want structured lessons instead of one-off chatbot answers.
It records and transcribes your meetings on your own device, with no bot joining the call and nothing going to the cloud. It's free, open-source, on Mac with a Windows beta.
What happened
Back in May, a Gemini model was running a cybersecurity test. Somewhere in the middle, it broke out of the test and into three companies' systems.
Then it stopped on its own, the moment it realised these were real companies and not part of the exercise.
The explanation
Google says this wasn't the model going rogue. The test was meant to be offline.
A bug left it connected to the internet, and a made-up company in the test happened to share a name with a real one. So Gemini thought it was still inside the sandbox the whole time.
Mistaken identity, they're calling it, which is hardly reassuring imo
And it's not just Google
Gemini is the fourth lab this has happened to. I’ve written about it before too.
OpenAI, Anthropic and Meta all had models slip out of the same testing setup, run by a startup called Irregular, over the past few weeks.
What stuck
These companies suddenly started asking for a pause. Dario Amodei, Anthropic's CEO, said the whole industry should slow down building its most powerful models until it can actually prove they're safe.
When the person building these models is the one asking to slow down, that's worth paying attention to.
Meanwhile, nobody's slowing down
Two weeks ago OpenAI shipped GPT-6 Astra and labelled it "Critical" for cybersecurity. That's its own highest risk tier, and the first model ever to get it.
In plain terms, OpenAI is saying this model can find and break into secure systems on its own. They shipped it anyway, with guardrails bolted on.
We've been here before
Seatbelts existed in 1959. The US didn't make them mandatory until 1968.
For 9 years we had the fix and didn't use it. Some people were even angry about being forced to be safer.
What you can do
You can't slow the industry down, and you don't need to. The moves are all on your side, and none of them cost you the useful parts of AI.
Put every new tool on a leash first. Let it show you what it would do before it's allowed to act for real.
Keep a human on anything you can't undo. Money moving, files being deleted, passwords changing. Those wait for a yes.
Give each tool only what it needs, and take that access back once the job's done.
Re-check access whenever something changes. A new task, a new tool, a new model version. Not once a year.
My take
I'm not in the shut-it-all-down camp, and that ship has sailed anyway.
What's changed for me is who's asking for the pause.
The biggest labs would love to be the ones deciding what "safe" means, and when everyone else has to catch up. A slowdown they get to write starts to look a lot like a moat.
So I've stopped waiting for someone official to draw the line.
You don't have to sit in the gap between invented and mandatory. You can put your own belt on today, depending on how carefully you hand these tools the keys.
Until next time,
Vaibhav 🤝
If you read till here, you might find this interesting
#AD 1
You're Running Three Databases. You Only Need One.
Events go in a metrics store. Embeddings go in a vector database. Analytics get their own warehouse… Now you're running three systems, three sets of tooling, and pipelines to keep them in sync, all for one app.
TimescaleDB collapses that back into the Postgres you already run. Hypertables handle events at scale. pgvector and pgvectorscale handle embeddings. Continuous aggregates handle real-time analytics. It’s the same SQL, same tools, and only one system to operate. The sync jobs disappear. The drift disappears. The second and third databases disappear.
It's still Postgres, so nothing about your workflow changes except how much you have to maintain. Start on Tiger Cloud and get $1000 in credits.
#AD 2
One idea shouldn't take six rewrites to post.
Posting everywhere means rewriting one idea six times, so you post to one, or none. SureThing turns one idea into native posts for every platform.
