In partnership with
If your model suddenly got smarter overnight, what would you assume?
2 weeks ago, DeepSeek's API started giving noticeably better answers.
By next morning, a rumour had swept across X that DeepSeek was secretly routing traffic to Anthropic's Claude Fable 5 to collect distillation data.
It was wrong. DeepSeek was grayscale-testing the V4 GA checkpoint before the official launch.
The outputs improved because their model had improved. Then DeepSeek dropped the official V4 Flash on July 31 while Claude Opus 5 shipped a week earlier.
I ran three identical coding tasks on both. They got the same answers and that should have been the end of the comparison, but it wasn't.
The difference was in what each model left behind after the answer. So I set up 3 tests on the same codebase to see if that difference holds up. But first:
Partner with us
DevTools of the Week
An open-source engine that runs Gemma 4 26B in around 2GB of RAM on any M-series Mac. It was the most upvoted developer launch on Hacker News this week.
An open toolkit that distils frontier open models against your own agent traces to build cheaper models that keep improving over time. It is maintained by Experiential Labs.
Open-source voice dictation for macOS that transcribes on-device using Apple's Speech framework, with no model to download. Built and maintained by Frigade.
A composable, GPU-accelerated library for building interactive plots that stay fast under heavy data. It comes from the team behind the Reflex Python web framework.
The setup
It was the same scaffolded node.js repo for both models: a REST API with auth, CRUD, and a notifications module.
I planted one bug in the due-date reminder logic, invisible from comments or docs. A 400-entry fabricated changelog with one specific fact buried between entries 200 and 240.
DeepSeek V4 Flash: $0.14 per million input tokens.
Claude Opus 5: $5 per million.
A 36x gap on paper.
I tracked cost, but the question was whether I could ship the output without re-checking it.
Test 1: build a feature cold
Same brief to both: add recurring tasks with automatic next-occurrence generation on completion. Full diff required, tests included.
DeepSeek V4 Flash: $0.04, 4m 29s. Worked off a linear todo list, shipped 8 tests, all green.
Claude Opus 5: $1.40, 4m 15s, 314 lines added, 19 tests. It called out edge cases that weren't in the brief.
Spawn only fires on a pending-to-completed transition, so reopening and re-completing a task doesn't duplicate the next occurrence and month-end dates clamp instead of drifting.
It also caught an unrelated duplicate key bug sitting in the same file, fixed it, then asked whether to update the README since the instructions said not to touch unrelated files.
My take: Both outputs compile and pass. DeepSeek built what I asked for and Opus built what I asked for, then told me about the assumptions I didn't know I was making.
If I'm reviewing the diff myself, both are fine. If this code goes into a repo nobody will touch for 3 months, Opus needs less follow-up.
Test 2: find a needle in a 400-entry changelog
One specific environment variable and its default value, buried deep with the full repo attached.
DeepSeek V4 Flash: Correct. NOTIFICATION_POLL_INTERVAL, default 30000, entry 1.0.0-215, quoted exactly. $0.01, 24.5 seconds.
Claude Opus 5: Also correct, and same. $0.39, 53 seconds.
But it also flagged that PORT shows up 13 times across the codebase as boilerplate with no documented default.
Opus named it, explained why it ruled it out, and moved on to the answer.
My take: Both got it right. The difference is what Opus did with the thing that looked like it could have been the answer but wasn't.
Test 3: find and fix a bug with zero hints
One real bug in the due-date reminder logic and no pointers.
DeepSeek V4 Flash: Found it.
The dedup check in notificationService.js keys on taskId and type only, reschedule a task after a reminder has already fired, and the stale record blocks a new one for the new date permanently.
It shipped the fix too: clearRemindersForTask function called from taskService.update() whenever dueDate changes. $0.03, 3m 48s.
Claude Opus 5: Same root cause, found on its own. It just got there differently.
Ruled out a race condition first, noting the single-threaded Node runtime makes it impossible in this block structure.
It checked the leap-year maths next, confirmed all date arithmetic passes and tested recurring-task collisions after that and eliminated them.
Only after those cuts did it land on the dedup key.
Then it wrote a repro script and ran it against the live code. Output confirmed the bug: after rescheduling, the notification count stayed at 1 where it should have been 2.
Its fix took a different approach.
Instead of clearing stale reminders after the fact, it stamps the due date on the notification itself and dedupes on that. A moved deadline gets its own identity instead of colliding with the old one.
It also flagged that non-canonical ISO timestamps could bring the bug back if clients send offsets like +05:30 instead of UTC.
Opus just didn't write the fix to disk and DeepSeek shipped an applied patch.
$0.92 vs $0.03.
My take: This test made me rethink the comparison.
DeepSeek found the bug, shipped the fix, and moved on. I believe the answer but I'm trusting the output because it “looks” right.
Opus found the same bug, then spent tokens proving it before proposing anything in the form of wrong theories eliminated with reasons, a repro script I can run myself and a fix that's harder to break later.
There's an Anthropic paper from last year called "Reasoning Models Don't Always Say What They Think."
They slipped hints into prompts and checked whether the model's chain of thought mentioned using them.
Claude 3.7 Sonnet mentioned the hint 25% of the time it used one and DeepSeek R1, 39%.
The rest of the time, the trace told a clean story that left out the thing that changed the answer.
DeepSeek shows you the full trace and Anthropic hides it. Their own research says visible traces leave out the key factor that influenced the answer more than 60% of the time.
Here, opus left behind a repro script that either confirms the bug or it doesn't. The trace might be unreliable but evidence it generated is not.
My Take
DeepSeek V4 Flash has been the most-used model on OpenRouter since May, doubled its share from 9% to 18%, and now pulls more traffic than Google and OpenAI combined.
That happened while Anthropic was publicly accusing Chinese labs of distilling Claude, but DeepSeek was the smallest actor named by a wide margin. (150,000 exchanges against MiniMax's 13 million)
I ran these tests expecting Opus 5 to justify the price gap. In a controlled setup it did but I don't live in controlled setups, and outside of them Opus 5 has been frustrating.
I've watched it flip-flop on trivial fixes for entire sessions and burn through context doing verification work that added tokens without adding clarity.
There's a Hacker News thread titled "Opus 5 is a really bad model" and the complaints in it line up with what I've been seeing in my own work.
DeepSeek V4 Flash reads the problem, answers it, and moves on.
It leaves less evidence behind, and Anthropic's own research says visible reasoning traces omit the key factor that influenced the answer more than 60% of the time anyway.
I trust Flash more right now because I can hand it a task and walk away, and I can't say the same for Opus 5.
If Anthropic sorts out the consistency, that changes because the reasoning depth is clearly there. But until then, Flash is my go-to now.
Until next time,
Vaibhav 🤝🏻
If you read till here, you might find this interesting
# Partner 1
Save 40% on 1000+ AI APIs
AI costs don't usually explode overnight. They grow quietly through duplicate requests, expensive routing and poor visibility.
Mesh API helps engineering teams spot the waste before finance does.
A global e-commerce company was able to reduce spend by 78%.
# Partner 2
Keep up with AI in 5 minutes a day
Keep up with AI in 5 minutes a day. MavSource aggregates updates from all major AI newsletters, podcasts, companies, AI labs, and hundreds of other sources — then summarizes the key updates and analyzes the trends shaping AI. Free. Daily.
