In partnership with
When you're picking a model for daily coding work, what decides it for you?
Grok 4.5 shipped last week, with Musk pitching it as "an Opus-class model, but faster, more token-efficient and lower cost." and OpenAI made GPT-5.6 generally available a day later, with Terra as its half-the-flagship-price tier.
Two value plays landing in the same week made the matchup pick itself, so I put them head to head on the same repo.
Time for some tests. But first, some catchup:
Partner with us
DevTools of the Week
An open-source visualisation language from Microsoft that lets AI agents generate charts and diagrams through a declarative spec instead of raw plotting code. Works across agent frameworks so the same output renders consistently wherever the agent runs.
Notion's system for running the full product development cycle in one place, from customer feedback through to a merged PR. Agents handle triaging, routing and summarising while the team keeps the judgment calls.
An open-source, self-hosted platform for cross-tool, multi-agent workflows with native MCP support and 500+ built-in integrations. Runs on your own infrastructure with your own API keys, so no vendor lock-in.
AI Cheatsheet 101
If today's piece left you wanting a proper handle on AI vocabulary, here is the best resource.
85 terms explained in plain English, structured so each section builds on the last.
Quick "test yourself" checkpoints between sections keep what you learn from evaporating, and 15 minutes of scrolling will leave you a lot more fluent than you were this morning.
Test 1: Fix 5 real bugs
Input:
I built a small FastAPI task-store service and planted 5 bugs on purpose.
To name, these included a priority field typed wrong, a missing 404 on a bad PATCH, a partial-update bug that clobbers unrelated fields, a missing title length cap and a status filter that gets silently ignored.
Eight tests, three passing, five failing on the starting repo.
The prompt to both models was identical:
Output:
Both models got to 8/8, fixed the root causes without taking any shortcuts.
Grok did it in 28.8 seconds while GPT-5.6 Terra took 1 minute 13.
Winner: Grok (speed)
Test 2: 10 tasks a working dev would run
Input:
I added 10 realistic prompts to the same repo.
The list includes a Postgres migration, thread-safety refactor, validator, rate limiter, a Slack summary for a non-technical PM, and GitHub Actions CI workflow.
For every output I asked one question: does this ship as-is, or does it need rework before you'd merge it.
Output:
Grok shipped 8 of 10, while GPT-5.6 Terra shipped 7 of 10.
Both failed the same task for the same reason: a rate limiter using per-process buckets that breaks the moment you run multiple workers, and neither model reasoned correctly about distributed state on the first pass.
That's not a Grok problem or a GPT problem.
Winner: Grok (count)
Total spend across the ten tasks:
Divide by what shipped and you get $0.016 per usable output from Grok versus $0.049 from Terra. 3x cheaper per thing you can merge.
Where this test setup has limits
I graded shippability using the models themselves.
Both flagged the same rate limiter flaw independently, which is a decent consistency signal, but a model has a blind spot for its own subtle mistakes even when it's not trying to cheat.
I'd hand-verify the migration and validator outputs before treating "8 out of 10" as final.
This is also just 1 repo on 10 tasks in the same session. It's a signal from a test.
Don't make a procurement call off this alone, and run your own version against your own codebase, that's the entire point of building the test.
My Take
On identical correctness, Grok was 2.5x faster and 3x cheaper per output you can merge. The capability gap everyone assumes, at least on dev tasks, didn't show up.
I went in expecting the standard tradeoff: Terra wins on capability, Grok wins on price, pick your poison.
Running the tests instead showed something narrower than that.
What surprised me was the rate limiter failure.
It's something a benchmark chart will never surface, because nobody benchmarks "does this model understand what happens when you run 3 copies of your API."
Only running the same task against both models side by side caught it.
If your workload is CRUD and infra glue, the stuff most of us ship daily, cost per shippable output is the important number, and right now it favours Grok.
Until next time,
Vaibhav 🤝🏻
If you read till here, you might find this interesting
# Partner 1
Six people doing the work. Your headcount is one.
Your finance close runs in #finance. Stripe and QuickBooks reconciled, runway updated, posted Sunday night without you asking.
Engineering review lands in #eng. Viktor pulled the open PRs, left comments on auth-refactor, flagged a dependency blocking api-pagination.
Campaign brief lands in #growth: Meta CPA up 18%, recommendation to pause broad match, a draft landing page already deployed for the variant test.
You hired him on day zero. He lives in Slack and Microsoft Teams alongside your contractors and investors, connects to 3,000+ tools, pushes back when you ship something dumb.
"Viktor is now an integral team member, and after weeks of use we still feel we haven't uncovered the full potential." Patrick, Director, Yarra Web.
# Partner 2
Think You Know What AI Does Next?
Which model leads the next benchmark? Which AI lab ships the next major breakthrough?
Kalshi lets you trade on real-world AI and technology events as the industry moves. If you follow launches, model updates, and benchmarks closely, put that knowledge to work.
Bonus credit varies from $15 to $500. Terms apply.
