one-cent architecture

In partnership with

What's handling the simple routing in your agent right now?

Every agent demo looks cheap, then you put it in front of customers and the bill shows up.

Take a simple support agent. It classifies emails, looks up order status, drafts replies and hands the weird cases to a human. In a demo, it runs smoothly and costs pennies.

In production, it sends the full conversation back to the model on every step, uses a top-tier model to decide whether an email is about shipping or a refund, and pays for reasoning on questions a few lines of code could have answered.

I've started calling the fix the $0.01 architecture. The number is a design target, so some requests will cost more and most should cost far less.

Tools of the Week

LiteLLM: one OpenAI-style interface for 100+ LLM providers, as a Python SDK or a self-hosted gateway with spend tracking.
Langfuse: open-source tracing, evals and cost tracking for LLM apps, so you can see which route each request took and what it cost you.
OpenAI Batch API: 50% off standard prices if you can wait up to 24 hours for results. Good for nightly evals or reclassifying an old backlog of tickets.

The basic idea

Most agent workflows are a chain of small decisions:

Is this a shipping request or a refund request?
Does this message contain an order ID?
Should I ask a follow-up question?
Is this output valid?
Do I need a model here?

Your smartest model doesn't need to touch most of these. The setup I'd use runs in this order:

Rules and cache first
A cheap model second
Validation in code
A stronger model only when validation fails
Human review at the edge

Here's how a request moves through it:

Any stage that produces a valid answer returns it right there, so a request only moves further down the chain when the stage before it couldn't handle it.

Expensive reasoning becomes an escalation path you pay for only when you need it.

Why this matters

The price gap between model tiers is why.

On OpenAI's current pricing, GPT-6 Astra costs $10 per million input tokens and $50 per million output tokens.

GPT-6 Luna, the most efficient model in the family, costs $0.10 and $0.50. That's a 100x difference for every token you send.

So I go through the workflow step by step and decide which part of it deserves intelligence.

A simple support agent

Here's a customer support router I'll use to show the pattern. It should:

answer order-status questions when it can
sort other requests into a few buckets
escalate anything unclear

Step 1: let code do the obvious work

Say a customer writes:

❝

"Where is order #12345?"

You don't need an LLM for that. You can pull out the order number, check that the message is asking about tracking, and look the order up directly.

import re

ORDERS = {
    "12345": "shipped",
    "67890": "processing",
}

def check_order(message):
    match = re.search(r"\border\s*#?\s*(\d{5})\b", message, re.I)
    if not match:
        return None

    if not re.search(r"\b(status|track|where|shipped)\b", message, re.I):
        return None

    order_id = match.group(1)
    status = ORDERS.get(order_id)

    if status:
        return f"Order {order_id} is currently {status}."
    return None

That path costs nothing in model spend.

In a production system you'd also verify the customer's identity and permissions before sharing order details, but the principle holds: if code can answer it reliably, let code answer it.

Step 2: use a cheap model for routing

Now take a message like:

❝

"I want to return my headphones."

This one needs some interpretation, but it's still a narrow task. We only need a label, so there's no reason to pay for a long chain of reasoning.

I hand it to GPT-6 Luna, which OpenAI built for focused, high-volume tasks like this one.

REPLIES = {
    "shipping": "Please send your order ID.",
    "refund": "Your refund request needs review.",
    "other": "Please share a bit more detail.",
}

def handle(message):
    # 1. Code first: zero model spend
    result = check_order(message)
    if result:
        return result

    # 2. Cheap model, reasoning off
    label = classify(message)

    # 3. Escalate only if the cheap model's answer failed validation
    if label is None:
        label = classify(
            message, model="gpt-6.1-sol", effort="low", max_tokens=2000
        )

    # 4. Human at the edge
    return REPLIES.get(label, "Human review required.")

What I'd add in production

Before I put this in front of customers, I'd add a few things:

spending limits per task
structured outputs, so the model returns a fixed schema you can check
logging for every route and every escalation
evaluation sets with labeled examples
prompt caching for policies and reusable context (cached input on Luna is billed at 10% of the normal input price)
human review rules for sensitive actions like refunds

And I'd track one number above the rest: cost per successfully completed task.

Cost per call and cost per token can look great while the agent is cheap and wrong. Cost per correctly handled task shows you quality and spend in the same number.

My take

For the last couple of years, most of the conversation has been about what models can do. The question I find more interesting now is which parts of a workflow need a model at all.

Answering it changes how you build.

Sometimes the right answer is a strong reasoning model, sometimes it's a cheap classifier, and a surprising amount of the time it's a regex and a database call.

I think the teams that figure out this split early will ship agents that can grow without the bill growing at the same pace.

Until then,
Vaibhav 🤝

If you read till here, you might find this interesting

#AD 1

How can AI power your income?

Ready to transform artificial intelligence from a buzzword into your personal revenue generator

HubSpot’s groundbreaking guide "200+ AI-Powered Income Ideas" is your gateway to financial innovation in the digital age.

Inside you'll discover:

A curated collection of 200+ profitable opportunities spanning content creation, e-commerce, gaming, and emerging digital markets—each vetted for real-world potential
Step-by-step implementation guides designed for beginners, making AI accessible regardless of your technical background
Cutting-edge strategies aligned with current market trends, ensuring your ventures stay ahead of the curve

Download your guide today and unlock a future where artificial intelligence powers your success. Your next income stream is waiting.

Get Your Guide

#AD 2

Take the prompts our creative team actually uses

The key to maintaining brand quality at scale? Mastering how to communicate with AI models and embedding them into your creative strategy.

Join our honest discussion with Ari Murray, Chief Digital Officer at Salt and Stone, and get 5 tips for expert LLM prompting with ready to run prompts.