In partnership with
Can you tell when something was written by AI?
Every teacher, recruiter and LinkedIn commenter is asking the same question right now: did a human write this, or did ChatGPT?
AI detectors have been guessing at it for years, and they get it wrong often enough that students have been flagged for essays they wrote themselves.
So I went looking at the other approach, where the model signs its own text while it writes.
The timing is interesting too, because the EU AI Act now asks generative AI providers to mark their outputs in a machine-readable way, text included, and the grace period for tools already on the market runs out on December 2.
With images you can slap a logo in the corner, but text has no corner, so the signature has to hide inside which words the model picks.
A 500-word article can read completely normal and still carry that pattern. Google DeepMind already does this at scale with SynthID Text.
Today I tried a simpler version called a green-list watermark, using Hugging Face Transformers on my laptop, with zero model training.
1. How do you watermark text?
Quick refresher: an LLM writes one token at a time, picking each one from a list of probabilities.
A green-list watermark uses a secret key plus the previous token to split the vocabulary into "green" and "red" tokens, then gives the green ones a small boost before the model picks.
The split changes at every step, so you'll never spot it by reading.
Anyone holding the key can count the green tokens, and if there are way more than chance would give you, the text was almost certainly watermarked.
2. Build a watermark
Install: pip install torch transformers
from transformers import (
AutoTokenizer, AutoModelForCausalLM,
WatermarkingConfig, WatermarkDetector
)
model_id = "openai-community/gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer.pad_token_id = tokenizer.eos_token_id
watermark = WatermarkingConfig(
greenlist_ratio=0.25,
bias=2.5,
seeding_scheme="lefthash",
hashing_key=15485863
)greenlist_ratio=0.25 puts about a quarter of the vocabulary on the green list at each step, and bias=2.5 is how hard we push the model toward those tokens.
Push harder and the watermark gets easier to detect, though the writing can start sounding a bit off. hashing_key is the secret, so treat it like a password.
3. Generate two samples
I ran the same prompt twice, once normally and once with the watermark switched on. I went with GPT-2 because it's tiny and runs fine on a laptop CPU.
prompt = "The future of artificial intelligence"
inputs = tokenizer(prompt, return_tensors="pt")
normal = model.generate(
**inputs, max_new_tokens=200,
do_sample=True, pad_token_id=tokenizer.eos_token_id
)
marked = model.generate(
**inputs, max_new_tokens=200,
do_sample=True, watermarking_config=watermark,
pad_token_id=tokenizer.eos_token_id
)
print(tokenizer.decode(normal[0], skip_special_tokens=True))
print(tokenizer.decode(marked[0], skip_special_tokens=True))Read both outputs side by side and you won't find anything that gives it away (GPT-2 rambles equally in both). The whole signature lives in which tokens got picked.
4. Detect the signal
Now the fun bit.
Hugging Face's WatermarkDetector rebuilds the green lists with the same key, counts how many green tokens showed up, and tells you whether that count is too high to be luck.
detector = WatermarkDetector(
model_config=model.config,
device="cpu",
watermarking_config=watermark
)
print("Normal:", detector(normal)[0])
print("Watermarked:", detector(marked)[0])A common z-score is (G - gamma*N) / sqrt(N*gamma*(1-gamma)), where G is observed green tokens, N is eligible tokens and gamma is the expected green fraction.
The Hugging Face detector flags anything above 3 by default.
Keep in mind a high score only tells you this specific watermark is present, and says nothing about text from models that never used it.
5. Try to break it
Time to play the cheater.
I ran the detector on three versions of the watermarked text: the original, a copy cut in half, and a copy I paraphrased with another model.
Each edited version needs to be tokenized again before testing, and if you want numbers you can trust, repeat it across 20 samples and log detection rates, false positives and sample lengths.
Shorter text gives the detector fewer tokens to count, so the score drops. Paraphrasing hurts even more, because rewriting swaps out the very tokens that were carrying the signature.
6. What about SynthID?
SynthID Text runs on the same principle with a fancier sampling method, and Google open-sourced it in 2024 (it works inside Transformers too).
Google says it survives light cropping and edits, and it struggles once text is heavily rewritten or translated.
Both methods share one limitation: the watermark goes in while the model writes, so you can't add it to a paragraph after the fact.
Tools I used this week
Hugging Face Transformers (what I used today, green-list and SynthID watermarking come built in)
SynthID Text (Google's open-source version, worth a look if you want something production-grade)
LLM Watermarking (the original green-list code from the University of Maryland paper that kicked all this off)
My take
Watermarking is a solid tool for a company that wants to prove which text came out of its own model, which is why Google tags Gemini output with SynthID.
It does nothing for text from models that never watermarked in the first place, and a decent paraphrase can wash it out.
So if a detector ever flags your work as AI, that alone shouldn't count as proof, and nobody should lose a grade or a job over it.
What blew my mind is that a tiny nudge to next-token probabilities, one no reader would ever notice, leaves a signature you can measure with a few lines of Python.
Hit reply and tell me: would you want your AI tools to watermark everything you generate? I'm torn on this one.
Until next time,
Vaibhav 🤝
If you read till here, you might find this interesting
#AD 1
Forget SpaceX. These 7 Stocks Could Be Next.
You may not have gotten into SpaceX early, but the space-investing story is far from over.
Record launches, falling costs, government contracts and new technologies are turning space into a commercial market. That growth is creating opportunities across satellites, defense, communications, data and in-space infrastructure.
This free report 7 Best Space Stocks to Own in 2026 reveals seven publicly traded companies positioned across this expanding economy. It explains how each business makes money, what could move its stock and which risks investors should understand.
SpaceX captured Wall Street’s attention. Now see which seven space stocks could benefit from what comes next.
#AD 2
Leave With a Framework for What Agents Own
Automating at random isn’t a system. Deciding what agents own and what people own is.
On October 28 at 11am EDT / 8am PDT, HubSpot for Startups and Dust are hosting a live webinar to show you how to decide which go-to-market work agents take on and which stays with your team. You’ll leave with:
A clear way to decide where a human stays in the loop
A post-call automation that updates your CRM for you
A Slack view of who owns each account and what happens next
