The Cheap Model Writes Well. Sometimes It Writes Nothing.eric

The dream is simple. You have a pile of small marketing writing, the launch tweet, the LinkedIn post, the short intro, and you want to hand it to the cheapest model that is good enough and move on. So this round we tried a genuinely low-cost model, one of OpenAI's budget options, against Gemini Flash, the fast model we had been testing all along. The writing came back good. The problem came back from a direction we did not expect.

A quick recap

This is the third round. In the first post a newer small model lost to Flash. In the second a bigger model beat it. This time the opponent is price. The challenger is gpt-6-luna, a low-cost model from a third lab, running the same three jobs under the same house-voice brief: a tweet, an image caption, and a one-hundred-and-fifty word intro.

The writing was close

On the words, the cheap model held up fine. It tied Flash on the tweet. It beat Flash on the image caption, where Flash keeps stepping outside the frame to talk about craft instead of describing the picture. And it ran level with Flash on the intro, hitting the length almost exactly. If we had stopped there, the story would be a happy one: the cheap model is good enough, maybe better on captions, so use it and save the money.

We did not stop there. The last two rounds taught us that a single sample lies, so we ran the intro five times for each model. That is where the real result showed up.

Then two of five came back empty

The two empty runs were not errors or timeouts. The model returned a perfectly valid response that contained zero words of text. It had spent its entire output budget, 2,048 tokens, on reasoning it never shows you, and ran out before it wrote a single sentence of the actual post. Flash does not reason heavily, so it fits inside that same budget every time and never comes back empty.

It is a budget problem, and that is the whole point

We raised the output cap to 4,096 tokens and the cheap model finished every time, three for three, clean one-hundred-and-fifty word intros. So the model can clearly do the job. It just needs room, and more importantly, it needs you to know that it needs room.

At the budget a reasonable person would set, the budget Flash lives inside comfortably, the cheap model failed silently two times in five. A pipeline that writes a LinkedIn post on a schedule would have published nothing, twice, and most likely would not have told anyone. An empty string is not an exception you catch. It is a valid response that happens to say nothing.

The real cost of a cheap model

The price per token is the smallest line item in the bill. Price the whole job instead.

You are paying for a token budget large enough to absorb hidden reasoning. You are paying for code that notices and retries an empty response. And you are paying for the reasoning tokens themselves, because "cheap per token" means a lot less when the model burns ten times as many tokens thinking as it does writing. Flash is not the cheapest per token, but it answered every time inside a small budget with no babysitting. For fire-and-forget automation, "never comes back empty" is worth more than a slightly sharper sentence.

So does Flash win the LinkedIn post

For that job, yes, but not for the reason you would guess. On the words it is close. Flash is a little punchier, the cheap model is a little tighter on length, and both are perfectly publishable. Flash wins on reliability at a sensible budget, which for an automated job is the thing that actually decides it. On captions the cheap model still wins. There is no single best writer here, only a best fit for the job and the plumbing wrapped around it.

The same limits as before

One job family, five runs each. Flash ran at a temperature of 0.7 while the cheap model ran at its own default, because it rejected the temperature setting. Raising the output budget removes the empty responses entirely. This is directional, not a benchmark, which is exactly why you should run your own job at the settings you will actually ship.

Three rounds now. A newer small model lost, a bigger model won, and a cheaper model wrote well but needed a seatbelt. The constant was never a particular model. It is that you have to run the real job yourself, at the real settings, before you trust any of them with your name.

Next time you price a model, open the token usage next to the output. The gap between what it wrote and what it spent is where the surprises live.