You probably don't need an LLM for that

A founder came to us wanting an LLM to categorise incoming support tickets. Reasonable request, obvious technology, and completely the wrong tool. The categories were fixed, there were eleven of them, and they had four years of labelled history sitting in their helpdesk export.

That is not a language-model problem. That is a text classifier — a model you can train in an afternoon, that runs in single-digit milliseconds, costs effectively nothing per prediction, and can be measured against a held-out set so you actually know how often it is wrong.

We built that instead. It was cheaper to build, an order of magnitude cheaper to run, and — the part people find surprising — more accurate on their data than the language model, because it had learned their eleven categories rather than reasoning about categories in general.

Why this keeps happening

For about three years, "AI" and "large language model" have been treated as synonyms. That is a recent accident of vocabulary. Machine learning was solving classification, forecasting, ranking and detection problems for a decade before ChatGPT, and it never stopped being the right answer to those problems.

The trouble is that language models are general, which makes them a plausible answer to almost anything. Plausible is not the same as correct. You can open a tin with a chainsaw.

The test we actually apply

Before writing a line of code, we ask four questions.

Is the output a fixed set of things? Categories, scores, yes/no, a number, a ranked list. If so, a trained model is probably the answer. Language models are the answer when the output is genuinely open-ended — prose, conversation, code, a summary in someone's voice.

Do you already have labelled examples? A few thousand rows of your own historical data beats any amount of prompt engineering. Your data encodes how your business actually behaves. A general model has to be told, imprecisely, in English, every single call.

How often does it run? At ten predictions a day, use whatever is fastest to build. At ten million, the difference between a model costing a fraction of a cent and one costing several cents is the difference between a viable product and a spreadsheet that never closes.

Do you need to know when it is wrong? This is the one people skip. A classifier gives you a confidence score and a confusion matrix. You can say "we are 94% accurate, and here is exactly which cases we fail on." Getting that same rigour out of a language model is possible but is real evaluation work, and most teams shipping LLM features have not done it. They have vibes and a demo.

When the language model genuinely wins

We are not anti-LLM — we run one in production ourselves. Reach for it when:

• The output is open-ended text, speech, or code.
• You have no labelled data and no realistic way to get it.
• The task needs world knowledge your data does not contain.
• Requirements change weekly and retraining a model each time would cost more than the inference does.
• You need something working this week and can optimise later. A language model is an excellent prototype for a problem you do not yet understand.

That last one matters. Sometimes the honest sequence is: ship the LLM version, learn what the task actually is, harvest labels from real usage, then replace it with a small model once the shape is clear. That is not indecision, it is how you buy information cheaply.

The hybrid answer

Most real systems end up as both. A cheap classifier routes and filters. A language model handles the genuinely open-ended remainder. A retrieval layer keeps both honest by grounding them in your documents rather than the model's memory.

The engineering judgement is not which technology is best — it is which part of this problem deserves which tool. That judgement comes from having built both kinds. We started with classifiers, NLP and Kaggle competitions in 2019, years before generative AI became the default answer to every question, and that background is why we are comfortable telling a client that the exciting technology is the wrong one.

What to ask whoever builds it

If you are evaluating an agency or a contractor for AI work, ask this: "When would you not use an LLM here?"

If they cannot answer — if every problem you describe gets the same architecture back — they have one tool. You will get that tool whether or not it fits, and you will pay for its inference costs for as long as the product lives.

The right answer sounds boring. Most good engineering does.