Context is king: Getting frontier-grade trend detection out of a cost-efficient model
We ran a global consumer brand's cultural trend detection on the lowest cost model in the family, matched what a frontier model surfaced, and made it consistent enough for a marketing team to act on.
Every model launch says the same thing: upgrade, upgrade, upgrade. Meanwhile the token bill climbs, and you're paying more each month to do the same work. On an always-on system that rereads thousands of social posts every couple of hours, that compounds into a real line item, one that grows every time a new model ships. For a consumer brand running trend detection across a portfolio, it's the number that decides whether the system stays switched on.
The reflex is to reach for the newest, biggest model. On a recent build for one of the world's largest consumer brands, we did the opposite. We shipped the lowest cost model in the family and got the same dashboard as the frontier one. The output quality came from the context and the evals around the model. The model tier turned out to be the wrong lever.
The short version: define what "good" means precisely enough to test it, feed the model concrete examples of good and bad, and validate the judge against human labels. Do that and a cheaper model reaches the expensive model's output quality, holds the same verdict run after run, and the model tier becomes a cost decision instead of a quality one.
Models don't define quality. People do, and evals capture that definition so a machine can hold to it.
Here's how that played out.
The problem: Quality is undefined, so no model can reliably hit it
There's an enormous amount of conversation happening online at any moment, and a brand's slice of what's relevant is buried in it. Most brands already have tools that surface signals. The trouble is volume: far more signals than anyone can read, and parsing them down to the few worth acting on takes time the moment doesn't give you. The relevant moments pass, and the organic engagement that was there goes to whoever moved first. The window for a trend-driven campaign is roughly two weeks, and manual triage rarely clears in time.
So the brand needed an always-on system. Every couple of hours, agents read thousands of social posts and surface a dashboard of the cultural moments worth acting on, while the window is still open.
Simple to describe. Then you hit the one word the whole thing depends on.
What actually is a "trend"?
Everyone has a different definition, and even our own team disagreed at first. That ambiguity is the real engineering problem, and it's bigger than it looks. If you can't define what a good result is, you can't test for it, you can't run it cost-effectively, and you can't trust it enough to act without a human re-checking every run. On a system that reruns every couple of hours, those last two decide whether it's viable at all. A frontier model on every run gets expensive fast. A dashboard nobody trusts gets re-triaged by a person, which drops you right back into the slow manual parsing you were trying to escape.
Evals turn quality from an opinion into a test
We started where most teams start: the biggest model and the client's documentation. Sonnet did a great job, especially at titling moments.
Bosnia and Herzegovina's first-ever knockout berth sparks a seismic fan roar.
Good copy. But were these actually trends by the brand's standard? A dashboard that's 40% noise is the same problem as before, handed to whoever has to read the dashboard. We had no way to answer the question, which meant we had no way to compare one model against another, or one prompt against another. Quality was still an opinion.
The client had already answered the question for us. Buried in their documentation were precise definitions, examples, and counterexamples of what counted as a cultural moment. We turned that into a test: an is_cultural_moment judge, scoring every candidate the same way every time.
The examples did more work than the instructions. If context is king, examples are the queen.
With the judge in place, quality became measurable, so we could compare models on equal terms. Sonnet surfaced more candidates than Haiku, 23 to 14. Once both ran through the judge, the gap nearly closed: 14 trends from Sonnet, 12 from Haiku, and every one of Haiku's made Sonnet's final dashboard.
Same dashboard. Roughly a third of the price.
Then the obvious objection: you've only shown Haiku agrees with Sonnet. How do you know the judge itself is right? You anchor it to people. Our engineer reviewed all 129 candidates by hand, keep or cut, then compared those calls against the judge. The question was never whether Haiku agreed with Sonnet. It was whether the judge agreed with the human holding the bar. In production that human is the brand team rather than the engineer, ideally several people on it, and the strongest evals come from the examples they all independently agree on.
Better context gets frontier-grade output from a cost-efficient model
The eval measured quality fairly. The next question was whether better context could raise it.
Out of the box, Haiku found 74% as many cultural moments as Sonnet. Instead of changing the model, we changed the specification. Three things moved the number.
Clarity. Spell out exactly what a cultural moment is, and what it isn't.
Focus. Judge the moment, not its popularity. Haiku was skipping thin topics because they weren't already trending, but a separate eval further down the pipeline already handled volume. At this stage the only question is whether something culturally interesting happened.
Examples. Back to the queen. We fed in the exact moments Haiku had been missing.
Two prompt iterations later, on the same full-pipeline A/B, Haiku went from 74% to 95% to 100% of Sonnet's detection rate, with no loss of precision. The extra moments were real ones rather than noise creeping back in: no match previews, no ads, no schedule spam. The dashboard grew from 12 to 20 trends.
The distance between the lower cost model and the frontier one was a context gap. We closed it with words.
A precise spec makes that quality repeatable
Matching on quality at a third of the cost is a good result. For a CTO it isn't the whole one. To ship the lower cost model you need the same answer every run, not just once.
We noticed it by accident first. Two runs over the same corpus produced 119 and 121 moments, and re-judging the same shortlist agreed 99% of the time, 114 of 115 verdicts. Interesting, and not yet proof.
So we ran it properly: a fresh 500-post benchmark, judged 100 times across a range of temperatures. Even at temperature 1.0, rolling the dice on every token, 96% of posts got the same verdict every time.
LLMs are non-deterministic by design. They're supposed to drift from run to run, and in earlier tests, even at temperature 0, we've seen much lower agreement than this. That's what made the result surprising. Given a definition precise enough, the model converges on the same call on its own. That's the behavior that makes AI usable where a decision can't quietly change between runs.
How we build this for consumer brands
The method generalizes past this one build, and it's the throughline in how A.Team builds agentic systems for consumer brands. Define quality with the people who own it, encode that definition as evals, engineer the context with real examples of good and bad, and let the specification carry reliability instead of the model tier. It's the same sequence whether the system is detecting cultural moments, scoring creative, or reading assortment signals.
Two things make it work in practice rather than in a benchmark. The definition has to come from the brand, not from the engineering team's best guess at what the brand means, which is why the client's own documentation was the unlock here rather than a cleverer prompt. And the judge has to be validated against human labels before anyone trusts it, or you've traded shipping without evals for judging without evals, one level removed.
This piece describes one engagement. The broader discipline it sits inside, detection through brand-fit scoring through activation, is covered in the trend dies before your brief is written, and the portfolio-level engagement numbers are in 88,000 posts in 9 minutes.
What it means for the invoice, and for trust
Models keep getting smarter and more expensive. Haiku didn't secretly become Sonnet on this build. It stayed exactly what it was. Once quality was defined precisely, raw model capability mattered far less than most teams assume.
Do that well and the payoff shows up twice: in the output and on the invoice.
There's a second payoff, and for a consumer brand it's the one that decides whether the system ships at all. A brand's definition of what counts as on-brand can't drift between runs. If the same post reads as a cultural moment on Tuesday and as off-brand on Thursday, nobody puts that dashboard in front of a brand director, and trend triage goes back to being a person and a spreadsheet. When the specification is precise enough, even a highly non-deterministic model lands on the same answer. You don't buy that consistency with a bigger model. You write it into the spec.
Get that right and the payoff is concrete: a dashboard a brand marketer can trust at a glance, cost-efficient enough to run every couple of hours, catching cultural moments that used to slip past while someone was still hunting through data.
So before you spend your way up the model ladder, try constraining instead. Write the eval that makes "good" unambiguous. The lowest cost model might already be good enough. It's been waiting for you to say what you actually want.
See how the consumer intelligence system works →
Christina Stejskalova is an Applied AI Engineer at A.Team focusing on social trends.
A.Team AI Solutions builds intelligence systems for Fortune 500 consumer brands. The engagement referenced is anonymized to role and business unit. Method detail comes from an A.Team AI Guild StackTalk.
For the rigor-minded: the model A/B and the prompt-tuning iterations were single full-pipeline runs per arm, tuned and validated on the same day's corpus. The consistency sweep is reported at a clean, pooled n=100 per temperature. Long sweeps occasionally hit transient API overload bursts where nearly every call fails. Those runs measure the API rather than the judge, and are excluded. The harness, the frozen corpus, and every per-run output are preserved.
Frequently asked questions
LLM-as-a-judge means using a model to score another model's output against a definition of quality you supply. The judge is only worth as much as its calibration, so you validate it against human labels before trusting it: have the people who own the definition mark a set of real candidates keep or cut, then check the judge's verdicts against theirs. On this build that meant hand-reviewing all 129 candidates. Skip that step and you've traded shipping without evals for judging without evals, one level removed.
On this build, yes, once quality was specified precisely. The frontier model was genuinely the better raw detector, surfacing 23 candidate moments against the cheaper model's 14. After both ran through the same validated judge, the dashboards were 14 and 12, and every one of the cheaper model's picks appeared in the frontier model's set. Two prompt iterations then closed the raw detection gap entirely, from 74% to 100%. The model was better at the step where being better didn't change what the user saw.
You write the consistency into the specification rather than buying it with a bigger model. Across a frozen 500-post benchmark judged 100 times, the judge returned an identical verdict for every post at temperature 0, and for 96% of posts even at temperature 1.0 where every token is a dice roll. The precision of the definition is what makes the model converge. The handful of posts that do flip tend to be the genuine gray zone in the rubric, which makes them a useful to-do list for sharpening it.
Less than most teams expect, and the shape matters more than the volume. Three changes moved this build from 74% to 100%: spelling out what does and doesn't qualify, telling the judge to assess the moment rather than its popularity because a later stage already handled volume, and feeding in concrete examples of the exact moments it had been missing. The examples did more work than the instructions.
You don't resolve the subjectivity in the engineering team. You source the definition from the people who own the brand standard, ideally several of them, and treat the examples they independently agree on as the anchor. On this build the client's own documentation already contained precise definitions, examples, and counterexamples, which became the eval directly. Where a brand hasn't written that down, producing it is the first piece of work, and it pays for itself the moment you need to compare two models or two prompts.
The trend dies before your brief is written
Detecting a social trend isn't the hard part. Scoring it for brand fit and turning it into a brief before the window closes is. Here's the discipline, and what an agent changes.
88,000 posts in 9 minutes
A consumer insights team was spending weeks monitoring trends their competitors were already acting on. Here's what changed when the system did the monitoring.
AI agents for CPG: What they actually do and why your data is the moat
AI agent use cases in CPG across pricing, promotion, demand, and retail media, and why the strongest ones run on your own data.
Build AI that compounds in value
A.Team's senior AI practitioners ship production workflows that hold up and improve over time. Tell us what you're building and we'll match you with engineers who've done it before.
Talk to A.Team