A new model drops, my group chat lights up, and for about ten minutes I think the product needs a rewrite.

Then I remember: customers are not buying the model name. They are trying to finish a task. The useful question is not “which model is smartest?” It is “which model completes this specific job reliably, fast enough, at a cost that leaves room for the business?”

Model news is a starting point, not a decision

There have been meaningful launches across the market in 2026. OpenAI announced GPT-6 Astra and expanded the family with Sol and Luna; Anthropic introduced Claude Sonnet 5; Google used I/O to introduce Gemini Omni and Gemini 3.5. Those announcements tell us what each provider wants us to notice. They do not tell us which model wins on your customers' messy input.

Before changing anything, list the job. Is it classifying a support ticket, pulling fields from a receipt, turning a voice memo into a draft, or suggesting next steps in a workflow? Write down the expected output and what a bad result would cost. “Good at AI” is not a requirement. “Returns the three fields our billing screen needs, or clearly asks for help” is closer.

Build a tiny eval from real-shaped examples

You don't need a 10,000-row benchmark to start. Make a set of 30–50 examples that look like actual usage: clean cases, weird cases, long cases, ambiguous cases, and a handful of “don't do this” cases. Remove personal data and secrets first. Ask someone who knows the job to mark what counts as correct.

Score each response on a few simple dimensions:

  • Did it produce the required fields or action?
  • Was the answer accurate enough for the use case?
  • Did it flag uncertainty instead of making something up?
  • How long did it take and how much did it cost?
  • How often did a person have to fix it?

If an evaluation doesn't change a product decision, trim it. The goal is not to win a model bake-off on social media; it's to prevent an avoidable customer headache.

A founder reviews a simple AI workflow on a laptop and notebook

Route the easy stuff cheaply

Most products have a mix of requests. Some are predictable: label a message, clean up formatting, suggest a short title. Others need deeper reasoning: compare several documents, explain a strange error, or plan an action that touches more than one tool.

Start with the smallest option that passes your eval. Send the unusual or high-impact cases to a stronger model, or offer a human review. A routing rule can be boring and explicit: if confidence is low, the input is long, or the action changes money or access, slow down and ask for another check.

Do not add a “smart” router until you can explain the problem it solves. A second model call adds latency, cost, and another failure mode. First try simple boundaries, clear schemas, and graceful fallbacks. Tiny product teams do not earn points for diagram complexity.

Put a ceiling on cost before it surprises you

Estimate the cost for a normal customer month: average requests, average input size, output length, retries, and any tool calls. Include the cases that make the prompt larger, like a long thread or a pile of uploaded documents. If the product charges $15 a month but one power user can burn through $12 in inference, you have a pricing and product-design question.

You can set limits without making the experience mean. Summarize old context, cap oversized uploads, cache repeated work, run expensive steps only when asked, and show the user what happened. Track cost per successful outcome, not just cost per API request. A cheap answer that gets rejected three times isn't cheap.

Keep the model replaceable

Put the provider call behind a small adapter and keep your app's request and response shape stable. Save a versioned prompt, schema, and eval result for each release. Log useful metadata like model name, latency, and failure category, while avoiding sensitive user content unless you have a clear reason and the right consent.

Then test a replacement with the same examples before you flip traffic. Maybe the newest model is better at long context, while an older fast model is perfectly good at your classification task. Treat a model like a dependency you can swap, not your entire product identity.

A one-afternoon selection plan

  1. Write the job and the worst believable failure.
  2. Build 30–50 redacted, representative examples.
  3. Compare two candidates on quality, speed, and cost.
  4. Add a fallback for uncertain or high-impact cases.
  5. Save the eval so the next shiny launch has to earn its way in.

The model landscape will keep moving. Your advantage is not predicting the next announcement. It's having a quick, honest way to decide whether a new capability solves a customer problem better than the system you already shipped.