AI engineering
LLMsystemsthatsurviverealusers
Agents, retrieval and structured extraction — built with an eval suite, so you can tell whether a change made things better or only different.
Most teams can get a convincing demo in a fortnight. The gap between that and something customers depend on is where projects die: no evals, no cost ceiling, no idea why quality moved. That gap is the work we do.
You might recognise
- A prototype that impresses in a demo and embarrasses you in production
- Retrieval that was good in July and nobody can explain why it is worse now
- Model spend that scales faster than revenue and has no ceiling
- No way to prove a prompt change helped, so nobody dares touch it
How we work through it
- 01
Build the eval set first
Before touching the system we assemble a graded set of real cases from your data. Everything after this is measured against it. Teams that skip this step are guessing, and they usually cannot tell they are guessing.
- 02
Fix retrieval before prompting
Most quality problems blamed on the model are retrieval problems: bad chunking, stale indexes, an embedding that never matched the query distribution. We measure it in isolation first.
- 03
Harden the loop
Timeouts, fallbacks, structured outputs with validation, retries that do not amplify load. What the system does when a call fails matters more than what it does when everything works.
- 04
Instrument the spend
Per-tenant cost attribution, hard token ceilings, cache layers on the paths that repeat. You should be able to answer what a given customer costs you.
What you are left holding
Everything ships into your repositories and your cloud. There is no runtime dependency on us after handover — that is the point.
- A graded eval set built from your own data, with CI wired to it
- The production system, in your repos and your cloud
- Cost and latency dashboards with per-route attribution
- A written record of what we tried, what worked, and what we ruled out
- A handover session with the engineers who will own it
Questions we get
- Do you work with our existing prototype or start over?
- Almost always we start from what you have. Rewriting is occasionally correct but it is rarely the fastest route to production, and your prototype encodes real decisions worth keeping.
- Which models do you use?
- Whichever fits the task, the latency budget and the price. We build the eval harness so the model is a swappable decision rather than a permanent bet.
- What if quality is still not good enough?
- Then the eval set tells us that early and cheaply, which is the point of building it first. Sometimes the honest answer is that the task is not yet reliable enough to ship, and we would rather tell you in week two than month five.
Have a problem shaped like this?
Thirty minutes, no deck. Bring the thing that is not working and we will tell you what we would do about it — including when the answer is that you do not need us.
We reply within one business day