Practice 04 / Applied AI
Applied AI systems.
We put language models into production behind measurements you can trust. Retrieval, serving, evaluation, and guardrails, built so you can prove the system is correct before it faces a user. When a task cannot be made measurable and safe, we say so.
What we build
Four building blocks, assembled to fit.
Each block stands on its own and connects to the others. We start from the outcome you need and work backward to the smallest system that delivers it under measurement.
Retrieval and RAG systems
Answers grounded in your own documents, tickets, and records instead of the model's training data. We build the ingestion, chunking, and embedding pipeline, tune hybrid retrieval (dense vectors plus keyword), and add reranking so the right passages reach the prompt. Every answer carries citations, and the system refuses when it cannot find support.
- Grounded generationCitations on every claim, with a refusal path when retrieval comes up empty.
- Hybrid retrievalVector plus keyword search with reranking, tuned against a labelled query set.
- Fresh indexesIncremental re-embedding so the index tracks source changes, not last quarter's snapshot.
Model serving and inference infra
The layer that turns a model into a service with a latency budget. We serve on Triton with dynamic batching, run models on GPU or CPU depending on cost and load, and put a queue in front so a traffic spike degrades gracefully instead of timing out. Autoscaling is tied to real queue depth, and every request is traced end to end.
- Batched inferenceDynamic and continuous batching on Triton to lift throughput without wrecking tail latency.
- Right-sized computeQuantized and distilled models where accuracy allows, to cut GPU spend.
- BackpressureQueues, timeouts, and load shedding so overload is bounded, not a cascade.
Evaluation, guardrails, and observability
The part most teams skip and later regret. We build a graded evaluation set from your real cases, wire it into CI so a change that drops quality cannot merge, and run input and output guardrails at request time to block prompt injection, unsafe content, and unsupported claims. In production we log inputs, retrieved context, and outputs so every answer can be audited.
- Graded eval setsLabelled cases with pass thresholds, run on every change and on a schedule against live traffic.
- Runtime guardrailsInjection detection, content policy, and grounding checks on input and output.
- Full traceabilityPrompt, context, and response logged so any answer can be reconstructed and reviewed.
ML platform and MLOps
The plumbing that makes AI repeatable rather than a one-off heroics project. Versioned datasets and prompts, a model registry, reproducible training and fine-tuning jobs, and deployment through the same review gates as the rest of your code. When a model or prompt changes, you can see exactly what changed, why, and how the evaluation numbers moved.
- Versioned everythingDatasets, prompts, and models tracked in a registry with lineage from data to deploy.
- Reproducible jobsFine-tuning and batch inference as pipelines you can rerun and audit, not notebooks.
- Gated rolloutCanary and shadow traffic with automatic rollback on an evaluation or latency regression.
How a model reaches production
From a promising idea to a gated release.
Frame the task and the failure modes
We define what a correct answer looks like, what a wrong one costs, and where a human has to stay in the loop. If the task cannot be graded, we stop here and tell you why, before you spend a budget on it.
Build the eval before the feature
We assemble a graded set from your real cases and set pass thresholds. This is the ruler we measure every later change against, so improvements are visible and regressions are caught in CI, not in front of users.
Serve, guard, and observe
We put the model behind Triton with batching and backpressure, wrap it in input and output guardrails, and instrument it so p95 latency, refusal rate, and eval scores are on a dashboard from day one.
Roll out behind a gate
Shadow traffic first, then a small canary, with automatic rollback if the eval score or latency budget slips. Once it holds, we widen the rollout and hand you the runbook, or run it for you under an SLO.
Where we say no
We ship AI where it is measurable and safe.
Not every problem is an AI problem, and not every AI feature belongs in production. If a task has no gradable notion of correct, if the cost of a wrong answer is high and cannot be contained by review, or if a smaller deterministic system would do the job better, we will tell you plainly. We would rather lose the line item than ship something we cannot measure and you cannot trust.
Measurable
If we cannot grade it, we do not ship it. Every feature ships with an eval set and a threshold.
Safe
Guardrails on input and output, a refusal path, and a human checkpoint wherever the stakes require one.
Honest
When a simpler system beats a model, we build the simpler system and say so up front.
Related work
This is what it looks like in production.
A retrieval assistant we built and evaluated behind measurable guardrails, running against real claims every day.
Questions we get
Honest answers on the hard parts.
We build a graded set from your real cases, with each case labelled by someone who knows the domain, and a clear pass threshold. That suite runs in CI on every change and on a schedule against live traffic, so a prompt tweak or a model swap that drops quality is caught before it reaches users. We track accuracy, refusal rate, and latency together, because a system that is accurate but too slow, or fast but wrong, is still broken. As reviewers correct outputs in production, those corrections feed back into the eval set, so the ruler gets sharper over time.
You cannot fully stop a model from inventing things, so we design around it. Answers are grounded in retrieved evidence and must cite the source they used, an output guardrail checks that the claim is actually supported by that source, and if nothing supports it the system refuses rather than guessing. For anything where a wrong answer carries real cost, a human reviews before the output is actioned. The goal is not a model that is always right, it is a system where an unsupported answer cannot reach the user unchecked.
Both, chosen on the constraints rather than fashion. Hosted API models get you to a working system fastest and are often the right call for lower-volume or exploratory work. We move to self-hosted models on Triton when data residency, cost at high volume, tail latency, or the need to fine-tune on your own data make it worth the operational weight. We keep the serving interface stable so you can switch providers or models without rewriting the application, and we benchmark both against the same eval set so the decision is made on numbers.
Your data stays under your control. For sensitive workloads we run self-hosted models inside your own environment, so prompts and documents never leave your boundary. Where a hosted API is appropriate, we use providers with no-training and retention terms that fit your obligations, strip or tokenize sensitive fields before they leave, and document exactly what is sent and stored. Retrieval indexes, logs, and traces live where you decide, with access controls and retention policies set to your rules, not defaults. See our privacy policy for how we handle information during an engagement.
Start
Have an AI feature that has to be right?
Tell us the task and where a wrong answer would hurt. We will tell you honestly whether it can be made measurable and safe, and how we would build it if it can.