06
AI Integration
Getting an LLM to do something impressive once is easy. Getting it to do the right thing reliably, at a cost you can predict, with a sensible answer when it fails — that's the engineering part.
Where AI features usually go wrong
- It works in testing and behaves differently with real user input.
- Costs that scale in a way nobody modelled.
- No handling for rate limits, timeouts or refusals.
- Retrieval that returns confidently irrelevant results.
- Output parsed by hope rather than by schema.
What I actually build
- 01
The right shape for the problem
Not every feature needs an agent. Often a single well-constrained call, or no model at all, is the correct answer — and I'll say so rather than building the more interesting thing.
- 02
Structured, validated output
Schema-enforced responses, validated before they reach your application. Parsing free text and hoping is the source of most AI-feature bugs.
- 03
Retrieval that actually retrieves
For RAG: chunking, embedding and ranking chosen for your content, then evaluated. "It seems to work" is not an evaluation.
- 04
Failure and cost control
Timeouts, retries with backoff, fallbacks, token budgets and logging — so a provider incident degrades your feature instead of taking down your product.
How long it takes
A well-defined feature is usually three to five days. It depends on the requirements — retrieval systems sit at the longer end, because building one is fast and evaluating it honestly is not.
Questions about this
Can you work with FastAPI?
Yes. FastAPI is my usual choice for Python services, particularly anything with an AI or data component. I’m comfortable across the surrounding ecosystem too — async workers, migrations and typed request models.
Can you work with Node.js?
Yes. Node is one of my primary backend stacks, including APIs, background jobs, integrations and the deployment around them.
What do you need from me?
Less than you’d think. A repository or a deployed URL, some idea of what’s broken, and access to whatever is failing — logs, the hosting dashboard, an error message. If you don’t have all of that, send what you do have and I’ll tell you what else I need.
Your AI-built app doesn’t need another prompt.
Sometimes it needs an engineer.
Send the messy version. That’s usually the useful version.