MODULE 01 OF 12 · RELEASE 1 · OUTLINE
Model Gateways & Provider Abstraction
One front desk for every AI service you use, so swapping one out does not mean rewriting your app.
This module is an outline. The plain-language explanation and the lab specification below are final. The full written lesson and its runnable lab repository are still being prepared, and enrolment is not open.
Think about the reception desk at a big office. You walk in, say who you want to see, and hand over your parcel. You never learn which lift the receptionist takes, which floor the person sits on, or what happens if they are out to lunch. If the company moves floors tomorrow, you still just talk to reception.
So what is this really about?
Your app should talk to AI services the same way. Rather than every part of your code knowing how to phone a particular AI company — their exact format, their quirks, their error messages — you build one "reception desk" that everything goes through. Behind that desk you can switch companies, add a backup, or handle a service being busy, and the rest of your app never notices.
QUICK CHECK
Your AI provider doubles its prices overnight. Which setup lets you move to a cheaper one fastest?
The words you will hear
- Provider
- The company whose AI you are using. There are several, and they all speak slightly differently.
- Gateway
- Your reception desk: the one piece of code allowed to talk to providers directly.
- Abstraction
- Hiding the messy differences behind one simple way of asking, so callers do not care who answers.
- Rate limit
- A provider saying "you are asking too fast, slow down" — your desk handles that, your app does not see it.
By the end you will be able to
- Describe why talking to a provider from everywhere makes change expensive.
- Build one interface that hides provider differences.
- Handle a busy or broken provider inside the gateway, not in your features.
How it is put together
A single gateway module sits between application code and every provider SDK. Callers pass a neutral request — messages, a schema, a budget — and receive a neutral result. Provider-specific translation, retries, timeouts and token accounting live inside. Nothing outside the gateway imports a provider SDK, which is the property that makes provider choice reversible.
Where you start
A project with one working provider call written directly into a route handler — the shape most codebases actually start in. The lab is the extraction.
The lab
- Define the neutral request and response types, including a structured error rather than throwing raw provider exceptions.
- Move the existing call behind the interface without changing behaviour, proving it with the tests that already pass.
- Add a second provider implementation and a configuration switch.
- Add a per-request timeout and a bounded retry with backoff, inside the gateway.
- Record tokens and latency per call so cost is attributable later.
What goes wrong, on purpose
The scenario. The primary provider returns 429 for a sustained period, then begins returning 500s.
What should happen. The gateway retries within its bound, surfaces a structured exhaustion error, and never blocks a request thread indefinitely. Calling code handles one error type regardless of which provider failed.
How you prove it works
- A test double simulating 429 then success proves the retry path.
- A test asserting no file outside the gateway imports a provider SDK.
- A test swapping providers by configuration with identical assertions passing for both.
What you walk away with
A gateway module with two provider implementations, a passing suite that does not know which provider is configured, and per-call token and latency figures in your logs.