Learning path
Governing Enterprise AI Traffic with AI Gateways
Skill Level
Intermediate
Duration 4 hours
Updated Oct 19, 2026
About this learning path
Learn what an AI gateway is and why enterprises need one, then design one for production: token-aware cost controls, guardrails, resilience, compliance, and agent governance. Real-world lessons from Uber, Grab, and Cloudflare show what works. Finish hands-on with Kong AI Gateway, configuring authentication, budgets, prompt governance, load balancing, failover, and MCP tool access.
Your instructors
Craig KemmererWorld Wide TechnologyTech Solutions Arch II, ATC
Corey WanlessWorld Wide TechnologyPrincipal Solutions Architect
Prerequisites
- Working knowledge of REST APIs and HTTP.
- Familiarity with API gateways or reverse proxies.
- Basic LLM concepts.
- Enterprise security and identity fundamentals.
- Command-line and Linux comfort.
What you'll learn
- Explain what an AI gateway is and how it differs from a traditional API gateway, including why tokens, payload inspection, streaming, and provider inconsistency demand purpose-built controls.
- Compare the four gateway models (purpose-built, API-gateway extension, cloud-native, and build-your-own) and the hosted, self-hosted, and hybrid deployment options.
- Make the business case for an AI gateway using cost control, security and compliance, reliability, developer velocity, and readiness for agentic workloads.
- Design token-aware rate limits, hierarchical budgets, and priority tiers that control spend instead of counting requests.
- Centralize credentials with virtual keys and federate identity so provider keys never reach application code.
- Apply guardrails and content safety at the gateway, balancing inline latency against asynchronous detection and knowing where gateway filtering falls short.
- Design for resilience with failover chains, retries, circuit breakers, and the distinction between failover and fallback.
- Plan logging, audit, and regulatory alignment including the EU AI Act, NIST AI RMF, and data residency considerations.
- Choose a deployment topology that separates control plane from data plane, avoids the gateway becoming a single point of failure, and resists lock-in.
- Apply lessons from production deployments at Uber, Grab, Cloudflare, Wealthsimple, and Bloomberg, and follow a four-phase rollout sequence.
- Configure Kong AI Gateway hands-on to enforce API key authentication, consumer groups, token budgets, enterprise system prompts, load balancing, and failover.
- Govern agentic tool access by converting a REST API into MCP tools at the gateway and applying per-tool access control.