Solution overview

The F5 AI Gateway solution matures your AI infrastructure in multiple ways. Below are a few of the benefits of running the F5 AI Gateway, from security to resiliency to performance. This is how the F5 AI Gateway can assist:

  • Inspecting and filtering client requests and responses across multiple models
  • Preventing malicious inputs from reaching an LLM back-end
  • Ensuring that responses are safe to send to clients, providing guardrails no matter the LLM
  • Ability to begin collecting data to retrain or fine-tune models to be more relevant to the consumers
  • Identifies a user through profiles or information, providing a level of trust that data will not be uploaded or exposed to the wrong audience
  • Can route traffic to different models, including the benefit of upgrading or replacing models without downtime.
  • Ability to leverage GPUs or pure software for performance gains - this particular lab runs on NVIDIA GPUs

This lab will provide access to an OpenShift cluster running the F5 AI Gateway solution. We will walk through how the F5 AI Gateway routes requests to different models by either allowing them to pass through or, more importantly, securing them via prompt injection checking. We have also added a couple of other tests that will allow for language detection of that input that the F5 AI Gateway can also detect.

Lab diagram

Technologies