Deploying AI agents at scale introduces significant challenges, including high compute costs and latency bottlenecks—especially in performance-critical environments. Balancing model accuracy with efficiency often requires complex workflows and ongoing manual intervention. 

The NVIDIA Data Flywheel Blueprint provides a systematic, automated solution to refine and redeploy optimized models that maintain accuracy targets while lowering resource demands. This blueprint establishes a self-reinforcing data flywheel, using production traffic logs and institutional knowledge to continuously improve model efficiency and accuracy. 

What's Included in the Blueprint 

This blueprint automates continuous optimization of AI agents using NVIDIA NeMo microservices for data curation, customization, and evaluation. It systematically evaluates multiple models and automatically surfaces the most efficient option that meets defined latency, cost, and accuracy criteria. The architecture is adaptable to a wide variety of reasoning and task-specific use cases. 

Key Benefits 

• Reduce Latency and Cost: Identifies smaller models that are empirically equivalent, enabling deployment of more efficient NIM microservices while maintaining performance.
• Continuous Improvement Loop: Enables ongoing evaluation without retraining or relabeling—true "flywheel" behavior that runs indefinitely as new traffic flows in.
• Data-Driven Decisions: Provides real comparisons across models using real traffic, backed by evaluator scores.
• Standardized Optimization: Any application can opt into the flywheel with minimal effort, making it a foundational component for a wide variety of use cases.

Key Features 

• Production Data Pipeline: Collects real-world data from AI agent interactions and curates datasets from configurable log stores for evaluation, in-context learning, and fine-tuning.
• Automated Model Experimentation: Leverages a deployment manager to dynamically spin up candidate NIMs from a model registry—including smaller or fine-tuned variants—and run experiments such as in-context learning and LoRA-based fine-tuning.
• Semi-autonomous Operation: Operates without requiring any labeled data or human-in-the-loop curation.
• Evaluation with NVIDIA NeMo Evaluator: Evaluates models using custom metrics and task-specific benchmarks (e.g., tool-calling accuracy), leveraging large language models (LLMs) as automated judges to reduce the need for human intervention.
• LoRA-SFT Fine-Tuning with NVIDIA NeMo Customizer: Runs parameter-efficient fine-tuning on models using real-world data.
• REST API Service: Provides intuitive REST APIs for seamless integration into existing systems and workflows, running continuously as a FastAPI-based service that orchestrates underlying NeMo microservices.
• Easy Deployment with Helm Chart: Simplifies deployment and management of the Data Flywheel Blueprint on Kubernetes setup with a unified, configurable Helm chart.