Explore
Select a tab
1 result found
ATC+
Red Hat AI Inference Sandbox
Serve a large language model on a single NVIDIA L40S GPU twice, with Ollama and with the Red Hat AI Inference container. Benchmark both with one user and with a crowd, compress a model to FP8 with LLM Compressor, and compare the results.
Advanced Configuration Lab
•Intermediate