Which GPUs does your company need for AI in its own building?

Each task has a standard AI workload per user per working day: tasks, model calls, input and output tokens, and for some tasks meeting audio, indexed text and reranked passages. The busiest hour sets the size: interactive work peaks there, work that can wait a few minutes is spread over the day and batch work runs at night.

For every GPU and model the calculator works out how fast the GPU reads prompts (limited by compute) and writes answers (limited by memory bandwidth and by the memory each conversation takes), keeping every user at or above the minimum answer speed. A model runs on as few GPUs as it fits on, or on twice as many when that is cheaper per token.

Every model in use stays loaded, so their weights add up in GPU memory. The GPUs needed is the highest of what the memory, the busiest hour and the night jobs ask for. The cheapest GPU per month, hardware written off plus electricity, is recommended.

The method was checked on 20 Oct 2026 against NVIDIA's published TensorRT-LLM throughput on the RTX PRO 6000 Blackwell Server Edition: for a mixture-of-experts model it comes within 1 to 7%, and for a dense 70B model it stays 20 to 28% below the measured figure, so it errs on the safe side.