◀Stamp a Copy, Save the Bill
MATH BerriAI/litellm #44341 · ★ 63kfix(proxy): stamp the client alias on a copy of each streamed chunk so pricing sees the deployment modelA real, public pull request, signed as made with Devin. See it on GitHub ↗
IN THE WORLD

Fixes $0 billing bugs when companies stream AI responses through model aliases.

REACH

Built by startup BerriAI and 1,700+ contributors, LiteLLM has 63,000+ GitHub stars and 92.2 million monthly downloads. Over 2,500 companies, including Netflix, NVIDIA, and Lemonade, use it to route AI traffic.

WHAT IT MEANS

Companies using model nicknames won't accidentally record $0 for streamed AI replies anymore. Billing and usage logs now match non-streamed requests, while unpriced custom models still bill $0.

  • BerriAImakes it
  • Netflixuses it
  • NVIDIAuses it
RED · how I did it
  • Start with $10 from the 2 non-streamed requests.
  • GONE Renaming streamed chunks in place hides the real model price, so every streamed request adds $0.
GREEN · how I do it now
  • Start with $10 from the 2 non-streamed requests.
  • NEW Only a copy is renamed for the user, so every streamed request is billed at its real $5 price.

WHERE In the LiteLLM admin dashboard on the request logs and team budget screen after streaming chat responses.

SCENE You manage an engineering team using a custom nickname for a $5-per-request AI model. Your team already ran 2 regular non-streamed chats ($10 total) and now runs extra streamed chats.

Streamed requests
How many streamed chat requests your team sends through the model nickname. (0 requests to 10 requests)
Logged team spend
The total spend recorded on your team's budget dashboard.
Fixed
2 non-streamed requests already billed at $5 each ($10 base), plus streamed requests worth $5 each