Aiven cuts AI token costs by 80 percent with open-weight models

Aiven, a European open source database service provider based in Finland, has saved 80 percent of its internal AI token costs by moving to open-weight models. The company implemented an internal gateway to help developers work with these models more easily instead of having strict token budgets. This gateway was built around the open source framework LiteLLM and added new features such as budgeting and access management. Aiven runs the models on rented GPUs and self-hosts them, offering employees access to different open-weight models from Western and Asian sources as well as frontier models.

Ville Lehto, Aiven's VP of strategy, stated that the company wanted to manage its own token costs and control its destiny regarding AI usage. He noted that a fraction of users are heavy users whose spend could be significantly saved by using open-weight models. Aiven uses metrics such as cost per task alongside token usage because the performance of open-weight and frontier models varies. The company continues to use frontier models to help plan work but uses open-weight models for the majority of actual tasks.

Aiven plans to offer the gateway as a service to customers to help them manage token costs. Its customers include German ecommerce company Conrad, UK payment processing company Dojo, cyber security vendor Sophos, and communications firm Avaya.