RTFM: Linux, DevOps, and system administration

We’re currently setting up our own hardware server where we want to run self-hosted models.

We’re getting ready to launch a self-hosted LLM, and at the testing stage the general idea is to send client requests simultaneously both to the “default production model” like GPT-5.6 and to the model running on our own server.

We have a server where we’re going to run self-hosted LLMs.

We have an Ubuntu server running one of our services.

A few days ago, I ran into an interesting situation with LiteLLM: on the one hand, the metrics showed a lot of errors “from the provider”, while on the other hand, the traces and alerts showed only a single error.

We got a new instance, a hardware server that will run our self-hosted LLMs.

OpenRouter recently announced a 50% discount on OpenAI, so we decided to give it a try.

Following up on the previous post Valkey: running in Kubernetes – Helm, monitoring, ACL – we set up Valkey/Redis there, now it’s time to add MongoDB.

We’re launching a new service for the project, and this service needs Redis and MongoDB.

This is the second part of the LiteLLM monitoring series – in the previous one, we covered the general integration with VictoriaStack and looked at the metrics and traces we get from LiteLLM (see LiteLLM: metrics, traces, and integration with the Victoria…