We look back at our fruitful collaboration with Mistral AI, a rising star of French AI. We support them in designing, deploying and running 24/7 their multi-cloud Kubernetes platform for the inference of their LLMs (Large Language Models).
Running AI in production at scale
Since its founding in 2023, the French startup has shown strong ambitions in generative AI: delivering high-performance open-weight models with an ethical, sovereign and responsible approach.
Turning this vision into an operational product - conversational agents like “Le Chat”, a public API, or services for partners - required a scalable, highly available inference platform capable of meeting the needs of millions of users. It was in this context that Enix was brought in for its recognized expertise in cloud and DevOps, as well as its specialization in the managed services of critical platforms. Our support began during the architecture design phase, continued through the platform’s deployment, and has carried on ever since with 24/7 Run and operational maintenance.
A multi-cloud architecture built for performance and resilience
From the earliest phases of the project, technology choices were guided by the specific requirements of production LLM inference: the ability to handle large volumes of requests with horizontal scalability, minimal latency, continuous availability, and optimized use of costly GPU resources.
"Our goal was clear: to provide Mistral with a cloud native platform capable of delivering consistent performance even under heavy load, operated by our teams, with flawless service availability.", explains Alexandre Buisine, one of Enix's managing partners.
The platform designed by Enix is built on a dual public cloud foundation and on Kubernetes, using the managed Kubernetes services from Azure (AKS) and Google (GKE). This multi-cloud approach allowed Mistral AI to adapt to GPU availability across different providers, ensure resilience by avoiding any single point of failure, and avoid dependency on a single provider.
Following a logic of advanced automation, the Kubernetes clusters are deployed using infrastructure-as-code solutions suited to cloud services: Terraform and Terragrunt. Applications are containerized with Docker and packaged with Helm, while updates and deployments rely on a GitOps strategy powered by the cloud native technology Flux.
AI inference-specific optimizations
One of the project’s key challenges was integrating AI-specific optimizations, which are often complex to implement in managed cloud environments that are by nature more closed than vanilla Kubernetes services: this included storage and deployment of large models, as well as autoscaling optimizations for GPU nodes, among others.
The deployed platform delivers the inherent benefits of Kubernetes, such as resilience through self-healing, intelligent node autoscaling, smoother application updates, and easier maintenance.
On the storage side, high-performance and durable solutions were put in place to support the deployment of large models, with fast access to critical data. The databases used (SQL-based) are also sized to handle high load peaks while maintaining minimal latency and high availability.
Monitoring, metrics and 24/7 Run
Since the platform went into production at the end of 2023, Enix has been handling its day-to-day managed services. This includes the core services required for high-quality operational maintenance and management:
- infrastructure monitoring (alerting, metrics, logs) and real-time supervision (cloud, Kubernetes, databases, etc.);
- proactive performance management (optimizations, capacity planning);
- functional and security updates;
- automation of operational tasks (scripts, DevOps tooling);
- incident management and 24/7 technical support.
Enix also uses its cloud native metrology platform to surface the right indicators to Mistral AI’s teams, with custom business dashboards tailored to their needs: inference performance monitoring, GPU consumption, application errors, SLAs, and more.
A close collaboration in service of a shared vision
The collaboration between Enix and Mistral AI goes beyond the traditional client-provider model. It takes the form of joint technical management and seamless communication through a shared Slack channel, as if the Enix teams were part of Mistral AI’s internal teams.
This way of working, characteristic of the premium managed services offered by Enix, guarantees direct access to senior engineers, with no intermediaries or complex escalation chain. This enables continuous platform optimization, greater operational agility, fast problem resolution, and a level of responsiveness that would be impossible to achieve with more traditional managed services and organizations.
Beyond that, this collaboration is built on shared values. Mistral AI’s commitment to ethical, sustainable and open source artificial intelligence resonates with Enix’s core values: transparency, technical excellence, and active contribution to the open source ecosystem.
Collaborating with Mistral is, for us, an opportunity to put our expertise at the service of a company that embodies our values and helps showcase France on the international stage. In these complex times, we're proud to show that we can play our part within the French tech industry.", highlights Alexandre Buisine.
Enix has thus been able to meet the scalability, performance and resilience challenges posed by Mistral AI, all while staying true to its DNA: delivering custom cloud architectures built on robust open source technologies, and guaranteeing high-quality 24/7 Run.
The road traveled together
Brought on board in 2023 to set up the first inference infrastructure in under 10 days, Enix rose to the challenge and had the privilege of contributing to the creation of Mistral AI’s public services, which are now used extensively.
At the time, Nvidia H100 GPUs were still in very short supply among cloud providers, and support was sometimes lacking. The Enix teams had to implement a workaround for a blocking bug on Azure AKS: related to the H100 GPU instance drivers, Microsoft’s official fix was not released until five months later.
This project reflects the way Enix builds and takes ownership of custom infrastructure to support the technological ambitions of a company like Mistral AI.
Beyond the technical success, a dynamic of trust and co-construction has developed between the technical teams of both companies. Now jointly managed by both entities, operations were previously handled solely by the Enix team, giving Mistral AI time to build up its own reinforced teams.
Enix remains committed to standing alongside Mistral AI to adapt and evolve this platform, with the same level of rigor, agility and commitment as always!
