A major player in French e-commerce, Rakuten France relies on critical infrastructure to run its marketplace and support its teams’ development cycles. It hosts QA, staging, and production environments, along with numerous application services, databases, microservices, and Kubernetes clusters.
To improve the performance of its environments, achieve long-term cost control, and gain operational autonomy, Rakuten France entrusted Enix with the design, deployment, hosting, and managed operations of a new virtualization platform based on Proxmox VE.
Initially intended for QA and development environments, the project proved reliable and relevant enough at scale to become the foundation of a 17-node production cluster. Rakuten France now runs close to 1,000 machines across its two Proxmox VE clusters.
Replacing a costly, underperforming cloud
Rakuten France’s QA and development environments previously ran on a public cloud operated by another provider. This infrastructure no longer met the teams’ needs. Performance was slowing down testing and development cycles, while support lacked responsiveness. For equivalent capacity, the cost was also estimated at nearly three times higher than that of a dedicated, managed infrastructure. Rakuten France therefore wanted to build a new platform addressing four main objectives:
- Improve the performance of QA and development environments;
- Make infrastructure costs more predictable;
- Automate the creation of environments;
- Benefit from responsive, experienced technical support.
A Proxmox VE platform built to measure
After analyzing Rakuten France’s needs, we selected Proxmox VE as the virtualization foundation. This open source solution met the automation and flexibility requirements, without the per-core licensing costs imposed by some established vendors. The architecture and hardware were sized according to the applications’ real-world constraints. Several monolithic Java workloads notably require strong single-thread performance.
Our involvement covered the entire project:
- Defining the architecture, and selecting and sizing the hardware;
- Configuring the servers and designing the network;
- Deploying the Proxmox VE clusters;
- Setting up distributed Ceph storage;
- Developing Infrastructure as Code tooling;
- Setting up monitoring and access management;
- Hosting and managed operations of the platform.
On-demand environments with Terraform
The platform was designed to automate the creation and teardown of QA and development environments. With Terraform, teams can deploy environments on demand while ensuring the consistency and reproducibility of configurations. The platform also integrates ProxLB, an open source tool that automatically balances virtual machines across the Proxmox VE hypervisors. Terraform plans can therefore coexist with VM live-migrations without creating conflicts between the declared infrastructure and its actual state.
From a first four-node cluster to nearly 1,000 virtual machines
The project started with a first QA platform made up of four nodes. After validating its operation and performance, Rakuten France progressively expanded it into a seven-node cluster dedicated to QA, development, and staging.
Each Dell server in this cluster is equipped with:
- Two AMD EPYC 7443 processors;
- 1 TB of RAM;
- Two 15 TB NVMe disks used by Ceph.
The architecture pools redundancy at the cluster level and keeps enough spare capacity available to withstand the complete loss of a hypervisor.
The success of this first platform then gave Rakuten France the confidence needed to extend Proxmox VE to its production workloads. The company now operates:
- A seven-node Proxmox VE cluster for QA, development, and staging, with distributed Ceph storage and automated provisioning via Terraform;
- A 17-node Proxmox VE cluster for production, hosting around 500 virtual machines.
In total, the two clusters host nearly 1,000 virtual machines.
A foundation for the marketplace’s critical services
The Proxmox VE clusters host a wide range of components essential to the Rakuten France marketplace: Data services running on PostgreSQL with Patroni, MariaDB, Couchbase, Redis, and RabbitMQ messaging; Containerized microservices with Docker and Kubernetes clusters; DNS services based on PowerDNS.
High availability is mainly handled at the application level. The Proxmox VE foundation, for its part, is designed as a compute layer built to be as stateless as possible.
Persistent application and database data is hosted on separate physical systems, with their own backup and restore mechanisms.
The storage for the QA and development cluster relies on Ceph, while the production cluster uses a SAN iSCSI infrastructure inherited from the previous environment.
Costs cut threefold
Replacing the old cloud with a dedicated Proxmox VE platform delivered immediate benefits, both technically and economically.
The cost of the QA and development infrastructure was cut by a factor close to three compared to the previous solution. Rakuten France also now benefits from a more predictable economic model, with no hard-to-anticipate variable pricing and no per-core licensing constraints. The company also retains more freedom to adapt its hypervisors’ capacity to its needs.
Performance has markedly improved. Teams now have faster, more consistent environments, which eases testing and speeds up development cycles.
Close, hands-on support, from design to 24/7 operations
Beyond the technology choice, Rakuten France was looking for a partner able to understand its application constraints, design a tailored architecture, and support it over time. Thanks to our Proxmox VE expertise and our 24/7 managed services model, we support Rakuten France across the platform’s entire lifecycle, from design to operations. Rakuten France benefits from direct access to our engineers, who work across the Proxmox VE virtualization layer as well as the hardware, network, and distributed storage.
This closeness has made it possible to progressively build a platform suited to the company’s real-world usage, rather than transposing a standardized infrastructure model.
A proven platform, now used in production
The QA and development cluster was initially meant as a first step to validate the relevance of Proxmox VE for Rakuten France’s needs.
Its reliability, performance, and operational flexibility turned this first platform into a proof of concept at scale. Rakuten France thus chose to extend Proxmox VE to a 17-node production cluster hosting the marketplace’s core services.
What started as a targeted replacement of a QA cloud environment has thus become a strategic virtualization foundation, able to support the company’s current and future needs.
