Objective
Run virtual machines and containers reliably, with the ability to do maintenance on nodes and to recover services after a failure.
Architecture
The cluster consists of three Proxmox VE nodes connected to a managed switch and to a network dedicated to cluster services. Each node has local ZFS storage replicated to the other nodes; the HA manager restarts protected resources on a surviving node after a failure.
- Managed switch is connected to Node 1
- Managed switch is connected to Node 2
- Managed switch is connected to Node 3
- Cluster network is connected to Node 1, logical link
- Cluster network is connected to Node 2, logical link
- Cluster network is connected to Node 3, logical link
- Node 1 is connected to Replicated ZFS storage
- Node 2 is connected to Replicated ZFS storage
- Node 3 is connected to Replicated ZFS storage
- Node 1 is connected to VMs and containers
- Node 2 is connected to VMs and containers
- Node 3 is connected to VMs and containers
Details
Details
The cluster groups several Proxmox VE nodes into a single management entity: configurations, virtual machines, and containers are visible from any node. The diagram shows the typical three-node setup, the minimum needed to keep quorum even after one node fails.
- Network: management and VM traffic go through a managed switch; a dedicated network carries the cluster services (Corosync) and replication.
- Storage: local ZFS pools on the nodes, with asynchronous replication of the protected machines.
- High availability: the HA manager restarts protected resources on a surviving node; watchdog-based fencing prevents double activation.
- Maintenance: updates are done on a rotating basis, moving workloads first with live migration.
Current setup
- Nodes: three, the minimum that lets quorum survive the failure of a single node. They are mismatched machines (not an identical kit): the characteristics of each are in Physical nodes and hardware.
- Version: Proxmox VE 9, updated regularly (packages and kernel), reading the release notes before every minor version upgrade.
- Storage: each node has one or more local ZFS pools (size and number vary from node to node, depending on the available disks) that host the operating system and the VM disks; machines protected by the HA manager are replicated asynchronously to a second node every few minutes, so recovery after a failure starts from a recent state rather than from the last backup.
Further reading
Creating the cluster and how quorum works are described in Three-node Proxmox VE cluster: setup, quorum, and first checks.