Note: The description in the article is based on Veeam Backup & Replication 11a, licensed using Veeam Universal License (VUL), similar to Enterprise Plus.
Architecture and Components
The core of Veeam Backup & Replication is the image backup of virtual machines VMware or Hyper-V, which are natively supported. For backing up physical machines (or even VMs) or selected applications, agents and plugins are used. Backing up each environment/platform (simplified VMware, Hyper-V, agents) works slightly differently, but we control it from one console using similar jobs. The basic architecture elements are the same but can be used or deployed differently. Therefore, Veeam does not have general documentation, but the description is separate for each platform. Some features and functions are only available for VM backup, with the most options for VMware backup.
The following description tries to be general, yet some things only apply to VM backup. Most information about architecture and components can be found in the documentation User Guide for VMware vSphere, User Guide for Microsoft Hyper-V.
Basic Components
Documentation of all components Backup Infrastructure Components
- Veeam Backup Server - control server, Backup Server
- virtual or physical, must run on Windows OS
- center for configuration and management (system administration)
- does not process data but manages infrastructure components
- controls and schedules jobs and resource usage, orchestrates backup and replication
- Veeam Backup & Replication Configuration Database - uses an instance of Microsoft SQL Server, which can run locally (Express version can be installed with the server) or remotely, stores configuration, job settings, etc. in the DB
- consists of several services and components
- Veeam Backup Proxy - data mover, Backup Proxy
- contains Veeam Installer Service and the main component Veeam Data Mover
- reads and sends data (Data Mover)
- provides an optimal path for backup traffic and efficient data transfer
- reads data (e.g., VM from production storage), then compresses and deduplicates (possibly encrypts) it and sends it to the repository (or another special component)
- facilitates communication between the data source and the backup target
- using multiple Backup Proxies, we can scale the backup infrastructure
- Veeam Backup Repository - backup storage, Backup Repository
- receives data and provides storage for backups and metadata
- processes data according to the type of storage and stores it
- can be a Windows or Linux server, NAS device, deduplication appliance, or object storage
- can use various file systems (NTFS, ReFS, XFS, etc.), ReFS or XFS with Fast Clone enabled is recommended
All three components can be located on one server or can be separated.
A special component is the source (Source Host) for backup. Generally, we can add a physical or virtualization server to the Veeam backup infrastructure and assign it a specific role.
- Virtualization Servers and Hosts - virtualization servers and hosts, Virtualization Servers and Hosts
- VMware vSphere Server (ideally vCenter Server, or ESXi host)
- Microsoft Hyper-V Server (Hyper-V cluster, SCVMM Server, Hyper-V host)
- Microsoft Windows server
- Linux server
- VMware vCloud Director
- Microsoft SMB3 server
Some Other Components
- Mount Server
- for restoring OS files or application items from VM
- to access files and items stored in the backup file, Veeam mounts the backup to the Mount Server
- only after mounting can files be retrieved and copied to the target destination
- Guest Interaction Proxy
- located between the Backup Server and the processed VM
- needed if the backup job performs application-aware processing, file system indexing, or transaction log processing
- Veeam vPower NFS Service
- service running on Windows, which allows functioning as an NFS server
- Veeam creates an NFS datastore where it publishes VMware VM files, which are still stored in the backup storage, so it is an emulation, the NFS datastore is connected to the ESXi host and allows starting the VM
- used for Instant Recovery
Deployment
For a simple deployment, we install Veeam Backup & Replication, and all its components and services, on a single server with Windows OS. For VMware backup, it may be suitable under certain conditions if it is a virtual server (allows using Virtual appliance transport mode).
All components involved in the backup job form a data channel for data transfer. The source host and backup storage create two endpoints for data flow. Data processing is performed by Veeam Data Mover. It uses a two-service architecture, one Veeam Data Mover manages interaction with the source host and the other manages interaction with the backup storage. They maintain a stable connection between them.
VMware vSphere Backup
VMware ESXi uses only a small VMkernel, so Veeam cannot run its components on it (unlike Hyper-V, which runs on a full OS). We must install Backup Proxy on the Backup server or separately. Depending on the architecture, we can choose different transport modes, which affect how (and where) data is transferred over the network. There is also direct storage integration (Direct Storage Integration), which allows Backup from Storage Snapshot.
Detailed description is in the article Veeam Backup & Replication - Backup from Storage Snapshot.
Transport Modes
Documentation Transport Modes
This is the method used by Veeam Data Mover (Backup Proxy) to read VM data from the source and write it to the target. The mode used affects the efficiency and duration of the backup job.
- Direct Storage Access
[san]- reads/writes data directly from/to shared storage (Storage) via SAN (bypasses ESXi servers and LAN), where VM or backup data is located, must have direct access to the storage via hardware or software HBA, Direct SAN access or Direct NFS access - Virtual Appliance
[hotadd]- uses the VMware SCSI HotAdd feature, which allows attaching devices to a VM while running, during backup, disks are attached to the Backup Proxy, data is read/written directly from the datastore instead of copying over the network, usable if the Backup Proxy is in a VM - Network Mode
[nbd/nbdssl]- least efficient, data is read over the network using the Network Block Device (NBD) protocol, 10 Gbps Ethernet is recommended
Note: The transport mode used (abbreviation) can be seen in the list of backup job actions at the end of the action Using backup proxy...
About VMware Backup
- if the ESXi host is managed by vCenter Server, it is recommended to connect the vCenter Server, not the standalone ESXi, the VM can then travel between ESXi servers
- during backup jobs, Veeam constantly communicates with the vCenter Server, which must be available with good network connectivity
- Veeam is designed for virtual environments, works on the virtualization layer, and uses images for VM backup
- for creating a backup, no agent is needed inside the VM's operating system, but the VMware vSphere Snapshot feature is used
- Snapshots have a certain performance impact on the infrastructure, creating them causes a brief pause of the VM, opening a Snapshot increases I/O load, and removing it causes the highest I/O load and may also cause a brief pause of the VM
- when backing up a VM, Veeam asks VMware vSphere to create a VM Snapshot and this Snapshot is used as the data source for the backup
- Veeam copies VM data from the source datastore at the block level, compresses and deduplicates it, and stores it in the backup storage in a proprietary Veeam format
How Backup Works
- Backup Job starts, a list of tasks to be processed is prepared (each VM disk is one task)
- resource scheduler checks available resources, assigns Backup Proxy and Backup Repository
- Veeam Data Mover on Backup Proxy and Backup Repository creates a connection for data transfer
- Veeam asks VMware vCenter Server to create a VM Snapshot (disks are switched to read-only state, changes are written to delta files)
- the source Veeam Data Mover reads VM disks and transfers the data to the backup storage (target Veeam Data Mover) using one of the transport modes. If it is an incremental backup, only the changed blocks since the last backup are read using CBT. If CBT is not available, Veeam Metadata is used to detect changed blocks
- during data transfer, the source Veeam Data Mover performs additional processing (filters out zero data blocks, swap file blocks, and excluded file blocks from the backup), compresses the data, and transfers it
- after data reading is complete, Veeam requests the confirmation (Commit) of the Snapshot
Microsoft Hyper-V Backup
Hyper-V runs on a complete Windows Server operating system, allowing the Backup Proxy to run directly on the source Hyper-V server. Backup is based on using Backup Proxy servers, which can be (set per Job):
- On-Host - directly the Hyper-V server where the VM runs, acts as a Proxy, processes VHD(X) and transfers data to the repository, loads the server during processing
- Off-Host - another (special) Hyper-V server serves as a Proxy, must be a physical server with the Hyper-V role
Note: To add a System Center Virtual Machine Manager (SCVMM) server, the corresponding version of the Virtual Machine Manager Console must be installed on the Veeam Server. SCVMM setup.exe cannot be run if the computer is not in a domain (we are not logged in with a domain account), it can be bypassed from the command line with setup.exe /client /i /IACCEPTSCEULA.
About Hyper-V Backup
- if the Hyper-V host is a member of a cluster, it is recommended to connect the cluster (or SCVMM server), not the standalone Hyper-V server, the VM can then travel between Hyper-V servers. It is not necessary to connect the SCVMM server, but it is sufficient to connect the cluster managed by SCVMM
- Veeam is designed for virtual environments, works on the virtualization layer, and uses images for VM backup
- for creating a backup, no agent is needed inside the VM's operating system, but the Microsoft VSS Snapshot and Checkpoint feature is used
- when backing up a VM, Veeam asks Microsoft Hyper-V to create a Snapshot / Checkpoint VM and this Snapshot is used as the data source for the backup
- Veeam copies VM data from the source volume at the block level, compresses and deduplicates it, and stores it in the backup storage in a proprietary Veeam format
- to track changed blocks, Hyper-V uses Resilient Change Tracking (RCT), but it is available only from Microsoft Hyper-V Server 2016 (in a cluster, all nodes must be on version 2016 and the cluster functional level as well, VM configuration version must be 8). Otherwise, the Veeam CBT driver can be used
How Backup Works
- Backup Job starts, a list of tasks to be processed is prepared (each VM disk is one task)
- resource scheduler checks available resources, assigns Backup Proxy and Backup Repository
- Veeam Data Mover on Backup Proxy (on-host or off-host) and Backup Repository creates a connection for data transfer
- Veeam asks Microsoft Hyper-V VSS to create a Volume Snapshot or VM Checkpoint (depending on the hypervisor version)
- the source Veeam Data Mover reads VM disks and transfers the data to the backup storage (target Veeam Data Mover) using one of the transport modes. If it is an incremental backup, only the changed blocks since the last backup are read using CBT. If CBT is not available, Veeam Metadata is used to detect changed blocks
- during data transfer, the source Veeam Data Mover performs additional processing (filters out zero data blocks, swap file blocks, and excluded file blocks from the backup), compresses the data, and transfers it
- after data reading is complete, Veeam requests the execution of cleanup operations
Deployment Topology and Recommendations
I did not find any simple document describing the optimal topology for specific practical situations/requirements. Even Best Practice articles describe everything very generally. Of course, it depends on a large number of factors. The following is a summary of the information mentioned above.
Sizing parameters of the server (CPU and RAM) are provided by Veeam according to the currently running jobs for each server role.
Note: Related topics are described in a separate article Veeam Backup & Replication - data transfer and backup or restore speed.
All-in-One (Simple Deployment)
For a simple solution, we use one virtual or physical server (Backup Server) where we install all components and services of Veeam Backup & Replication. We can back up any source, and data is copied over the LAN network. Processing takes place either on the Hyper-V server or our Backup Server. Backups are stored on a local Repository, and depending on the type of connected storage, the data flows through a specific channel (e.g., iSCSI network).
Role Separation (Advanced Deployment)
If we want to optimize network and (virtualization) server load, we can deploy some roles on separate servers (there can be more than one server for a role). An important decision, even for all-in-one deployment, is whether to use a physical or virtual server. And which Proxy mode to choose.
We describe only the main roles here, such as the database, which can be placed on a separate Microsoft SQL Server.
Backup Proxy
For VMware, it is important whether the Backup Proxy will be a physical server or a VM, as this determines which transport modes can be used. The most optimal setup should be a physical server in Direct SAN mode along with Backup from Storage Snapshot. All backup data flows from the source to the Backup Proxy, requiring a high bandwidth connection. It is recommended to place it as close to the source as possible. Here, deduplication and compression occur, so less data flows to the Backup Repository (some materials state 50%).
For Hyper-V, the default Backup Proxy runs directly on the Hyper-V server where the backed-up VM is located. Thus, data is read directly from the storage and does not need to be transferred to the Backup Proxy. Deduplication and compression occur here, which loads the Hyper-V server, and the smaller data is sent to the Backup Repository. If server load is an issue, we must choose Off-Host Backup Proxy, which has other disadvantages.
Backup Repository
The last basic role is the Backup Repository. Optimized data is transferred from the Backup Proxy to the Backup Repository, still involving a large amount of data. Therefore, it is important to consider how data flows and which network it loads. The simplest (most optimal) setup is if it is combined with the Proxy (located on the same server), so data does not have to travel over the LAN network. This is not possible for Hyper-V backup. Generally, it is recommended to use a physical server (for VMs, it is recommended not to use VMFS disks but Direct Attached iSCSI).
We must not forget about the actual storage - disk array (Storage) where the Backup Repository stores data. The transport medium (SAN network) and the storage itself will be loaded. Generally, it is recommended to use different resources for backup than for production. For storage, it is clear that it must be a different disk array than where the backed-up data is stored.
There are no comments yet.