VMware vSAN – A closer Look [Part 4 – Fault Domains and Stretched Clusters]

If you would like to read any other chapter’s of this blog series, click the links below:

• Part 1 – Introduction and Licensing
• Part 2 – Architecture and Hardware
• Part 3 – Data Availability
• Part 4 – Fault Domains and Stretched Clusters
• Part 5 – Failure Events
• Part 6 – Compression, Deduplication and QoS
• Part 7 – Monitoring and Reporting

In part 3 I discussed how vSAN protects data using at the software layer within the boundaries of a cluster. If there is a requirement to deploy vSAN over multiple racks or sites then fault domains or stretched clusters can provide that additional layer of redundancy. 

Fault Domains

A fault domain usually refers to a collection of hardware devices that would be impacted by an outage. One common example of this is a server rack. Say you had a twelve host vSAN cluster and all the hardware was installed to the same rack. If the power failed to this rack then you would have a very bad day.

To mitigate this risk we could instead split the twelve hosts across multiple racks and configure each rack as a fault domain. Now vSAN will consider the rack location when placing data across multiple hosts making vSAN rack aware. If a single rack failed in this example there would be sufficient copies of the data available to maintain access to virtual machines.

So far I’ve discussed how we can configure vSAN to make data highly available in a single site. To further protect the data we can use vSphere replication to make an asynchronous copy of the data to a DR site, which may be a second site you own or even a cloud service. Typically, this type of deployment would involve a manual failover process, or at least human intervention to click the big red button failover button if using Site Recovery Manager. Some applications may need a higher level of protection and this is where stretched clusters can help

Stretched Clusters

Now before we go into the detail it’s important to note for this type of deployment you need some serious bandwidth and a third site to act as a witness. Below are the minimum requirements as stated by VMware

If you don’t have 10Gb connectivity between your sites then this option isn’t for you, but it could be in the future so let’s look at how it works.

A vSAN stretched cluster utilises vSphere HA to automatically restart virtual machines at the secondary site if an outage is detected at the primary site. This is done with no data loss but remember there will be a small amount of downtime whilst the virtual machines are restarted. As with any synchronous based replication the writes must be committed at both sites before the write is acknowledged back to the virtual machine. This is the primary reason why the bandwidth and latency requirements are so strict as anything not meeting this will likely result in performance issues.

The witness host must be a standalone host dedicated to the stretched cluster. This serves as a tiebreaker when the network connection is lost between the two sites and prevents a split-brain scenario.

Returning to the pFTT policy that was discussed in part 3 configuring a stretched cluster in vSAN 6.6 enables a new availability policy called secondary level of failures to tolerate. Prior to vSAN 6.6 there was only the option to mirror data between sites when using a stretched cluster. This meant that if a site were to fail there would be no local redundancy on the copy of the data at the second site. If there are at least four hosts at each site there is the option to configure the following policy settings.

pFFT

• defines the number of host and device failures that a virtual machine can tolerate across the two sites.

sFFT

• defines the number of host and object failures that a virtual machine object can tolerate within a single site.

Affinity

• this is only available if the pFFT level is set to 0 and allows a virtual machine to be restricted to one site in the stretched cluster. One reason you may wish to configure this is restrictions with application licensing.

Depending on what level of redundancy you require will have an impact on how much capacity is consumed on the vSAN datastore. The table below gives a few examples of the capacity required to store a 100GB virtual machine.

As you can see vSAN has been designed to cope with large deployments where there may be a physical constraint to overcome such as coping with entire rack or site failures.  The next logical step is to look at how vSAN handles failures such as the loss of a host or disk and this will be covered in part five.

More in the series

• Part 1 – Introduction and Licensing
• Part 2 – Architecture and Hardware
• Part 3 – Data Availability
• Part 4 – Fault Domains and Stretched Clusters
• Part 5 – Failure Events
• Part 6 – Compression, Deduplication and QoS
• Part 7 – Monitoring and Reporting