Smarter VM Placement with Nutanix AHV - Part 2

Overview


Took me a bit longer to get back around to writing Part 2, life get's in the way sometimes!

Following up on Part 1 where I covered the benefits (and some pitfalls) of Affinity and Anti-Affinity polices with AHV, this post will cover the configuration and validation of these policies. Affinity and Anti-Affinity policies are crucial for ensuring application availability, performance, and compliance by controlling the placement of virtual machines on specific hosts or preventing them from residing on the same host.

Lab Environment

My lab environment consists of a 4-node cluster, with just a handful of workloads. I have two Domain Controllers (MainDC01 and MainDC02), as well as 2 SQL Server nodes in an AAG (MAINSQL01 and MAINSQL02).


Creating VM-Host Affinity and VM-VM Anti-Affinity Rules

Let's dive into creating both VM-Host Affinity Policies and VM-VM Anti-Affinity Policies. In a larger environment, you might have more policies based on the workloads, but we'll keep this post simple to give a good foundation and focus on the following scenarios:

  1. Affinity: Ensuring a SQL Server only runs on two of the 4 hosts within the cluster.
  2. Anti-Affinity: Ensuring that Domain Controllers remain on separate hosts.

VM-Host Affinity Policies

Quick Recap: VM-Host Affinity Policies control the relationship between specific VMs and physical hosts. Configuring these polices can be done through one of three methods:

  1. ACLI
  2. Prism Element
  3. Prism Central

A few things to consier when configuring VM-Host affinity settings:

  1. VMs with VM-Host affinity policies can only be migrated to the hosts specified in the affinity policy.
  2. A virtual machine cannot be powered on or migrated to a host that does not conform to requirements of the affinity policy as this policy is enforced mandatorily.
  3. VMs configured with host affinity settings retain these settings if the VM is migrated to a new cluster.
  4. The VM-host anti-affinity policy is not supported.

The last one is an important one to remember, you can't omit a host from where a VM may run.

aCLI (Command Line)

Using ACLI to set the VM-Host affinity allows flexibility in doing en-masse, however there is limited validation via this method. In my case, I am binding my 2 SQL servers to hosts 3 and 4, as shown below.

1# Enable VM-Host affinity
2acli vm.affinity_set MAINSQL01,MAINSQL02 host_list=172.20.120.23,172.20.120.24
3
4# Unset VM-Host affinity
5acli vm.affinity_unset <vm_name> (or can use \* to unset all VMs)

To validate the affinity is set for the VM, you can do this in Prism Element. Here we can see that nodes 3 and 4 were applied.

Prism Element (GUI)

So having gone through the ACLI method, since we have to validate through Prism ElementWithin Prism Element, an administrator can define the VM-Host affinity policies during the VM create or update operation. This is done a per-vm basis.

  1. Select the VM to enable VM-Host affinity, click Update.
  2. Scroll down to the VM Host Affinity section.
  3. Click the Set Affinity button, and select the desired hosts. In my scenario, I have selected hosts cl2-node3 and cl2-node4 as my desired hosts.

Prism Central (GUI)

VM-Host Affinity policies, when created within Prism Central use a category-based assignment, rather than a per-VM assignment. The image below provides a quick view as to the workflow for creating these policies via PC.

  1. Create a category (or categories) to associate to a VM. In my case, outside of the usual categories that I use, I will create a Category:Value pair of HostWorkloads:SQL, to which I will assign those hosts I want to host SQL workloads.
  2. After creating my Hosts category, I apply the category value to the hosts by using the Managing Categories function on the Hosts page, applying the specific category to each host.
  3. Before fully applying the policy, I ensure that my 2 SQL instances have the Microsoft_SQL value applied, which in my case is using the AppType system category.
  4. Finally, we create a new VM-Host Affinity policy, and indicate that I want any VMs with the Category:Value pair of AppType:Microsoft_SQL to run only on hosts with the Category:Value pair of HostWorkloads:SQL.

Now that the VM and Hosts are associated to this VM-Host Affinity policy, we can validate compliance with the policy, and in my case, we show compliant.

We can also validate the specifics of our worklaods to the hosts they are running on, which in my case was only cl2-node3 and cl2-node4.


VM-VM Anti-Affinity Policies

Quick Recap: VM-VM Anti-Affinity Policies define the relationship between two or more virtual machines by either keeping them apart. These policies are commonly used to optimize availability, performance isolation, or clustered workload behavior.

Nutanix AHV supports VM-VM Anti-Affinity policies through both the legacy CLI-based approach and the GUI-based method in Prism Central. VM-VM Anti-Affinity polices are not possible through Prism Element.

Prior to the AOS 7.0 and pc.2024.3 releases, VM-VM anti-affinity policies could only be managed via the Acropolis CLI (aCLI). With more recent releases, administrators can create category based VM-VM anti-affinity policies from Prism Central, rather than the CLI method. The CLI method is still useful in environments where Prism Central isn't deployed or where automation scripts are preferred.

aCLI (Command Line)

1# Create a VM Group
2acli vm_group.create Domain_Controllers
3
4# Add VMs to the group
5acli vm_group.add_vms Domain_Controllers vm_list=MAINDC01,MAINDC02
6
7# Apply an Anti-Affinity rule to the group
8acli vm_group.set_rule Domain_Controllers anti_affinity=True

Now it's great that we can programatically set Anti-Affinity policies via ACLI, however notice the output that we receive when doing so, telling us this will be deprecated. Good to know!

1"Deprecation Warning: VM group operations through ACLI will be removed in future releases. Use Prism Central based anti-affinity policies instead."
2Domain_Controllers: pending
3Domain_Controllers: complete

Now that these are setup, we can validate the results with the following commands. You can find additional information in the commands reference guide:

1# Lists all VM groups
2acli vm_group.list
3
4# Retrieves information about a VM group
5acli vm_group.get Domain_Controllers
6
7# Lists VMs configured to the VmGroup
8acli vm_group.list_vms Domain_Controllers

VM-VM Anti-Affinity Prism Central (GUI)

As we saw from the warning when creating Anti-Affinity polices using ACLI, these are being deprecated, and recommended using Prism Central. With Prism Central, Nutanix has provides a visual interface to create and manage VM-VM affinity and anti-affinity rules.

As the image above highlights, it's a 3-part process to creating Anti-Affinity Policies.

First, we want to create categrories to define the VMs we want to apply to. In my case, I'm using the built-in SharedService category of AD, and have applied that to both of my domain controllers.

Second, we create the policy and define the category or categories to be compliant with this policy.

Once the policy is applied, the current state of compliance will be validated, the active compliance can be quickly seen.


Final Thoughts

Affinity and anti-affinity policies are powerful tools in the Nutanix AHV arsenal that enable administrators to optimize workload placement for availability, performance, and compliance requirements. As we've seen throughout this series, the evolution from CLI-only management to the category-based approach in Prism Central represents a significant improvement in usability and scalability.

The key takeaways from implementing these policies include understanding when to use VM-Host affinity for resource optimization (like keeping SQL workloads on specific high-performance nodes) versus VM-VM anti-affinity for availability (ensuring critical services like domain controllers remain separated). While the CLI methods provide automation capabilities, the Prism Central GUI offers better visualization, compliance monitoring, and easier management at scale.

As Nutanix continues to deprecate the legacy ACLI approaches in favor of the category-based policies, now is an excellent time to transition your infrastructure management to leverage these more intuitive and maintainable methods. Remember that these policies should be part of a broader infrastructure design strategy, not just tactical fixes - proper planning during the initial deployment will save significant effort in ongoing operations.

As always, thanks for taking the time to read this, and I'd love to hear your thoughts!

Smart policy. Smarter placement. Stronger outcomes.