Automating with Guest Scripts in Nutanix DR

Overview

Advanced automation through in-guest scripts takes disaster recovery from good to great. While Recovery Plans handle infrastructure orchestration, there are application-specific configurations that require execution inside the guest operating system; DNS reconfiguration, IP changes, and service initialization as a few examples. This post explores how to implement in-guest scripts with Nutanix Guest Tools to eliminate manual intervention during failover operations.

📖 Disaster Recovery in 2025 Series - Part 8 This post is part of my comprehensive disaster recovery series. New to the series? Start with the Complete Guide Overview to see what's coming, or catch up with Part 1 - Why DR Matters, Part 2 - Modern Disaster Recovery, Part 3 - Nutanix DR Overview, Part 4 - Protection Policies, Part 5 - Recovery Plans, Part 6 - Testing DR, and Part 7 - Planned vs. Unplanned Failover.

In the previous posts of this series, we've explored how Protection Policies ensure your data is continuously replicated, how Recovery Plans orchestrate failover operations, and how testing and failover execution work in practice.

But here's where disaster recovery moves from good to great: advanced automation through in-guest scripts. While Recovery Plans handle VM power-on sequencing and network mapping beautifully, there are application-specific configurations that require execution inside the guest operating system itself.

In Understanding DNS Changes in Nutanix DR Recovery Plans back in October 2024, I introduced the concept of using custom scripts to automate DNS server modifications during failover. Today, we're going to expand on that foundation with improved scripts, better error handling, and additional automation scenarios that eliminate manual intervention during recovery operations.

In-Guest Scripts and Recovery Automation

Why In-Guest Scripts Matter

Nutanix Recovery Plans provide excellent orchestration at the infrastructure layer: they handle VM power-on sequences, network mappings, and recovery point selection automatically. However, there are configuration changes that must happen inside the guest operating system that the hypervisor simply can't manage.

The key principle is simple: If a post-failover task can be scripted, in-guest scripts can automate it.

This capability dramatically accelerates recovery by eliminating manual intervention for any configuration that differs between sites. Common scenarios include:

  • DNS server reconfiguration: Pointing VMs to site-appropriate DNS infrastructure (we'll focus on this example throughout this post)
  • Application service initialization: Restarting services with recovery site configurations
  • Monitoring infrastructure: Reconfiguring agents for the recovery site
  • Site-specific certificates: Updating SSL certificates or certificate authorities

Without automation, these tasks require manual intervention during failover operations, adding precious minutes (or hours) to your Recovery Time Objective and introducing the possibility of human error during high-stress disaster scenarios.

While we'll use DNS reconfiguration as our primary example in this post with real production scripts you can adapt, remember that the same in-guest script capability applies to any scriptable post-failover task. The automation framework is identical; only the script content changes to match your specific requirements.

Prerequisites: Nutanix Guest Tools and Script Placement

Before Recovery Plans can execute in-guest scripts, you need to prepare the environment:

Nutanix Guest Tools (NGT) Installation

In-guest script execution requires Nutanix Guest Tools to be installed on every VM that needs script automation. NGT provides the communication channel between Prism Element and the guest operating system.

NGT is available for both Windows and Linux operating systems and should be installed during VM provisioning as a standard practice for any workload that will be part of a DR strategy.

Installation Methods:

  • Prism Element or Prism Central: Install NGT on individual VMs through the VM management interface
  • Bulk Installation: Use automated deployment methods for installing NGT across multiple VMs simultaneously

Choose the method that best fits your environment size and operational workflow. For DR-protected workloads, ensure NGT is installed on all VMs that require in-guest script automation during failover operations.

Script File Paths and Naming Conventions

Scripts must be placed in specific locations with exact naming conventions for Nutanix DR to recognize and execute them. The paths differ between Windows and Linux, and between production and test failover operations:

Windows VMs
  • Production failover script path: C:\Program Files\Nutanix\scripts\production\vm_recovery.bat

  • Test failover script path: C:\Program Files\Nutanix\scripts\test\vm_recovery.bat

Important: The scripts, production, and test folders typically don't exist by default after NGT installation. You must create these directories manually. Additionally, some Windows environments require the .bat extension even when saving as a batch file; if your script doesn't execute, try renaming to include .bat explicitly.

Linux VMs
  • Production failover script path: /usr/local/sbin/production_vm_recovery

  • Test failover script path: /usr/local/sbin/test_vm_recovery

Linux scripts should be executable shell scripts (typically bash) with appropriate execute permissions (chmod +x).

Enabling Script Execution in Recovery Plans

Even with NGT installed and scripts in place, Recovery Plans won't execute them unless explicitly enabled:

  1. In your Recovery Plan, select the VM that should execute scripts
  2. Click Manage Scripts for that VM
  3. Enable script execution

This per-VM or Category configuration gives you granular control over which systems execute automation during recovery operations.

DNS Automation: Building on the Foundation

Back in October 2024, I published Understanding DNS Changes in Nutanix DR Recovery Plans, where I introduced the challenge of automating DNS server reconfiguration during failover. The scripts I shared then worked well, but based on real-world usage and feedback, I've refined the approach with better error handling, more flexible configuration, and improved logging.

Currently, Nutanix Recovery Plans do not provide native DNS modification capabilities within the Recovery Plan configuration itself. This differs from solutions like Zerto, which allows both IP and DNS modifications to be defined on a per-VPG (Virtual Protection Group) basis. While Recovery Plans excel at network mapping and IP address changes, DNS reconfiguration must be handled through in-guest scripts.

This is where in-guest script automation becomes critical for achieving truly automated failover operations. Let me walk through the updated implementation.

The Challenge: Site-Specific DNS Infrastructure

Here's the scenario that's common in most DR configurations:

  • Primary Site: VMs use DNS servers 172.20.19.253 and 172.20.19.254 on the 10.10.16.0/24 network
  • Recovery Site: VMs should use DNS servers 172.20.20.253 and 172.20.20.254 on the 10.10.216.0/24 network

When Recovery Plans fail VMs over to the recovery site, network mapping changes the VLAN association, but the DNS server configuration inside the guest OS remains unchanged. Without automation, VMs continue trying to reach primary site DNS servers, resulting in slow or failed name resolution and broken application functionality.

The solution is to detect the VM's current IP address after failover and automatically reconfigure DNS servers to match the recovery site infrastructure.

Implementation Architecture: Two-Script Approach

The DNS automation solution uses a two-script architecture:

  1. vm_recovery.bat: The required entry point that Nutanix DR calls during failover
  2. Set-DnsConfiguration.ps1: The PowerShell script that performs the actual DNS reconfiguration

This separation provides flexibility; you can add multiple PowerShell scripts for different post-failover tasks, all orchestrated through the single vm_recovery.bat file.

The Entry Point: vm_recovery.bat

The batch file serves as the orchestration layer, creating the logging infrastructure and calling PowerShell scripts with proper error handling. Here's the core execution section:

 1REM Script 1: DNS Configuration
 2echo ------------------------------------------------------------- >> "%batchLog%"
 3echo Script 1: Set-DnsConfiguration.ps1 >> "%batchLog%"
 4echo ------------------------------------------------------------- >> "%batchLog%"
 5powershell -ExecutionPolicy RemoteSigned -File "%scriptDir%Set-DnsConfiguration.ps1" -LogDir "%logDir%"
 6set "scriptExitCode=%errorlevel%"
 7if %scriptExitCode% equ 0 (
 8    echo   Result: SUCCESS ^(Exit Code: %scriptExitCode%^) >> "%batchLog%"
 9) else (
10    echo   Result: FAILURE ^(Exit Code: %scriptExitCode%^) >> "%batchLog%"
11    set "overallExitCode=%scriptExitCode%"
12)

Key Features:

  • Execution Policy: Uses -ExecutionPolicy RemoteSigned to allow locally created scripts to run while maintaining security by requiring downloaded scripts to be signed
  • Exit Code Tracking: Captures and logs the exit code from each PowerShell script
  • Extensibility: The script includes commented examples showing how to add additional scripts for other post-failover tasks
  • Timestamped Logging: Creates batch-level logs with format vm_recovery_batch_YYYYMMDD-HHMM.log

The batch file also sets up the logging directory structure:

 1set "scriptDir=%~dp0"
 2set "logDir=C:\Program Files\Nutanix\scripts"
 3
 4REM Create the log directory if it doesn't exist
 5if not exist "%logDir%" (
 6    mkdir "%logDir%"
 7    if errorlevel 1 (
 8        echo ERROR: Failed to create log directory: %logDir%
 9        exit /b 1
10    )
11)

This ensures all logs are stored in a consistent location regardless of where the scripts are installed.

The DNS Logic: Set-DnsConfiguration.ps1

The PowerShell script handles the actual DNS reconfiguration logic. It's designed to be site-aware and idempotent, only making changes when necessary.

Site Configuration:

First, define your site-specific subnet and DNS mappings:

 1# Define subnet and DNS information for Site A and Site B
 2$sites = @{
 3    "SiteA" = @{
 4        "Subnets" = @("172.20.21.0/24")
 5        "DNSServers" = @("172.20.11.254")
 6    }
 7    "SiteB" = @{
 8        "Subnets" = @("172.20.121.0/24")
 9        "DNSServers" = @("172.20.14.254")
10    }
11}

You can define multiple subnets per site and multiple DNS servers per site by adding them to the arrays:

1"Subnets" = @("172.20.21.0/24", "172.20.22.0/24")
2"DNSServers" = @("172.20.11.254", "172.20.11.253")

Network Adapter Selection:

The script automatically identifies the primary active network adapter based on the lowest interface metric:

1# Get the primary active network adapter (sorted by InterfaceMetric for priority)
2$activeAdapter = Get-NetAdapter |
3    Where-Object {$_.Status -eq "Up"} |
4    Sort-Object -Property InterfaceMetric |
5    Select-Object -First 1

This ensures the script modifies the correct network interface, even on multi-homed systems.

Subnet Matching and DNS Updates:

The script includes a proper CIDR subnet matching function (Test-Subnet) that correctly calculates network boundaries. Once it identifies which site the VM is in based on its current IP address, it checks if DNS servers need updating:

 1# Compare DNS server settings (order-independent)
 2$desiredDNS = $siteData["DNSServers"]
 3
 4# Check if DNS arrays are equivalent (same servers, any order)
 5$dnsMatch = $true
 6if ($null -eq $currentDNS -or $currentDNS.Count -ne $desiredDNS.Count) {
 7    $dnsMatch = $false
 8}
 9else {
10    foreach ($dns in $desiredDNS) {
11        if ($dns -notin $currentDNS) {
12            $dnsMatch = $false
13            break
14        }
15    }
16}
17
18if ($dnsMatch) {
19    Write-Log "DNS servers are already correctly configured for $site. No changes needed."
20}
21else {
22    Write-Log "DNS mismatch detected."
23    Write-Log "  Expected: $($desiredDNS -join ', ')"
24    Write-Log "  Current:  $($currentDNS -join ', ')"
25    Write-Log "Updating DNS servers for $site..."
26
27    # Set new DNS servers
28    Set-DnsClientServerAddress -InterfaceIndex $interfaceIndex -ServerAddresses $desiredDNS
29    Write-Log "DNS servers updated successfully to: $($desiredDNS -join ', ')"
30}

Important behavior: The script is idempotent, only making changes when DNS servers don't match the expected configuration. This prevents unnecessary modifications during test failovers when DNS may already be correct.

Verification and Logging:

After making DNS changes, the script verifies the update was successful:

 1# Verify the DNS settings after update
 2Start-Sleep -Milliseconds 500  # Brief pause for changes to take effect
 3$verifyDNS = (Get-DnsClientServerAddress -InterfaceIndex $interfaceIndex -AddressFamily IPv4).ServerAddresses
 4
 5# Validation logic here...
 6
 7if ($verifyMatch) {
 8    Write-Log "DNS update verification: PASSED"
 9}
10else {
11    Write-Log "WARNING: DNS update verification FAILED"
12    Write-Log "  Expected: $($desiredDNS -join ', ')"
13    Write-Log "  Actual:   $($verifyDNS -join ', ')"
14}

Each execution creates a timestamped log file (Set-DnsConfiguration_YYYYMMDD-HHMMSS.log) with detailed information about the DNS check and any changes made, making troubleshooting straightforward.

Validating the Scripts Before Deployment

Before trusting these scripts to execute automatically during a real failover event, you should thoroughly validate them in your environment. Here's a systematic approach to testing:

1. Manual Execution Testing

Test the scripts manually before integrating them with Recovery Plans:

1# Navigate to the script directory
2cd "C:\Program Files\Nutanix\scripts\production"
3
4# Execute the batch file manually
5.\vm_recovery.bat

This allows you to observe the execution behavior and review the generated log files without waiting for a failover event.

2. Review Log Files

After manual execution, examine both log files created:

  • Batch execution log: C:\Program Files\Nutanix\scripts\vm_recovery_batch_YYYYMMDD-HHMM.log

    • Verify the batch file successfully called the PowerShell script
    • Check the exit codes (0 = success, non-zero = failure)
    • Confirm the overall execution status
  • PowerShell execution log: C:\Program Files\Nutanix\scripts\Set-DnsConfiguration_YYYYMMDD-HHMMSS.log

    • Verify the script correctly identified your network adapter
    • Confirm the current IP address was detected properly
    • Check which site (if any) was matched based on subnet
    • Review whether DNS changes were needed and if they were applied successfully
    • Look for the "DNS update verification: PASSED" message if changes were made

Example Batch Execution Log:

 1=============================================================
 2vm_recovery.bat - Execution Started
 3=============================================================
 4Date/Time: Sun 11/23/2025 16:36:00.63
 5Script Directory: C:\Program Files\Nutanix\scripts\production\
 6Log Directory: C:\Program Files\Nutanix\scripts
 7
 8-------------------------------------------------------------
 9Script 1: Set-DnsConfiguration.ps1
10-------------------------------------------------------------
11  Result: SUCCESS (Exit Code: 0)
12
13=============================================================
14vm_recovery.bat - Execution Completed
15Overall Status: SUCCESS
16=============================================================

This batch log shows successful execution with exit code 0, confirming the PowerShell script ran without errors.

Example PowerShell Execution Log:

 12025-11-23 16:36:01 - -------------------------------------------------------------
 22025-11-23 16:36:01 - DNS and IP Configuration Checker - Version 2.0
 32025-11-23 16:36:01 - Script initiated to verify and update DNS settings if needed
 42025-11-23 16:36:01 - Log File: C:\Program Files\Nutanix\scripts\Set-DnsConfiguration_20251123-163601.log
 52025-11-23 16:36:01 - -------------------------------------------------------------
 62025-11-23 16:36:02 - Primary Network Adapter: Ethernet (Index: 4, Metric: )
 72025-11-23 16:36:03 - Current IP Address: 172.20.21.11
 82025-11-23 16:36:03 - Current DNS Servers: 172.20.11.254
 92025-11-23 16:36:03 - Matched 172.20.21.11 to subnet 172.20.21.0/24 in SiteA
102025-11-23 16:36:03 - DNS servers are already correctly configured for SiteA. No changes needed.
112025-11-23 16:36:03 - -------------------------------------------------------------
122025-11-23 16:36:03 - SUMMARY
132025-11-23 16:36:03 - -------------------------------------------------------------
142025-11-23 16:36:03 - Adapter:          Ethernet
152025-11-23 16:36:03 - IP Address:       172.20.21.11
162025-11-23 16:36:03 - Previous DNS:     172.20.11.254
172025-11-23 16:36:03 - Current DNS:      172.20.11.254
182025-11-23 16:36:03 - Changes Made:     No
192025-11-23 16:36:03 - Log File:         C:\Program Files\Nutanix\scripts\Set-DnsConfiguration_20251123-163601.log
202025-11-23 16:36:03 - -------------------------------------------------------------
212025-11-23 16:36:03 - Script completed successfully.

This PowerShell log demonstrates the script's idempotent behavior. The VM is at SiteA (172.20.21.11), the DNS configuration already matches SiteA's expected DNS servers (172.20.11.254), so no changes were made. The detailed summary at the end provides a clear audit trail of what was checked and the outcome.

3. Verify DNS Configuration

After the script runs, manually verify the DNS settings match expectations:

1# Check current DNS servers
2Get-DnsClientServerAddress -InterfaceAlias "Ethernet" -AddressFamily IPv4
3
4# Or use ipconfig
5ipconfig /all

Compare the displayed DNS servers against what you defined in the $sites configuration within Set-DnsConfiguration.ps1.

4. Test Idempotency

Run the script multiple times in succession:

1.\vm_recovery.bat
2.\vm_recovery.bat
3.\vm_recovery.bat

Review the logs from each execution. After the first run, subsequent executions should log "DNS servers are already correctly configured" and make no changes. This confirms the script is idempotent and won't cause unnecessary modifications during test failovers.

5. Test Failover Scenario

The most thorough validation is to test during an actual DR test failover:

  1. Enable script execution for a test VM in your Recovery Plan
  2. Execute a test failover to the recovery site
  3. Once the VM powers on, RDP/SSH into the VM
  4. Check the log files in C:\Program Files\Nutanix\scripts\
  5. Verify DNS servers were updated to match the recovery site configuration
  6. Test DNS resolution: nslookup your-domain.com
6. Validate Script Permissions

Ensure the scripts have appropriate permissions and can be executed by the SYSTEM account (which Nutanix Guest Tools uses):

  • The vm_recovery.bat file must be executable
  • The Set-DnsConfiguration.ps1 file must be readable by SYSTEM
  • The log directory must be writable by SYSTEM

You can test this by running the script as SYSTEM using PsExec:

1# Download PsExec from Sysinternals if not already available
2# Run the script as SYSTEM to simulate NGT execution
3PsExec.exe -s -i "C:\Program Files\Nutanix\scripts\production\vm_recovery.bat"
7. Troubleshooting Common Issues

If the scripts don't execute as expected during failover, check these common issues:

  • Scripts not executing at all: Verify script execution is enabled in the Recovery Plan for the VM
  • PowerShell execution policy errors: The batch file uses -ExecutionPolicy RemoteSigned, which requires locally created scripts to be unsigned but blocks unsigned downloaded scripts. If you're getting execution policy errors, verify no GPO is overriding this setting or consider the security implications of using -ExecutionPolicy Bypass instead
  • Permission denied errors: Ensure SYSTEM account has read/write access to the script directory and log directory
  • DNS not changing: Verify the subnet configuration in $sites matches your actual network addressing
  • Wrong adapter modified: Check the interface metric priority; the script targets the adapter with the lowest metric

Best Practice: Document your validation results and keep a record of successful test failovers. This documentation becomes valuable during actual disaster scenarios when you need confidence that automation will work as expected.

Wrapping Up: From Manual to Automated Recovery

In-guest scripts transform Nutanix disaster recovery from good to great by eliminating manual post-failover tasks. While Recovery Plans excel at orchestrating VM power-on sequences and network mapping, there are critical guest OS configurations that require automation inside the VM itself.

The key principle we explored is simple but powerful: If a post-failover task can be scripted, in-guest scripts can automate it. This capability dramatically reduces Recovery Time Objectives and eliminates human error during high-stress disaster scenarios.

What We Covered

Throughout this post, we've explored:

  • Why in-guest scripts matter: Understanding the gap between infrastructure orchestration and guest OS configuration
  • Prerequisites: NGT installation requirements and script file placement conventions for Windows and Linux VMs
  • DNS automation as a working example: A two-script architecture (vm_recovery.bat and Set-DnsConfiguration.ps1) that detects site location based on IP address and automatically reconfigures DNS servers
  • Script implementation details: How the batch orchestration layer calls PowerShell scripts with proper error handling, logging, and exit code tracking
  • Validation methodology: A systematic seven-step approach to testing scripts before trusting them in production failover scenarios
  • Real-world examples: Actual log file output showing what successful script execution looks like

Key Takeaways

  1. Start with one automation: DNS reconfiguration is a common pain point that's straightforward to automate. Use it as your starting point.

  2. Leverage the framework: The vm_recovery.bat orchestration framework can call multiple PowerShell scripts sequentially, each handling a different post-failover task.

  3. Build idempotent scripts: Scripts should check current state before making changes, preventing unnecessary modifications during test failovers.

  4. Validate thoroughly: Test scripts manually, validate with actual DR test failovers, and document your results before trusting them in production scenarios.

  5. Think beyond DNS: While we focused on DNS automation, the same framework applies to service initialization, monitoring agent reconfiguration, certificate updates, and any other scriptable post-failover task.

Download the Scripts

Ready to implement DNS automation in your environment? Download the current version of the Nutanix DR Guest script package:

Download: Nutanix DR In-Guest Scripts

The package includes:

  • vm_recovery.bat - The orchestration batch file with logging and error handling
  • Set-DnsConfiguration.ps1 - The DNS reconfiguration PowerShell script with subnet detection
  • Example configuration for two-site DR scenarios
  • Comprehensive inline documentation

Remember to customize the subnet and DNS server configurations in Set-DnsConfiguration.ps1 to match your specific environment before deployment.

Next Steps

As you implement in-guest script automation:

  1. Install NGT on all VMs that will be part of your DR strategy
  2. Customize the scripts with your site-specific subnet and DNS configurations
  3. Test manually before enabling in Recovery Plans
  4. Start small with a single test VM in your Recovery Plan
  5. Validate during DR tests and document the results
  6. Expand gradually to additional VMs and automation scenarios

In-guest scripts aren't just about DNS. Consider what other post-failover tasks your team currently performs manually. Service restarts? Configuration file updates? Monitoring agent reconfigurations? If it can be scripted, it can be automated through this same framework.

The goal is simple: when disaster strikes, you want recovery operations to be as automated and reliable as possible. Every manual step eliminated is one less opportunity for error during the most critical moments.

What post-failover tasks are you looking to automate? Working on implementing in-guest scripts and have questions? Feel free to reach out to me at mike@mikedent.io.

Thanks for reading!


This post is part of my ongoing disaster recovery series. For more in-depth coverage of Nutanix DR capabilities, check out the complete series overview.