Virtualizing Microsoft Active Directory Domain Services (AD DS)-Windows 2012 on vSphere Best Practices

Hi All ..

Active Directory Domain Services (AD DS) is the core of our IT Infrastructure nowadays. It's the authentication and authorization center of any IT Infrastructure. Being here since 1990's, AD DS has been through a great development till reached this version on Windows 2012 with many new features. Luckily, some of these are only for helping virtualizing Domain Controllers (DC's) with min. effort and to leverage all of Virtualization advantages and features. These features are:

1-) VM Generation ID (Virtualization Safeguard): An ID that's generated and add to any Windows 2012 VM when being deployed on a supported hypervisor (vSphere 5.0 U2) .vmx file. It's used to trace VM life-cycle. When it's power on for first time, a new ID is generated using a special generator driver on Windows 2012 by default. It changes only when the VM is either imported, deployed from a template or a copy, hot/cold cloned, hot/cold snapshot-ed, restored from a backup or replicated to another VM. This ID is saved in AD Computer Object attributes of the VM as "msDS-GenerationId Attribute" and that helps the AD to keep track of the VM version and if the VM is restored or reverted from different version. This allowed us to use the next features: AD DC Snapshot and AD DC Cloning.

2-) AD DC Snapshot: Before Windows 2012, any try to use a snapshot with DC would lead to a corruption in DC database. That's because of a known issue called "USN Rollback" which leads to creating different objects on different DC's with the same SID. Long story short, each DC has to maintain a version number of its DB called "Update Sequence Number - USN". When creating an object on a DC for example, the USN will be incremented by 1 and hence, the DB -on this DC- will have a higher version number and that's why any other DC will accept a replication from that DC. When reverted from snapshot, USN will be reverted to old state. When the admin create new objects, the USN on the creator DC will increment till reach -or even pass- the same USN on any other DC, but with different objects and will not either initiate or accept any replication because it maintains the same USN -till it passes the USN of other DC's-, but with different objects. With using VM Generation ID, AD DC's now have a VM version which is used with USN to validate that each DC maintain its DB version and VM version. If a DC is reverted from snapshot, its VM Generation ID will change and hence, its DB will have a new combination of USN and VM Generation ID. This alerts other DC's to initiate a two-way replication to acquire any objects created on the reverted DC and push any missing objects to it.

3-) AD DC Cloning: In Windows 2012, Cloning of a DC is a new feature which facilitates the scaling of AD Infrastructure when needed. It also uses the VM Generation ID to maintain the version of the DC's DB and hence, prevents the corruption due to USN Rollback Issue. Using Windows PowerShell, a XML file will be created to save the configuration needs to be deployed when the DC machine is cloned. Then, after cloning and configuration, a new DC is created with a new VM Generation ID in the same point of time of the source DC. This feeds the need for replication of AD objects and extremely useful in large environments or in case of DR process. For the cloning process step-by-step guide, check thispaper.

When virtualizing AD DC using vSphere, it's recommended to follow Microsoft Best Practices in sizing as well as leverage any feature provided by vSphere to provide the needed level of performance and availability. AD DC isn't a performance-intensive application and its availability is easy to maintain at the required level. The following are a collection of MS and VMware best practices when virtualizing an AD DC based on Windows 2012. I divided these best practices into six categories that align with the known Design Qualifiers: Availability, Manageability, Performance, Recoverability and Security in addition to another qualifier: Scalability.

Availability:

1-) Try to separate FSMO roles between many AD DC VMs and separate these VMs on different host using VMs Anti-Affinity Roles.

2-) Try to separate all DC VMs on separate back-end Storage Arrays. If not available, try to host one DC VM on a local datastore as a protection in case of back-end shared storage failure. Keep in mind that, DC VM on local datastore won’t use features like: HA or vMotion.

3-) Try to separate your DC VMs on many clusters on many physical racks or blade chassis using Soft (Should) VM-Host Anti-affinity rule. At least, dedicate a management cluster separated from production cluster.

4-) Make sure to set HA Restart Priority for all DC VMs in your HA Cluster(s) to High in order to be restarted first before any other VMs in case of host failure.

5-) Try to use VM Monitor to monitor activity of the AD DC VMs and restart them in case of OS failure.

Manageability:

1-) Time Sync:

Time Synchronization is one of the most important things in AD DS environments. As stated by VMware Best Practices to Virtualize AD DS on Windows 2012here, it’s recommended to follow a hierarchical time sync as follows:

- Sync the PDC in Forest Root Domain to an external Startum 1 NTP servers.

- Sync all other PDCs in other child domains in the forest to the Root PDC or any other DC in the Root Domain.

- Sync all ESXi Hosts in the VI to the same Startum 1 NTP Server.

- Sync all workstations in every domain to the nearest DC in their domains respectively.

To configure the PDC to time-sync with an external NTP server using GPO, check thislink.

Also, it’s recommended to disable time-sync between VMs and Hosts using VMware Tools totally (Even after uncheck the box from VM settings page, VM can sync with the Host using VMware Tools in case of startup, resume, snapshotting, etc.) according to the followingKB.

This will make the PDC VM only sync its time with the configured time source using GPO.

2-) Use Best Practices Analyzer (BPA):

It’s recommended to use BPA for AD DCs to make sure that your configuration is coherent with Microsoft recommended configuration. In some cases and for valid reasons, you can drift from Microsoft recommendations. For more information, refer to thislink.

3-) Use AD DS Replication Tool:

This tool, offered by Microsoft for free, can help detect any issue in replication between all DCs in your environment and show them and the related KB articles to solve these issues. It’s the next generation from REPADMIN CLI tool. Download it from thislink.

4-) Snapshots:

As mentioned above, using AD DC on Windows 2012, you can use snapshots without worrying about reverting to old snapshot and the related USN Rollback issue. AD DC on Windows 2012 leverages the new VM-Generation ID feature that makes the AD DC Virtualization aware and hence, any hot/cold snapshot can be used to revert to it safely. Check VMware Best Practices to Virtualize AD DS on Windows 2012herefor more information about VM-Generation ID and related Virtualization Safeguards.

Performance:

1-) vCPU Sizing:

This assumes that the primary work of the directory is user authentication. For any additional workload, like Exchange Server, additional vCPUs may be required. Capacity Monitoring is helpful to determine the correct vCPUs required.

2-) Memory Sizing:

Memory of AD DC can help in boosting the performance by caching AD DB in the RAM, like any other DB application. Ideal case is to cache all AD DB in the RAM for max. performance. This is preferred in environments that have AD integrated with other solutions, like Exchange servers. The following guide line is a start:

For the correct sizing of RAM, start with min. required and use Windows Performance Monitor to monitor “Database/DB Cache Hit%” for lsass service after extended period after deploying this DC. Add RAM if required using vSphere Hot Add feature (Keep in mind that you have to enable it before starting up the DC VM). When the RAM is sized correctly enough to cache proper portion of DB, this ratio should be near 100%.

Keep in mind that, this is only for AD Domain Services, i.e. additional RAM is required for the Guest OS.

3-) Storage Sizing:

The following equations are general for sizing the required storage size for a DC:

“Storage required= OS Storage+ AD DB Storage+ AD DB Logs Storage+ SYSVOL Folder Storage+ Global Catalogue Storage+ Any data stored in Application Partition+ Any 3rd Part Storage”

“AD DB Storage= 0.4GB for any 1,000 users≈ 0.4MB*Total No. of Users”

“AD DB Logs Storage= 25% of AD DB Storage”

“SYSVOL Folder≈ 500MB+”          May increase in case of high no. of GPOs.

“Global Catalogue Storage= 50% of AD DB for any additional Domain”

“Any data stored in Application Partition": is to be estimated.

“Any 3rd Part Storage": includes any installed OS patches, paging file, backup agent or anti-virus agent.

The following table shows the Read/Write behavior of each of AD DC components:

Keep in mind that for large environments with many integrated solutions with AD, separation of OS, AD DB and AD DB Logs on many disks is recommended for IO separation on many disks and vSCSI adapters. In such environments, caching most of AD DB on RAM will give a performance boost.

4-) Network Sizing:

AD DC VM should have a VMXNET3 vNIC which gives the max. network performance with least CPU load and this should be sufficient on 1GB physical network. The port group to which AD DC VM is connected should have a teamed physical NICs for redundancy.

Recoverability:

1-) Try to use a backup software that is VSS-aware to safely backup your AD DB. AD DC on Windows 2012 can be backed up with a backup software that uses VSS to backup and restore entire DC VM, because AD DC on Windows 2012 leverages the new VM-Generation ID feature that makes the AD DC Virtualization aware and hence, any restore process of entire DC VM can be done safely. Check VMware Best Practices to Virtualize AD DS on Windows 2012herefor more information about VM-Generation ID and related Virtualization Safeguards.

2-) Make sure to backup any DC System State. System State contains AD DB, AD DB Logs, SYSVOL Folder and any other OS critical component like registry files.

3-) For DR, you can use native AD DCs replication to replicate the AD DB between the main site and the DR site. This approach requires min. management overhead and good DR capability. This approach only lacks the ability to protect the five FSMO role holders.

4-) Another approach for DR is to leverage VMware SRM with VM-Generation ID capability on Windows 2012. This approach helps to continuously replicate AD DC VMs using SRM Replication or Array-based Replication and failover in case of disaster. This allows to protect FSMO roles holders as well as provide AD infrastructure to failed-over VMs in the DR site.

Security:

1-) All security procedures done for securing physical DCs should be done in DC VMs, like: Role-based Access Policy and hard drive encryption.

2-) Follow VMware Hardening Guide (v5.1/v5.5) for more security procedures to secure both of your VMs and vCenter Server.

Scalability:

1-) For greater scalability, try to upgrade your AD DCs to Windows Server 2012. AD DC on Windows 2012 leverages the new VM-Generation ID feature that makes the AD DC Virtualization-aware and hence, it can be cloned easily and any cloning process can be done safely. Check VMware Best Practices to Virtualize AD DS on Windows 2012herefor more information and cloning process step-by-step guide. Cloning can help in case of urgently needed expansion in AD DC infrastructure, DR process or testing. It also cuts down heavy network utilization by AD DCs in replication of entire DB to the new promoted-from-scratch DCs. Keep in mind that Cold Cloning is the only one supported by both VMware and Microsoft. Hot Cloning isn’t supported in production by either VMware or Microsoft.

I wish that this comprehensive -long unintentionally- guide helps you in virtualizing this simple but important application and providing the required level of availability and performance. For more information or detailed explanation of any item, check the References section.

Share the knowledge ...

References:

-- Virtualizing MS Business Critical Applicationby Matt Liebowitz and Alex Fontana.

-- MS Infrastructure Planning and Design Guide (IPD).

-- Virtualizing AD Domain Services on vSphere

-- vSphere Design Sybex 2nd Editionby Scott Lowe, Kendrick Coleman and Forbes Guthrie.