Horizon 7.13.1 Instant Clones and Office GPOs Not Applying After Publising Snapshot

[TeejMaximus]

Environment: ESXI 6.7, Horizon 7.13.1, DEM 9.9.0.905, Windows 10 21H2, Microsoft 365 (Shared Computer Activation), Teams Machine-wide Install. 4 Pools that share the same golden image and current snapshot.

So we have a GPO in place for Office Shared Computer Activation that automatically activates Microsoft 365 and configures an Outlook profile for a new user the first time they log into the VDI. This GPO does this without any prompts to the user. When all is working as it should this GPO has zero issues.

However, sometimes (maybe 50%), when we publish a new snapshot to a pool, and log into the new clone, we notice is Teams does not automatically sign the user in.  If Teams does not auto sign in a user, then that's the red flag that let's us know the Office GPO is broken (for some reason they go hand and hand).  Thus, a new user that signs into the pool will get all the prompts for manual Office activation.  And even if the user goes through all the prompts to complete activation, it still fails stating that Office is unlicensed. 

To fix this problem what we have to do is disable provisioning in the affected pool, select all the machines and remove them so there are zero available, then re-enable provisioning to let it recreate all the machines. Then, if we log into the pool Teams will auto sign in as it should, and the Office GPO works for new users. That's the problem and fix in a nutshell, however sometimes it's a bit more complicated.

Occasionally we have to disable provisioning and delete/recreate machines in a different pool that shares the same golden image, as sometimes publishing a new snapshot in Pool A causes the broken Office behavior in Pool B, even though Pool B was previously working. Therefore, if we publish a snapshot in one pool, we essentially have to log into all other pools that share that same golden image make sure we see Teams auto sign in the user. If Teams launches as expected, great!  However If one or more pools are broken, which can be easily identified by Teams not auto signing in at login, then we have to start the process of deleting the machines in those pools and let them recreate, then sign in to make sure it's fixed.

When everything is working, it stays working until we publish a new snapshot. When it's broken, it will stay broken and it seemingly ignores any GPO changes we make until we take the steps above to fix the broken pool, and once we do that then everything start working again.  Has anyone seen similar behavior or have any ideas as to what may be the underlying issue? 

[BenTrojahn]

..

[GTO455]

Did you try running a gpupdate /force before sealing your image?

I don't know if its required, but I have recently added that to my cleanup script. I haven't really seen a difference in my case (my master image is not in the domain) but it I guess it can't hurt.

[TeejMaximus]

I've manually done that on the template VM after it's joined to the domain before it powers off to be cloned, however it had no effect.  I would expect this to be an all or none type deal if we have something mis-configured, but since it breaks about 50% of the time when we publish a new snapshot I feel it's pointing to some other anomaly within Horizon/ESXi that we haven't discovered. Like some bug that's still out in the wild.

[GTO455]

Is your template VM in the same OU as your pool?

We ran into something similar recently when we switched the network for our master image. We switched the network, created a new snap and provisioned the pool. The VM's came up on the right network, but none could get IP's. I had to actually power on the master, let it register with the host, get an IP, seal the image again, take a new snap and republish the pool.

[TeejMaximus]

Yes.  We have an Image Publish Computer Account specified for each pool that's in the same OU as the clones, so the template is always in the same place.

It's just strange that the fix involves deleting the all VMs and letting them recreate.  I mean, what's the difference between an instant clone going through the normal process of being destroyed and recreated after a user logs out (which never fixes it) vs. me going in and destroying all the clones after disabling provisioning and then enabling provisioning to let them recreate (which fixes it)?  I'm not changing the replica that the clones are being created from.

[GTO455]

Are you also deleting the computer accounts in AD when you delete the VM's and have them recreate? 

[TeejMaximus]

No, I've not been deleting the computer accounts in AD.  Also, I just tried the disabling provisioning on a pool and deleted one of the clones to see what would happen to the AD computer object, and was not deleted (which I guess is expected behavior). The AD computer objects present have been in place since the first time the corresponding clone was created according to the object properties.

[sjesse]

Thats a setting, look for  "reuse computer objects" in the pool settings. I don't know if deleting them will help or not, but its worth trying. If you delete one in AD then delete the clone, it should recreate the computer object.

[sjesse]

I'd also try removing your parent image from the domain, add it again, and push out a new image. Should refresh the trust which there could be a wierd issue with.

[vBritinUSA]

Have you checked to make sure the policies are being replicated between the DC's? 

[TeejMaximus]

I tried unchecking the option to reuse computer objects and did some testing, however even after the computer accounts being deleted and recreated in AD we still ran into the issue.  Our parent image isn't domain joined, however we have tried creating a brand new parent image but no luck.

[BenTrojahn]

Im not sure what first reply, but it went along to line of verifying infrastructure and how DHCP scopes/lease times/replication, AD sites/zones, DNS registration/replication  all impact how well instant clones work. Try to simplify as much as possible, we use a dedicated AD site and large VLAN for provisioning, specify the AD site/DCs for provisioning and use those same for primary DNS etc..   Ive had wonky DCs cause issues with sysvol repl and or slow or no GPO application.

We also do not have domain joined golden images, but we have found as best practice to make the first deployment to a dedicated image deploy farm higher in the OU structure that the rest of your farms.  Then set a post sync customization script to reboot after provisioning. 

the only other policy issues we have had is with DEM occasionally crashing GPP processing at which point there is no fix without a system reboot/recover.  

[TeejMaximus]

Yes, all policies are being replicated between DCs.  Running gpresult on an affected clone shows that the GPO is being applied.  I also checked the DC where GPO settings were being pulled from to make sure they were accurate.