Hosts have HA issues after ESXI upgrade

[hendersp3]

After upgrading ESXi hosts from 6.5 build 5969303 to 6.5 7388607 the HA agents have issues.  Windows vCenter Server 6.7 

vSphere HA agent for this host has an error.  The vSphere HA agent is not reachable from vCenter server.

They were fine before the host upgrade as well as during the upgrade.  It wasn't until the 4th host was put into maintenance mode that this happened and the running vms couldn't evacuate because the remaining 3 hosts in the cluster had HA errors. 

Does anyone know what I can look at to see what caused this?  I  do not want to just reconfigure for HA, but would like to know if this can be avoided.  This happened three different times. 

I have been all over the logs for a few days now and don't really see much.  Maybe I don't know what to look for. 

Thank You for any help you can provide.

[MikeStoica]

Did you tried to restart the management agents? Networking is ok?

[rajen450m]

Hi,

Hope it could be due to network connectivity between your vcenter server and agent on ESXi host, please check the watchdog process on the ESXi host and reconfigure vSphere HA on that host.

Please login to ESXi host from webclient and check the status of sfcbd-watchdog service and start/restart it.

You should refer to vpxa.log which contains information about the agent communication details with vCenter Server.

Have updated vCenter server also?

It is good to migrate to VCSA rather than Windows vCenter Server.

[hendersp3]

Thanks for teh replies so far.  The network is 100% fine and re-configuring HA or removing the host from the cluster and adding it back gets the agents working again. 

So the vib for the HA agent vmware-fdm is not included in any ESXi package and gets pushed down by the vcenter. An ESXi upgrade apparently does not initiate an agent reload and you must do this manually in my case.  I admit I rarely perform host upgrades anymore as I have changed roles now but can anyone validate this?  Does upgrading ESXi (even minor version) brake HA until you reconfigure it or disconnect\reconnect the host to vcenter? 

Thanks