VMware Cloud Community
SSpiers
Contributor
Contributor

vCenter 7 error - no healthy upstream

Hi All,

I'm working on brand new ESXi 7.0.0 deployment and have run into an issue when deploying vCenter 7.

When browsing to the vCenter UI I receive a "no healthy upstream" error.

I can access vCenter Server Management without issue.

I've done some googling and have been pointed towards certificate issues however my Leaf and Root certs are showing as valid.

All of the services which are set to Automatic start are running without any errors or warnings.

Is there a documented fix for this issue?

76 Replies
romesds157
Contributor
Contributor

I've done the same thing, restart vcenter server service from the Management Console. It looked like it was going to timeout because of the blue spinning wheel of death but it started, came online as "healthy" and i was able to access everything as usual! 

0 Kudos
AmitGN
Contributor
Contributor

I also faced the same problem.

In my case.

1. log directory was full, so increased the log partition ( after increasing the log partition also same error continued )

2.  after the recreation of the certificate problem is solved.

cmd:  /usr/lib/vmware-vmca/bin/certificate-manager

Cukal42
Contributor
Contributor

AmitGN response is correct, running certificate-manager fixes it. 

This kb details how to check for expired certificates: https://kb.vmware.com/s/article/82332

This kb details how to run certificate-manager: https://kb.vmware.com/s/article/2097936

 

0 Kudos
vmkfix-SSA
Contributor
Contributor

 
0 Kudos
vmkfix-SSA
Contributor
Contributor

0 Kudos
riscoto
Contributor
Contributor

I have tried restarting all my services from vMware Server Management. And it worked on my end. I can now use vSphere Client and manage multiple Physical Server and VM Servers.

0 Kudos
AkmalAza
Contributor
Contributor

Hello everyone, I encountered a similar issue with our vCenter UI, where the upstream status appeared unhealthy. We are utilizing linked mode for three vCenters. Numerous solutions were suggested, including DNS and NTP configurations, which we attempted but yielded no results. Additionally, we tried increasing the vCenter's memory and allocating a specific amount for the vpxd services, but the improvement lasted only for a period of 24 to 48 hours. To resolve this, we simply rebooted our SRM (Site Recovery Manager). It turned out that SRM had a bug causing the vpxd service to become busy. After rebooting both SRM and vCenter, the services were refreshed, and everything has been running smoothly for over two weeks without any interruptions even with reduced the memory allocation on the vCenter. my suggestion is to take note whatever connection connected to your vCenter and powered it off and bring it back on by one to eliminate and detect which appliance that causing it.   

0 Kudos
AkmalAza
Contributor
Contributor

you also need to clear up your vpxd crash dump log as each time the vpxd stopped it will create 1 files which in my case around 10GB per file.

0 Kudos
ThereAreSomeWho
Contributor
Contributor

Thank you, this appears to have resolved my issue after updating to 7.0 Update 3n

0 Kudos
dvbsolutions
Contributor
Contributor

I had the same issue.  I deleted and installed vcenter twice before I read this solution.  The problem I had was that I was shutting down my other host (which had my DC and DNS) , so that I could add the host into Vcenter.  I had to put the host in maintenance mode prior to adding it into vcenter.  When I launched vcenter I got the upstream error.  I exited maintenance mode and started my DC, and Vcenter launched correctly. 

0 Kudos
promaster
Contributor
Contributor

Thanks got this error after the latest update, the dns whas changed to 127.0.0.1. now vcenter works.

0 Kudos
ecerronj
Contributor
Contributor

Soy Eduardo Cerrón, También me sale el error, "no healthy upstream", he intentado resolverlo viendo varios problemas similares, pero, sin éxito, alguien puede proporcionarme mas información, por favor.

0 Kudos
genemcsween
Contributor
Contributor

This turned out to be an expired STS certificate chain in my environment keeping vmware-stsd from starting.  At the CLI, I ran service-control --status and saw the service was stopped.  The script in this article (Checking Expiration of STS Certificate on vCenter Servers (79248) (vmware.com)) threw a bunch of errors because it couldn't connect to STS.

I ran the script from this article Article Detail (vmware.com) then ran option 8 from this article How to use vSphere Certificate Manager to Replace SSL Certificates (2097936) (vmware.com), rebooted and I'm up and running again!!

I now have a reminder set in my email to renew this cert when it gets 60 days out from expiring.

0 Kudos
mariejosv
Contributor
Contributor

YES this is the FIX. After 7 attempts this little step fixed my problem

0 Kudos
mariejosv
Contributor
Contributor

After 7 attempted deployment that failed due to the same error, the NTP fixed my problem.

0 Kudos
IK-2
Contributor
Contributor

Check DNS, and all IP's that are set, and all physical network connections. Also, if you use a switch or switches, reboot the one that is passing thru the network to your datacenter on the same subnet. Most likely this is either a network issue or a big bug in a release.

0 Kudos
Sina_Rastgouyan
Contributor
Contributor

You can solve this problem by restarting the VMware Vsphere Client service or starting it manually using your vcsa management panel.

0 Kudos