Using vcenter 6.7 Administration - > Certificates have added root CA certificate of Letsencrypt and replaced Machine certificate with signed one provide certificate and key
After reboot vcenter doesn`t start anymore:
2019-12-19T17:22:23.429Z info vpxd[05606] [Originator@6876 sub=ThreadPool] Entering worker thread loop
2019-12-19T17:22:23.430Z info vpxd[05605] [Originator@6876 sub=ThreadPool] Thread enlisted
2019-12-19T17:22:23.430Z info vpxd[05605] [Originator@6876 sub=ThreadPool] Entering worker thread loop
2019-12-19T17:22:23.459Z error vpxd[05321] [Originator@6876 sub=Main opID=CheckCertificateExpiry-6058ed8] Unable to get certificate count for APPLMGMT_PASSWORD from VECS localhost, error: 0
2019-12-19T17:22:23.548Z info vpxd[05332] [Originator@6876 sub=ThreadPool] Spawning additional worker - allocated: 144, idle: 19
2019-12-19T17:22:23.553Z info vpxd[05617] [Originator@6876 sub=ThreadPool] Thread enlisted
2019-12-19T17:22:23.553Z info vpxd[05617] [Originator@6876 sub=ThreadPool] Entering worker thread loop
2019-12-19T17:22:23.572Z warning vpxd[05113] [Originator@6876 sub=LSClient] Caught exception while getting service with Id :e2136204-f25b-4a2b-a5ac-67b473cfd253. N7Vmacore9ExceptionE(Cannot initialize service registration stub)
--> [context]zKq7AVECAAAAAGC34QAOdnB4ZAAA4AArbGlidm1hY29yZS5zbwAAWCUbABWWGAH5kWV2cHhkAAHOlWUB9qFlASkvoAIqhQJsaWJhdXRoemNsaWVudC5zbwABvdeeAToJVAGKaFQBGcZSA5AFAmxpYmMuc28uNgABpb5S[/context]
2019-12-19T17:22:23.573Z warning vpxd[05113] [Originator@6876 sub=LSClient] Caught exception while retrieve endpoint. N7Vmacore9ExceptionE(Cannot initialize service registration stub)
--> [context]zKq7AVECAAAAAGC34QAPdnB4ZAAA4AArbGlidm1hY29yZS5zbwAAWCUbABWWGAH5kWV2cHhkAAE4l2UBKJllASuiZQEpL6ACKoUCbGliYXV0aHpjbGllbnQuc28AAb3XngE6CVQBimhUARnGUgOQBQJsaWJjLnNvLjYAAaW+Ug==[/context]
2019-12-19T17:22:23.574Z warning vpxd[05113] [Originator@6876 sub=LSClient] endpoint not found for Product: com.vmware.cis, Type: cs.inventory
2019-12-19T17:22:23.574Z warning vpxd[05113] [Originator@6876 sub=LSClient] Caught exception while getting service with Id :e2136204-f25b-4a2b-a5ac-67b473cfd253. N7Vmacore9ExceptionE(Cannot initialize service registration stub)
--> [context]zKq7AVECAAAAAGC34QAOdnB4ZAAA4AArbGlidm1hY29yZS5zbwAAWCUbABWWGAH5kWV2cHhkAAHOlWUB9qFlASkvoAI3hQJsaWJhdXRoemNsaWVudC5zbwABvdeeAToJVAGKaFQBGcZSA5AFAmxpYmMuc28uNgABpb5S[/context]
2019-12-19T17:22:23.575Z warning vpxd[05113] [Originator@6876 sub=LSClient] Caught exception while retrieve endpoint. N7Vmacore9ExceptionE(Cannot initialize service registration stub)
--> [context]zKq7AVECAAAAAGC34QAPdnB4ZAAA4AArbGlidm1hY29yZS5zbwAAWCUbABWWGAH5kWV2cHhkAAE4l2UBKJllASuiZQEpL6ACN4UCbGliYXV0aHpjbGllbnQuc28AAb3XngE6CVQBimhUARnGUgOQBQJsaWJjLnNvLjYAAaW+Ug==[/context]
2019-12-19T17:22:23.597Z warning vpxd[05113] [Originator@6876 sub=LSClient] endpoint not found for Product: com.vmware.cis, Type: cs.inventory
2019-12-19T17:22:23.718Z warning vpxd[05113] [Originator@6876 sub=VpxdAuthClient] [ConnectAndLogin] Failed to loginBySamlToken: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: 49:68:90:15:2C:75:C6:7C:C7:B4:55:EB:87:E2:E6:29:92:21:A8:72
--> ExpectedThumbprint:
--> ExpectedPeerName: localhost
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.)
--> [context]zKq7AVECAAAAAGC34QANdnB4ZAAA4AArbGlidm1hY29yZS5zbwAAWCUbAP6dGACeQCIAaXEiABtFIgDTSSIAOaIjAHFvIwA6ciMAnVYrAdRzAGxpYnB0aHJlYWQuc28uMAAC3Y4ObGliYy5zby42AA==[/context]
2019-12-19T17:22:23.719Z info vpxd[05113] [Originator@6876 sub=VpxdAuthClient] fallback to loginByCertificate
2019-12-19T17:22:23.729Z error vpxd[05113] [Originator@6876 sub=ServerAccess] Remote login failed: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: 49:68:90:15:2C:75:C6:7C:C7:B4:55:EB:87:E2:E6:29:92:21:A8:72
--> ExpectedThumbprint:
--> ExpectedPeerName: localhost
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.)
When resetting certificates using /usr/lib/vmware-vmca/bin/certificate-manager it works again
There is no ESXi host connected to vCenter just in case...
Looks like the cert is incorrectly configured
* Host name does not match the subject name(s) in certificate.
Run the below commands and make sure all 3 gives you hostnmae of vCSA
1. PNID of the vCenter server: # /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
2. Hostname of vCenter server: # hostname -f
3. And SAN (Subject Alternative Name) field of machine ssl cert: # openssl x509 -in machine.cer -noout -text | grep DNS:
same here... cant figure out why
tail -f /var/log/vmware/vpxd/vpxd.log
--> ExpectedThumbprint:
--> ExpectedPeerName: localhost
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.)
the following command gives same result
/usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text | grep -A1 Alternative
openssl x509 -in <path_to_certificate_file> -noout -text | grep -A1 Alternative
hostname -f
Did you ever find a fix for this issue? Having the exact same issue here.
I had the same issue for past few weeks. Updated to 6.7.0.44000 and looks like this is resolved. I've run my playbook for renewing letsencrypt certificates a bunch of times, rebooted vcsa and everything seems to be stable so far.
It could be also due to duplicate certificate in the trusted root store . Try running the below command and match the serial numbers . If you find duplicate serial numbers then you would have to remove them.
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text| grep -A 6 -i alias |less
Was this fixed??
I am facing the same issue. Followed the thread but nothing seems to work
vcenter 6.7.0.52000
have the same issue after renewal letencrypt certificate:
vcenter.yyy.com
# /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
vcenter.yyy.com
# hostname -f
vcenter.yyy.com
# /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store MACHINE_SSL_CERT --text | grep -A1 Alternative
X509v3 Subject Alternative Name:
DNS:vcenter.yyy.com
--> PeerThumbprint: 98:FE:16:42:E3:CF:43:2B:63:C5:9D:79:9C:77:FB:BD:B2:2A:07:FA
--> ExpectedThumbprint:
--> ExpectedPeerName: localhost
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.)
--> [context]zKq7AVECAAAAAAt9JgENdnB4ZAAAPFYrbGlidm1hY29yZS5zbwAAJEUbAFqxGAAehyIA9b4iAJuLIgBTkCIAie8jAMG8IwCKvyMA+asrAdRzAGxpYnB0aHJlYWQuc28uMAAC7Y8ObGliYy5zby42AA==[/context]
2022-05-19T13:40:08.483Z info vpxd[36481] [Originator@6876 sub=VpxdAuthClient] fallback to loginByCertificate
2022-05-19T13:40:08.487Z error vpxd[36481] [Originator@6876 sub=ServerAccess] Remote login failed: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: 98:FE:16:42:E3:CF:43:2B:63:C5:9D:79:9C:77:FB:BD:B2:2A:07:FA
--> ExpectedThumbprint:
--> ExpectedPeerName: localhost
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.)
--> [context]zKq7AVECAAAAAAt9JgENdnB4ZAAAPFYrbGlidm1hY29yZS5zbwAAJEUbAFqxGAAehyIA9b4iAJuLIgBTkCIAie8jAMG8IwCKvyMA+asrAdRzAGxpYnB0aHJlYWQuc28uMAAC7Y8ObGliYy5zby42AA==[/context]
2022-05-19T13:40:08.488Z error vpxd[36481] [Originator@6876 sub=AuthzStorageProvider] [AuthzStorageProvider::CreateAuthzMgr] Failed to connect to IS: <N5Vmomi5Fault17HostCommunication9ExceptionE(Fault cause: vmodl.fault.HostCommunication
--> )
--> [context]zKq7AVECAAAAAAt9JgESdnB4ZAAAPFYrbGlidm1hY29yZS5zbwAAJEUbAFqxGAEGEFR2cHhkAAGu9FoBmOJjAV+voAG6mKACru4BbGliYXV0aHpjbGllbnQuc28AAlcHAgLSjgICsoYCAdkvnwFSJ1QBooZUAfnjUgPgBgJsaWJjLnNvLjYAAYXcUg==[/context]>
2022-05-19T13:40:08.490Z error vpxd[36481] [Originator@6876 sub=Default] Failed to instantiate AuthzStorageProvider: N5Vmomi5Fault17HostCommunication9ExceptionE(Fault cause: vmodl.fault.HostCommunication
--> )
--> [context]zKq7AVECAAAAAAt9JgESdnB4ZAAAPFYrbGlidm1hY29yZS5zbwAAJEUbAFqxGAEGEFR2cHhkAAGu9FoBmOJjAV+voAG6mKACru4BbGliYXV0aHpjbGllbnQuc28AAlcHAgLSjgICsoYCAdkvnwFSJ1QBooZUAfnjUgPgBgJsaWJjLnNvLjYAAYXcUg==[/context]
any thoughts?