We are unable to add one ESXi host in VC. Giving attached error.
VPXD is showing below logs.
icate verification is disabled, so connection will proceed despite the error
2014-11-08T06:23:40.971+10:00 [10296 verbose 'SSL SoapAdapter.HTTPService'] User agent is 'VMware vim-java 1.0'
2014-11-08T06:23:40.971+10:00 [10296 verbose 'Default'] [VpxVmomi] Invoking [waitForUpdatesEx] on [vmodl.query.PropertyCollector:propertyCollector] session [528e6787-82ab-bf4e-3ccb-4e8d652b50e7(521c762d-2875-a801-43a0-0f2ed2e69621)]
2014-11-08T06:23:40.987+10:00 [03520 verbose 'Default' opID=8FD16342-00000205] [VpxVim] Local and Remote Version are the same. Talking with version vim.version.version7
2014-11-08T06:23:41.033+10:00 [13332 verbose 'DAS' opID=SWI-80e21c46] [FdmManager::SendPendingFdmUpdates] No master to send updates to. Will retry.
2014-11-08T06:23:41.111+10:00 [13652 warning 'ProxySvc Req54900'] Connection to localhost:31000 failed with error class Vmacore::SystemException(No connection could be made because the target machine actively refused it. ).
2014-11-08T06:23:41.127+10:00 [13652 verbose 'ProxySvc Req54903'] New proxy client TCPStreamWin32(socket=TCP(fd=8068) local=127.0.0.1:80, peer=127.0.0.1:60379)
2014-11-08T06:23:43.077+10:00 [10296 verbose 'SoapAdapter.HTTPService'] User agent is 'VMware VI Client/4.0.0'
2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxVmomi] Invoking [retrieveContents] on [vmodl.query.PropertyCollector:propertyCollector] session [522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)]
2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLroList::ThreadMainEntry] Unlocked _propProvider to call into external code
2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLRO::LroMain] Invoking _currentMain
2014-11-08T06:23:43.077+10:00 [13652 info 'Default' opID=D5520D1B-000010C8] [VpxLRO] -- BEGIN task-internal-79722 -- -- vmodl.query.PropertyCollector.retrieveContents -- 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)
2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default' opID=D5520D1B-000010C8] [VpxVmomi] Invoke done: vmodl.query.PropertyCollector.retrieveContents session: 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f
2014-11-08T06:23:43.077+10:00 [13652 verbose 'SoapAdapter.HTTPService' opID=D5520D1B-000010C8] HTTP Response: Complete (processed 1131 bytes)
2014-11-08T06:23:43.077+10:00 [13652 info 'Default' opID=D5520D1B-000010C8] [VpxLRO] -- FINISH task-internal-79722 -- -- vmodl.query.PropertyCollector.retrieveContents -- 522f9a8f-c325-5f3b-a5e4-cad79cc8fc0f(5243ef82-3fe2-dd25-5e17-ad56685d2cbd)
2014-11-08T06:23:43.077+10:00 [13652 verbose 'Default'] [VpxLRO::ClearCurrentMain] Clearing _currentMain
2014-11-08T06:23:44.325+10:00 [13872 verbose 'SoapAdapter.HTTPService' opID=c1bade25] HTTP Response: Complete (processed 1004724 bytes)
2014-11-08T06:23:44.653+10:00 [13872 info 'Default' opID=c1bade25] [VpxLRO] -- FINISH task-internal-79718 -- -- vmodl.query.PropertyCollector.retrieveContents -- 525144ec-a5fb-44c0-d24a-1a182c7c7afb(5273c6e1-09a6-3392-72a2-15b32eb62009)
2014-11-08T06:23:44.653+10:00 [13872 verbose 'Default'] [VpxLRO::ClearCurrentMain] Clearing _currentMain
From vCenter, can that host be resolved/pinged? Not the solution, but have you tried adding it via IP and seeing if that works?
Yes, We can ping it from both side, VC to host and vice versa.
Also tried adding it by IP as well as FQDN. No Luck!
You can try restarting the management agents. See this for more info. If you have console access, its probably easiest to login and perform it from there.
VMware KB: Restarting the Management agents on an ESXi or ESX host
Are you able to directly connect to the ESXi host using VI client.
Can you paste the hostd logs ?
It will little crazy to hear but ESXi OS Re-installation will be fastest solution
Can you chek host bios date and time...
If Bios date and time wrong, sometimes i see that issue...
Hi,
2014-11-08T06:23:41.111+10:00 [13652 warning 'ProxySvc Req54900'] Connection to localhost:31000 failed with error class Vmacore::SystemException(No connection could be made because the target machine actively refused it. ).
Can you verify the Port number are open in the Vcenter machine and try to restart the Vcenter services in vcenter
When I logged into DCUI, "Lockdown Mode" option is grayed out.
Anything else what we can try?
"/var/log/hostd.log" shows below logs while adding to VC.
2014-11-09T22:32:57.043Z [2D681B90 verbose 'Proxysvc Req00005'] New proxy client SSL(TCP(local=10.228.10.172:443, peer=10.228.2.93:59047))
2014-11-09T22:33:00.779Z [FFED1AC0 verbose 'Cimsvc'] Ticket issued for CIMOM version 1.0, user root
2014-11-09T22:33:32.474Z [2D681B90 verbose 'ResourcePool ha-root-pool'] Root pool capacity changed from 32196MHz/254530MB to 32196MHz/254529MB
2014-11-09T22:33:36.587Z [2D681B90 verbose 'DvsManager'] PersistAllDvsInfo called
2014-11-09T22:33:46.008Z [2D681B90 verbose 'Default'] Timed out reading between HTTP requests. : Read timeout after approximately 50000ms. Closing stream TCP(local=127.0.0.1:8307, peer=127.0.0.1:63341)
2014-11-09T22:34:32.476Z [2DAC2B90 verbose 'ResourcePool ha-root-pool'] Root pool capacity changed from 32196MHz/254529MB to 32196MHz/254530MB
Also, Just to add on this,
Regards,
Pranay
Seems network issue between vc and the host .
Are you able to telnet 902 and 443 port from vc.
Both port 443 & 902 are opened.
Default Gateway is not reachable from host, however same gateway is reachable from other hosts in cluster.
Also, I can ping few IPs in same segment where few are not pingable. Though the same IP addresses are reachable from other ESXi hosts.
Hello,
How many uplink connected on the management vswitch?
If its 2, can you try only 1 uplink...
Hi,
Would it be possible to check the networking configuration? If you're using cisco switches with an etherchannel config, try disabling the etherchannel for now until getting the host into vCenter.
For now, I'd try using one NIC enabled in VMware with the proper VLAN for management and see if you can ping the gateway. You can also compare the configuration between a working host and this problematic host to see if everything matches.
Hope this helps.
From all the comments I understand that you are confident that ESXi host has Networking problem with below symptoms:
Assumptions:
You have identical network configuration for this ESX box like other working servers
You have PNIC's enabled with required VLAN's same as other working servers
vSwitch 0/ Management vSwitch - has valid PNICs enabled
Next steps:
Confirm Network Adapters are plugged into proper Physical Network switch port
Valid drivers are installed for Network Adapters - Try to update the new drivers or firmware as per compatability matrix
Physical Network switch port has required settings for VMware Virtualization like disable STP, configure Portfast
Replace the Network cables if they found faulty
Run full diagnostics for the server HW and open a case with Hardware vendor