Esxi 5.5u1 added iscsi storage adapter- reboot, and now vSphere Client could not connect to "my ip" An unknown error occurred. The server could not interpret the client's request. The remote server returned an error (503) Server unavailable

[mervincm1]

I didn't even connect it to an iSCSI target, not device.

I can ping my host

when I open up http://"hostip" in a web browser, i get a 503 service unavailable.

restarting the host gets me nowhere.

SSH opens sort of, but can't login

console appears OK

vSphere Client can't connect

If I reset to defaults from the console it's ok, but once I reconfigure the host, this error will return.

I tried re-install from DVD

I am completely patched to date via SSH esxcli

This happens on both my hosts, although they are nearly identical Lenovo thinkserver TS140s with broadcom 10Gig NIC and intel integrated NIC

It almost always seems to happen the next time I reboot after enabling iscsi support

The only weird thing I have is that my integrated NIC is an intel 217A and I have to use a special VIB to get it to be usable in ESXi

Client is win 8.1

here are my install notes

Install on USB/SSD using custimized ISO with i217 NIC driver, reset configuration and reboot

Management NIC set to NIC0:1Gig

Management IP: hostIP/24 GW: my gateway

DNS:dns on windows vm1 , dns on windows vm2

Hostname:ESXi1.sub.myregistereddomainname Custom DNS Suffixes:sub.myregistereddomainname

Reboot

Patch to date (https://www.youtube.com/watch?v=_O0Pac0a6g8)

Upload the VIB and .zip to a datastore by using the vSphere Client

Get them from (https://www.vmware.com/patchmgr/findPatch.portal)

Start the ESXi SSH service and make a Putty SSH connection to the ESXi server.

Put the ESXi server in maintenance mode,

example command : esxcli software vib install -d /vmfs/volumes/ESXi2-2/patch/ESXi550-201404020.zip

Re install the Intel 217 NIC driver if removed by patch

Change the ESXi host acceptance level to Community Supported,

command: esxcli software acceptance set -–level=CommunitySupported

Install the VIB,

command:esxcli software vib install –v /vmfs/volumes/datastore1/net-e1000e-2.3.2.x86_64.vib

command: reboot

Connect via VSphere client

-Storage

Verify/fix/create local storage. VMFS5

-Networking

vSwitch0

Verify vmnic0 (1GbE)

Rename VM Network Port Group to “Core”

Rename Management Network Port Group to Core-VMkernel  Management Traffic.

-Time configuration

Enable NTP Client  Start and stop with the host. set 0-3 ntp.org time servers

DNS and routing

Virtual machine Startup and Shutdown

-enable –continue immediately if tools start –Shutdown action –guest Shutdown – both delays to 10 secs

Security Profile

Services

SSH – start – enable start and stop with host

Host Cache Configuration

-Properties on boot SSD – allocate 40GB for host cache.

Surpress SSH warnings:

Advanced settings, UserVars, UserVars.SuppressShellWarning, change 0 to 1.

Storage Adapters

-Add – Add Software iSCSI adapter

[NealeC]

Have you tried the following KB?
You config could have become corrupted during the reboot.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100556...

Also check the host can be resolved from your DNS server and that the host can nslookup itself (or blank out resolv.conf) and restart the mgmt service.

Other KBs and articles point at restarting the mgmt agents, but without really having a root cause.  You can try a simple restart of the mgmt agent from the DCUI if you have console access or via SSH.

If that does solve it then you still probably have a config issue elsewhere.

Let me know if that works.

Chris

[mervincm1]

It definitely can't resolve itself since the 2 DNS configured both run on VM's that didn't yet start (run on that host)

I can restart from scratch and then point it to a physical box running DNS that the ESXi host will be able to resolve itself from.

Is this critical once you add iSCSI? I rebooted many times before this point w/o this access issue.

My DNS is pretty straight forward.  I have a registered domain.ca.  I use domain registrar's DNS as authoritive (I think that's the right term) updated dynamically to be my WAN interface public IP for that space.  I don't actually have any hosts defined, just the root (@)  It's sole purpose is so I can use name to assist in VPN access from external.  My internal windows domain is one level deeper inside.domain.ca.  My Windows DC and another VM are authoritive for that space, and I have another physical system with DNS.  These are all just for internal resolution, My internal DNS's and the Registar DNS do not know or forwards to each other.  I have no need to ever resolve hostname.inside.domain.ca from the Internet.  Hopefully that's not a problem here.

no functional SSH, so that's not a possibility.  console access works, so I will look at that.

I understand a rebooted host accomplishes a management agent restart, so I didn't pursue that since, well a host reboot makes no difference!.

I needed access so I rebuild the host already (without iscsi!) but I want to get this fixed, so I will break it again!

thanks!

[mervincm1]

Downloaded a fresh copy of the DVD (5.5Up1 rollup2 1623387) to install from

-avoiding the modified ISO I made (using ESXi customizer) to add in the modified vib to support the Intel i217 NIC

-configured only the physical DNS server, skipped filling in the custom DNS suffix

-skipped applying any patches

So far so good! its been very stable, I have even added in some iSCSI devices from my ReadyNAS, it's looking good!

[mervincm1 · Accepted solution]

I think I determined what I did wrong.  I actually applied two patches when only one was appropriate. I started with 5.5u1rollup2 and then applied both ESXi550-201404001 and ESXi550-201404020.  strangely I didnt seem t o have any issues till I worked with iSCSI.