High latency from only one host

[vChr1s]

Hello.  I recently upgraded to vsphere 5.1.0b, running build 1021289 on my esxi hosts.  I run NFS over 10 GbE on my Netapp with Intel X520-DA2 SFP+ nics in my hosts, and a HP Procurve 10 GbE network environment.  NIOC, SIOC, and VAAI are enabled.

It seems since the upgrade I had one host that was performing very poorly.  I was getting numerous alerts about disk latency and esxtop is reporting guest latency upwards of 20,000 ms during its peaks, although it is typically around 2,000 to 10,000 ms.  All of the latency issues are with a single host.  Esxtop on my other hosts are reporting <1 ms gavg/cmd with an occasional spike around 1.5 ms.

I have replaced the cables, moved ports on the switch, and upgraded the 10 GbE drivers from the Intel ixgbe 3.11.32 to version 3.14.3.

I am running out of places to look.  Any ideas?  Thanks.

[Nitzan_S]

Hi Chris.

Two questions:

1. Have you tried to disable the NIOC and then check the latency?

2. Same with SIOC.

[vChr1s]

Hi. Thanks for the reply. I should have came back to update this. Turns out it was a bad nic on the 10gbe mezzanine card on the netapp. Had a bunch of crc errors. Thanks.

[Nitzan_S]

Glad your issue is solved.