Disk write latency is too high

[ARK85]

Hi,

On one of our virtual machine which is a file server we are getting continous alert of Disk write latency is too high.

“Disk write latency is too high  The threshold for the Physical Disk\Avg. Disk sec/Write performance counter has been exceeded.”

This virtual machine has many disks which are all RDM's  except one which is the c:\ drive.

Can anyone let me know what is the reason behind this. This is the first time we are getting this alert on this VM.

Thanks

[MKguy]

It means exactly what it say: The disk latency threshold was exceeded. What's the threshold value and what average and peak latency are you having on this VM's RDMs? Also, about how many average IOPS is this VM generating (check performance charts or (r)esxtop for these values).

High physical disk latency, as clearly implied by it's name, generally points in the direction of a storage performance issue. There can be a million causes for this ranging from from disk failures or RAID rebuild operations, to SAN-misconfigurations, or simply the physical disks hitting their IO limits etc.

Please share a bit more details about your storage setup in addition to the values mentioned above.

[ARK85]

Hi,

We have found out the disk latency value is high on the c:\ drive which is not an RDM. There are few volumes in the machine which have been mounted on c:\ drive. Attached is the screenshot from disk monitor. I hope that the issue is related to storage, and our storage team is working on this.

If you can let us know, how to proceed further from VMware point of view.

[MKguy]

This screenshot doesn't really say anything about latency. Like I said before, check (r)esxtop or vSphere Client performance charts for Disk, and Datastore latency counters. Also provide more info on your storage systems and layout.

[ARK85]

Hi,

Please find the attached screenshot. We are using EMC Clarion.

[MKguy]

First, did you take these screens when the issue occured?

Next, the 2nd image will only show latency of your VMFS datastore related VMDKs, which in your case would only be the C:\ drive. The first image is only showing disk throughput per LUN but not the crucial part about latency.

Select the host currently running the VM and check the advanced disk performance charts for the respective RDM LUN identifier there.