Slow disk speed on new ESX 3.5 install

[evandena]

I have an old IBM x345 server that I am trying ESX 3.5 on. The server is as follows:

2x Xeon 2.6Ghz, 4GB RAM, 3x 74GB 10k RPM HD’s w/ ServeRAID 5i.

The problem is the disk speeds I am seeing with a fresh install of ESX 3.5 (currently in trial mode). I seem to only be able to get 3.5MBPS throughput, and I believe it is the disk that is the bottleneck.

Here’s the testing I have done:

1) Setup a Win 2k3 Std server and shared out a folder. When writing or reading to the shared folder (which is ESX local disk), I transfer at 3.5MBps on a gig connection.

2) Attached a NAS as a datastore in ESX. I then attached a drive to that NAS and transferred files to it from a computer on the network. I had higher throughput, about 40MBps, which is expected. In this test no files hit disk. Through on nic, out the other.

3) Doing a simple SCP transfer to the ESX box yields only 3.5MBps, no more.

4) I installed Win2k3 and Ubuntu directly to the server (no ESX involved) and had normal transfer speeds and disk writes.

5) Tried RAID 1 and 5. Tried using onboard raid instead. No change.

So it appears ESX is the limiting factor. The hardware is fully capable of reaching much higher speeds.

Is anyone able to offer any help?

Thanks.

[spex]

Have a look at the following article:

http://www.tuxyturvy.com/blog/index.php?/archives/37-Troubleshooting-VMware-ESX-network-performance....

Regards

Spex

[evandena]

Thanks for the reply.

I read the article, but I am not experience quite the same problems as this writer.

I checked my IRQ's and there is nothing conflicting with either vmnic.

[Rumple]

SCP to the box and using normal linux file copies will not give you any usable results. The Service console is a VM once ESX is running and it is given a low priority to the disk.

If you are doing testing of I/O create a vm and install IOMeter to test how the different writes work. Using vmkfstools will probably give you close results as well since its the vmkernel doing the writes to the vmfs volumes (which is where you need to test sppeds, not in the ext3 file system

Also ensure you have write back enabled on the RAID controller as that will make huge difference in speed when doing writes (although i suspect it is since you are seeing normal disk writes with Windows.

Another thing to take into consideration is that esx is really designed to allow you to take a bunch of under utilized VM's and combine them to bring up the total ROI of a piece of hardware. If you need high sustained resources (disk, memory,cpu) then the vm is probably not great for virtualizion anyhow.

About this - 2) Attached a NAS as a datastore in ESX. I then attached a drive to that NAS and transferred files to it from a computer on the network. I had higher throughput, about 40MBps, which is expected. In this test no files hit disk. Through on nic, out the other

This is probably accurately testing vmfs throughput as it sounds like you created a VM on the NAS and transfered files into it. Or did you create a vmfs volume and use winscp or something to copy files to it (or did you just read off the vmfs). There is disk activity happening somewhere there, not just network.

[evandena]

Thanks for the detailed reply Rumple.

To answer your question and more clearly state my testing; I had a Win2k3 on local disk (slow transfer), but then attached another "disk" from the NAS (vmfs) as a secondary drive in Windows. Transfer to this disk was much much faster.

I understand your arguments against SCP through the service console as a speed test, but it just seems odd that I can only reach 3.5 MBps. I'm fairly involved with ESX at work, and in the past when we ran VM's off local disk we used SCP to tranfer vmdk's between hosts at much higher speeds than I am seeing... even on similar hardware.

I'll try IOMeter tonight. I ran HD Tune, but got crazy results because the disk is virtualized. Running HD Tune in Win 2k3 with no ESX I got around 60MBps write to the disk.

[evandena]

Ok, I ran some IO Meter tests. Regardless of the number of workers I am running, I can only get abotu 16-20 MBps.