Storage performance bottleneck after P2V

[niriwoing]

I recently P2Ved a windows 2000 advanced server running SQL 2000 to ESXi 4.1 U1. After P2V the database server is not performing as it should. Query which was executing in 20 - 25 seconds on source server takes more than 60 secods to execute and the application times out. KAVG and DAVG values are not very high around 3-5. Device queue is showing 100% utilization and it has another 80-100 requests queued. When I check from inside guest OS it also shows high disk read queue length.

When we check the performance on source server, it has high disk queue length values as well but application does not timeout. On source server the database files are located on SAN LUN and on VM we are using copy of original LUN as RDM. The disk has 42% file fragmentation. The database file in question has size of about 35 GB and has 168k fragments. Can someone suggest a way to resolve this?

P.S: We already tried defragmentation but OS cannot complete defragmentation of DB file even when SQL server services are stopped.

[marcelo_soares]

You can try to change the SCSI controller from buslogic/LSI Parallel to LSI SAS. This will improve a bit the IO performance. Also, check another points, like memory reservation (reserve 100% of the RAM of the VM and remove all limits) and CPU limitation (no limits for CPU also). You can try to change the VM VMDK from one LUN to another, including to a local ESX disk to check if you have changes on the IO performance.

[niriwoing]

Thanks for your reply Marcelo. As far as CPU and memory are concerned there are no limits defined. I already tried moving the VM VMDK to different LUN but that did not help. I will try and use LSI SAS controller and see if it helps.

Ok....so I tried LSI SAS controller type but problem is I cannot locate driver for windows 2000. Is there driver available for LSI SAS controller type?

[christianZ]

Hi,

after p2v you should check the vm if there are any unused drivers (network, disk, etc.). In the dos window "set devmgr_show_nonpresent_devices=1" and then "devmgmt.msc".

You should remove any unused driver - but make a snapshot before  (on not running vm!).

What scsi driver do you see in the device manager?

What OS there?

Here you can try to set the vmware scsi driver (by windows 2003 - if not already done).

Reg

Christian

[niriwoing]

Hi Christian....have removed all phantom devices. Its windows 2000 advanced server. I also tried vmware scsi driver but no improvement.

[christianZ]

Well I checked my one of the older w2000 vms:

- bus logic scsi controller with vmware driver here; alternative could the lsi logic with original lsi logic 1020/1030 (as far I remember) - but problem with boot

(you could first add a second scsi controller and install the lsi driver)

- network flexible with vmware accelerated AMD PCNet driver

- VGA Vmware Svga II driver

- check the cpu number; less is better here (e.g. 2 are better than 4)

Under certain circumstonces this vm won't be running optimal.

Just my thoughts.

Reg

Christian

[niriwoing · Accepted solution]

So here is what happened....during migration we selected HW version 7 but after migration it was 4. After migration nobody noticed it (sad). But when we upgraded HW version to 7 latency was reduced. Queue length though was very high but TSQL performance is ok now.

So why did the HW version change to 4? Our guess, there was an attempt to virtualize these servers 2 years back using P2V. But it failed and when they rolled back vmware tools were not removed properly....we can see vmware tools service(automatic but not started) on physical server but installation directory is missing. So when we migrated it automatically changed hardware version to 4. Weird conclusion but this is what I think. Or is it that when you migrate win2k, converter automatically sets hardware version to 4?

Any ideas anyone?