Hi cgrace70,
Thank-you for you input,
Not sure if it was related but soon after we started seeing the issue we had a disk failure. Once swapped out the issues went away. Looking at our monitoring software both controllers are working correctly are being utilised with around 20/40% load.
Our configuration:
Two isolated switches - each on a different VLAN - 2 and 3
Configured Controller ports in the following manner:
Port 1 Cont A - VLAN 2 (IP 192.168.15.1)
Port 2 Cont A - VLAN 3 (IP 192.168.16.1)
Port 3 Cont A - VLAN 2 (IP 192.168.15.2)
Port 4 Cont A - VLAN 2 (IP 192.168.16.2)
Port 1 Cont B - VLAN 2 (IP 192.168.15.3)
Port 2 Cont B - VLAN 3 (IP 192.168.16.3)
Port 3 Cont B - VLAN 2 (IP 192.168.15.4)
Port 4 Cont B - VLAN 2 (IP 192.168.16.4)
On the ESX host we enabled two interface as VMKernals - 1 in 15 range and another in 16 range.
Set all LUNS to round-robin.
Another way that i have seen to-do this is to give each port a different VLAN:
Port 1 Cont A - VLAN 2 (IP 192.168.2.1)
Port 2 Cont A - VLAN 3 (IP 192.168.3.1)
Port 3 Cont A - VLAN 4 (IP 192.168.4.1)
Port 4 Cont A - VLAN 5 (IP 192.168.5.1)
Port 1 Cont B - VLAN 2 (IP 192.168.2.2)
Port 2 Cont B - VLAN 3 (IP 192.168.3.2)
Port 3 Cont B - VLAN 2 (IP 192.168.4.2)
Port 4 Cont B - VLAN 2 (IP 192.168.5.2)
We did not try this as our ESX boxes only had two interfaces that we could assign to iSCSI.
We tested the performance of the SAN within VMWARE using Iometer. I found the perfomance to be worse than expected. After some research we found the following:
Create smaller logical drives (4+1, 6+1) rather than a single logical drive across all disk. (unless IOPS is key requirement for a single application) - We found that if a single logical drive was used, over time as more VM's are added the contention to read/write affected performance as the same 24 spindles are used by all devices. By using smaller logical drives - this isolated contention to the spindles allocated within the logical drive. If utilisation was high for a specific application and multiple logical drives have been created then it does not affect other VM's perfomance if located on another logical drive.
Also create Multiple LUNS within the Logical drive - We found that the LUN is locked during Read \ Write operation. So multiple Luns allowed for better throughput
Another issue we found was a VMWare one, when testing 100% sequential read the most throughput we could get was 100Mbits ps. Using the CLI we set the IOPS to 1 across the Luns which increased the performance to 200Mbps. We found that when using round-robin - VMWARE sent 1000 iops across one interface then another 1000 down the other. After setting it to 1 it spread the load after each IOP increasing the performance
Not sure if this will help you in any way, but thought i would share it - if you have any question please ask
Tim
