Hello everyone,
I'm planning to introduce an iSCSI solution in my setup, and wanted to fly the design plan by you to see if it makes sense and if anyone has any suggestions for improvement.
Current situation:
2x Quad Core Dell sc1430 server, with 8 GB ram, single integrated GB network adapter, internal 2610sa SATA RAID controller (6 ports), and 4x 80GB SATA disks in RAID 10 configuration. There are currently 3 of these in use, running around 35-40 VMs altogether.
All VMs are configured with 2 VCPUs, 256 MB ram, and are heavily loaded (75% - 80% CPU utilization) for about 8 hours every day. For other 16 hours they sit there turned on but don't do much.
For each VM, disk activity peaks at around 5 Mbits per second (writes), not much on disk reads (except on startup, and these don't get rebooted for months). Network traffic is also negligible.
Because of nature of work they perform, and different scheduling times, it is important for me to be able to dynamically balance the load across hosts. Currently I do this manually and even though it's not the end of the world, I'm planning on expanding my setup with at least 3 more servers of same configuration (short term, more (up to 10) may be expected within a year or so), and growing number of VMs to at least 75 (short term), therefore I figured centralized storage and dynamic load balancing would be in order.
Situation after expansion:
- 6 servers of above config, with single SATA drive used for booting ESX, not used for any VM storage.
- Servers equipped with additional 100Mbps NIC for their regular network traffic
- 1Gbps NIC used exclusively for iSCSI traffic
- Main storage box and the center of the universe: storage server running as iSCSI target :
- 20 SATA 80GB hotpluggable drives
- 3x 2610sa controllers, each in charge of 6 SATA disks in RAID10 configuration (usable space rougly 3x80GB) (total 18 disks). VMs are spread equally across these 3 arrays.
- 2x 80GB SATA disks in mirror used as boot for storage server
- Two 1Gbps NICs, one used exclusively for iSCSI network, the other for regular network traffic.
- iSCSI network will run on completely separate gigabit network, no other network traffic.
I am undecided about what OS/iSCSI target software to run on storage server. Bear in mind I'm solely responsible for this setup and non-Windows skills are on the "can follow step-by-step written instructions" level. Preference is Windows, however if performance is double under some preconfigured eaasily deployable iSCSI enabled flavor of Linux I may consider that.
I am also debating what software to use for iSCSI target if I go Windows route. There are a few that are available, I haven't had a chance to do a decent performance test with any of them, and certainly not with planned setup. Any input here is greatly appreciated.
Also, I have no idea what to expect in terms of CPU load on iSCSI target storage server. As I'm building this from chassis and available parts, I'm planning to canibalize AMD 64 3200+ single core CPU with 1,5 GB ram from another machine. Obviously this being as important as it will be, I can beef this up as required. Chassis supports standard server boards, so even dual socket is an option.
Notes:
- I am not concerned with single disk in ESX hosts, I can tolerate losing a host to disk failure as long as my VMs get restarted on other available hosts.
- With 75 VMs writting at 5Mbits/sec, I'm looking at ~400 Mbits = 50MBytes / sec requirement on the iSCSI box. While I'm sure 3 arrays will handle this easily, I'm not sure at all if a single Gb NIC can handle all that (SCSI gets encapsulated into TCP and is sent over network, there is overhead here, don't know how much).
Should I be looking at additional Gb NICs on iSCSI box side, and somehow create more capacity? Fatpiping, direct crossover to hosts using multiport NICs? What about mainboard bus, can it transfer all that data from disks to NICs?
Many thanks to all who can share their knowledge and experience with these things.
Cheers
