VMXNET3 :: "Real" performance gain for low-bandwidth server?

[virtualeye]

Hey all.

Running a Dell R610 ESXi 4.1 host with 32GB RAM and 6X quad core CPU.

Currently just have to guests, 2003 server, and a CentOS 5.7 web server.

Host load is minimal, web server does @60 to 100GB bandwidth per month.

Now, given the low-bandwidth, is there any performance gain in using vmxnet3 vNIC, or is e1000 the better call (and more reliable)?

Should mention that the host sits behind a Cisco ASA 5505, so 100MB connectivity via ASA on both DMZ and vmkernel subnet.

We also have a gigabit switch that both host and physical backup server connect to.

Ideas appreciated, we have vmxnet3 vNIC in place now, some tcp connections (smtp and tunneled 3306 connections) are incredibly slow, so looking to make some tweaks to get near-physical NIC performance

Thanks

[rickardnobel]

In your situation the VMXNET3 card would probably not increase the performance, you could use the E1000 if you like. However, could you describe the problem some what more? Some TCP sessions are slower than expected - what kind of throughput do you get?

[virtualeye]

@Ricnob,

we're getting terrible tcp performance for smtp traffic (as in 10-15 seconds to send a simple "hello" text email), and tunneled tcp traffic over port 3306 is fairly horrible as well. Downloading email over port 110/143 is also quite slow although not as bad as sending email.

Have read that disabling LRO can help improve TCP performance so will try that in the AM.

The host machine (Dell R610) is a complete beast spec-wise compared to our old bare metal machine (SC 1425), so lack of resources is not at all an issue (again just 2 VMs running, low-load, and on separate physical NICs to boot).

Should point out that we are also getting several hundred kernel errors/day on the vmxnet3 vNIC in CentOS guest. Today I discovered a fix for this issue which requires upgrading our host from 4.1 to 4.1 update 2; thus the question: is there any real gain/enhancement/stability/benefit/etc to using vmxnet3 over e1000 in a lightly loaded host?

I assume the answer is no, not really, but, you know, I like the bleeding edge, spent $5K on this machine and enjoy the thought of squeezing max performance out of it (even if that gain is not realized in web server performance, but something as mundane as copying multi-gigabyte vmdk files over the network to separate backup server)

Basically trying to justify the latest & greatest without shooting myself in the foot 😉

[rickardnobel]

I do understand the goal of total optimization. However, you seems to have quite serious problems with this machine and if changing to E1000 is the solution then of course do it. Both E1000 and VMXNET3 should be able to get very near wirespeed network traffic. VMXNET3 should cost less CPU and has some additional features, but in your light loaded servers this would not matter.

Add a E1000 card to the VMs and try. Just set some other IP on both VMs and try some network performance test.

[virtualeye]

Yes, agreed I should go e1000 route.

The problems are not catastrophic, web traffic is plenty snappy, the only complaint I have heard from clients is smtp, which I experience as well. Of course, I don't sit there waiting for emails to send, I do something else. Tunneling localhost over port 3306 is only for SqlYog and not really noticable except with large-ish query results, something that existed on the bare metal machine as well.

I'm assuming upgrading to update 2 is not a bad thing, nor is disabling LRO, which itself may be the cause of tcp issues (according to a vmware KB).

I'm going to give the above a try in the AM; if that does not fix tcp issues, then to e1000 it is.

In general was looking for what the actual benefits of vmxnet3 adapter is vs e1000 in non-enterprise-SAN setups (i.e. where you do not have a physical 10gb ethernet connection to work with).

If vmxnet3 will not hinder performance (after switch to update 2 and disable LRO at host level) and provide some auxilliary gain, perfect, at some point I'll find a real use for the latest & greatest. And if not, I'll be satisified with the stability of tried & true e1000 😉