Intel Pro/1000 PT Quad PCIe doesn't work...

[jbsengineer]

Just an FYI. There were some threads about this but I couldn't find them.

We got some new Dell 2950's in with the PCIe architecture. With Intel having the new Quad PCIe NIC we decided to wing it and see if it wouldn't atleast work in our test labs. From what I've read it's the same chipset. However it doesn't work. ESX won't load a driver for it.

So my question is how do I let VMware know that atleast I would love to see it supported in the next release?

[Paul_Lalonde]

If you go into /etc/vmware and edit the vmware-devices.map file (ie. change the PCI device ID for an existing Pro/1000 adapter entry to the device ID of your PCIe card) you should be able to get it to work.

Paul

[jbsengineer]

Good thinking Paul. Unfortunately ESX still doesn't seem to load the driver for it. I was easily able to get the device ID from /proc/pci and changed the vmware map file though.

Any other files need to be changed?

[Paul_Lalonde]

What happens if you force the load of the driver?

Ie. vmkload_mod /usr/lib/vmware/vmkmod/e1000.o debug=5

Does it show up?

Paul

[jbsengineer]

Error:

"Failed to load module: Invalid argument"

Tried with and without debug=5.

Josh

[jbsengineer]

Error in the systems log is:

kernel: e1000: 10:00.0: e1000_sw_init: Unknown MAC Type

Any ideas? You are close!

[jbsengineer]

and the VMKERNEL log says:

"Initializationg failed with status -19" on the e1000 module.

Darn

[Paul_Lalonde]

Hi,

Can you determine which Intel chipset is used on that PCIe card? If it's reasonably compatible with the e1000 driver, I might be able to add support for it to the e1000.o vmkernel module.

Let me know!

Paul

[Rumple]

Off topic, but...

methinks Paul needs to go work for vmware...

Or else he used to....

[jbsengineer]

The Dual Intel Pro/1000 PT PCIe which is supported runs the 82571eb(We currently have these available to put in the system). The quad runs two 82571gb's.

Link to the dual specs:

http://www.intel.com/network/connectivity/products/prodbrf/pro1000_pt_dualport_server_adapter.pdf

Link to the quad specs:

http://www.intel.com/network/connectivity/products/prodbrf/314100.pdf

Good luck, !

[Paul_Lalonde]

Excellent... thanks for that info. I'm poring through the driver source and am off to a good start. But I need to know the PCI device ID for the 4-port card... do you have this on-hand?

Thanks!

Paul

[Paul_Lalonde]

Hey Rumple... greetings from Hamilton, Ontario!

Nope, don't work for VMware.... just a guy applying age-old logic (and rusty fifteen-year-old C skills) to a relatively simple problem!

Regards,

Paul

[jbsengineer]

device id= 10a4

[Paul_Lalonde]

Perfect... that's exactly the device id listed in the Intel Windows driver .INF file I downloaded, so at least there aren't any variations of it.

I'll plug this into the code and post a patch within a few hours. Stay tuned!

Regards,

Paul

[Paul_Lalonde]

Ok, here you go. Download this patched e1000 driver:

http://home.cogeco.ca/~plalonde2/e1000.tgz

Uncompress it in a temporary folder:

tar xfz e1000.tgz

Make a backup copy of your existing ESX e1000.o vmkernel driver:

mv /usr/lib/vmware/vmkmod/e1000.o /usr/lib/vmware/vmkmod/e1000.bak

Copy the patched e1000.o into the vmkernel module directory:

cp e1000.o /usr/lib/vmware/vmkmod/

Set read-only permissions on the module:

chmod 444 /usr/lib/vmware/vmkmod/e1000.o

Add device support for this Intel NIC:

echo device,0x8086,0x10a4,nic,82571GB Quad Gigabit Ethernet Controller,e1000.o >> /etc/vmware/vmware-devices.map

Restart your server and voila! Your quad-port Intel GigE PCI-Express card should work!

Let me know how it works.

Regards,

Paul

[jbsengineer]

IT WORKS! I'm impressed Paul. If I could shake your hand I would. Very impressed!

The only difference I see that would make it appear "unsupported" after the reboot was that in the Network Adapters through the VI Client the Adapter is labeled as "Unkown". But it works just fine. I was able to vmotion over a few machines that used the port group on the Quad.

Well DONE!!!

Thanks Paul!

[Paul_Lalonde]

Glad to hear it works!

However, I'm curious to see if you reboot your VirtualCenter whether or not the gigabit interface gets 'labelled' properly.

Also... worst case, try removing the ESX from the VC database and then adding it back in. I have a feeling this is a setting on the VC side, not the ESX side.

Further, what happens if you use the VI Client to connect directly to the ESX Server IP, as opposed to the VC Server IP? Does the label show up?

Anyway, glad it's working for you! The addition I made simply provided for detection of the device ID 0x10A4 and should not impact reliability or cause support issues.

Regards,

Paul

[Paul_Lalonde]

Also, double check the /etc/vmware/vmware-devices.map entry you added and just make sure it looks proper. Perhaps the label field got jumbled up?

Paul

[jbsengineer]

I had already tried to directly connect the VI Client to ESX and it showed the same Unknown. The vmware-devices.map entry looks fine. Must get that information from somewhere else.

[Paul_Lalonde]

Can you ensure the following lines are included in your /etc/modules.conf file?

alias eth0 e1000

alias eth1 e1000

alias eth2 e1000

alias eth3 e1000

It looks like ESX may pull device name information from loaded Linux kernel modules...

Try this and let me know. Thanks!

Paul