ESXi 8u1 and BCM57412 NetXtreme-E 10Gb problem

[tomvitale]

hi there,

I've two phisical NICs enabled, one Broadcom Corporation NetXtreme BCM5720 that's work perfectly:
1) vmnic0 , ntg3 , 00:11:22:33:44:55 , Enabled , 1000 Mbps, full duplex

and a BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller not working properly:
2) vmnic5 , bnxtnet , 11:22:33:44:55:66 , Enabled , 10000 Mbps, full duplex

ESXi 8.0 Update 1 recognizes it and shows me that it goes to 10000Mbps. I created a vSwitch and a Port group and associate it with a new virtual machine, but it doesn't seem to communicate with "the outside" (other device in the same vlan).

Do you have any ideas or tests to perform?

- - - - - - - -

[root@esxi-host:~] esxcfg-nics -l
Name PCI Driver Link Speed Duplex MAC Address MTU Description
vmnic0 0000:18:00.0 ntg3 Up 1000Mbps Full 00:11:22:33:44:55 1500 Broadcom Corporation NetXtreme BCM5720 Gigabit Ethernet
.....
vmnic5 0000:5e:00.1 bnxtnet Up 10000Mbps Full 11:22:33:44:55:66 1500 Broadcom BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller

[a_p_ · Accepted solution]

Without further details, one can just guess.

It could be related to a configuration issue, like e.g. tagged ports on the physical switch, and no/wrong VLAN-ID on the port group (or vice versa), as well as a possible driver issue (see https://kb.vmware.com/s/article/93580).

André

[acpzehetal · Accepted solution]

Without any further logs or informations an educated guess is the Broadcom firmware.

in the last few months there were a lot of posts mentionning bugs in several versions of BCM 574xx NICs.

Sometimes the settings of "Autonegotiate speed and dublex" are not working or the VLAN ID flag has to be reconfigured. Even a wrong or not supported cable can produce that issues.

https://techdocs.broadcom.com/us/en/storage-and-ethernet-connectivity/ethernet-nic-controllers/bcm95...

Unfortunately there are a lot of causes. If you can provide some logs from the host, we can narrow down the errors origin.

[degvm · Accepted solution]

Please use https://kb.vmware.com/s/article/1027206 to identify driver and Firmware of the cards:

esxcli network nic get -n vmnic5

compare the driver and firmware with the following, stated from https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=44441&vcl=true

bnxtnet version 223.0.192.0 Firmware 22.31.11.61

https://customerconnect.vmware.com/en/downloads/details?downloadGroup=DT-ESXI80-AVAGO-BROADCOM-BNXT-...

That might be the right firmware/driver combination. Other possible issues might be VLAN tagging, but also the combination of switch settings with the right/wrong cable. Please check cable type and Tranceiver/SFP.

[tomvitale]

thanks @a_p_ @degvm  @acpzehetal 

I followed the https://kb.vmware.com/s/article/93580 and upgraded the network card driver (and check firmware/driver combination) and set the right vlan value. Now it works perfectly, thanks again to all three of you