Intel(R) Ethernet Controller E810-C esx.problem.net.connectivity.lost on boot

[vencatoitc]

Hi all,

something weird happens after deploying two hosts (Dell PowerEdge R650xs) direct attached to a SAN (Dell PowerStore 500T) via Intel(R) Ethernet Controller E810-C for SFP; when they boot trigger the following errors visible on the VCSA and in host's /var/log/vobd.log:

2022-09-07T09:59:01.701Z: [iscsiCorrelator] 189191497us: [vob.iscsi.discovery.connect.error] discovery failure on vmhba64 to 192.168.121.10 because of a network connection failure.
2022-09-07T09:59:01.701Z: [iscsiCorrelator] 189192359us: [esx.problem.storage.iscsi.discovery.connect.error] iSCSI discovery to 192.168.121.10 on vmhba64 failed. The iSCSI Initiator could not establish a network connection to the discovery address.
2022-09-07T09:59:01.701Z: An event (esx.problem.storage.iscsi.discovery.connect.error) could not be sent immediately to hostd; queueing for retry.
2022-09-07T09:59:01.712Z: [iscsiCorrelator] 189203183us: [vob.iscsi.target.connect.error] vmhba64 @ vmk2 failed to login to iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-a-1ff956b8 because of a network connection failure.
2022-09-07T09:59:01.712Z: [iscsiCorrelator] 189203533us: [esx.problem.storage.iscsi.target.connect.error] Login to iSCSI target iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-a-1ff956b8 on vmhba64 @ vmk2 failed. The iSCSI initiator could not establish a network connection to the target.
2022-09-07T09:59:01.712Z: An event (esx.problem.storage.iscsi.target.connect.error) could not be sent immediately to hostd; queueing for retry.
2022-09-07T09:59:01.720Z: [iscsiCorrelator] 189210815us: [vob.iscsi.target.connect.error] vmhba64 @ vmk2 failed to login to iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-b-151fcaaf because of a network connection failure.
2022-09-07T09:59:01.720Z: [iscsiCorrelator] 189211081us: [esx.problem.storage.iscsi.target.connect.error] Login to iSCSI target iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-b-151fcaaf on vmhba64 @ vmk2 failed. The iSCSI initiator could not establish a network connection to the target.
2022-09-07T09:59:01.720Z: An event (esx.problem.storage.iscsi.target.connect.error) could not be sent immediately to hostd; queueing for retry.
2022-09-07T09:59:07.392Z: [netCorrelator] 194882715us: [vob.net.pg.uplink.transition.down] Uplink: vmnic3 is down. Affected portgroup: ISCSI Network 2. 0 uplinks up. Failed criteria: 128
2022-09-07T09:59:07.392Z: [netCorrelator] 194882752us: [vob.net.vmnic.linkstate.down] vmnic vmnic3 linkstate down
2022-09-07T09:59:08.003Z: [netCorrelator] 195494442us: [esx.problem.net.connectivity.lost] Lost network connectivity on virtual switch "vSwitch-Iscsi1". Physical NIC vmnic2 is down. Affected port groups: "ISCSI Network 1"
2022-09-07T09:59:08.003Z: An event (esx.problem.net.connectivity.lost) could not be sent immediately to hostd; queueing for retry.
2022-09-07T09:59:08.270Z: [netCorrelator] 195761333us: [vob.net.vmnic.linkstate.up] vmnic vmnic3 linkstate up
2022-09-07T09:59:08.371Z: [netCorrelator] 195862168us: [vob.net.pg.uplink.transition.up] Uplink:vmnic3 is up. Affected portgroup: ISCSI Network 2. 1 uplinks up
2022-09-07T09:59:08.993Z: [iscsiCorrelator] 196483644us: [vob.iscsi.target.connect.error] vmhba64 @ vmk2 failed to login to iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-b-1e8fb719 because of a network connection failure.
2022-09-07T09:59:08.993Z: [iscsiCorrelator] 196483812us: [esx.problem.storage.iscsi.target.connect.error] Login to iSCSI target iqn.2015-10.com.dell:dellemc-powerstore-crk00222107199-b-1e8fb719 on vmhba64 @ vmk2 failed. The iSCSI initiator could not establish a network connection to the target.
2022-09-07T09:59:09.002Z: [netCorrelator] 196493261us: [esx.problem.net.connectivity.lost] Lost network connectivity on virtual switch "vSwitch-Iscsi2". Physical NIC vmnic3 is down. Affected port groups: "ISCSI Network 2"
2022-09-07T09:59:09.002Z: An event (esx.problem.net.connectivity.lost) could not be sent immediately to hostd; queueing for retry.
2022-09-07T09:59:10.002Z: [netCorrelator] 197493087us: [esx.clear.net.connectivity.restored] Network connectivity restored on virtual switch "vSwitch-Iscsi2", portgroups: "ISCSI Network 2". Physical NIC vmnic3 is up

The issue is that the alarm is (red) visible on the host page in VCSA interface but not in the ESXi interface.
So, could those multiple entries "Event (...) could not be sent immediately to hostd; queueing for retry." be the cause? Sort of "I (ESXi) got an error and I queue it because VCSA has not been started yet; the error got resolved but I (ESXi) still have the error queued to be sent via hostd"?

Thanks,
Alessandro