Not sure if it's the right place but it's definitively network related so here goes.
The environment:
VCF4.2 build and setup following the latest VVD. The only thing which we altered was to not use BGP as the physical layer currently does not support it. So we setup static routing instead on NSX-T. This shouldn't be a problem as the workload are running on VLAN segments, not overlay. We have a Mgmt domain with 4 nodes and a WL domain with 8 nodes.
The issue:
Migrating a VM from the WL domain to the Mgmt domain breaks the network connection. On the WL domain, the VM is connected to a NSX-T VLAN segment. On the destination, the Mgmt domain, it's connected to a traditional DVS portgroup.
After the XVM (cross vcenter vMotion), sometimes the NIC will be disconnected and I'm not able to connect it and get the following error:
Failed to connect virtual device 'ethernet0'.
Sometimes the NIC will be disconnected as if nothing is wrong, but in the guest OS, it says cable unplugged (Ubuntu).
The workaround:
If I can't connect the NIC, changing the NIC to another network solves the issue.
If there is no connection in the Guest OS, I need to power off the VM, vMotion it to another host and power it on again.
I already had a look with VMware Support and we noticed, using packet capture, that the ESXi host is sending out ARP requests but never gets a reply. So they concluded it's not VMware related and that the problem is somewhere on the physical layer.
This feels like the same issues we are having migrating VMs from the old VCF2 environment to VCF4 as explained in another post.
