New UDT (Unified Data Transport) in vSphere 8. When exactly does it come into play?

[UnivLorraine]

Hello,

I observe significant differences in transfer speed of Storage vMotion tasks, depending on whether the virtual machine (whose disks are to transfer) is powered-off or powered-off.

When the VM is powered-on, the transfer speed reaches 400 MB/s.
When the VM is powered-off, the transfer speed barely reaches 165 MB/s, that is 1.3 Gb/s, which is the exact limit of what NFC can provide, as stated here [1].

Here below are the tests I performed and the time taken to transfer 16 GB from one datastore to another (all ESXi in the cluster can see the source and the destination datastores and VAAI XCOPY is being used).

- VM powered-off, vMotion/SvM task is "Change storage only". Transfer time: 2'37"
- VM powered-off, vMotion/SvM task is "Change both compute resource and storage". Transfer time: 2'15"
- VM powered-on, vMotion/SvM task is "Change storage only" Transfer time: 59"
- VM powered-on, vMotion/SvM task is "Change both compute resource and storage" Transfer time: 1'08"

What I would like to know is if UDT came into play here or not and whether or not it should have.

VMware support says that UDT only comes into play when the vMotion/SvM task is of type "Change both compute resource and storage" and both ESXi don't see each other datastores (source and destination). I find it hard to believe.

My cluster is a vSphere 8 cluster with 'vMotion' and 'Provisioning' services enabled a vmkernel port group working on the 'Default' TCP/IP stack, so UDT requirements should be met.

Can someone please:

- enlighten me on how and when UDT intervenes?
- tell me if UDT brings an improvement for the turned off VMs that do not change host (Change storage only).
- tell me how to check from the logs that UDT is involved
- help me understand why the cold transfer speed is still lower than the hot one.

Best regards,
Frédéric.

[1] https://core.vmware.com/resource/vsphere-vmotion-unified-data-transport

[UnivLorraine]

Hello,

Replying to myself. UDT seems to work when vMotion/SvM task is of type "Change both compute resource and storage" and both ESXis don't see each other's datastores (source and destination) and of course when Provisionning service is set on vMotion interface.

Once the migration completes, /var/log/vmkernel.log of the destination ESXi should show a line like this one:

[root@esxi:/var/log] grep UDT vmkernel.log
2023-09-07T15:12:15.941Z In(182) vmkernel: cpu22:2099338 opID=8203293f)Migrate: 132: udtUUID = 52bb8e17-baa5-99c1-0986-91b805fd3754: Clean up UDT session

Not sure if UDT should also come into play under other conditions/circonstances.

Regards,
Frédéric.

[Igubu]

Hi,

Waited for U2 to be released - and made the change a few weeks ago. As part of the deep dive performance notes, also came across this and thought to test it on a few clusters.

Same result - source/destination storage needs to be unique  - so non shared - for UDT to work.  Doesn't seem to make sense to me as a feature, and neither the release notes or the few videos about it mentions the requirement, so have a case open with VMware to have a look at it - maybe they will release a fix, or update the documentation.

My tests all done with NFS storage, have not tested this with different types of storage yet.