ESX 7 NUMA problem

[wickyR]

Hello,

I have been given to analyse a problem related to a NUMA vCPU placement and memory allocation.

The physical server is based on Intel Xeon E7-8880 V4 having 2 Processor sockets and 22 processor cores per socket with 88 logical processors.

The output of the command sched-stats -t numa-clients shows very imbalanced distribution of vCPUs and significant remote memory access. (Please see attached image). The system has NUMA.PreferHT=1 already set.

My question is, why vCPUs are unevenly distributed and there is significant remote memory access? Where should I look for more information? 

Thank you very much for your time and answers.

Roshan

[vbondzio]

NUMA clients migrate across physical NUMA nodes due to changes in load, contention and for locality optimization. This doesn't have to be an issue but it might under some circumstances. See https://kb.vmware.com/s/article/2097369

For a better overview, run:

for numaOption in $(sched-stats -h | sed -n 's/.*:    \(n.*\)$/\1/p'); do echo -e "\nsched-stats -t ${numaOption}"; sched-stats -t ${numaOption}; done

and post the text on pastbin.com / link that here.

You might also want to run / pastebin the output of: https://github.com/vbondzio/sowasvonunsupported/blob/master/pci2numa.sh

[wickyR]

Thank you very much for your answer!