ESXi CPU question

[ytlevine]

I have a server/computer running ESXi 5.1 (was originally set up with 5.0, then upgraded to 5.1, if it makes a difference).

I have two VM's on there - a Windows 7 Pro VM, and a CentOS/cPanel VM.

I have a AMD A6-3650 Quad Core 2.60GHz CPU in the machine (I bought it from another company, so I couldn't choose the hardware), and it shows up as 4 cores in ESXi.

I originally had both machines set to use all 4 cores, but I was having issues on the CentOS/cPanel VM and was getting high CPU usage emails, so I changed the Windows 7 VM to only use 2 cores, but the CentOS/cPanel VM is still set to use all 4 cores.

So I have 2 questions:

In general, is there any reason not to have VM's overlap with the amount of CPU cores that they're assigned to? Meaning, in theory, is there any reason why not to assign two VMs to both use all the available CPU cores?

Also, if I want to give the CentOS/cPanel machine more CPU usage, am I better off keeping it like it's currently set up, and to have that VM assigned to all 4 cores, and have the other VM assigned to just 2 cores? Or should I be setting the CentOS/cPanel VM to use either 2 or 3 cores, and set up the Windows 7 VM to use 1 or 2 cores, so that they don't overlap?

Thanks!

[chriswahl]

In general, is there any reason not to have VM's overlap with the amount of CPU cores that they're assigned to? Meaning, in theory, is there any reason why not to assign two VMs to both use all the available CPU cores?

The core's thread can only be scheduled by one VM at a time. If a particular VM is using a high quantity of compute resources, the remaining VM and hypervisor will be starved for CPU. Additionally, the 4 vCPU VM has to schedule all four vCPUs at the same time, leaving nothing for the hypervisor and other VM. I typically like to make sure that the hypervisor has one core to itself.

Also, if I want to give the CentOS/cPanel machine more CPU usage, am I better off keeping it like it's currently set up, and to have that VM assigned to all 4 cores, and have the other VM assigned to just 2 cores? Or should I be setting the CentOS/cPanel VM to use either 2 or 3 cores, and set up the Windows 7 VM to use 1 or 2 cores, so that they don't overlap?

Because you only have 4 cores total, I wouldn't advise creating a 4 vCPU VM as it will rob the entire server of compute power. Sizing beyond that will simply entail monitoring the usage of compute by your VMs and adjusting accordingly. If the CentOS VM really needs 4 entire cores, then you will need a beefier CPU. Start small (1 core) and work your way up until the resources required by the VM are met.

[ytlevine]

Thanks for the reply.

So if I were to set the CentOS/cpanel VM to use 3 cores (I could use 2, but I would assume I'll need more), how many should I set the Windows VM to use?

Can I still use 2?

Thanks!

[frankdenneman]

In addition to Chris his answer, be aware that CPU scheduling is a transient process in ESX.The moment the vCPU does not use a CPU, it will be descheduled from the pCPU and the VMkernel allows other vCPUs to be scheduled. Therefore Virtual machines can contain multiple virtual processors. In the past vSMP virtual machine could experience latency due to the requirement of co-scheduling. Co-scheduling is the process of scheduling a set of processes on different physical CPUs at the same time. In vSphere 4.1 an advanced co-scheduling (relaxed co-scheduling) was introduced which reduced the latency radically.

However ESX still needs to co-schedule vCPUs occasionally.This is due to the internal working of the Guest OS. The guest OS expects the CPUs it manages to run at the same pace. In a virtualized environment, a vCPU is an entity that can be scheduled and unscheduled independently from its sibling vCPUs belonging to the same virtual machine. And it might happen that the vCPUs do not make the same progress.

If the difference in progress of the VM sibling vCPUs is too large, it can cause problems in the Guest OS. To avoid this, the CPU scheduler will occasionally schedule all sibling vCPUs. This behavior usually occurs if a virtual machine is oversized and does not host multithreaded applications. The "impact of oversized virtual machine series” offer more info on right-sizing virtual machines.

http://frankdenneman.nl/memory/impact-of-oversized-virtual-machines-part-3/

http://frankdenneman.nl/vmware/impact-of-oversized-virtual-machines-part-2/

http://frankdenneman.nl/vmware/impact-of-oversized-virtual-machines-part-1/

http://frankdenneman.nl/memory/sizing-vms-and-numa-nodes/

[ytlevine]

The information all makes sense, but then how am I supposed to best configure those two VMs?

Thanks.

[jdptechnc]

Having some overlap in and of itself is not a huge deal, but for the reasons that others have stated, it is a good practice to only assign the number of CPU cores that are really necessary, especially if you are constrained in the number of total physical cores available.  If you had more cores to work with, or if you were in a cluster that could dynamically reallocate a VM to a host with less contention, it wouldn't be as big of a deal.

I would try to scale back your cPanel to 2 vCPUs and see how it performs.  If it performs acceptably, leave it there; as you will enounter less memory overhead with less cores per VM.  How to best configure your VM for performance is trial and error, but it is best practice to start off with less and increase as needed.