VM memory

[pearlyshells]

I've recently attended the VMware Course and while I "thought" I understood the memory resourcing section, after reading more about it I'm not absolutely sure I did.

At the crux of my confusion is the term "swap file". It appears to me that there are two swap files that I need to separate. The first is the Swap File in the Guest O/S and the other is what VMware terms the Virtual Machine Swap File. (actually, I'm just validating my thoughts here)

If I understand what I'm reading, the Balloon driver (vmmemctl) is associated with reclaiming Unused memory from the Guest O/s but the ballooning technique actually relies on a Swap File (this would be the Virtual Swap File....difference between the Allocated VM memory and the Reserved memory). This Virtual Swap File must be equal to or greater than the difference between the Allocated and reserved Virtual memory and is (and, this is the part that I'm not sure of) located on the VM hard disk (vmdk file?) and the swap file is colocated with the VMX file. This is a MUST since the ballooning technique will actually reclaim unused memory (presumably back to a larger ESX pool for other VMs) and cause the guest to use the Virtual Swap File on the local disk to think it still has that memory. And, then when the VM actually needs more memory it then uses ballooning to make the Guest think it needs some of the memory from its Virtual Swap file and then gets back the reclaimed memory from the larger pool. Is that correct?

So, the question is: if this is correct....................what good is the guest O/S virtual memory settings?

Also, all my VMs were configured with defaults and the VM memory settings are set to an "unlimited" LIMIT. Based on my reading, this may not be a good thing and I should really be customizing the LIMIT based on what I allocate to the VM (i.e. if I allocate 512mb....I should set the limit to , at least, 1024 or greater). Is this also a correct presumption?

Thanks in advance.................from a Novice ESX admin.

[AntonVZhbankov]

Baloon driver is used to reclaim memory, and it absolutely doesn't care if memory is used or not.

Let's take spherical VM in vacuum. VM has 1 GB RAM configured, and 512 reserved. That means: this VM will always have 512MB of allocated physical memory. What happens when physical memory on host is 100% loaded? Host juct can't give VMs more memory and we have to choose which VM is more important than others.

This is done through shares mechanism. Let's assume we have no overhead, so all the memory on host is available for VMs. 10GB (total memory) minus 5GB (sum of all reservations) = 5GB to share between VMs. So each VM have some weight - 'shares', ESX sums all shares and divide memory based on shares.

5GB (available memory) = 40 000 shares (total shares sum). By default VM has 2 000 shares (if I'm not mistaken), so ESX will give this particular VM 5GB / 40 000 * 2 000 = 256 MB of physical memory. 768MB of physical memory available for our VM, but it thinks that it has 1GB! And then balloon driver inflates and take this 256MB.

Actually it just says to ESX - 'I took 256MB' and ESX subtracts 256MB from 'memory required'. Guest OS sees that 256MB is already allocated, so there is need for swapping. Why can't we use external swap (on ESX level) first? Because ESX doesn't know about all these memory structrures inside VM, and we lay the burden on guest OS to decide what it can move to swap first and it cannot move no matter what the reason. ESX can see if Guest VM actively use 100% memory, but it can't see what is there. Maybe there is 800MB disk cache which can be halved and no one notice - guest OS only knows.

After all tools exhausted (or maybe we just forgot to install vmware-tools) ESX swapping starts.

What is limit? VM can not get resources more than a limit even if there are no other VMs and ESX is 5% loaded. For example, if VM has 1GB configured and 512MB limit, then there always be 512MB balloon. Do not use it unless you are a kind of hosting provider and you want to allocate resources precisely and dynamically without VM shutdown. And even then think twice and don't forget you've limited some VM.

---

VMware vExpert '2009

http://blog.vadmin.ru

[vmmeup]

Just to add you should be aware that in ESX when you power on a VM it's created a vswap file for the VM that is the same size as the amount of RAM you give the VM. This is important to know because it plays in to sizing datastores appropriately to accommodate the vswap files for VM's. Here is more information on this:

Sid Smith

-

---

VCP, VTSP, CCNA, CCA(Xen Server), MCTS Hyper-V & SCVMM08

• Don't forget to award points for correct and helpful answers.;-)

[pearlyshells]

Anton

Thanks very much for your help. Can I ask one more question of you? I know that as a VM powers up that there is going to be ballooning of the memory on the VM. However, in one of our VMs, our NetIQ AppManager also shows that during this time there is excessive paging on this VM.

Is this to be expected? In reading the paper provided by another respondent and by your comments, I get the impression that ballooning is supposed to alleviate paging. So, if the VM has been allocated enough memory, ballooning should not induce paging, correct? So, I'm thinking I need to boost the memory on this VM.

[avlieshout]

First to correct vmmeup a little bit. The size of the vm swap file is equal to the vm's limit minus the vm's Reservation (vSwap size = Limit - Reservation)

Ballooning is a mechanism utilized by ESX te reclaim vm memory. By inflating the balloon if forces the VM to start paging (so this behaviour is expected) if there is not enough memory on its free list.

If there is no memory pressure inside the vm, then there is no need for the OS to page out memory

Bottom rule. If you see ballooning happening then you are running out of memory resources on ESX host / or resource pool level.

You don't want ballooning to happen because it impacts performance, but it's allways "better" than ESX host swapping (using the vSwap file).

-Arnim van Lieshout

-

---

Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[AntonVZhbankov]

Ballooning is intended to start paging on guest OS level instead of ESX. Because guest OS knows better what to page.

If memory load level for this particular VM is less than "Configured minus ballooned" then paging won't start.

To inflate balloon in unused memory and don't affect loaded VMs there is "unused penalty". When counting memory unused page costs by default 3 times more than used.

---

VMware vExpert '2009

http://blog.vadmin.ru

[pearlyshells]

Interesting. Didn't get that from any of my readings. So, I should ignore the Excessive paging message from NetIQ AppManager.

[pearlyshells]

herein where I get confused or , perhaps, I am misunderstanding.

Some say that ballooning does not reclaim memory.

the paper indicates (or, I interpret it to say) that ballooning occurs BEFORE paging begins and I interpret that to mean that it's a good thing.

I see in the paper that the swapfile size is as you indicate while it also says that when a VM powers up it automatically creates the swapfile and the size is equal to the RAM (or, did I get that from someone in the forum)

Or, am I mixing up terms??

[avlieshout]

Vswap file Size is allways Limit - Reservation. In the most configurations this equals to the RAM size.

You should not ignore the excessive paging as it is an indicator that the vm has insufficient memory. You should give more memory to the vm or set a reservation on it.

But beware!. If you see ballooning your system, it is suffering from memory pressure and you should look into that first. Try adding more memory to your ESX host or add another server in the cluster.

If you using Resource pools see if you can add more memory to the resource pool.

If you don't look into that first, giving the specific vm more memory is like moving the paging problem to another vm.

the paper indicates (or, I interpret it to say) that ballooning occurs BEFORE paging begins and I interpret that to mean that it's a good thing.

The paper is referring to ESX paging here (or swapping). Don't mix up ESX paging with OS paging. And indeed ballooning is a good thing in contrast to ESX paging, because it is much less tedious to the vm's performance as ESX paging.

-Arnim van Lieshout

-

---

Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[pearlyshells]

The VM is on an ESX host with 32gb of RAM. This host is part of a cluster (HA/DRS). The cluster has 64gb RAM. the total memory being allocated to all the VMs in the cluster is 5gb. So, it appears to me that there is plenty of memory in the cluster and the host.

As much as I am reading the Resource manual and other papers along with the advice I am getting, it is obvious to me that Resourcing ESX is complex and a bit hard to understand for a novice like me. What I do understand is that ballooning is a normal process , at least, during power up. My concern is whether the NetIQ AppManager Excessive Paging is normal alongside the power-up ballooning that is occuring. If the ballooning, is indeed 3x more during this time, then I can understand that paging will occur and that I could ignore it during power up (at least, the NetIq message).

the confusion for me is : when is ballooning not good, as I understand that it does occur when memory is NOT used (where the ballooning expands to claim the unused memory) or when more memory is needed by the VM (then the ballooning shrinks and it then reclaims memory, I think). But, from what you mention during this process, it should NOT page. So, if it induces Paging "at any time", it is a matter of concern. Am I correct?

[AntonVZhbankov]

Here is detailed description how ESX manages memory: http://www.vmware.com/pdf/esx3_memory.pdf

---

VMware vExpert '2009

http://blog.vadmin.ru

[pearlyshells]

Thanks Anton

This is the paper I've read. The confusion is the interpretation and comments I'm getting .....may be that all are correct and I'm the one misinterpreting the information

[avlieshout]

IMHO ballooning is allways BAD.

I never see ballooning on my vms when starting.

To view the ballooning activity on your ESX host:

run esxtop in your ESX console, then press "m" for memory statistics.

Press "S2" to refresh every 2 seconds

Then press "f" and then "i" to show the vmmemctl (ballooning) columns

MCTL?: is the ballooning driver active?

MCTLSZ: the size in MB that is actually reclaimed by the ballooning driver

MCTLTGT: the size in MB that is going to be reclaimed (targetted memory)

MCTLMAX: the maximum size that the ballooning driver can reclaim. Default is 65% of assigned memory.

Do you see any ballooning here?

If so, that shouldn't be there as far as the info you gave me on your environment.

Reclaiming memory is the process of the ESX host to get back memory it gave out to the VM. ESX has two methods for this:

1) ballooning

2) ESX host swapping(paging)

If memory is NOT overcommitted there is NO NEED for the ESX host to reclaim memory. From the information you gave there is no overcommitment.

You can check your ESX hosts's memory overcommitment on the first line of esxtop.

The number showed here is a percentage (e.g. 0.20 means 20% overcommitment). This should be 0.00 in your environment as you describe it here.

-Arnim van Lieshout

-

---

Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[AntonVZhbankov]

Ballooning should happen only 100% loaded host. If you got "Memory Ballooned" > 0 in situation when you have 5GB VM on 32GB host, check "memory limit" for VM and set it to "unlimited"

---

VMware vExpert '2009

http://blog.vadmin.ru

[pearlyshells]

This is all very revealing and enlightening for me. Thanks for being so patient

I checked and used ESXTOP as you suggested. I checked the VM information:

MCTL?: N

MCTLSZ: 0

MCTLTGT: 0

MCTLMAX: 507.00

But, this is "current" information and High Paging messages no longer are displaying in our NetIQ. The only time I have seen these messages is when the VM is powered on.

Thanks very much.

[pearlyshells]

The limit on the VM is already set to unlimited.

[avlieshout]

I see MCTL? = N. This means that there is no ballooning driver active inside your VM. So there will be no ballooning only ESX host swapping when there is memory pressure.

I suggest installing the VMware tools inside the vm.

-Arnim van Lieshout

-

---

Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[pearlyshells]

this gets more confusing. Accordng to my VC, VMtools is already installed. And, when I launch the VM the tools appear , in fact, to be loaded.

[avlieshout]

Are you sure you looked at the right line in esxtop? Maybe you looked at a line above or below by mistake.

-Arnim van Lieshout

-

---

Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[pearlyshells]

I redid ESXTOP and you must be right......it is Y