VMware VMUG Community
Lars_Laursen
Contributor
Contributor

Virtual machine memory usage

Hi

I have a virtual machine running on vSphere 4.0.0, 164009

Guest OS = Windows Server 2008 R2

VM Version = 7

Memory = 4096 MB

When looking at the Resource Allocation tab in vcenter, I can see that it currently has 3,8 GB Active, this is generates an alarm "Virtual machine memory usage"

When I look in the OS, I can only see that the virtual machine is using around 1 GB RAM.

Why is my Active amount of RAM so high ?

Any explanation is apreciated.

Regards

Lars

0 Kudos
13 Replies
dsoltesz
Enthusiast
Enthusiast

Lars,

If you open up Windows Task Manager you may see that the Active Memory as represented by Microsoft does not agree with vCenter.

This can be for a few reasons.

1.  After a reboot/startup event Windows touches all of the memory, triggering the vcenter alert event.  The event should clear after around 20 minutes when vCenter polls what memory has changed/touched since the last poll.

2.  The amount of memory listed as used in the graph Microsoft gives does not include the system cache.  It would be foolish for any OS not to take advantage of all the memory for both user applications and system cache.  This will lower the number of SAN/storage hits, which is far slower than memory.  To see how much memory is being used add in the memory listed as cached (also in task manager) to the number presented in the task manager graph.  Or in server 2008 just check out the Line that reads Free.

3.  After awhile on a server that is not active, the active memory displayed in vmware will be much lower than the active memory in Microsoft.  Microsoft does not unload items from memory if they are not being used, and there is spare memory available.  This is just more caching.  VMware lists active memory as memory that is changed during a specific time period, which on an inactive server will be low.

Doug

0 Kudos
Lars_Laursen
Contributor
Contributor

Hi Doug

Thanks for your reply.

I can see that my system is using alot of "System Cache" and the concepts you describe, with Windows not releasing memory is also clear.

What I do not understand is, that if I have a virtual machine which has 2 GB Ram and ths OS uses 500MB, and then 1.4 GB System cache.

Then the Active Guest memory in vCenter will display 1,9GB.

This will result in vCenter giving a alarm. (Red exclamationpoint)

If windows never releases the System Cache, then I will always see alarms on my VM's in vCenter.

If I give the VM 4 GB Ram, then over time the System Cache will grow to 3,4 GB and vCenter displays active guest memory as 3,9 GB and then the alarm is back!

What can I do so my environment will keep active guest memory for my VM's in a "normal" state ?

./Lars

0 Kudos
dsoltesz
Enthusiast
Enthusiast

Lars,

I can tell you what I have done in my environment.  I'm not saying that this is the best answer, but it works for us.

1.  I have edited the alarm definition for Guest Memory Usage under datacenter, alarms, definition.

     It is set to Warn if condition is above 85% for 15 minutes, or Alert if condition is above 95% for 10 minutes.

This way when a VM boots, it touches 100% of the memory, but over the first 10 minutes some of the memory is not used.  As long as 5% isn't not used after 10 minutes, the Alert trigger will not hit.  Then in the next 5 minutes, the amount of active memory should fall another 10% so that the warn flag should not hit.  Now if either of these conditions do hit, and stay on, I know that I probably need to give that VM more memory, as it is actively using the memory I have given it.

Most VMs will have a sweet spot.  I have a SQL server running Dynamics, that was given 4GB and was constantlly alerting.  I boosted it to 6GB, and it runs better and no alerts.  I'm sure the sweet spot was lower than 6GB, but if memory becomes an issue in the environment, vSphere has memory ballooning and memory compression techniques for pulling back unused memory.

Doug

PS.  To help you set the tresholds, try opening the advanced performance tab for a VM.  Switch it to memory and reboot the VM.  You will see the active memory spike (Red line in my graphs).  Then after a few minutes you will see it start to settle down.  For me this occurs in about 10 minutes, thus the tresholds I set above.

Lars_Laursen
Contributor
Contributor

Hi Doug

I think I will try to do some more trouble shooting to see if I can find the reason why the VM are blocking out memory.

First I will upgrade all my VM to newest hardware/vmtools

After that I will definen new LUN's so my I/O load will be more balanced.

Then I will se it has made a difference to the VM.

Lars

0 Kudos
Marquise
Contributor
Contributor

Hello,

I have this same problem. You did it solve?

Thanks

0 Kudos
MauroBonder
VMware Employee
VMware Employee

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101320...

*Please, don't forget the awarding points for "helpful" and/or "correct" answers. *Por favor, não esqueça de atribuir os pontos se a resposta foi útil ou resolveu o problema.* Thank you/Obrigado
0 Kudos
dsoltesz
Enthusiast
Enthusiast

Thanks, but those patches are from 2009 and I have had them applied to my system for a long time now and still see the same issues.  I'm even running 4.1 now and VMs still do the same things. 

Like I said in an earlier post, it is just to OS touching all the memory.  After a couple of minutes the usage settles provided a VM has enough memory.

Doug

0 Kudos
Lars_Laursen
Contributor
Contributor

Hi

No this issue has not been solved yet.

We are still in the process of upgrading vmware tool on all vm's

0 Kudos
admin_harting
Contributor
Contributor

Hello

We have the same problems and no solution. Most of our server are 2003 and we started to set up new 2008 R2 server. Before a few days I notized that issue with the memory usage. The private memeory usage which can't be shared to other virtual machines. That is not very useful and the VMware memory sharing is not a big feature.

We need to get this fixed, possible by a new version of VMware tools.

Thanks and best regards

Alex

0 Kudos
Lars_Laursen
Contributor
Contributor

Hi All

We are in the process of upgrading our ESX servers.

Take a look at this KB

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101320...

I will report back when we are finished with the upgrade.

Regards

Lars

0 Kudos
Bastien_P
Enthusiast
Enthusiast

Hi,

similar issue here.  the "cache/standby" memory take all the available memory of the VM so my ESXi servers finds themself almost at full memory use with 2008r2 server where he was at a bit more of half memory used with the same number of 2003/2008 server with same memory allocated.

I was wondering if someone has a solution for this ... I have read some explanation/solution but failed to find a vmware official KB on this issue.

thanks.

Remember: If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
AartK
Contributor
Contributor

We got the same issue, our private guest memory is at 100%.

All of our new deployed WIN2008 R2 servers have this issue on ESXi 4.1.0 348481 and vCenter 345043.

Anyone has a clue?

0 Kudos
orthohin
Enthusiast
Enthusiast

Please have a look..

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101320...

Orthohin 
Never trust a computer you can't throw out a window
0 Kudos