frequent 30-60-second lockup with solid disk activity

[ajw]

I have a problem where VMware Workstation locks up for 30 to 60 seconds, with solid disk activity. No reaction even to things like caps lock or num lock (VM is using extended keyboard; if I'm in the host it does show the caps/numlock lights - i.e., VMware is busy and doesn't pass the keyboard to the VM, but doesn't lock up the host)

Both host and client are Windows XP SP2; I'm running VMware Workstation 6.5.0 build 118166 (there is a newer build, but nothing in release notes looks applicable to this)

Notebook has 4gb of memory, 2Ghz Core 2 Duo.

2 drives; system on one, VMs on another (7200rpm, 16meg cache)

I tested both drives S.M.A.R.T. - well within 'good' parameters

I've tried everything I can think of with this - enabling/disabling (XP's) virtual memory on host and/or client, performance settings, changing VM's memory size, etc.

It happens at odd times - when I'm just typing (like during this sentence - it just did it) and happens in VMs that are various memory sizes. It seems to happen in all VMs, regardless of their memory size. (I have VMs ranging from 512megs up) It happens when I'm running just one VM or several - it seems worse when running more than one, but not linear; at least not bad enough to keep me from running more than one. (I usuallly am only running one or two VMs) It never happens if I'm not running VMware; if I'm just running apps on the host.

It doesn't seem to matter what applications I'm running in the client. Setting VMware to keep everything in memory doesn't help. Disabling memory page trimming doesn't help. (which shouldn't matter if I'm only running 1 VM, would it?) Logs are disabled. I have VMware set to use the maximum reserved memory (3456meg). It doesn't seem any worse if I run apps in host (usually just an editor, sometimes Firefox, sometimes Nero disk burning)

I'm baffled by it. It seems to be getting worse lately, but it may just be my annoyance level increasing... I'm not sure; I think it started way back when I switched to this notebook, but I'm not sure.

Does anyone have any ideas?

- Al Weiner -

[mikefoley]

Just for the record (and your sanity), I'm seeing the same things.

My host is a Dell D630, 4GB memory and Vista x64. I was running two VM's but rolled that back to just one VM with 1.2GB of memory.

I'm just guessing, but there doesn't seem to be a good way to measure what's happening is that someone (host or guest or both) are paging like mad. I say that because I was running sysinternals Process Explorer on the host when one of these events happened and the System process was busy.

mike

VCP/vExpert

[ajw]

Thanks for posting that! Good (uh, sorta... to know that others are seeing it too. Although if experts are seeing it and not resolving it... (how do I get a 'wry grin' smiley here...)

I note some commonalities between us:

- host has 4GB of memory

- both are Core 2 Duos

and differences:

- my host is 32-bit Windows XP SP2, your's is Vista x64

I agree with your guess - and I'm frustrated that there doesn't seem to be a way to (easily) profile the system-level disk activity. I see the same; everything attributed to system process. I strongly suspect VMware Workstation is doing it; for the last month or so I've disabled virtual memory (paging file) on the host. What it's doing is not clear - it'll happen even when I'm running a single 512meg VM which should fit entirely in memory so should never require anything except when the client uses the disk... Additional points: I run in 'quickswitch' mode - sometimes during lockup I can mouse to top of screen, VMware menu/toolbar drops down and sometimes I can minimize and get to host; other times I cannot - clicking anything on menu/toolbar just moves the mouse down to below it. Also, frequently when I do get to the host (during the lockup) the host's taskbar isn't updated until the lockup is over. (do you see this same behavior?)

Questions: are you running Workstation? (and which build?) and is this a VM that you were running (and created?) prior to 6.5.0 upgrade? my two main VMs were created under 5.x and upgraded to 6.x then to 6.5. Maybe it's time to (oh! dread!!) recreate them from scratch...

Another observation -most of my virtual drives are 2GBsplits, but my main "work" VM is unsplit drives, except for 'E:' which is split - and is where that XP's paging file is... just moved it to the unsplit 'D:'... see if that makes any difference... (I'm reaching for anything at this point...)

- Al -

[Scissor]

Please attach the vmware.log and .vmx file from the directory containing one of your Guests.

What AntiVirus do you have installed on your Host? Any 3rd party firewalls or other "security" software installed on your Host?

- - Are any of your Guests configured for 2 virtual CPUs? If so, change them back to 1 vCPU.

- - Configure your Guest CD-ROMs to start "disconnected".

- - Exclude the directory containing your Guests from the Host's AntiVirus real time scanner.

[cdc1]

One of Vista's features that can cause nice disk activity is System Recovery. This not only keeps track of changes to your system, but it also keeps track of file versions, which allow you to recover documents to a previous version in cases where you make drastic editing mistakes, for example. Unfortunately, this can consume a huge amount of disk space on drives where it is enabled. By default, I believe Vista reserves about 15% of available disk space on your drive right after Vista is booted up the first time after install (although, I've experienced situations where it has consumed 30GB of space on my C drive, which is sized at 60GB.) I would suggest that you disable System Recovery on your C: drive, as well as the drive holding your VMs on it (if it's enabled on that drive.) Anyway, with all of this disk space being consumed, it usually doesn't leave much room for things like pagefile management or temp file space, so fragmentation is more likely to occur, causing disk performance to take a noticeable hit in some cases, which can make things worse when disk activity happens.

Although the above is targeted at Vista hosts, XP also has the System Recovery feature, but without the VSS capability that its Vista counterpart has. I'd still look at disabling system recovery on XP, though, then defrag the drive, then re-enable system recovery if you really want it there.

Also, don't forget to turn of file indexing for those drives as well. This can also cause a lot of disk activity to take place.

Keep in mind, none of what I have suggested is guaranteed to fix the cause of your issues. It's just one more step in the troubleshooting process to try to figure out what's causing the problem. It's helped in my situation, but that doesn't mean it will in yours. Good luck!

Message was edited by: cdc

[ajw]

Hi Scissor,

Thanks for responding. I've attached a zip file containing the vmx file, vmware.log ("Copy of vmware.log"), as well as the last 3 logs (since they zip down so nicely

This is for my daily "work" VM - log is "Copy of" because I'm running that VM now.

As to your questions:

- all guests are configured for 1 CPU

- all start with CDROMs disconnected (except one but I rarely run that one)

- anti-virus configured excluding the entire "E:\VMware virtual machines" tree, as well as verified .vmx, .vmdk, etc are not (supposed to be...) scanned.

Anti-virus on both host and client is Avast! - I've disabled it on client without any joy; just disabled it on host because I don't remember if I did in the past...

No firewalls in host nor most of my VMs - I've got a VM that I'm testing Comodo in, but I don't run that normally. (daily use is typically just my "work" VM and/or my "home" VM; sometimes both)

- Al -

[ajw]

Hi cdc,

Thanks for your reply. I just checked - hmmmm... I have System Recovery enabled on both host and client - I'll test with both disabled. (I just disabled anti-virus on host; testing that now - and, amusingly..., I just got the lockup as I typed the word 'on' I thought I had it disabled on both; apparently not...

File indexing - definitely always off! Does anyone actually use that? I never have, at least...

Thanks for the suggestions - I'll post results as I test them.

- Al -

[Scissor · Accepted solution]

Looking over your logs I see that you seem to have VMware Shared Folders enabled. VMware Shared Folders have all sorts of strange issues. Try disabling/turning off your VMware Shared Folders to see if that makes a difference.

[ajw]

(voice of Geico gecko - which won't make any sense to international customers...

"oh, dear..."

I use shared folders a lot! Hmm... I have since the beginning; way back on Workstation 5.something. I'll try disabling them to see if that helps - will post results here...

- Al -

(Geico is an insurance company; they have a computer-generated gecko as a spokesman -uh- spokesgecko - in one advertisment, an executive is doing a "trust" exercise where he'll fall backwards and the gecko is supposed to catch him. Gecko looks up at big executive falling back onto little gecko and, in a British voice, says "oh, dear..."

[ajw]

Status update:

I've disabled anti-virus on the host machine, and disabled System Restore on the host. (still enabled on the guest) I did not reboot either host or client. (meant to reboot client but got busy...

I've been using my "work" VM (the only one running) heavily - doing some big builds, editing at the same time, etc. The lockup is still happening, but subjectively it is less frequent. I just checked on my previous notebook, and system restore is enabled on that machine; I never saw this lockup on that machine. (timeline is: I started with Workstation 5 on that machine, upgraded it to 6.0, later switched to this notebook (with 6.0), then upgraded this to 6.5)

More status later as I test further.

- Al -

[ajw]

Status update (after several days):

It's the VMware Shared Folders.

The day after I started this thread, I tried with System Restore and anti-virus (Avast!) disabled on both host and client - no joy.

I then disabled VMware Shared Folders - the lockups went away immediately! I really didn't expect that; I've been using shared folders for years, and the lockups only started a few months ago...

I kept running through the day, mostly with one VM ("work") running, sometimes with 2 ("work" and "home") running. Still kept running fine (I disabled shared folders on both of them). Late yesterday, I needed some files that were previously in shared directories, so I enabled it - and the lockup occurred a few minutes later! Got the files, disabled shared folders - all fine again.

I did the same last night - enabled shared folders and grabbed a few files. This morning I had several lockups - then realized I'd forgotten to disable shared folders! Disabled them - bingo! No more lockups.

Scissor, you get the prize for the right answer! I especially appreciate it because I would never have thought of shared folders as the problem.

Many thank-yous!

- Al -

[mikefoley]

Great job guys!