Re: Windows 2008R2 snapshot problem on 4.1 - Page 2

AntonVZhbankov · ‎08-09-2010

I have problem with snapshots on one of my VMs. And what's strange it started only last friday.

One of my DC's on Win2008R2 refused to take quiesced snapshot after clod migration from one cluster to another. Remaining DC, absolutely the same configuration still can take snapshots.

I've made new VM, built from nothing, clean installation, promoted to DC instead of failed on and problem still persists.

What I've found in Windows logs:

1) Volume Shadow Copy Service error: Unexpected error DeviceIoControl(
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 000000000000042C,0x00560000,0000000000000000,0,00000000003C7F70,4096,[0]). hr = 0x80070001, Incorrect function.

.

Operation:

Exposing Recovered Volumes

Locating shadow-copy LUNs

PostSnapshot Event

Executing Asynchronous Operation

Context:

Device:
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Examining Detected Volume: Existing -
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Execution Context: Provider

Provider Name: VMware Snapshot Provider

Provider Version: 1.0.0

Provider ID: {564d7761-7265-2056-5353-2050726f7669}

Current State: DoSnapshotSet

2) lsass (460) An attempt to write to the file "
?\Volume{b6c3b2c0-a388-11df-badc-0050568b0019}\Windows\NTDS\edb.log" at offset 7525888 (0x000000000072d600) for 512 (0x00000200) bytes failed after 0 seconds with system error 19 (0x00000013): "The media is write protected. ". The write operation will fail with error -1032 (0xfffffbf8). If this error persists then the file may be damaged and may need to be restored from a previous backup.

3) lsass (460) Unable to write to logfile
?\Volume{b6c3b2c0-a388-11df-badc-0050568b0019}\Windows\NTDS\edb.log. Error -1032 (0xfffffbf8).

---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda

pciccone · ‎09-24-2010

Thats not really an acceptable solution. What you have essentially done via disabling the UUID is turn off VSS snapshotting completely. VSS won't work with UUID disabled, it's well documented in the vDR documentation. You will notice when UUID is disabled and you make a snapshot check the application event viewer and you will see the VSS routines are never started at all.

BulletByte · ‎09-25-2010

Hi all,

i installed last night the VMWare Data Recovery VM. All VMs are Windows 2008 R2 and i have 3 different domains with 3 Domain Controllers in my farm (Datacenter). i use also the 4.1 version of vCenter and everything is a fresh install with the latest service packs available.The database is a MS SQL 2008 R2 Express on the same machine as the vCenter.

Ok, here we go:

i can backup everything, also 2 of the 3 domain controllers .BUT the 3rd domain controller has all the failures and error events as described above. I tried everything out, double checked the UUID settings...no success.

Any news from VMWare support on this?

Here some eventview entries:

-

Volumeschattenkopie-Dienstfehler: Unerwarteter Fehler "DeviceIoControl(
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 0000000000000498,0x00560000,0000000000000000,0,0000000000416790,4096,[0])". hr = 0x80070001, Unzulässige Funktion.

.

Vorgang:

Wiederhergestellte Volumes werden verfügbar gemacht

Schattenkopie-LUNs werden gesucht

PostSnapshot-Ereignis

Asynchroner Vorgang wird ausgeführt

Kontext:

Gerät:
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Erkanntes Volume wird überprüft: Existing -
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Ausführungskontext: Provider

Anbietername: VMware Snapshot Provider

Anbieterversion: 1.0.0

Anbieter-ID: {564d7761-7265-2056-5353-2050726f7669}

Aktueller Status: DoSnapshotSet

Protokollname: Application

Quelle: VSS

Datum: 25.09.2010 14:41:19

Ereignis-ID: 12289

Aufgabenkategorie:Keine

Ebene: Fehler

Schlüsselwörter:Klassisch

Benutzer: Nicht zutreffend

Computer: MYVM

Beschreibung:

Volumeschattenkopie-Dienstfehler: Unerwarteter Fehler "DeviceIoControl(
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 0000000000000498,0x00560000,0000000000000000,0,0000000000416790,4096,[0])". hr = 0x80070001, Unzulässige Funktion.

.

Vorgang:

Wiederhergestellte Volumes werden verfügbar gemacht

Schattenkopie-LUNs werden gesucht

PostSnapshot-Ereignis

Asynchroner Vorgang wird ausgeführt

Kontext:

Gerät:
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Erkanntes Volume wird überprüft: Existing -
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Ausführungskontext: Provider

Anbietername: VMware Snapshot Provider

Anbieterversion: 1.0.0

Anbieter-ID: {564d7761-7265-2056-5353-2050726f7669}

Aktueller Status: DoSnapshotSet

Ereignis-XML:

<Channel>Application</Channel>

</System>

<Data>DeviceIoControl(
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 0000000000000498,0x00560000,0000000000000000,0,0000000000416790,4096,[0])</Data>

<Data>0x80070001, Unzulässige Funktion.

</Data>

<Data>

Vorgang:

Wiederhergestellte Volumes werden verfügbar gemacht

Schattenkopie-LUNs werden gesucht

PostSnapshot-Ereignis

Asynchroner Vorgang wird ausgeführt

Kontext:

Gerät:
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Erkanntes Volume wird überprüft: Existing -
?\fdc#generic_floppy_drive#6&3b4c39bd&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Ausführungskontext: Provider

Anbietername: VMware Snapshot Provider

Anbieterversion: 1.0.0

Anbieter-ID: {564d7761-7265-2056-5353-2050726f7669}

Aktueller Status: DoSnapshotSet</Data>

</EventData>

</Event>

-

lsass (468) Versuch, in Datei "
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log" bei Offset 9932288 (0x0000000000978e00) für 512 (0x00000200) Bytes zu schreiben, ist nach 0 Sekunden mit Systemfehler 19 (0x00000013): "Das Medium ist schreibgeschützt. " fehlgeschlagen. Fehler -1032 (0xfffffbf8) bei Schreiboperation. Wenn dieser Zustand andauert, ist die Datei möglicherweise beschädigt und muss aus einer vorherigen Sicherung wiederhergestellt werden.

Protokollname: Application

Quelle: ESENT

Datum: 25.09.2010 14:41:20

Ereignis-ID: 482

Aufgabenkategorie:Allgemein

Ebene: Fehler

Schlüsselwörter:Klassisch

Benutzer: Nicht zutreffend

Computer: MYVM

Beschreibung:

lsass (468) Versuch, in Datei "
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log" bei Offset 9932288 (0x0000000000978e00) für 512 (0x00000200) Bytes zu schreiben, ist nach 0 Sekunden mit Systemfehler 19 (0x00000013): "Das Medium ist schreibgeschützt. " fehlgeschlagen. Fehler -1032 (0xfffffbf8) bei Schreiboperation. Wenn dieser Zustand andauert, ist die Datei möglicherweise beschädigt und muss aus einer vorherigen Sicherung wiederhergestellt werden.

Ereignis-XML:

<Channel>Application</Channel>

</System>

<Data>lsass</Data>

<Data>

</Data>

<Data>
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log</Data>

<Data>-1032 (0xfffffbf8)</Data>

<Data>Das Medium ist schreibgeschützt. </Data>

</EventData>

</Event>

-

lsass (468) In Protokolldatei '
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log' konnte nicht geschrieben werden. Fehler -1032 (0xfffffbf8).

Protokollname: Application

Quelle: ESENT

Datum: 25.09.2010 14:41:20

Ereignis-ID: 408

Aufgabenkategorie:Protokollierung/Wiederherstellung

Ebene: Fehler

Schlüsselwörter:Klassisch

Benutzer: Nicht zutreffend

Computer: MYVM

Beschreibung:

lsass (468) In Protokolldatei '
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log' konnte nicht geschrieben werden. Fehler -1032 (0xfffffbf8).

Ereignis-XML:

<Channel>Application</Channel>

</System>

<Data>lsass</Data>

<Data>

</Data>

<Data>
?\Volume{d88f5f65-c8a1-11df-8e28-005056b0001e}\Windows\NTDS\edb.log</Data>

<Data>-1032 (0xfffffbf8)</Data>

</EventData>

</Event>

BulletByte · ‎09-25-2010

... and also this as the last entry from VSS

-

Protokollname: Application

Quelle: VSS

Datum: 25.09.2010 14:41:20

Ereignis-ID: 8229

Aufgabenkategorie:Keine

Ebene: Warnung

Schlüsselwörter:Klassisch

Benutzer: Nicht zutreffend

Computer: MYVM

Beschreibung:

Von einem VSS Writer wurde ein Ereignis mit dem Fehler "0x800423f4, Für den Generator wurde ein nicht vorübergehender Fehler festgestellt. Wenn der Sicherungsvorgang wiederholt wird,

tritt der Fehler höchstwahrscheinlich erneut auf.

" zurückgewiesen. Änderungen, die während der Verarbeitung des Ereignisses an den Writerkomponenten vorgenommen wurden, sind für den Anforderer nicht verfügbar. Zugehörige Ereignisse der Hostanwendung des VSS Writer finden Sie im Ereignisprotokoll.

Vorgang:

PostSnapshot-Ereignis

Kontext:

Ausführungskontext: Writer

Generatorklassen-ID: {b2014c9e-8711-4c5c-a5a9-3cf384484757}

Generatorname: NTDS

Generatorinstanz-ID: {7e4e9f06-19c8-438d-8c4c-e90d7e309f8d}

Befehlszeile: C:\Windows\system32\lsass.exe

Prozess-ID: 468

Ereignis-XML:

<Channel>Application</Channel>

</System>

<Data>0x800423f4, Für den Generator wurde ein nicht vorübergehender Fehler festgestellt. Wenn der Sicherungsvorgang wiederholt wird,

tritt der Fehler höchstwahrscheinlich erneut auf.

</Data>

<Data>

Vorgang:

PostSnapshot-Ereignis

Kontext:

Ausführungskontext: Writer

Generatorklassen-ID: {b2014c9e-8711-4c5c-a5a9-3cf384484757}

Generatorname: NTDS

Generatorinstanz-ID: {7e4e9f06-19c8-438d-8c4c-e90d7e309f8d}

Befehlszeile: C:\Windows\system32\lsass.exe

Prozess-ID: 468</Data>

</EventData>

</Event>

pciccone · ‎09-25-2010

You are correct, I just re-read the thread and it appears we have two different VSS writer problems, one that deals with ADAM and one that deals with NTDS. Both with the same fail reason of "System error 19 (0x00000013): "The media is write protected. ". The write operation will fail with error -1032 (0xfffffbf8)."

lucmorin · ‎09-25-2010

Actually, these errors are pretty consistent with what the OP reported and with what we're getting here.

It's a bit difficult to pinpoint the real source of the problem, as everytime we try to take a quiesced snapshot, we have about 4-5 log entries that are either warnings or errors.

For example, the "floppy" related error message, and also the "edb" file being protected error, then the VSS warning about a transient problem.

I'll try to post the relevant entries monday when I'm back at work, and the we can compare with others.

Regards

pciccone · ‎09-25-2010

My apologies, I got side tracked with the ADAM VSS writer and lost scope to the initial report of the NTDS writer. I updated my response above.

lucmorin · ‎09-30-2010

To the OP and others with same problem:

Discussing the issue with VMWare, we have found that on the VMs that are experiencing this problem we're missing file VCBRequestor.dll in folder:

C:\Program Files\VMware\VMware Tools\Drivers\vss

There should be two dll in this folder, the other one being VCBSnapshotProvider.dll

Could you guys check to see if this file is also missing on your systems ?

It seems that for some reason, VMWare tools doesn't install this file (and maybe others).

Regards

kswail · ‎09-30-2010

Interesting idea sir!

On my Windows 2008 R2 DC which has the backup problems, I do NOT have a copy of "VCBRequestor.dll" installed.

The only file in that directory is "VCBSnapshotProvider.dll".

What's odd is that NONE of my servers have this file, but all of them work correctly for application aware snapshots. It only fails on the DC.

I wonder if perhaps this file is from older versions of VMware Tools and no longer installed?

cheers,

kswail

jcwuerfl · ‎09-30-2010

Device:
?\fdc#generic_floppy_drive#

so do you have a Floppy connected to the VM? Have you tried removing that?

OmniTech · ‎10-06-2010

I have the same problem.

Since it seems that VMWare and Windows 2008R2 is not bugfree, I'll install 2003R2...

lucmorin · ‎10-06-2010

I have an SR opened with VMWare, and it has been elevated, so I'm hoping they will find the cause AND the cure.

It basically seems to be that some files don't get properly installed/registered on Windows 2008 R2. So far the service reps have not been able to reproduce it on their setup, so it might be something to do with ESXi 4.1 interacting funny on some hardware. Who knows.

I'll post whenever there's progress on this.

BulletByte · ‎10-06-2010

Device:
?\fdc#generic_floppy_drive#
so do you have a Floppy connected to the VM? Have you tried removing that?

U asking me? No, there was never a floppy drive. I run the Backup creation process again (and therefor the snapshot to copy the backup with Data Recovery) with the same results and the same error messages.

No progress!

jcwuerfl · ‎10-06-2010

No more directed at @AntonVZhbankov I think but even if its defined and connected on the VM that could cause an issue?

mronsman · ‎10-11-2010

Count me in on this one. Trying to do VDR on vCenter itself as a VM. The DB is remote. Fails on the snapshot every time. Please post if anyone has gotten this to work yet......

Thanx,

Matt

BulletByte · ‎10-12-2010

Nope, no chance... still the same. It would be nice if we could have ANY statement from VMWare about this issue

mrjlturner · ‎10-12-2010

*Duplicate post* my apologies

mrjlturner · ‎10-12-2010

Matt,

If you read through the thread, you'll find that I had luck w/ backing up the vCenter Server (w/ SQL Express on server) using VDR 1.2 on and ESXi 4.1 host by adding the following: (taken from the thread & modified slightly)

Enable Windows 2008 Virtual Machine Application Consistent Quiescing

1 Start the vSphere Client, and log in to a vCenter Server.

2 Select Virtual Machines and Templates and click the Virtual Machines tab.

3 Right‐click the Windows 2008 virtual machine for which you are enabling the disk UUID attribute, and select Power > Power Off. The virtual machine

powers off.

4 Right‐click the virtual machine, and click Edit Settings.

5Click the Options tab, and select the General entry in the settings column.

6Click Configuration Parameters… The Configuration Paramters window appears.

7Click Add Row.

8 In the Name column, enter disk.EnableUUID.

9In the Value column, enter FALSE.

10 Click OK and click Save.

11 Power on the virtual machine.

By setting the option to 'FALSE' I am able to take successful snapshots and backups.

Thanks!

mrjlturner

pciccone · ‎10-12-2010

UUID as disabled = crash consistent backups (VSS disabled/bypassed).

Here is an article on this fact: http://vknowledge.wordpress.com/2010/08/18/application-consistent-quiescing-and-vdr/

mrjlturner · ‎10-12-2010

Pciccone,

Dually noted my friend. I have posted my results simply so that folks may be able to get "some" backups vs. no backups whatsoever against the vCenter server. I am having a hard time believing that VMware has yet to have addressed anyone in this thread. I hope that changes soon...

Thanks again!

mrjlturner

jcwuerfl · ‎10-12-2010

Still currious if a floppy drive is connected, and if that was removed if it would work after.

Since it was trying to write to the floppy drive?

Device:
?\fdc#generic_floppy_drive