datastore issue after power failure

[twelve1212]

Hi - not very experience at all with ESXi.  Running 6.7.0 (Build 8169922).

Had a power cut last night and now the datastore isn't accessible.

It look like its recognising the drive here but then its showing as 0 bytes/free space on other parts.

None of my VMs are accessible but the VM list is showing 32 VMs..

These VMs are labelled random numbers between 37 and 76.

The data was nothing irreplaceable so I could even redo everything but have no idea how to proceed with even accessing the drive

appreciate any help!

[e_espinel]

Hello.
Execute the following commands and show your results in this post

esxcli storage core adapter rescan --all  (this command does not show anything, it just rescans the devices)

esxcfg-scsidevs -c

esxcli storage core device list

 esxcli storage filesystem list

esxcli storage vmfs snapshot list

[twelve1212]

esxcfg-scsidevs -c

Device UID Device Type Console Device Size Multipath PluginDisplay Name
mpx.vmhba1:C0:T5:L0 CD-ROM /vmfs/devices/cdrom/mpx.vmhba1:C0:T5:L0 0MB NMP Local ASUS CD-ROM (mpx.vmhba1:C0:T5:L0)
t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500 Direct-Access /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500 1907729MB NMP Local NVMe Disk (t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500)

esxcli storage core device list

mpx.vmhba1:C0:T5:L0
Display Name: Local ASUS CD-ROM (mpx.vmhba1:C0:T5:L0)
Has Settable Display Name: false
Size: 0
Device Type: CD-ROM
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/cdrom/mpx.vmhba1:C0:T5:L0
Vendor: ASUS
Model: DRW-24D5MT
Revision: 1.00
SCSI Level: 5
Is Pseudo: false
Status: on
Is RDM Capable: false
Is Local: true
Is Removable: true
Is SSD: false
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: unknown
Attached Filters:
VAAI Status: unsupported
Other UIDs: vml.0005000000766d686261313a353a30
Is Shared Clusterwide: false
Is SAS: false
Is USB: false
Is Boot Device: false
Device Max Queue Depth: 31
No of outstanding IOs with competing worlds: 31
Drive Type: unknown
RAID Level: unknown
Number of Physical Drives: unknown
Protection Enabled: false
PI Activated: false
PI Type: 0

PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false

t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500
Display Name: Local NVMe Disk (t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500)
Has Settable Display Name: true
Size: 1907729
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500
Vendor: NVMe
Model: Samsung SSD 970
Revision: 2B2Q
SCSI Level: 6
Is Pseudo: false
Status: on
Is RDM Capable: false
Is Local: true
Is Removable: false
Is SSD: true
Is VVOL PE: false
Is Offline: false
Is Perennially Reserved: false
Queue Full Sample Size: 0
Queue Full Threshold: 0
Thin Provisioning Status: yes
Attached Filters:
VAAI Status: unknown
Other UIDs: vml.0100000000313231465f423438315f354233385f323530300053616d73756e
Is Shared Clusterwide: false
Is SAS: false
Is USB: false
Is Boot Device: true
Device Max Queue Depth: 2046
No of outstanding IOs with competing worlds: 32
Drive Type: unknown
RAID Level: unknown
Number of Physical Drives: unknown
Protection Enabled: false
PI Activated: false
PI Type: 0
PI Protection Mask: NO PROTECTION
Supported Guard Types: NO GUARD SUPPORT
DIX Enabled: false
DIX Guard Type: NO GUARD SUPPORT
Emulated DIX/DIF Enabled: false

 esxcli storage filesystem list

Mount Point Volume Name UUID Mounted Type Size Free
------------------------------------------------- ----------- ----------------------------------- ------- ------ ---------- ----------
datastore1 5c7d5a34-c96d250b-efff-2c4d5444c155 false VMFS-6 0 0
/vmfs/volumes/ec4c427c-28a7cfbc-7157-26a28ade9d4e ec4c427c-28a7cfbc-7157-26a28ade9d4e true vfat 261853184 261844992
/vmfs/volumes/be49d074-267f609a-dcd2-e900458ae55e be49d074-267f609a-dcd2-e900458ae55e true vfat 261853184 113827840
/vmfs/volumes/5c7d5a2d-950d693e-517b-2c4d5444c155 5c7d5a2d-950d693e-517b-2c4d5444c155 true vfat 299712512 80486400
/vmfs/volumes/5c7d5a34-ea46b3b4-0bbb-2c4d5444c155 5c7d5a34-ea46b3b4-0bbb-2c4d5444c155 true vfat 4293591040 4275568640

esxcli storage vmfs snapshot list

Didn't return anything.

Thanks!

[continuum]

Check wether the 1.8tb vmfs partition still has the vmfs-magic number d0 0d.
Use hexdump -C againstpartition 3.

Ulli

[twelve1212]

How?   no idea what the command for partition 3 is.

Added you on skype too.. twelve-1212

[continuum]

hexdump -C /dev/disks/naa-number:3 >> /tmp/hex.txt

run that for a few seconds and  then show the first few lines.

Ulli

[twelve1212]

Doesnt work..

stalls for a few seconds and returns:

hexdump: /dev/disks/t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500:3: Input/output error

[continuum]

I/O error before the vmfs magic number ? - sorry this means :
A: you declare defeat here
B: you must switch to a Linux LiveCD fr further troubleshooting

Next step: find out which area is blocked by the I/O error ...f
If only the range from 1 mB offset to 21 mB offset is damaged the datastore may still be repairable..

Ulli

[twelve1212]

Is the physical drive dead or can it be wiped and start over?

Is so, how to I proceed in just formatting?   the data on the drive is all replaceable so not a big deal and if it saves me buying a new drive.

[continuum]

If possible wipe the disk with zeroes - at least the first 5 GB.
That will improve the chance to re-use it

dd if=/dev/zero of=/dev/disks/naa-number bs=1M count=5000
make sure to specify correct device !!!!!!!!

Ulli

[twelve1212]

That didn't work either.

Does this mean the drive is completely dead and needs replacing?  or is there any other way to completely wipe it and start over?

giving this error: dd: can't open '/dev/disks/t10.NVMe____Samsung_SSD_970_EVO_2TB_________________121FB4815B382500': Function not implemented

is that even the right command?  the t10.nvme bit i mean..  it gives the same error no matter what I type.