RDM to Local Disk (or RAID volume)

[tlduong]

Hi,

How does one create an RDM to a locally connected drive (in my case, a volume on an Adaptec 5805 RAID controller) that does not have a "VMWARE SCSI id" or unique identifier. Whenever I try it, I get the following error:

Failed to create virtual disk: Invalid argument (1441801).

FYI, the target rdm file was to a folder existing in a VMFS partition. The array volumes can be seen by ESX (one of the volumes is being used as a VM Store). The other volumes are formatted NTFS and have data on them.

I tried scsi-passthrough as a deviceType in the vmx but the VI client stops the VM from powering on and throws a message that says something about the device not being a non-disk scsi device. I also tried rawDisk as a device type from VMWare server but that didn't work either.

Is there anyway to manually create the RDM link without going through vmkfstools? Anyway to trick ESX into "thinking" the volume has a VMWARE SCSI id? Should I be asking Adaptec about this?

I was able to easily do a RDM on a local disk on the same server but it was one that had a unique identifier (SATA drive on ICH9R).

Any help would be greatly appreciated. This feature is very handy in VMware Server and I guess from reading around, ESX 2.5 allowed scsi-passthrough for disks but ESX 3.x doesn't anymore. It would be very tedious to re-move all the data in the NTFS volumes into VMDK files because we don't have spare space of this size and time.

Thanks!

[jhanekom]

I know this was discussed a few times on the forum some time back and definitely was a problem (more like a bug than a dropped feature, from what I can tell.) What version of ESX are you running specifically, and which patches?

Any particular reason you would not want to use vmkfstools to create the RDM link?

[tlduong]

We're running 3.5 without any patches yet (the server was just brought up very recently in dev). Do you know for sure that a patch will help this (I hope it's a bug)? The reason why I want to see if I could do it manually is because I believe vmkfstools checks for the unique identifier before it "actually" makes the link.

[lamw]

It sounds like he's running 3.5 base, this is not including the patches that were release after 3.5 release and not including 3.5 Update 1. I don't recall looking at the recent 3.5 patches that this was an issue it had resolved from any of the KBs. Probably still a known issue or bug , as I've seen previous post in the forum without any real solutions. Curious to see if this can be resolved or remediate in 3.5

[Texiwill]

Hello,

For a Local LUN you can not really make an RDM, instead you can direct access the LUN by hooking it up as a disk using a scsi generic device. Just like you would do for a tape drive. At least I have always had to do this to get things to work.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[tlduong]

Hi,

I tried using the generic disk method through vmx file in the console (the generic scsi device option is greyed out in the VM settings dialogue) but was presented an error that said something about the scsi device not being a non-disk device. Let me know if i'm doing something wrong...

This was my entry in the VMX file:

scsi0:2.present = "true"

scsi0:2.fileName = "/vmfs/devices/disks/vmhba2:3:0:0"

scsi0:2.deviceType = "scsi-passthru"

I get the following error when starting the VM up in viclient:

/vmfs/devices/disks/vmhba2:3:0:0 (scsi0:2.fileName) isn't a valid non-disk scsi device.

Failed to configure scsi0.

[lamw]

Here are the valid options for deviceType:

scsi0:0.deviceType = "plainDisk"

scsi0:0.deviceType = "PhysicalDrive0"

scsi0:0.deviceType = "rawDisk"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:0.deviceType = "scsi-passthru"

scsi0:0.deviceType = "scsi-nonpassThru-rdm"

scsi0:0.deviceType = "scsi-passthru"

scsi0:0.deviceType = "scsi-passThru-rdm"

[Texiwill]

Hello,

You need to use the /dev/ name of the volume and not a vmhba name. If there is a vmhba name for the volume then you may have additional issues.

Use 'fdisk -l' to get the name of the device, say /dev/sdaX, then modify the .vmx file... I tend to add a separate SCSI adapter for RDMs.

scsi2.present = "TRUE"

scsi2:1.present = "TRUE"

scsi2:1.deviceType = "scsi-passthru"

scsi2:1.fileName = "/dev/sdX"

Note this is discussed on page 385 of my book which is Safari enabled for those who have access to Safari.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[tlduong]

So, I tried this out as you suggested:

scsi1.present = "true"

scsi1.sharedBus = "none"

scsi1.virtualDev = "lsilogic"

scsi1:0.present = "true"

scsi1:0.fileName = "/dev/sde"

scsi1:0.deviceType = "scsi-passthru"

But now it is giving me a weird error:

The attempted operationg cannot be performed in the current state (Powered Off).

[Texiwill]

Hello,

Any 'scsi' entries within the VMX would be handy as well as the output from fdisk -l. Is this already mapped to a vmhba device if so you have other issues, specifically, it may not be possible for both to happen at the same time. Please also run:

esxcfg-vmhbadevs -l

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[tlduong]

Hi,

Here is the output you requested:

=======================Relevant scsi entries within vmx:=======================

scsi0.present = "true"

scsi0.sharedBus = "none"

scsi0.virtualDev = "lsilogic"

scsi1.present = "true"

scsi1.sharedBus = "none"

scsi1.virtualDev = "lsilogic"

scsi0:0.present = "true"

scsi0:0.fileName = "Rhapsody.vmdk"

scsi0:0.deviceType = "scsi-hardDisk"

scsi0:1.present = "true"

scsi0:1.fileName = "Rhapsody_1.vmdk"

scsi0:1.deviceType = "scsi-hardDisk"

scsi0:1.redo = ""

scsi1:0.present = "true"

scsi1:0.deviceType = "scsi-passthru"

scsi1:0.fileName = "/dev/sde"

scsi1:0.redo = ""

===============fdisk -l output:===========================

Disk /dev/sda: 400.0 GB, 400088457216 bytes

255 heads, 63 sectors/track, 48641 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sda1 * 1 13 104391 83 Linux

/dev/sda2 14 650 5116702+ 83 Linux

/dev/sda3 651 48305 382788787+ fb Unknown

/dev/sda4 48306 48641 2698920 f Win95 Ext'd (LBA)

/dev/sda5 48306 48374 554211 82 Linux swap

/dev/sda6 48375 48628 2040223+ 83 Linux

/dev/sda7 48629 48641 104391 fc Unknown

Disk /dev/sdb: 1148.8 GB, 1148884877312 bytes

255 heads, 63 sectors/track, 139677 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 3 78328 629145600+ 7 HPFS/NTFS

/dev/sdb2 78328 110962 262140280+ 7 HPFS/NTFS

/dev/sdb3 110963 139677 230653237+ 7 HPFS/NTFS

Disk /dev/sdc: 214.7 GB, 214729490432 bytes

255 heads, 63 sectors/track, 26106 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdc1 3 15668 125829120+ 7 HPFS/NTFS

/dev/sdc2 15668 20889 41945388 7 HPFS/NTFS

/dev/sdc3 20890 26105 41897520 7 HPFS/NTFS

Disk /dev/sdd: 107.3 GB, 107346919424 bytes

255 heads, 63 sectors/track, 13050 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdd1 3 13051 104814592 7 HPFS/NTFS

Disk /dev/sde: 214.7 GB, 214729490432 bytes

255 heads, 63 sectors/track, 26106 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sde1 3 26107 209680384 7 HPFS/NTFS

Disk /dev/sdf: 233.7 GB, 233754853376 bytes

255 heads, 63 sectors/track, 28419 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdf1 1 28419 228275553+ fb Unknown

I believe they are already mapped to a vmhba device.

esxcfg-vmhbadevs -l doesn't work for me but here is the same command without -l

vmhba2:0:0 /dev/sdb

vmhba2:1:0 /dev/sdc

vmhba2:2:0 /dev/sdd

vmhba2:3:0 /dev/sde

vmhba2:4:0 /dev/sdf

vmhba32:0:0 /dev/sda

[Texiwill]

Hello,

THanks for the information...

You want to use /dev/sde1 as the fileName.

scsi1:0.fileName = "/dev/sde1"

That should solve your issue, you need to mention the partition as well as the device letter.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[tlduong]

Hi,

I just tried to make the change that you suggested but I still got the same message when trying to start up the virtual machine.

[lamw]

Has anyone actually verified that these configuration changes indeed allow for an RDM to local disk? Looks like dltuong is just trying every combinaton one, and suggestions one after another?

Which platform of ESX has this been confirmed/verified on? If someone can provide what "vmware -v" states along with an output to possibly patch level (esxupdate -l query), this might have been an added feature or not?

[tlduong]

Does anyone else have any suggestions?

[danielharvey]

Hi guys,

I had this problem in ESXi. After lots of searching I've got it working. I'm running a Dell 2900 with RAID and wanting to directly mount a RAID target from within a VM. The documentation ( and the man page) for vmkfstools states that you can specify devices of the form /vmfs/devices/disks/vmhba1:1:0. This just did not work for me.

What worked using the Remote CLI interface:

1. Use esxcfg-vmhbadevs to "print a mapping of vmhbaX:X:X names to console /dev/ names"
   e.g.
   >
   esxcfg-vmhbadevs --verbose --username root --server 192.168.0.251

2. Use the mapped device name to create the RDM
   e.g.
   >
   vmkfstools --verbose --username root --server 192.168.0.251 -r /vmfs/devices/disks/vml.02000000006001c230c848e2000ebfe71ddbd83de6504552432035 /vmfs/volumes/datastore1/openfiler-2.3/pool.vmdk
   

:smileyinfo: Important note: I first tried using the name "pool-rdm.vmdk" but this seemed to confuse ESX and it did not recognise it. I tried again with "pool.vmdk" and it worked.Dunno, but the "-" hyphen seemed to confuse it.

PS The "generic SCSI" option suggested by Texiwill was not available for me as all of the devices were considered by ESXi as "disks".

[Texiwill]

Hello,

On regular ESX the device you want to make an RDM can NOT be seen as a vmhba device. That is the only caveat. Seems ESXi reverses that. But try the ESXi approach it may work.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[BoKohut]

danielharvey,

First like to thanks you for taking the time to post. I too have now spent about 7 hours trying to get this to work and your experiences along with you instructions make me "feel" like I have gotten nearest to the goal but still not successful. I would like to inquire about some data that I find may be relevant to your situation and surely relevant to clarify to others, including me.

First I will give my hardware and software setup and my goal: Hardware == HP DL360 Dual 3Ghz Xeon with the onboard HP Two disk RAID Mirror and an Adaptec 2810SA with two RAID sets (0) Five disk RAID 5 (1) Two disk RAID Mirror, Software == Fresh install Esxi 3.5.0 Update 2, Goal == Use an Ubuntu VM with Zoneminder ( ) to have raw disk access to the Adaptec (1) Two disk Mirror. The onboard RAID houses the default datastore1 location while the Adaptecs I am trying to get raw access too. I too installed the remote cli after reading your entry on how you "got it to work" but I have not yet succeeded. I am quite flustered since in VMWare Server it seemed so easy for raw disk access.

I have had no problems with ESXi seeing the RAID sets and even used them to test by creating VMFS storage volumes on them. I have however had no luck using the esxcfg-vmhbadevs command to implicitly produce the results which you seem to have received based on your line item entry for your second command vmkfstools. What I mean by "implicity" is my results do not include output with a "vml.NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" (not exact length, just for show) file to be able to map too. I followed the instructions using the /vmfs/devices/disks/vmhbaA.B.C.D and I too got the same error. I always like to follow the exact steps, errors and all, to know I am on the correct path. When I received the error I was certain your instructions would lead me to success, I was disappointed. I considered the fact your ESXi may be older, not to old though since your post is recent. I would like to point out I tried other previous steps in this post as well without success, all tricks led to the "isn't a valid non-disk scsi device" as seen by tdluong.

So i got creative too which I did not find in my internet searches. First I would like to say I am a very experienced software developer, I always get my hands dirty and have been using VMware products since beta. Using the --verbose from the exscfg-vmhbadevs output I saved the soap response to an xml file. Within this response is data relevant to many setup and hardware items with theESXi environment. I located the SsciLun entry which is my device I wish to get raw access too as seen here:

<scsiLun xsi:type="HostScsiDisk">

<deviceName>/vmfs/devices/disks/vmhba2:1:0:0</deviceName>

<deviceType>disk</deviceType>

<key>key-vim.host.ScsiDisk-vmhba2:1:0</key>

<uuid>0000000000766d686261323a43303a54313a4c30</uuid>

<canonicalName>vmhba2:1:0</canonicalName>

<lunType>disk</lunType>

<vendor>Adaptec</vendor>

<model>surveillance</model>

<revision>V1.0</revision>

<scsiLevel>3</scsiLevel>

<serialNumber>unavailable</serialNumber>

<standardInquiry>0</standardInquiry>

etc....

I found very few entries on the net that show a file /vml.NNNNNNNNNNNNNNNNNNNNNNNNNNNN referenced as you had but I, as I am writing this post, do not see such a file using the available Remote Cli intereface. I did find a site though that had some verbiage, whether correct or not, stating information about the uuid. So being creative I used the uuid in my vmkfstools. call but this to produced the same error stated as tdluong, so much for being sneaky.

As you can read, having provided some background from my experiences, I am at a loss of what your esxcfg-vmhbadevs command execution produced or your setup to have the vml.* files. If your setup is only a server with RAID with what you achieved raw disk/ scsi pass-thru access to I am wondering what I am missing. What I have done or not done to not be able to achieve what you have, which reads to me based on the title of this forum thread "RDM to Local Disk (or RAID volume)", as it sound to me as though you have RDM'd to a RAID volume locally without a SAN or any other requirements.

I understand these methods are most likely not supported and since I am not using this for production or business I do not mind. It is my home lab and any help, right or wrong, is appreciated. (Preferably right :] )

[bychang]

Hi guys,

I'm trying to figure out how to map my RAID array as a generic SCSI disk to a VM on my ESX 3.5 server.

I'm trying to consolidate most of ours servers to a new IBM x3400 running ESX 3.5. One of the servers is a file server that has a 2TB RAID array - which we have physically moved to the new server and cold migrated the linux server. Now we just need to attach the RAID array to the VM to complete the migration.

I know we've done this before with ESX 2.x, but it seems like I can't do that now with ESX 3.5.

I tried some of the suggestions that Edward gave - manually editing the vmx file and got the same error messages that previous users were getting.

The ESX service console already knows about the array and has mapped it to vmhba0:1:0:0. But since it's a RAID array on an IBM ServeRAID controller, RDM does not seem to work (probably since it does not provide a UUID/serial number for the array).

output of esxcfg-vmhbadevs

vmhba0:0:0 /dev/sda

vmhba0:1:0 /dev/sdb

vmhba0:2:0 /dev/sdc

Is there a way to "unmap" the array from the service console so that I can put it as a generic SCSI disk (/dev/sdb) in the VM? Or is there a utility that can generate a serial number so that the VMware service console can uniquely identify it so RDM will work?

I really hope there is a solution, otherwise we may have to move to VMware Server to make this work.

Thanks!

-Bill

[dfir]

BoKohut,

Seems like you're struggling just as much as me to get this to work.

Have you managed to get anything related to RDM to work?

Best regards,

Jesper