VMFS3 does not see the new expanded space

[largie]

Hi!

Sorry if this issue have been solved before. After hours searching I have not been able to solve this little problem.

First of; My Hardware RAID is a Adaptec 3805 and I have a RAID 6 Logic drive that has recently been expanded to 2,7TB (earlier size were 1,82TB) .The physical rebuild is now of cause finished and I have rebootet the server

When I look in the VI-client under Configuration -> Storage Adapters I see a device called vmhba3 with the capacaty of 2,7TB. But when I go to the Configuration -> Storage then the old VMFS3.11 partition is still 1,82TB and I cannot find any options to extend/expand/resize or simular here. Name of paritition is vmhba3:1:0:1

If I check the Properties for the drive vmhba3:1:0:1 I get no info regarding the size of the partition and by pressing "Add Extend" I get an error saying: "Error during the configuration of the host: Failed to get disk partition information".

Second I tried to "Add Storage" but of cause there is no new "Disk/LUNs" available since the name and everything is the same, only the more size are available.

Any tips on how to resize the VMFS-partition. Is it possible todo so and keep data during process?

I don't have available space to copy the VMDK-files, hopefully there are some clever heads out there who know. In advance thanx!

[Rubeck]

You can't expand an existing VMFS LUN....

Your options are to configure a new LUN and add this as an extent or create a new VMFS volume.

And by the way.... The maximum size for a single VMFS is 2TB. By adding extents you can reach 64TB (32x 2TB)

/Rubeck

[largie]

Thnx for the reply Rubeck.

Will it be possible to reconfigure the LUN and keep the existing data?

For what I understood from your answer is to create a new VMFS like vmhba3:1:0:2, that is 1TB in size, right?

I mentioned earlier that I could not see any more disk/luns in the add storage wizard. Are there any good guides on how to add extra LUN on the same logical RAID-drive?

[Rubeck]

I dont know if that's possible.... That has to be up to your Adaptec controller...

You cant add more than one VMFS on the same logical RAID drive...

You have to create a new RAID set and then you add an extent or create a new VMFS... vmhba3:1:1

(vmhbax:y:z:v)

x=Hostbusadapter

y=LUN

z=Disk

v=Partition

/Rubeck

[largie]

That is pretty annoying :_|

So even if I copied all the data from the RAID to other discs and wiped the old partition I can not make a VMFS3 disk that is 2,7TB afterwards. The limitation is 2TB (as stated here http://www.vmware.com/pdf/vi3_301_201_config_max.pdf)

These limits forces me to split up the RAID6 and then lose more space for parity, making the new drives totaly obsolite.

I though ESX was a enterprise high-end system that should be able to handle big RAIDs. Looks like a simple RAID-card is "little league" and a SAN is required.

[Rubeck]

Also... remember to format the VMFS volume with the correct block size, depending on the size of .vmdk files you plan to put on it.

(Default: 1MB = 256GB maximum per. vmdk)

A bit of planning needs to be done before installing ESX and setting up datastores..

/Rubeck

[Texiwill · Accepted solution]

Hello,

Also remember that More LUNs is often better than a single large LUN. Never fill a disk more than 80%, and most LUNs only contain 12-15 VMs. Also, avoid using extents unless absolutely necessary. 1 LUN <=> 1 Partition <=> 1 VMFS (unless it is local disk).

Those are the best practices.... To size a LUN you want to look at the size of the VM and see where you fall (I have posted a formula for this in other posts in this Forum). I standardized in one office on 512GB LUNs and another 750GB LUNs. These work well and give me the 12-15 VMs per LUN for best performance.

There is a common misconception about LUNs that a big one is so much better and generally it is not always the case. As stated, a bit of planning needs to be done. I woul


Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[largie]

Thank you for answering this pretty complete.

It seems like ESX is best use for SANs (or simular) and not integrated RAID-cards. Since it is not possible to easly split up a big pool into smaller virtual discs without loosing to much on parity-disks.

I've only seen demos on how todo it on SANs and ESX is perfect for such use, but on local raid-cards it become "to small" with a 2TB limit. It forces you to use an external NAS or a simple fileshare-server to actually get an amount of data a small designer company (or simular) needs.

Edit:

Please check this out: http://www.applicationdelivery.co.uk/blog/tag/vmfs-limits/

It states that a single VMFS volume can be 64TB. I'm not talking about the VMDK-files and the 2TB filesize limit, I'm talking about the VMFS-volume that is a logicaldrive on my RAID card for 1,8TB which has been increased to 2,8TB where I can see the total capacity to 2,8TB but I cannot change the size of the VMFS-volume. Has there been a misunderstanding?

[darren_wardle50]

I don't know if this is any help but I have managed to add an extend to an expanded logical drive on a Smart Array P400 in a DL380 G5 running ESXi.

However I did get an error after clicking finish and found I had to shutdown all the VMs in order to add the extent. Also the free space did not appear when I tried to create another VMFS partition as you'd expect but it did appear when I clicked Add Extent from the volume properties. I was only adding 2 extra 146Gb drives to an existing 2x 146GB drive mirror to create a 292GB RAID 10

It may be your getting that error message because your total LUN size is over 2TB, in which case your only option is to start again by creating 2 seperate logical drives on the array and either run them separately or add the second one as an extent and restore your VMs.

[Dave_Mishchenko]

Edit:

Please check this out: http://www.applicationdelivery.co.uk/blog/tag/vmfs-limits/

It states that a single VMFS volume can be 64TB. I'm not talking about the VMDK-files and the 2TB filesize limit, I'm talking about the VMFS-volume that is a logicaldrive on my RAID card for 1,8TB which has been increased > to 2,8TB where I can see the total capacity to 2,8TB but I cannot change the size of the VMFS-volume. Has there been a misunderstanding?

You can have up to 64 TB, but that would be using extents (i.e. joining 32 - 2 TB LUNs). If you have 2.8 TB of local storage you would need to break that into arrays smaller than 2 TB. Depending on the RAID controller you could still create a RAID5 arrays across all drives but you would just limit the maximum size of each array to 2 TB.

[largie]

OK, so there is no way around this problem then.

I have to split my 8x500GB RAID6 to two smaller RAID5 solutions like 4x500GB + 4x500GB. That will give me two logical drives on around 1,35TB which will be displayed as two seperated LUNs.

My only concern about this solution is the rate of failure is much higher running two parrallell RAID5 solutions and not RAID6.

Thank you for all the help.

Hopefully in the future the 2TB limit shall be avoided by adding a "virtual SAN" solution so you can run a big pool of disks on a local RAID-card and with the safty of your own choice.

[Texiwill]

Hello,

This really depends on the RAID controller in use. Which one are you using?

On mine I can have multiple RAID LUNs spanning the same set of drives. Also the general rules I stated do not necessarily work when discussing local storage as well. Yes you should still have 12-15 VMs per LUN. With local storage the 1 VMFS per LUN actually goes away. You could create a 2.7 TB LUN and host 4 VMFS' on that LUN using 4 partitions. I would not, but it is possible to do so. But remember that only works for Local Storage as SCSI Reservations do not come into play with Local storage ONLY remote storage.

If you can split disk sets into different RAID volumes I would use that over the multiple partitions.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[largie]

I use the Adaptec 3805 adapter (http://www.adaptec.com/en-US/products/sas_tech/value/SAS-3805/)

Now as far as I understand there is no support for LUNs greater than 2TB, or is that only on fibre-channels?

IF there are an solution to keep the RAID6 solution, even with more partitions, I would prefer that solution.

[Texiwill]

Hello,

You are correct there is a 2TB limit... Can you use the management console to setup a logical drive that spans all the physical drives but is only 1/2 the normal disk space? The raid controller I use will allow me to do that. The best solution would be to see if the Adaptec can also setup multiple LUNs like this.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education.

CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354

As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[Luke_J_Smith]

The 3805 can do "Multiple arrays per disk drive". When creating or migating/expanding the array just set the size to the size you want (&lt; 2TB) and then the remainder of the drives are available for another array. Note that the first array will perform a lot better due to the way disk drives work as the data on the begining of the disk is on the outside, where the surface rate is higher.

You will need to destroy the LUN you have now though, as you cant shrink them.

You can even make the arrays different RAID types.

Cheers, Luke

[Jon_Vickers]

Hey, can you tell me more about how you managed to get access to the available space after extending an existing Raid Array

I have a dell Poweredge 2950. I added 2x146 to an existing 4-146 Raid Array. I cannot get ESX to recognize the new space.