Please bear with me as I try to explain this problem fully.
I have a server running ESXi. It has a number of VMs and has been working. This server has an LSI RAID controller and had a single hard drive attached to this controller. I am allergic to non-redundant drives, so I added a second drive and, from the RAID controller's boot BIOS (pre-boot of the server itself) I created the mirror and had the controller "rebuild" the second drive, copying an image of the first drive over it. The controller did so and displays a single virtual volume (in its BIOS) as a RAID 1 volume comprised of the two physical disks.
At this point I would expect any server OS to see the single volume. However, ESXi, on boot, sees both physical hard disks (seemingly ignoring the controller's supposed configuration, which is strange to me), and since they both contain identical copies of the ESXi install, it purple-screens complaining about two identical UUIDs. When I bypass this error with the command-line boot option "overrideDuplicateImageDetection", ESXi boots from one of the two drives. vSphere shows both drives as physical drives mounted in the system.
I am relatively certain that the RAID card is maintaining the mirror. The second drive is not part of a datastore. I'd prefer ESXi see the mirror as the single logical volume the RAID card is supposedly offering, but I will settle for ESXi ignoring the second drive altogether.
Any ideas?
Thanks in advance for your assistance,
H.
Hi,
Some RAID controllers operates in RAID or Non-RAID depending on the configuration. If no RAID are created it will operate as generic SCSI cards and when a RAID is created then it will become a proper raid controllers. When you installed the ESXi the card was probably operating in a non-RAID mode and later you converted to a RAID controller mode. Please check the drivers loaded in the ESXi kernel and if necessary please include the RAID controller module and re build the boot image.
Is it possible to let me know the model number of LSI RAID you are using and the output of kernel module listing. I think these dual mode controllers were possible in old server models, not sure they are still available. I used to see this kind of issues with Linux in olden days.
It seems the RAID controller in your system is not a real hardware RAID controller, but a software (fake) RAID controller, which requires drivers in the OS to operate. ESXi only supports hardware RAID controllers which transparently present the physical disks as a logical volume to the OS.
André
Hi,
Some RAID controllers operates in RAID or Non-RAID depending on the configuration. If no RAID are created it will operate as generic SCSI cards and when a RAID is created then it will become a proper raid controllers. When you installed the ESXi the card was probably operating in a non-RAID mode and later you converted to a RAID controller mode. Please check the drivers loaded in the ESXi kernel and if necessary please include the RAID controller module and re build the boot image.
Is it possible to let me know the model number of LSI RAID you are using and the output of kernel module listing. I think these dual mode controllers were possible in old server models, not sure they are still available. I used to see this kind of issues with Linux in olden days.
Thank you - that makes a lot of sense. The card was definitely in non-RAID mode when ESXi was installed - I only had one drive at the time. Can you tell me how to check for the RAID module or to get an output of the kernel modules?
H.
Dear a.p.,
if it were a software RAID, how could I build the RAID and create the mirror as well as rebuild the mirror drive pre-boot from the RAID card's BIOS, with no OS in RAM?
H.
esxcli system module list or vmkload_mod -l will list the whole modules ,
Can you please let me know the exact hardware model ( RAID controller) you are using ?
Output of esxcli system module list command:
VMware offers supported, powerful system administration tools. Please
see www.vmware.com/go/sysadmintools for details.
The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ #
~ #
~ # esxcli system module list
Name Is Loaded Is Enabled
----------------------------- --------- ----------
vmkernel true true
chardevs true true
user true true
procfs true true
vfat true true
vmkapi_socket true true
vmkapi_v2_0_0_0_vmkernel_shim true true
vmkplexer true true
vmklinux_9 true true
vmklinux_9_2_0_0 true true
vmklinux_9_2_1_0 true true
random true true
usb true true
ehci-hcd true true
hid true true
ipmi_msghandler true true
ipmi_si_drv true true
ipmi_devintf true true
procMisc true true
iscsi_trans true true
etherswitch true true
netsched true true
cnic_register true true
e1000e true true
igb true true
vmkapi_v2_0_0_0_iscsi_shim true true
vmci true true
vmkapei true true
healthchk true true
teamcheck true true
vlanmtucheck true true
heartbeat true true
shaper true true
lldp true true
cdp true true
ipfix true true
tcpip3 true true
dvsdev true true
vdl2 true true
dvfilter true true
lacp true true
vmkapi_v2_0_0_0_dvfilter_shim true true
dvfilter-generic-fastpath true true
vmkstatelogger true true
svmmirror true true
cbt true true
migrate true true
esxfw true true
dm true true
nmp true true
vmw_satp_local true true
vmw_satp_default_aa true true
vmw_psp_lib true true
vmw_psp_fixed true true
vmw_psp_rr true true
vmw_psp_mru true true
libata_92 true true
libata_9_2_0_0 true true
usb-storage true true
vmkapi_v2_0_0_0_nmp_shim true true
hbr_filter true true
libfc_92 true true
libfcoe_92 true true
libfc_9_2_0_0 true true
libfcoe_9_2_0_0 true true
rste true true
lvmdriver true true
deltadisk true true
sunrpc true true
nfsclient true true
vprobe true true
vmkibft true true
vmfs3 true true
~ #
I am not certain of the RAID controller model -it's an LSI controller, but I'd have to reboot to see any info about it and I've done too much of that lately - don't want to reboot again until I absolutely have to.
H.
RE: the controller... does MegaRaid 2 sound familiar?
H.
Exactly , This is the card which i have seen in old dell servers which works in both modes ( SCSI and RAID). In your driver list there is no megaraid driver loaded. If i am not wrong i hope you are using two SATA drives to create a RAID1. And VMFS is hosted on a different storage ?
As you can see your controllers are still using libata controllers which makes the system to work as normal scsi cards. Can you please make sure this server you are using is supported by VMware for the specific version.
Now to go further i need following details: The version of ESX you are using, and server model you have. Also lspci -v output from console the esxi.
We have two work arounds :
1. Re-built the boot image ( depends on ESX version). to include the Megaraid2 controller driver - This is a tricky one and need to be very careful not to break the server which is currently booting.
2. Re-install the ESX preserving the VMFS, the new installation will detect the megaraid as it now operates in raid mode. This will be a cleaner and easier method.
Good luck
AKU
hoppah wrote:
Dear a.p.,
if it were a software RAID, how could I build the RAID and create the mirror as well as rebuild the mirror drive pre-boot from the RAID card's BIOS, with no OS in RAM?
H.
For many years now even software RAID controllers have had pre-boot BIOS support for creating and maintaning volumes. However they still require an OS, usually Windows, in order to actually use the drives in RAID mode.
@Westcoaster:
Ugh, that would suck. I sure hope that's not the case here. Although ESX appears to be linux underneath - why couldn't one drop a driver in there for such a situation?
H.
@Aku: ESXi version is "5.1.0 #1 SMP Release build-1065491 Mar 23 2013 10:12:46 x86_64 GNU/Linux"
~ # lspci -v
00:00:00.0 Host bridge Bridge: Intel Corporation Xeon E5/Core i7 DMI2 [PCIe RP[00:00:00.0]]
Class 0600: 8086:3c00
00:00:01.0 PCI bridge Bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1a [PCIe RP[00:00:01.0]]
Class 0604: 8086:3c02
00:00:03.0 PCI bridge Bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode [PCIe RP[00:00:03.0]]
Class 0604: 8086:3c08
00:00:05.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Address Map, VTd_Misc, System Management
Class 0880: 8086:3c28
00:00:05.2 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Control Status and Global Errors
Class 0880: 8086:3c2a
00:00:05.4 PIC Generic system peripheral: Intel Corporation Xeon E5/Core i7 I/O APIC
Class 0800: 8086:3c2c
00:00:1a.0 USB controller Serial bus controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2
Class 0c03: 8086:1d2d
00:00:1c.0 PCI bridge Bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 [PCIe RP[00:00:1c.0]]
Class 0604: 8086:1d10
00:00:1c.2 PCI bridge Bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 3 [PCIe RP[00:00:1c.2]]
Class 0604: 8086:1d14
00:00:1c.3 PCI bridge Bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 4 [PCIe RP[00:00:1c.3]]
Class 0604: 8086:1d16
00:00:1c.4 PCI bridge Bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 5 [PCIe RP[00:00:1c.4]]
Class 0604: 8086:1d18
00:00:1d.0 USB controller Serial bus controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1
Class 0c03: 8086:1d26
00:00:1f.0 ISA bridge Bridge: Intel Corporation C600/X79 series chipset LPC Controller
Class 0601: 8086:1d41
00:00:1f.3 SMBus Serial bus controller: Intel Corporation C600/X79 series chipset SMBus Host Controller
Class 0c05: 8086:1d22
00:00:1f.6 Signal processing controller Signal processing controller: Intel Corporation C600/X79 series chipset Thermal Management Controller
Class 1180: 8086:1d24
00:01:00.0 PCI bridge Bridge: Intel Corporation C608/C606/X79 series chipset PCI Express Upstream Port
Class 0604: 8086:1d74
00:02:08.0 PCI bridge Bridge: Intel Corporation C608/C606/X79 series chipset PCI Express Virtual Switch Port
Class 0604: 8086:1d3f
00:03:00.0 Serial Attached SCSI controller Mass storage controller: Intel Corporation Patsburg Dual 4-Port SATA/SAS Storage Control Unit [vmhba0]
Class 0107: 8086:1d68
00:03:00.3 SMBus Serial bus controller: Intel Corporation C600/X79 series chipset SMBus Controller 0
Class 0c05: 8086:1d70
00:03:00.4 SMBus Serial bus controller: Intel Corporation C608/C606/X79 series chipset SMBus Controller 1
Class 0c05: 8086:1d71
00:04:00.0 Ethernet controller Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic2]
Class 0200: 8086:105e
00:04:00.1 Ethernet controller Network controller: Intel Corporation 82571EB Gigabit Ethernet Controller [vmnic3]
Class 0200: 8086:105e
00:06:00.0 VGA compatible controller Display controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1)
Class 0300: 102b:0522
00:08:00.0 Ethernet controller Network controller: Intel Corporation I350 Gigabit Network Connection [vmnic0]
Class 0200: 8086:1521
00:08:00.1 Ethernet controller Network controller: Intel Corporation I350 Gigabit Network Connection [vmnic1]
Class 0200: 8086:1521
00:ff:08.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link 0
Class 0880: 8086:3c80
00:ff:08.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0
Class 0880: 8086:3c83
00:ff:08.4 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0
Class 0880: 8086:3c84
00:ff:09.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link 1
Class 0880: 8086:3c90
00:ff:09.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1
Class 0880: 8086:3c93
00:ff:09.4 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1
Class 0880: 8086:3c94
00:ff:0a.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 0
Class 0880: 8086:3cc0
00:ff:0a.1 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 1
Class 0880: 8086:3cc1
00:ff:0a.2 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 2
Class 0880: 8086:3cc2
00:ff:0a.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 3
Class 0880: 8086:3cd0
00:ff:0b.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Interrupt Control Registers
Class 0880: 8086:3ce0
00:ff:0b.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Semaphore and Scratchpad Configuration Registers
Class 0880: 8086:3ce3
00:ff:0c.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0
Class 0880: 8086:3ce8
00:ff:0c.1 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0
Class 0880: 8086:3ce8
00:ff:0c.6 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 0
Class 0880: 8086:3cf4
00:ff:0c.7 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 System Address Decoder
Class 0880: 8086:3cf6
00:ff:0d.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0
Class 0880: 8086:3ce8
00:ff:0d.1 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0
Class 0880: 8086:3ce8
00:ff:0d.6 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 1
Class 0880: 8086:3cf5
00:ff:0e.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Processor Home Agent
Class 0880: 8086:3ca0
00:ff:0e.1 Performance counters Signal processing controller: Intel Corporation Xeon E5/Core i7 Processor Home Agent Performance Monitoring
Class 1101: 8086:3c46
00:ff:0f.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Registers
Class 0880: 8086:3ca8
00:ff:0f.1 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller RAS Registers
Class 0880: 8086:3c71
00:ff:0f.2 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 0
Class 0880: 8086:3caa
00:ff:0f.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 1
Class 0880: 8086:3cab
00:ff:0f.4 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 2
Class 0880: 8086:3cac
00:ff:0f.5 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 3
Class 0880: 8086:3cad
00:ff:0f.6 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 4
Class 0880: 8086:3cae
00:ff:10.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 0
Class 0880: 8086:3cb0
00:ff:10.1 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 1
Class 0880: 8086:3cb1
00:ff:10.2 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 0
Class 0880: 8086:3cb2
00:ff:10.3 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 1
Class 0880: 8086:3cb3
00:ff:10.5 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 3
Class 0880: 8086:3cb5
00:ff:10.6 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 2
Class 0880: 8086:3cb6
00:ff:10.7 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 3
Class 0880: 8086:3cb7
00:ff:11.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 DDRIO
Class 0880: 8086:3cb8
00:ff:13.0 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 R2PCIe
Class 0880: 8086:3ce4
00:ff:13.1 Performance counters Signal processing controller: Intel Corporation Xeon E5/Core i7 Ring to PCI Express Performance Monitor
Class 1101: 8086:3c43
00:ff:13.4 Performance counters Signal processing controller: Intel Corporation Xeon E5/Core i7 QuickPath Interconnect Agent Ring Registers
Class 1101: 8086:3ce6
00:ff:13.5 Performance counters Signal processing controller: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 0 Performance Monitor
Class 1101: 8086:3c44
00:ff:13.6 System peripheral Generic system peripheral: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 1 Performance Monitor
Class 0880: 8086:3c45
~ #
What model is the LSI controller? I don't see it in the device listing. Is it possible the drives are connected to the Intel controller?
Hi, Apologiesfor delay. was busy and was not able to come back quickly.
Ok, For cisco servers you can try this image Cisco Custom ESXi 5.1.
These image will be custom built and tested to work with Cisco Servers. To make sure your Megaraid is loaded during re-install, Just make sure you have created the RAID 1 volume on the controller before you re-install the new ESXi image.
Hope this helps
AKU
Will this use my current license key or will I have to get a new one?
H.
It should be using the same license key
Dear Aku,
Thanks for all the feedback - I'm in the middle of several weeks of travel and haven't had time to rebuild the server, but I'm certain you've found the root of the issue. I appreciate the assistance and will update when I get time to reinstall.
H.