All nvme drives detected but not all usable. 2 given same "display name" name by NVMEDEV at boot.

[sjbylo]

Hi,

I am seeing all 4 nvme devices (on a PCIe card) in the ESXi 7.0.3 *hardware* list (as expected) but only 3 in the ESXi *storage* list. 

Only 3 devices are usable.   I would expect all 4 devices to be usable. 

In "vmkernel.log" I see that 2 devices are given the same "display name", e.g. :

   "NVMEDEV:5339 Set name of ns 1 to t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70"

The other 2 devices get their own unique names.


"0501021502187C70" is then allocated two paths:  

1. vmhba3:C0:T0:L0 

2. vmhba4:C0:T0:L0

ESXi is "VMware ESXi, 7.0.3, 21424296" 

There is one other SSD in the host, via ATA.  

 

I am able to see and use all 4 NVME drives in Windows and Linux.  

Please can you advise why I can't use all 4 nvme devices on ESXi 7.0?   Thank you! 

[sjbylo]

Here are some more infos:

---

[root@esxi4:/var/log] esxcli storage hpp path list

pcie.5d00-pcie.0:0-t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70
Runtime Name: vmhba4:C0:T0:L0
Device: t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70)
Path State: active
Path Config:

sata.vmhba0-sata.0:6-mpx.vmhba0:C0:T6:L0
Runtime Name: vmhba0:C0:T6:L0
Device: mpx.vmhba0:C0:T6:L0
Device Display Name: Local HL-DT-ST CD-ROM (mpx.vmhba0:C0:T6:L0)
Path State: active
Path Config:

sata.vmhba0-sata.0:0-t10.ATA_____AGI1T0G17AI178__________________________AGISAIVWK0600504____
Runtime Name: vmhba0:C0:T0:L0
Device: t10.ATA_____AGI1T0G17AI178__________________________AGISAIVWK0600504____
Device Display Name: Local ATA Disk (t10.ATA_____AGI1T0G17AI178__________________________AGISAIVWK0600504____)
Path State: active
Path Config:

pcie.5c00-pcie.0:0-t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70
Runtime Name: vmhba3:C0:T0:L0
Device: t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70)
Path State: active
Path Config:

pcie.5b00-pcie.0:0-t10.NVMe____ADATA_LEGEND_850________________________52A6291502187C70
Runtime Name: vmhba2:C0:T0:L0
Device: t10.NVMe____ADATA_LEGEND_850________________________52A6291502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________52A6291502187C70)
Path State: active
Path Config:

pcie.5e00-pcie.0:0-t10.NVMe____ADATA_LEGEND_850________________________D201021502187C70
Runtime Name: vmhba5:C0:T0:L0
Device: t10.NVMe____ADATA_LEGEND_850________________________D201021502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________D201021502187C70)
Path State: active
Path Config:

----

[root@esxi4:/var/log] esxcli storage hpp device list
t10.ATA_____AGI1T0G17AI178__________________________AGISAIVWK0600504____
Device Display Name: Local ATA Disk (t10.ATA_____AGI1T0G17AI178__________________________AGISAIVWK0600504____)
Path Selection Scheme: FIXED
Path Selection Scheme Config: {preferred=none;}
Current Path: vmhba0:C0:T0:L0
Working Path Set: vmhba0:C0:T0:L0
Is SSD: true
Is Local: true
Paths: vmhba0:C0:T0:L0
Use ANO: false

t10.NVMe____ADATA_LEGEND_850________________________52A6291502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________52A6291502187C70)
Path Selection Scheme: FIXED
Path Selection Scheme Config: {preferred=vmhba2:C0:T0:L0;}
Current Path: vmhba2:C0:T0:L0
Working Path Set: vmhba2:C0:T0:L0
Is SSD: true
Is Local: true
Paths: vmhba2:C0:T0:L0
Use ANO: false

t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________0501021502187C70)
Path Selection Scheme: FIXED
Path Selection Scheme Config: {preferred=vmhba3:C0:T0:L0;}
Current Path: vmhba3:C0:T0:L0
Working Path Set: vmhba4:C0:T0:L0, vmhba3:C0:T0:L0
Is SSD: true
Is Local: true
Paths: vmhba4:C0:T0:L0, vmhba3:C0:T0:L0
Use ANO: false

mpx.vmhba0:C0:T6:L0
Device Display Name: Local HL-DT-ST CD-ROM (mpx.vmhba0:C0:T6:L0)
Path Selection Scheme: FIXED
Path Selection Scheme Config: {preferred=none;}
Current Path: vmhba0:C0:T6:L0
Working Path Set: vmhba0:C0:T6:L0
Is SSD: false
Is Local: true
Paths: vmhba0:C0:T6:L0
Use ANO: false

t10.NVMe____ADATA_LEGEND_850________________________D201021502187C70
Device Display Name: Local NVMe Disk (t10.NVMe____ADATA_LEGEND_850________________________D201021502187C70)
Path Selection Scheme: FIXED
Path Selection Scheme Config: {preferred=vmhba5:C0:T0:L0;}
Current Path: vmhba5:C0:T0:L0
Working Path Set: vmhba5:C0:T0:L0
Is SSD: true
Is Local: true
Paths: vmhba5:C0:T0:L0
Use ANO: false

[sjbylo]

Can anybody help?  Is this a FAQ?  Apologies if it is.  

[sjbylo]

I may know the cause now, just in case anybody else has the same problem.....

I see an error in Linux with a slightly older kernel: 
There is an error "globally duplicate IDs for nsid 1" which causes RHEL to "skip" one of the drives and not use it. 

This helps explain the problem:
https://unix.stackexchange.com/questions/711739/globally-duplicate-ids-for-nsid