Error adding more than one PCI Passthrough device to vm with add-PassThroughDevice

[tgbennett]

I have a Cisco B200M4 with 2 NVIDIA M60 GPU's and am trying to add  the four GPU's to a VM as PCI passthrough devices.

I am able to configure the GPU's in the host as Passthrough devices and I can add the GPU's to a vm through the fat client and web client. When I try to add all four to a vm two are added then the other two throw an error.  Add-PassthroughDevice The maximum number of PCI devices has been reached for VM.

I tried

$esx  = Get-VMHost -Name $MyHostName

$MyVM  = Get-VM -Name $MyVMName

$esx | get-passthroughdevice | Where {($_.State -eq "Active") -and ($_.Name -like "NVIDIATesla*")} | add-PassThroughDevice -VM $MyVM

and

$MyGPUs = $esx | get-passthroughdevice | Where {($_.State -eq "Active") -and ($_.Name -like "NVIDIATesla*")}

$MyGPUs | foreach{add-PassThroughDevice -VM $MyVM -PassthroughDevice $_ }

I also tried adding the third by itself after adding two. add-PassThroughDevice -VM $MyVM -PassthroughDevice $MyGPUs[2]

Thanks.

[LucD]

Which PowerCLI and ESXi version are you using?

This bug has been reported before.

See for example  How to add more than two devices with PowerCLI

I just tried this again with PowerCLI 6.5.2 in a vSPhere 6.5U1 environment, and I don't get that error.
But it could be caused by other settings I might not have in this environment.

[tgbennett]

I saw that article. there is also one in the VMWare communities. Re: powercli 3rd pci device, order irrelevant fails.

I am running PowerCLI 6.3 Release 1 build 3737840 and ESXi 6.0U2 build 3620759. I can try updating PowerCLI.

I have been hesitant to update due to a change that was made for a command I used frequently in my scripts and the upgrade requires me to change the command in my prod scripts. I think it was something to do with being a module instead of a snapin.

thanks

[LucD]

Yes, since PowerCLI 6.5R1 it's only modules, no more PSSnapin.
And since 6.5.1 (which is not the same as 6.5R1), PowerCLI is located on the PowerShell Gallery.

See Welcome PowerCLI to the PowerShell Gallery – Install Process Updates

Note: I'm not sure there was a fix in the more recent PowerCLI builds, nothing is mentioned in the Release Notes.

On the other hand, I don't seem to have the problem with 6.5.2.

But the issue could be caused by other limitations which I don't happen to have in my test setup.

[tgbennett]

LucD,

Instead of using the xxx-PassThrouhDevice cmdlets could I access the devices by dot-sourcing them. Similar to

$esx = Get-VMHost -Name $esxName

$ptMgr = Get-View -Id $esx.ExtensionData.ConfigManager.PciPassthruSystem

$ptConfig = New-Object VMware.Vim.HostPciPassthruConfig

$ptMgr.UpdatePassthruConfig($config)

Like you did with configuring passthrough devices in a host here How to create a VMDirectPath IO (Paththrough) device by using PowerCLI

but for a VM?

-tim

[LucD]

Yes, that would be possible, but you would be using a ReconfigVM method, since it's done on the VM.

Alan did a post on that some time ago, see ADDING A VGPU FOR A VSPHERE 6.0 VM VIA POWERCLI

[tgbennett · Accepted solution]

For future reference I was able to add more than two passthrough PCI devices.In my case four GPUs. This is the code I came up with

$MyHostName = "<esx host name>"

$esx  = Get-VMHost -Name $MyHostName

$MyVMName = "<vn name>"

$MyVM  = Get-VM -Name $MyVMName

# Add GPUs to vm

$MyGPUs = $esx | get-passthroughdevice | Where{($_.State -eq "Active") -and ($_.Name -eq "NVIDIATesla M60")}

$MyGPUs | foreach{

    $spec = New-Object VMware.Vim.VirtualMachineConfigSpec

    $spec.deviceChange = New-Object VMware.Vim.VirtualDeviceConfigSpec[] (1)

    $spec.deviceChange[0] = New-Object VMware.Vim.VirtualDeviceConfigSpec

    $spec.deviceChange[0].operation = 'add'

    $spec.deviceChange[0].device = New-Object VMware.Vim.VirtualPCIPassthrough

    $spec.deviceChange[0].device.key = "-100"

    $spec.deviceChange[0].device.backing = New-Object VMware.Vim.VirtualPCIPassthroughDeviceBackingInfo

    $spec.deviceChange[0].device.backing.deviceName = "NVIDIATesla M60"

     $spec.deviceChange[0].device.backing.id = $_.ExtensionData.pcidevice.id

     $spec.deviceChange[0].device.backing.deviceId = "13f2"

     $spec.deviceChange[0].device.backing.systemId = $_.extensiondata.systemid

     $spec.deviceChange[0].device.backing.vendorId = "4318"

     $vmobj = $myVM | Get-View

    $reconfig = $vmobj.ReconfigVM_Task($spec)

}