passthrough in combination with IOMMU

[bartcuijpers]

Hi,

I'm unable to launch a VM which has IOMMU enabled and is connected to a PCI passthrough device (using ESXi 8).
The error is:

WARNING: PCIPassthru: 7487: Couldn't setup PCI passthru IOMMU domain because the IOMMU can't reach the entire VM's memory range (lastIopn = 0x7ffffff, lastPPN = 0x1ffffffff)

When I disable IOMMU it does work, but in my opinion the performace isn't as fast as expected.

[ptarnawski]

Hey @bartcuijpers ,

I presume you are talking about  Input-Output Memory Management Unit, hence from the error you provided, it looks like you are getting indicates that the IOMMU is not able to map the entire memory range of the virtual machine to the PCI passthrough device. This can happen if the virtual machine has a lot of memory, or if the PCI passthrough device is not compatible with the IOMMU on your ESXi host.

Try this:  

1. Check the amount of memory that your virtual machine has allocated. If it is more than the amount of memory that the IOMMU can handle, you will need to reduce the amount of memory allocated to the virtual machine.
2. Check the compatibility of the PCI passthrough device with the IOMMU on your ESXi host. Some PCI passthrough devices are not compatible with all IOMMUs. You can find a list of compatible devices on the VMware website.
3. If the PCI passthrough device is compatible with the IOMMU on your ESXi host, you can try updating the BIOS on your host. Some BIOS updates include improvements to the IOMMU.

Also try this : 

• Make sure that the PCI passthrough device is not in use by another virtual machine.
• Reboot your ESXi host.
• Update your VMware tools.

Good luck

[bartcuijpers]

Thanks for your response.

I'm sorry to say that the issue only occurs on ESXi 8.
I've tested the exact same system and VM config on ESXi 7, and the VM starts perfectly.

In general, does exposing IOMMU to the OS increase performance? Or shouldn't I bother about it too much?

Bart

[ptarnawski]

Hey Bart,

In general, exposing IOMMU to the OS can increase performance, especially for VMs that are using high-performance devices such as GPUs.

However, there are some cases where exposing IOMMU can actually decrease performance. This can happen if the VM is not configured properly or if the device is not compatible with IOMMU.

If you're running performance-critical workloads that require direct hardware access, it's worth exploring. However, for more general workloads, the performance impact might not be substantial enough to warrant the added complexity.