Verified working procedure • RTX 5060 Ti • Ollama VM

Proxmox GPU passthrough for an Ollama local LLM server.

Operational checklist and command guide for binding an NVIDIA GeForce RTX 5060 Ti to vfio-pci, passing it into an Ubuntu Ollama VM, and exposing Ollama to a Hermes/Godzilla VM over LAN/API.

Execution Progress0 of 0 complete

16Sections

RTX5060 Ti

VFIOTarget driver

Step 1

Confirm GPU exists

Verify Proxmox sees the RTX 5060 Ti GPU and audio function.

Verified

GPU appears as 01:00.0 Audio function appears as 01:00.1

Confirm GPU visibility

lspci | grep -i -E "nvidia|vga|3d"

Verified result

01:00.0 VGA compatible controller: NVIDIA Corporation GB206 [GeForce RTX 5060 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GB206 High Definition Audio Controller (rev a1)

Step 2

Confirm IOMMU

Verify Intel VT-d / IOMMU is active before passthrough work.

Verified

DMAR/IOMMU enabled is visible BIOS VT-d / IOMMU confirmed if needed

Check IOMMU messages

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi

Verified result

DMAR: IOMMU enabled
DMAR: Intel(R) Virtualization Technology for Directed I/O

Step 3

Check driver state

Confirm current kernel driver and module state.

Action

Current kernel driver reviewed nouveau/nvidia/vfio module state recorded

Check current GPU drivers

lspci -nnk -s 01:00.0
lspci -nnk -s 01:00.1
lsmod | grep -E "vfio|nouveau|nvidia" || true

Desired driver

Kernel driver in use: vfio-pci

Step 4

Host driver rule

Do not install NVIDIA drivers on the Proxmox host.

Action

NVIDIA driver not installed on Proxmox host Host role limited to VFIO passthrough

Do not install on Proxmox host

apt install nvidia-driver nvidia-smi

Step 5

Load VFIO modules

Load vfio, vfio_iommu_type1, and vfio_pci at boot.

Action

vfio.conf module file created vfio module file verified

Create VFIO modules file

cat > /etc/modules-load.d/vfio.conf <<'EOF'
vfio
vfio_iommu_type1
vfio_pci
EOF

Verify VFIO modules file

cat /etc/modules-load.d/vfio.conf

Step 6

Bind GPU to VFIO

Bind PCI IDs 10de:2d04 and 10de:22eb to vfio-pci.

Action

PCI IDs 10de:2d04 and 10de:22eb configured vfio-pci binding file verified

Create VFIO binding config

cat > /etc/modprobe.d/vfio.conf <<'EOF'
options vfio-pci ids=10de:2d04,10de:22eb disable_vga=1
EOF

Verify VFIO binding

cat /etc/modprobe.d/vfio.conf

Step 7

Blacklist host GPU drivers

Prevent nouveau/nvidia from grabbing the GPU.

Action

nouveau and nvidia host modules blacklisted blacklist file verified

Create GPU blacklist

cat > /etc/modprobe.d/blacklist-gpu.conf <<'EOF'
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nvidia_uvm
options nouveau modeset=0
EOF

Verify blacklist

cat /etc/modprobe.d/blacklist-gpu.conf

Step 8

Rebuild and reboot

Rebuild initramfs and reboot into VFIO state.

Action

initramfs rebuilt host rebooted and reachable by Web UI/SSH

Rebuild initramfs and reboot

update-initramfs -u -k all
reboot

Step 9

Verify passthrough readiness

Confirm both GPU functions use vfio-pci.

Verified

01:00.0 uses vfio-pci 01:00.1 uses vfio-pci nouveau is not loaded vfio modules are loaded

Post-reboot verification

echo "=== GPU ==="
lspci -nnk -s 01:00.0

echo "=== GPU AUDIO ==="
lspci -nnk -s 01:00.1

echo "=== MODULES ==="
lsmod | grep -E "vfio|nouveau|nvidia" || true

Success criteria

01:00.0 GPU        Kernel driver in use: vfio-pci
01:00.1 GPU Audio  Kernel driver in use: vfio-pci
nouveau is not loaded
vfio modules are loaded

Step 10

Create Ollama VM

Use q35, OVMF/UEFI, CPU host, VirtIO, 32GB+ RAM.

Action

Ubuntu Server 24.04 VM planned q35/OVMF/CPU host selected RAM and CPU allocation checked

Recommended VM settings

OS: Ubuntu Server 24.04 LTS
Machine: q35
BIOS: OVMF / UEFI
CPU Type: host
Disk Controller: VirtIO SCSI
Network: VirtIO
RAM: 32GB+ recommended
CPU Cores: 8+ recommended

Step 11

Add PCI device

Add RTX 5060 Ti with All Functions, PCI-Express, ROM-Bar, Primary GPU.

Action

GPU PCI device added All Functions and PCI-Express enabled ROM-Bar and Primary GPU selected

Proxmox GUI path

Ollama VM → Hardware → Add → PCI Device

Required options

All Functions: checked
PCI-Express: checked
ROM-Bar: checked
Primary GPU: checked

Step 12

Install NVIDIA in VM

Install NVIDIA driver inside the Ollama VM only.

Action

NVIDIA driver installed inside VM nvidia-smi shows RTX 5060 Ti

Install NVIDIA driver inside VM

sudo apt update
sudo apt install -y ubuntu-drivers-common
ubuntu-drivers devices
sudo ubuntu-drivers autoinstall
sudo reboot

Verify NVIDIA inside VM

nvidia-smi

Step 13

Install Ollama

Install Ollama, run a model, verify GPU usage.

Action

Ollama installed Test model runs nvidia-smi shows GPU usage

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Verify and run test model

ollama --version
ollama run llama3.2:3b
nvidia-smi

Step 14

Expose Ollama API

Bind Ollama to 0.0.0.0:11434 and test from Hermes VM.

Action

OLLAMA_HOST override added Ollama service restarted Hermes VM can curl /api/tags

Edit Ollama service override

sudo systemctl edit ollama

Service override content

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Reload and test

sudo systemctl daemon-reload
sudo systemctl restart ollama
ss -tulpn | grep 11434
curl http://OLLAMA_VM_IP:11434/api/tags

Step 15

Troubleshooting

Fix nouveau binding, IOMMU, display loss, and host driver mistakes.

Action

VFIO troubleshooting commands available IOMMU boot parameters documented Host display loss warning understood

If GPU is still using nouveau

cat /etc/modprobe.d/blacklist-gpu.conf
cat /etc/modprobe.d/vfio.conf
cat /etc/modules-load.d/vfio.conf
update-initramfs -u -k all
reboot

IOMMU boot parameters

intel_iommu=on iommu=pt
amd_iommu=on iommu=pt

Step 16

Final verified state

Confirm vfio-pci, IOMMU enabled, nouveau unloaded, vfio modules loaded.

Verified

GPU and audio use vfio-pci IOMMU enabled nouveau not loaded vfio modules loaded

Final verified state

01:00.0 RTX 5060 Ti GPU       Kernel driver in use: vfio-pci
01:00.1 RTX 5060 Ti Audio     Kernel driver in use: vfio-pci
IOMMU                         enabled
nouveau                       not loaded
vfio modules                  loaded

Verified hardware/kernel

NVIDIA GeForce RTX 5060 Ti
GPU PCI ID:   10de:2d04
Audio PCI ID: 10de:22eb
Proxmox kernel: 6.17.2-1-pve

Operational warning

Host display and driver boundary

After the GPU is bound to VFIO, Proxmox may lose local monitor output if the RTX 5060 Ti was the only display adapter. Manage Proxmox through Web UI or SSH. Install NVIDIA drivers inside the Ollama VM only.

Important