VFIO - Update! w/Pop!_os -- PCIe Passthrough even with identical graphics cards

Forum Post:
forum.level1te...
This will guide you through setting up your Linux computer to host a Windows virtual machine with a real graphics card by showing you how to pass through one of your graphics devices. This guide works even if you have identical graphics cards in your host system.
**********************************
Thanks for watching our videos! If you want more, check us out online at the following places:
+ Website: level1techs.com/
+ Forums: forum.level1tec...
+ Store: store.level1tec...
+ Patreon: / level1
+ L1 Twitter: / level1techs
+ L1 Facebook: / level1techs
+ Wendell Twitter: / tekwendell
+ Ryan Twitter: / pgpryan
+ Krista Twitter: / kreestuh
+ Business Inquiries/Brand Integrations: [e-mail]
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to [e-mail].
-----------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons...

[@redneckrestoration9385]

As always, thanks for everything you do, especially linux related

[@Speccy48k]

For the audio passthrough, the Scream project is fantastic (especially when using Looking Glass)

[@motoryzen]

My favorite channels on youtube include Level1linux, Level1techs, Chris Titus Tech, Joe Collins for these for linux content. Among these guys, it's a safe bet most of your linux needs are covered somewhere at some point.

[@nukedathlonman]

OH - PCIe pass through with identical models - NICE!!!

[@ufoludek]

I still have to watch the video but wasn't this a tesla? It supports SR-IOV.

[@johnbuscher]

@@ufoludek no, it's two RTX 2080 cards.

[@lordswaggity1213]

Are you planning on ever levelling up so we can get Level2Linux? 🤔🤨

[@stranger7968]

Need to compile a new kernel with a level2linux support for that.

Linus is Level0

@Gabe Mendoza No

[@sardarasif1858]

Awww yes ! Man I love these pass through videos 🤩 keep it up man

[@samuelschwager]

Windows faster in a VM than on bare metal just sounds wrong. But I take your word for it!

[@rlbk3649]

i don't know what you were talking about but it sounded cool

[@socialistsuccubus822]

"Oh yeah, btw, ESR let me borrow his computer" you realize what a humble brag that is to some nerds? He's practically a celebrity to us

[@Pirxel]

This is still a pretty hard process unfortunately :( looking forward when you beautiful geniuses will make this a bit more noob friendly ;)

[@aktw1234]

that's just Linux in general lol

[@svampebob007]

@@aktw1234 Indeed, it's really not "that hard", but [it] sure isn't noob friendly! if you have a good guide, and do it 1000 times before succeeding you'll actually learn something that will help you in the next battle to get [insert trivial task] working! The worst part is dealing with people that just reply "hurr durr there's a good man page try reading it" Some times the man page isn't noob friendly either! it's the whole "Linux" mentality that takes some getting use to. I'm almost a year on Linux only and I'm definitely not going back any time soon :) examples of non noob friendly things is search for "replace comma with new line with sed" and you'll get 20 different answers, and just a few of them will work.

[@ferror18]

it mostly reading instruction correctly its not any of us could figure it out on our own

[@MrEiselekd]

One passthrough scenario (that I prefer) is to use only one GFX card, boot into the system _headless_ (no gfxcard for Linux by blacklisting the drivers in initramfs) then start Windows KVM on boot with passthrough of the single card so Windows is controlling your screen and input devices. After that you use ssh in cygwin to log into Linux host or if you need a GUI in Linux after all you can install and use No-machine to connect and get a Linux desktop. No-machine forwards audio and usb devices, so you are quite flexible. You can use a Grub menue to select weather you want to start normal Linux or Windows-on-top-of-Linux. This is for setups where Linux is your main platform but you cannot run CAD programs like Altium inside a Virtual Machine under Linux because the emulated GPU performance of the virtual machine is too bad.

[@3lH4ck3rC0mf0r7]

I'd like to see a series detailing the beauties of the Linux kernel :D

[@RenegadeJr100]

6:48 Actually, I found that if you enable the nVidia nView desktop manager, it will fix the Display Port/Nothing Attached issue without using a dongle. VNC used to stop working on remote machine when monitors were powered off, this seems to fix it.

[@bristolemotv]

These videos are amazing. The information I've received is incredible. I'm now a fully AMD man again, because of the BS Nvidia is doing *yet again* on Linux, and because AMD > Intel now. I'm going to eventually build a fully AMD virtualization beast later this year with KDE Neon based on the coming Ubuntu LTS with Linux kernel 5+. I love KDE and playing games, so it'll be perfect for me. It's gonna be an awesome time.

[@damian007567]

can i use a old gpus for passthrough? i don't want to game so the performance doesnt really matter. The card would be a AMD HD 7870 As far as i know only my mainboard has to support imoo right?

[@cappyo]

It works without issues with an 7750 for me. You're fine

[@cidadaoPPT]

AMAZING content! Love these! MOAR!! 😍

[@r3pents7]

HELL YEAH BROTHER!

[@recreationalplutonium]

I love ESR because he loves guns.

[@CheapBastard1988]

I'm excited to dive into this topic when the 3900x is released and I finish the custom loop water cooling for it.

[@emmy4mt12]

More off topic rants, less VFIO. Thx :)

[@brandonphilander661]

Would the same process apply for other Linux distributions? I would like to do KVM vfio gpu passthrough on Void Linux.

[@alkeryn1700]

my favorite way of switching devices was evdev passthrough.

[@StefanUrkel]

Level1Linux: I want to use 2 Radeon VII passed through to Windows VM for video editing (Davinci Resolve). Is the reset bug fixed? Would it be worth the hassle or just do away for the complexity and time investment needed? My other option is to go with a Mac... Davinci Resolve does run on Linux but only specific distributions are supported.

[@LastINAC]

So what do we think would be better performing: passing through a SATAIII SSD for the VM or creating a qcow2 on the same NVME as the host OS? NVME benchmarks 3 times faster than the SATAIII SSD.

[@texasdeeslinglead2401]

Awe heck yes , we've moved up to Frasier music. I'm cool with that.

[@Javierm0n0]

I'm seeing more and more stuff coming out that windows 10 is more borked than people originally knew.

[@steveskipper6473]

Is it just me or does Eric Raymond look like an elderly version of Rich Evans?

[@kkeanie]

Oh yes! Just ask Rich if he likes robot subs

[@mikeyz8775]

Ill check out the forums.

[@cidsapient7154]

with the way linux runs today im not sure why devs are focusing on windows linux today is as simple to use as windows just a few years ago some distros are easily up to windows XP level of development and for ppl that only browse the internet/social media, play flash game, or watch videos theres no point in windows

[@Blink_____]

Won't happen until user adoption hits a critical junction point. It's like asking why car dealers still sell pure gasoline vehicles instead of only electric or only hybrid.

[@KingHalbatorix]

@@Blink_____ no, there are many good reasons to use gasoline vehicles, none the least of which is low cost of maintainance, better infrastructure support, higher energy density and more user control over the vehicle. I would not even consider buying something like a tesla until 3rd party repair is a viable option, let alone being able to control the car for yourself. For someone who seems interested in linux I find it strange that you wouldn't have a visceral repulsion to the way tesla forces software updates on its users; they are legitimately worse than microsoft when it comes to both user rights and repairability. Hybrid and all-electric vehicles are still only viable in major cities.

[@markdavenportjr5129]

Yay another Linux Vid!! :D

[@eldizo_]

Lots of issues from users right in this video! I wonder if it'll ever be plug and play without any hassle or extra hoops to get the VM running. Dual booting is still the way to go in my eyes, even if I don't boot into Windows that much anymore.

[@silvernode]

I think we need a full on PCIe passthrough refresh because I want to know if any progress has been made that makes the set up easier to do.

[@Mastermind12358]

I like the idea of virtual machines, I can install my games on a virtual box and never have to install them again. I don't have to bloat my system with emulation software and stuff like that. I consider it a way to keep things clean. I just setup a new system? Let me just install this virtual machine software and I can run all of my old Windows games without a problem. I sure hope PCIe passthrough gets more noob friendly and easier in the future.

[@DanDishonored]

8:40 "Windows doesn't handle the four node NUMA configuration of the 2990WX as well as Windows" :-D

[@wendelltron]

I meant as well as Linux. Sorry :D

[@DanDishonored]

@@wendelltron Nice video. I wish to have other OS in virtual, but I have two Nvidia cards and some guy from California wont let me to use proprietary driver on host system :-/

[@seanbrisson7928]

5:36 OH So tahts why! Thanks dude

[@japrogramer]

I'm going to get the 12 core 24 thread ryzen thing that's coming out this month. What motherboard would allow for iommu pass thru ?

[@Level1Techs]

Tomorrow (tm)

[@japrogramer]

@@Level1Techs Are y'all going to do a video on iommu on that new hardware? I can wait since I'm building the PC purely for cyberpunk 2077.

[@eliotrulez]

I would also like a video guide with ryzen 3000 :) A real tutorial please.

[@JoelMMcKinney]

Wendell - the ambassador to human evolution.

[@jh-lp7cg]

If everyone moves to Linux, there will be no distraction for the hackers and virus writers. Let them have their Windows target.

[@Suerlink]

Doing my part. I look forward to hearing about an Nvidia solution by the community in the future! Nvidia's VM check is pure evil......

[@MrPunkassfuck]

Tried this? Setting the vendor-id to random 12 chars. forum.level1techs.com/t/solved-nvidia-code-43-on-win10-vm/142795/21

[@tin2001]

Nvidia think tightarse IT people will stick a dozen GTX 1080's into servers and run virtual desktops when they sell an expensive product for that. Its an attempt at profit protection while not actually understanding what people are trying to do.

[@bulcub]

I am going to be running davinci resolve from my hp laptop and I do have a server to commit to pci/gpu passthrough, plus I have huge storage to save videos too. I will be using linux for everything and wanted to know will I still need 2 video cards? or could I get away with 1 so I can do rendering only? Thanks

[@ChrisDecrease]

Hey, Wendell! May we have an updated video when you have time?

[@AayVy]

what about crossfire/sli in a vm?

[@Marin3r101]

Yeah same thing happened before when AMD got to dual core first on server and intel got it on desktop but 2 cores is a far cry from 64 thread 32 core power houses like TR2990WX.

[@texasdeeslinglead2401]

Hold up , another war thunder player?

[@desther7975]

I ended up being unimpressed by Pop!_OS. There was screen tearing on every system I tried it on. Fedora remains my distro of choice.

[@TheFiXaS]

Can you tell name and model of that keyboard with touchpad looks nice . Thank you :)

[@MatthewHensley8304]

Windows will soon go the way of the dinosaur's!!!

[@stranger7968]

with the amount of money bags and lobbying microsoft has/does its very unlikely. Maybe, just maybe, android with desktop support will just overtake the PC market in the future, but for now probably not.

[@lifebarier]

4:29 what font is that?

[@gh975223]

Are you able to assist with a reset type bug on Xeon Phi on PCI pass through? aka power off VM with Xeon Phi attached and the who computer locks up! (needs to be in VM as i can only get it to work in Red Hat 7.3)

[@bijeshbalakrishnan9118]

i have a dell R510 with Dell PERC H800 SAS RAID Controller VVGYD PowerVault 1gb Cache and Battery and hp storage array D2700. i installed proxmox and it can see the storage array. I tried to install freenas but the vm cannot see my array disks. can you help

[@stikkgreen5237]

Is there absolutely, without question, no possible way to have the GPU default back to the linux host when qemu/kvm is shutdown, and allow the card to be used with passthru again after reboot for example?

[@TheBoltcranck]

Level1. Prestige ∞

[@wlmcgrann]

Yess... and if you can build this interocitor (VFIO ) and prove your intelligence, we will send for you via our private flying saucer! Seriously, it is far more simple just to build a dual boot machine. I mean I get it! I hate what MS windows has become too! I long for the golden days of winXP! What is the point of running widows inside of a linux box?!?! Seriously, game developers need to get a clue!... the PC master race is done with Microcrap!

[@MrBiky]

I am having troubles with booting on a passed through SATA SSD. Hoping this new guide will have an answer to what my mistake was (if it was indeed a mistake, posted on the forum a few days ago, nobody knew the solution, I need to rebump my post).

[@Retalak]

Imagine being an ancap. Lol.

[@MrBiky]

@@Retalak I prefer not to talk politics on tech channels, but what's wrong with wanting all interactions between people to be voluntary, ie: consensual?

[@Retalak]

@@MrBiky The problem is that is antithetical to capitalism. Freedom of association can only occur in communism, a society in which the means of production are commonly owned and we produce for consumption rather than exchange. Private ownership of the means of production is directly opposed to free association.

[@MrBiky]

@@Retalak We see property differently. I'd like to live in a society where I can keep what I worked hard to own and where I can exclude others from using the things that I own and where others do the same with their property. This respect for property is respect for the persons who own that property - meaning that we associate freely with each other not only without violating other people's bodies, but without violating their self-ownership. Certain things exist because people exist. If people didn't exist, property wouldn't exist, because it's the labor that went into it that made it to begin with. To capitalists, communists are using force in order to steal our rightful property. To communists, capitalists are "using force" in order to keep / secure / exclude other from using their property (that's what property is, exclusion of use as one sees fit, as one _wills_ it). You should probably be more familiar than me to "property is theft", said by Joseph Proudhon. These are very different viewpoints of the world we live in. No matter what kind of property one owns, it was his work that went into acquiring it, so he deserves it. If I had to live in a society where only certain property is allowed, I'd definitely be disincentivized from obtaining property that would end up being forcefully stolen from me. But I guess that would describe a socialist society, not a communist one, where absolutely no property ownership is allowed (at least, if you apply your views consistently, this is the logical conclusion of communism). With that said, I wouldn't use force to stop you from living in a society where everybody is sharing their property with everybody. However, I'd definitely defend myself and my property if one tried to steal from me.

[@TheRangeControl]

What happened with SR-IOV with the radeon card?

[@liamphillips2918]

I'm curious if you could do this with a windows LTSC version VM. I would be all over that.

[@gsevla]

Hello I have an Acer Aspire 5 notebook with Intel Integrated Graphics and GeForce Mx130. Is it possible to passthrough my intel graphic card to an Mac OS virtual machine using qemu and kvm? I have vtx and vtd enabled and (probably) muxed system, but i already tried a bunch of things and could not bind intel graphics and vfio.

[@aa-vb9tj]

It's hard to do that sometimes on laptops and especially if nvidia is involved.

[@justincase9471]

VFIO-PCI fix for POP! 20.04: www.reddit.com/r/VFIO/comments/g8vdd3/vfio_broke_on_ubuntu_2004_upgrade/

[@peterjansen4826]

Windows updates twice caused a system-breaking bug on my Windows7 installation, both times directly after the first boot after the update. The bug: no interaction between the mouse pointer/cursor (whatever you want to call it) and the windows, I couldn't grab and drag a window, I couldn't activate a window-button. It wasn't the mouse, that worked fine. Strangely opening the task manager briefly recovered the normal interaction between the mouse pointer and the windows and window-buttons, I could drag another window or click a button but only once, twice, maybe three or four times and then I had to close the task manager and open it again. That made me afraid to update Windows7. Later I 'upgraded' to Windows10 and that upgrade wouldn't work, it kept hanging at 99%. When it finally did work the system was extremely slow and little responsive. I wanted to go back to Windows7 but MS couldn't recover it. It said that something was wrong with the windows.old folder but I never touched it other than opening it and looking in it (just looking, I was very careful). So I did a clean install of Windows7 and didn't touch Windows10 again on my desktop computer. Back then gaming on Linux was not an option yet (not with my hardware anyway) so I sticked with Windows7 for some time but I got more interested in using Linux. Previous year I decided to switch to Linux for everything outside gaming and now most of the gaming can even been done on Linux. In short, Microsoft basically chased me to Linux. The perfect operating system does not exist but Linux is a lot better than Windows as an operatig system. Forget about which software does and does not run on both operating systems, just the OS itself is a lot better. Shorter boot times, shorter loading times for almost everythin, around half the RAM usage, faster suspension and shutdown time, the graphical/non-graphical user-interface which you prefer (whatever it is), easier management of software and where on Windows I struggled to find programs for certain tasks (capturing MiDi, modifying PDF documents on Linux I easily find software for all my needs. For the occasional game I will keep booting to Windows as long as it is Needed but when it is not I much rather stay in Linux.

[@socialistsuccubus822]

WAIT. You're on the kinda basis with Eric Raymond that you just.. borrow his computer?

[@joer8854]

I was wondering if you could help me. Obviously buying 2 video cards separately is easier than one but also having different video cards running different monitors is also more ideal as one card to run both isn't going to be ideal or as financially doable. I have a dual monitor setup and my motherboard is a B450M mortar titanium and one is a TV running at 1080 70hz and a monitor running at 1600x900 and I was trying to figure out if I can run 2 video cards one for each monitor since the TV only has to play videos and sports. The ram is 3000 Mhz 2x8 for 16 gigs and the CPU is a 2700x. I'm thinking of something like a rx590 or better for the monitor (I would like to go 1080p 144hz eventually) for the monitor and something like a 570 for the tv eventually.

[@wlmcgrann]

all you need to do is make your machine a dual boot system... just buy an extra drive...even a usb key will do these days with the speed of USB 3.1 10GB. 1) install two drives... one for linux, one for Mircrocrap. 2) press f8 as yur puter boots up 3) select which operating system you want to play a game on 4) your done... enjoy! =P

[@joer8854]

@@wlmcgrann my problem is how to push two monitors. I'm fine running just Linux.

[@SlopeiZCSGO]

would it not be better to make a partition of the size you need for windows, and duelboot?

[@obvious_humor]

No, because then you have to reboot. And you wouldn't be able to access your Linux environment while in Windows.

[@MrPunkassfuck]

I tried to follow the Fedora 26 guide, didn't work on Fedora 30. Found this instead: laketide.com/setting-up-gpu-passthrough-with-kvm-on-fedora/ Replace 'sudo vi' with your favorite text editor, mine is nano. So 'sudo nano'. That isn't installed on Fedora so do: sudo dnf install nano If you want that. Also, replace the lines that start with 'cat' with 'sudo nano'. cat just outputs text in a file, not useful in this case. You want to edit the textfile. Another thing. Trying to put the device ids in vfio.conf didn't seem to work so I stuck em in grub instead. You can find it at: /etc/default/grub. The Cmdline_Linux ending part should look similar to this: iommu=1 amd_iommu=on rd.driver.pre=vfio-pci pci-stub.ids=1002:687f,1002:aaf8. Of course replace it with your ids for GPU and GPUS HDMI/sound. If you have Intel CPU, I think it is intel_iommu=on instead. Followed the guide through to halfpage, worked wonderfully. I'm still struggling with virt-manager but 2nd GPU is blacklisted, right drivers are loaded (the stub-driver) and second screen is black. Just have to figure out the rest. Virt-manager just loads up some shell when the windows install is supposed to start.

[@skipad4306]

why system76 ? why not raptor with power9? just curious

[@David_Quinn_Photography]

I can play GTA5 on Linux? wow only thing I have issues with now is Sony Vegas 14

[@tin2001]

GTA 5 working in proton caused me to back up my data and reinstall Linux as the sole OS on my gaming PC the very next day.... I've had exactly 2 causes to consider Windows since - needing iTunes for an iPhone that wanted a reinstall, and for GTA4 retail (steam version supposedly works, but I own retail). 2 occasions in about 8 months isn't bad.

[@levifig]

What we need is to be able to share our big fat GPU, that we want running 90% on Linux, with the Windows VM, for those last 10% (without an X restart, obviously)… It makes no sense to get 2 beefy cards, and one of them sitting there 90% of the time, unused and idle… :(

[@Oswee]

I tried to get this "pass-through" thing working like for 4-6 times... on Fedora 29 and 30 with GTX 760 and RX580... but... man... every time it just f*'d up my system. Lutris/Steam are just piece of c*p IMO because... it just doesn't work out of the box. Booth times i tried to follow official instructions line by line, character by character... i just ended up by reinstalling whole system. It is impossible to clean them up from the system by just simply uninstalling as it is expected in 21st century. They leaving bunch of garbage behind all over the file system. To be honest, i got my only game i like to play (BF4) working in booth setups, but it was not playable. Lags was too terrible and in Lutris case it just crashed after ~10 min play. Overall, it is great that there is this move... but... as we see from KZbin and web overall - to get it work properly is not a mainstream thing. I seen just a few videos where people got decent performance out of this tech. And there is only few tutorials on the setup. This means that this is hard to get done for a regular users. If that would be simple, we would see tutorials and game plays all over the web. I am trying to set up Ansible playbook for my developer workstation setup. When i will get that done i will be able to easy reinstall it and so i will be able more freely experiment with this thing. (will not be afraid to f*k up my system). Currently, if i messed it up, it take almost whole day to reinstall and configure all the things i need for the development. Actually - it would be great if somebody would create GitHub repo with Ansible playbooks to set this thing up by just simply running that Ansible playbook from other PC. People would be able to commit setups for different distros as well. I hope, some day i will be able to write simple Go application to set this thing up. :)

[@BlitzIn4]

10:44 hit pause. hit play. freak out.

[@cybercat1531]

I've been doing this for ages with a little Linux init boot script. wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Using_identical_guest_and_host_GPUs

[@Gamez4eveR]

I think getting linux to work on my laptop is more difficult than getting macos to work on it lmfao

[@techzone2009]

why popos don't have a different desktop environment

[@Totalschaden-dd4bp]

what's wrong with gnome?

[@solitudesf8111]

@@Totalschaden-dd4bp whats right with gnome

[@motoryzen]

@@solitudesf8111 Amen. I prefer something more usable like mate or cinnamon. I love the speed of xfce but can't get my hotkeys and other things to work in it yet.

[@BobWya]

I read that as poo-poos... I guess you were talking about Gnome?

[@johnbuscher]

@@motoryzen The only thing I don't like from Gnome 3 is the lack of tiling options (ffs, even Windows lets me snap a program to each corner!) But it just werks otherwise. But a good i3 or dwm (awesome works too but I haven't used it) set up is pretty nice for easy of use, especially if you're on a smaller screen.

[@musmuk5350]

Cans you explains how me cans install GtA on windos 7?

[@finarfin9939]

Windows is trash. Its a glorified gaming OS. Anything else Linux does it far more efficiently.

[@obvious_humor]

Disagree that gaming is why people use Windows. DirectX was the last real holdout -- with DXVK, that's less of an issue nowadays. The REAL use case that Windows can do that Linux can't do well? That'd be professional programs. Especially for things like media studios and specialized engineering. CAD is pretty poor on Linux right now. Having replacements for things like Solidworks is still a long way away, unfortunately.

[@DavidCoutinhoCG]

This is specially good for CG stuff that just can't be done on linux, like 3dsmax, and some studios just use this crap, do i care?... newp

[@affabanana2]

First