Forbidden Router Part 2: Virtual OPNsense Install Tutorial Inside of Proxmox, no PCIE Passthrough

[@demanuDJ]

I have the same set up for pfSense on Proxmox, no PCI-E passthrough, only on bridges and as Virtio Devices inside VM. But I have LAN bridge attached also to my VMs and containers with no issues at all. Works like this since one year

[@tommybronze3451]

Have you tested that on opensense ? asking because I"m considering jumping the ship from pfsense to opensense ... abundoning the CE is not palatable for me. Also did you had to "disable checksum ofloading" ? I had to do it back in the day when I had pfsense on a VM.

[@demanuDJ]

​@@tommybronze3451 No I didn't but it should work the same way as on pfSense. I didn't have to disable checksum offloading. Now I'm using pfsense on bare metal

[@SataPataKiouta]

As someone who has struggled with OPNsense on Proxmox, I'd have to say superb tutorial. What kind of network card/nics do you suggest? I hear couple of contradicting opinions on Intel cards/nics. Some people say that they are the most all around and the only ones that work great with OPNsense. Others say that they have run into unbeatable issues with Intel cards/nics. 5:11 -> One other thing I do is select the type of Gues OS and set "Other". Since OPNsense is FreeBSD which is not essentially Linux. 16:12 -> Why UFS and not ZFS since your disc is already ZFS, and in general ZFS formatting gives some great benefits? Additional questions: Why set always active the physical NICs? Mine were set to not auto-start for a lot of time now and everything seems to work. What's the downside or what's the gain from auto-starting ? Same question for Firewall on vmbridges. Any particular reason to turn it off ?

[@drreality1]

Iommu tutorial would be great thanks

[@letstalkdev]

Thanks for explaining that I need a bridge reserved for the proxmox and other VMs. Was struggling with this, and wasn't really obvious for me. So basically if I have a 6 port network card, 1 port is proxmox and VMs, 1 port will be WAN and the rest 4 ports will be LAN. Did I get that right?

[@drreality1]

So now you have opnsense separated by a physical network from proxmox. The question is how to route VMs and proxmox management interface through opnsense? Cheers

[@overthinkingtech]

That's why my setup has 3 network interfaces. You can think of it like having a stand alone opnsense router with 2 nics and a proxmox box with 1 nic; all the routing can be done exactly the same as if proxmox and opnsense were physically separate devices. I will cover this in more depth in the next video in the series where I'll be setting up pi-hole and unbound.

[@drreality1]

@@overthinkingtech that’s great thanks, proxmox routing through opnsense seems to not click with me 😫

[@RocketBoom1966]

Excellent work! Thank you.

[@tommybronze3451]

Hey bud, about the problem that you've mentioned that VMBR having issues If you use it for more than just one VM. I think that you might have a problem when you do vlans inside of your vm, and let other people use it. I would suggest to add as many network interfaces as you want to have vlans into VM, while tagging those inside of proxmox and enabling "vlan aware". That way you VM will see (let say) 10 interfaces, while other VM's will be able to use same bridge as router VM.

[@greglonnon]

Do you know the performance difference between pci pass through and linux bridges? I would assume that for a gigibit nic most modern hardware wouldn't have a problem???

[@LampJustin]

First you won't be limited to 10G with virtio. It's just the linkspeed it's advertising but it can do a lot more. Secondly there's very little memory overhead in running a VM (a couple megs), but the VM will do caching of all sorts of things, but mostly Filesystem. So that's totally normal, the memory usage presented in the Opnsense is what it would use if you'd discard the cache. The cache will shrink when more memory is going to be allocated in the VM, so you won't see any crashes unless you do some DPI and stuff.

[@brettjensen125]

What are you adding storage?

[@jmgacousticmusic]

Thanks for the content. One thing im struggling to understand, do you not need a functioning router first that the proxmox server is connected to and the pc connecting to it to manage it? So at what point will the opnsense firewall take over the Lan

[@overthinkingtech]

For my setup I had an existing router and this was the replacement but if you don't have the existing router you can configure the IP address of the proxmox node and connected pc manually.

[@MSMusiXz]

Hey, thanks for your work. 👍 can someone explain me the reason for adding a new storage drive?

[@overthinkingtech]

You can use your main proxmox drive, however, if you have a drive failure this can quickly become a very big headache. Placing the vm's on a different drive simplifies drive replacement when a failure occurs and decreases the chances that the drive with proxmox on it will fail.

[@mjmeans7983]

I wish this were possible with only 2 physical NICs.

[@doddezx]

Im running it with 1 nic u can use VLANS

[@overthinkingtech]

@@doddezx I'm curious because I haven't tested it, how does vlan1 work? Proxmox documentation says to avoid it because of how it's implemented as a management vlan and I need it to work correctly in opnsense because of some down stream tp-link stuff that uses it as the management vlan.

[@overthinkingtech]

It is totally doable. One option is to pass a single nic through to opnsense and configure your WAN to be on a vlan instead of on a dedicated nic. You will want to pick a vlan that nothing else is on. Then connect your modem to an untagged port on your switch with the wan vlan, and make sure the port opnsense is on is tagged with the wan vlan.