Caddy Reverse Proxy using DNS Challenges and CrowdSec Multi-Server Config with OPNsense

[@crowdsec]

Awesome video! Glad to see the crowd power being demonstrated perfectly. The only side note I would add is I don't remember seeing how to add a caddy acquisition 👍 - Laurence CrowdSec Support

[@homenetworkguy]

Thanks! I've been wanting to do a multi-server CrowdSec configuration with my reverse proxy for quite a while but also wanted to try out Caddy so I finally sat down to learn more about it. What are you referring to about how to add a Caddy acquisition? Just making sure I didn't miss anything since I think everything seems to work well, but I also want to do things proper and better if I'm missing something important. This was a bit of a new experience but it was a fun rabbit hole to dig into.

[@crowdsec]

@@homenetworkguy No worries, so after installing the Caddy collection, you didnt show how to configure CrowdSec to monitor the caddy logs. In premise the Caddyfile which you used would mean that all logs would go to stdout which is fine but might be worth it to show how to configure the log file and how to setup CrowdSec to find the log files. - Laurence CrowdSec Support

[@homenetworkguy]

Ahh ok. I assumed Caddy produces log files and the CrowdSec agent just reads those logs based on the Caddy collection. Clearly there’s more to dig into! Haha. Thanks for the info!

[@cyrilpinto418]

Thank you for everything you do; I initially had it setup via Nginx Proxy Mgr using your guide, and was able to set this up using DuckDns. Thanks once again.

[@homenetworkguy]

You're welcome! I have moved everything on my network over to Caddy since I've seen where Nginx Proxy Manager doesn't always keep up to date with all of the latest Nginx security vulnerabilities so it ends up using older versions of Nginx which are vulnerable.

[@JasonsLabVideos]

Good video man !! Always full of good information and detail !

[@homenetworkguy]

Thanks! My favorite videos are demonstrating real world examples but they take more work to setup the environment and to cover all the details.

[@jocelyn-n-tech]

Thanks! I do things the same way you do by running LXC containers on Proxmox for everything instead of just spinning up a Docker container because I like to know how everything works under the hood.

[@homenetworkguy]

Nice! Plus you don't have to mess with Docker's networking/firewalling (you can use ufw without making any tweaks!). I find it very easy to put LXCs on various networks in Proxmox. I'm not quite sure how I would go about putting Docker containers on different networks (I would need to research that topic further to know how to do that since when I used Docker in the past, I used the containers on the same network).

[@janouchouchou]

Hey. Just a short question… why you didn’t use the official Opnsense Caddy (os-caddy) plugin? Are there any drawbacks?

[@homenetworkguy]

Funny you mention that. The Caddy plugin was released right after I had finished the video! Haha. Also I don’t know if you can add any custom modules because you have to compile a new executable file to add modules. That would be the biggest reason not to use it. I personally prefer to run the reverse proxy in the DMZ network because if the proxy gets compromised, at least it’s not running on the OPNsense box itself. Not sure how much extra security it buys but it might be a little more secure.

[@janouchouchou]

Yes, maybe that is the better solution. Thanks

[@hugobrito361]

What are your thoughts on setting up Caddy directly on OPNsense instead of having a dedicated VLAN and VM just for Caddy? This way, external traffic could be directed to an internal server without exposing web servers or services directly to the internet. Have you considered this approach?

[@homenetworkguy]

Either approach you are still exposing Caddy to the Internet. In my example, I only expose Caddy in the DMZ network to the Internet. My apps are on other internal VLANs so they’re not being directly exposed to the Internet. I created this guide like 1 week before the Caddy plugin was available on OPNsense. Haha. Also one thing I worry about with running a reverse proxy on OPNsense is what happens if the reverse proxy is compromised? Now you’re on the same box as the router/firewall… does that mean it’s easier to compromise the entire firewall at that point? I like the DMZ network approach because if the Caddy server gets compromised at least it can be contained to the DMZ network instead of potentially bad things happening on the OPNsense box. I like having that separation and it allows you to swap OPNsense for something else more easily in the future if you want.

[@Monviech]

@@homenetworkguy Theres GUI options available to run the Caddy plugin on OPNsense as www user/group, which can limit what an attacker could do if they break out of caddy. They wouldn't have elevated rights. Though, using a different server is always a good choice too to limit what an attacker could potentially do.

[@homenetworkguy]

@@Monviech Thanks for this info! I haven't dug into the plugin that much yet but that is good to know.

[@janiel471]

You got great tutorial as always. Thank you so much. ❤

[@homenetworkguy]

Thanks! You’re welcome!

[@Shrp91]

Love your videos and this channel!

[@homenetworkguy]

Thanks! I’m taking a short hiatus from technical guides to do some demos/projects with some sponsored products I’ve received lately, but I’ll get back to it after a couple more videos. Kinda of need a break even though “the Algorithm” might not like it as well. Haha.

[@Shrp91]

@@homenetworkguydo what you gotta do to enjoy yourself

[@videoproductionsecrets6705]

I'm getting SSL_ERROR_INTERNAL_ERROR_ALERT, what can cause it? I checked the certificate in acme, and it says "ok"

[@homenetworkguy]

Hmm, have you tried clearing your browser cache?

[@videoproductionsecrets6705]

@@homenetworkguy Yeah, I tried to use 3 browsers, and cleaned the cache. I installed HAProxy after that, and it works including the certificate. Don't know why I've got the error with SSL with Caddy

[@homenetworkguy]

Earlier I was slightly confused and thought you were commenting on the OPNsense Let’s Encrypt video.. (I always have lots going on..). I’m not yet an expert on troubleshooting Caddy but if the service behind the reverse proxy is HTTPS, you may have to include some additional settings to accept the self signed certificate, etc. I haven’t tested that yet since all my apps behind the proxy are HTTP (currently).

[@kimskaugvoll5499]

any reason not to use the caddy plugin in opnsense?

[@homenetworkguy]

The Caddy plugin was literally released a few days after I published this video! haha. I think it's great it was added, but there are a few reasons you might not to use it: 1. I'm not sure if you can add custom modules (I haven't looked yet) to extend the base functionality like I did in this video (to support Let's Encrypt as well as CrowdSec). 2. From a security point of view, I feel a bit uncomfortable if my web server/reverse proxy is compromised and the software is running on my firewall. This is more of a concern if you are publicly exposing the reverse proxy. I prefer to run my reverse proxy on my DMZ for public facing services (which I have very few things exposed and protected in various ways).

[@kimskaugvoll5499]

@@homenetworkguy that makes sense, I was just curious. Seeing I’m trying to decide between HAproxy and caddy, since kemp is throttling and can’t be updated without a subscription. I just have a couple of things I want to expose on my DMZ. Any recommendations? Mainly Nextcloud to begin with.

[@homenetworkguy]

I have 2 services exposed. I used a Cloudflare proxy and create firewall rules on my WAN interface to only allow connections from Cloudflare IP addresses (they publish a list you can use in your firewall rules). Then I have the CrowdSec module protecting Caddy and CrowdSec on OPNsense (in addition to Zenarmor) protecting my entire network. Also use ufw on the Caddy LXC in Proxmox to restrict access to only the SSH/HTTPS ports. I have OPNsense configured to isolate the DMZ network from the rest of my internal networks. I host my apps/services on a separate VLAN so only the Caddy reverse proxy lives on the DMZ network (so if it gets compromised, it would have to try to get to my services on a separate network).

[@kimskaugvoll5499]

@@homenetworkguy that’s sounds like a great way to do it. Do you have a tutorial on how to set up Cloudflare like that? And caddy/crowdsec is done like you do it in this video I’m guessing.

[@homenetworkguy]

I have a written guide for it, but I don’t have videos for my backlog of website content: homenetworkguy.com/how-to/deploy-nginx-proxy-manager-in-dmz-with-opnsense/

[@darksidediver17921]

Thanks for your videos they are truly helpful. With that said its has been 2 hours since I followed our guide step by step and still I have not gotten a valid Let's Encrypt cert can you point me in the right direction to resolve this?

[@homenetworkguy]

Thanks! You may need to check the logs to see if you have any errors. You can usually see the errors with the Caddy logs. Verify you’re using the correct API key for DNS challenges.

[@darksidediver17921]

@@homenetworkguy Thanks, for the response. I'm using the right API key as I generated a new one just for this project. I will read more through the documentation to figure out how to get to the logs to scrub them thanks again. If, I can't figure it out I will just wipe everything and start over..

[@geekcruz]

Awesome video, worked perfectly for me. I don't know if you noticed as well the plugin for OPNSense for caddy?

[@homenetworkguy]

Yes. I made this video like 1 week before the plugin came out. Haha. I personally prefer not to run it on the firewall because I worry about what happens if the reverse proxy gets compromised. The good thing about doing it this way is you can the proxy on a DMZ network and keep it isolated from the rest of your network. However someone assured me that it may not be that bad since it may be a less privileged user or the plugin runs in a jail or jail-like environment. I can’t recall the details off the top of my head.

[@rashedobaid]

Is it good practice to use a reverse proxy on top of any firewall or use its built in plugin like ACME for certificates?

[@homenetworkguy]

You could do that if you don't have or want to use a dedicated server for a proxy. I personally prefer to run a reverse proxy in the DMZ network for anything that is publicly hosted (which I try to minimize and ensure that I have it protected the best that I can-- most of my stuff is only accessible via VPN into my home network). The reason being is that if the reverse proxy gets compromised, it's not running on the same system as my firewall. I'd rather have it quarantined to the DMZ network. Recently since I switched to Caddy, I decided to run multiple Caddy servers-- one for public services on the DMZ and one for internal services that live on a separate APP network. Since I'm running the Caddy server in a LXC container, I just duplicated the container and put them on 2 separate networks so it was quick to set up multiple instances. This allows me to have public and private services separated but still be using secure connections even on my internal network (internally it's not quite as important but it's nice to get rid of browser warnings and it could help prevent sniffing traffic if something bad got on your network).

[@palimali3336]

A guide for caddy and crowsec only would also be helpful. There is non available :-)

[@homenetworkguy]

You mean a guide without including the OPNsense portion? The main difference is that you could run the LAPI on the same system that Caddy is hosted on (which is the default configuration of CrowdSec-- in the video I had to change it to use the OPNsense LAPI instead of the local default LAPI).

[@palimali3336]

@@homenetworkguy I will try it again, thanks for the great work

[@ronnybeer471]

Great Video. A video about HAProxy on OpnSense and Authelia on Proxmox LXC will be cool. 👍

[@homenetworkguy]

Thanks! I’ve thought about doing a HA Proxy guide before. Then I started using reverse proxies that are not hosted on the firewall itself (because it feels less risky in case of a compromise) and never went back to try it out. I know a lot of people like to use it because it’s built into OPNsense as a plugin.

[@sirdurex5581]

hey, nice video! May i ask why you dont use haproxy? BTW: Did you do some yt-vids for zenarmor? :)

[@homenetworkguy]

I tried using it a long time ago but I was getting frustrated getting it set up (I was probably missing something fundamental that I didn’t understand back then). Later I decided I’d rather have the reverse proxy on my DMZ network rather than my firewall in case it ever got compromised. I found Nginx Proxy Manager easy to use but more recently moved to Caddy after concerns that NPM doesn’t always keep up with the latest Nginx security vulnerabilities. I created this video right before the Caddy plugin came out for OPNsense (but as I mentioned before I prefer to have it in the DMZ). Yes, I do some video work for Zenarmor!

[@Monviech]

​@@homenetworkguy Additional Opnsense VM in the DMZ with the os-caddy plugin