HAProxy-WI: Run Lots Of Public Services On Your Home Server

linode.com/lev...
forum.level1te...
github.com/Aid...
haproxy-wi.org/
**********************************
Thanks for watching our videos! If you want more, check us out online at the following places:
Website: level1techs.com/
Forums: forum.level1tec...
Store: store.level1tec...
Patreon: / level1
L1 Twitter: / level1techs
L1 Facebook: / level1techs
Wendell Twitter: / tekwendell
Ryan Twitter: / pgpryan
Krista Twitter: / kreestuh
Business Inquiries/Brand Integrations: [e-mail]
IMPORTANT Any email lacking “level1techs.com” should be ignored and immediately reported to [e-mail].
-----------------------------------------------------------------------------------------------------------
Intro and Outro Music By: Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
creativecommons...

[@paulwratt]

thanx for this, I had just started investigating what I could use to set up this exact same senario for 2x seperate private networks, both on RPi's - the fact that it has already been prototyped for two seperate users/networks, AND documented, has already taken a load off my mind, especially the securty aspect of it all - cheers, and thanx again

[@JoaoSilva-gs5jb]

so much gold being spilled for free man, thanks uncle Wendel, you make us better IT Masters

[@antaishizuku]

I'd love to see you do a collaboration with Lawrence systems.

[@roxy-wi2201]

Nice video, thank you! Couple little remarks: HAProxy-WI doesn't take all stuff from SSH. Main part information provides from sockets. Also GUI can install haproxy, nginx and keepalived and configures your hosts, so you don't need install them by manually

[@xryz]

OMG I was literally wanting to do this and was research. Was going to make a forum post asking about software.

[@kurtpeterson4652]

I have been looking for some of these answers for almost as long as your video has been posted... Thank you!

[@TerenceKearns]

Dude this is ace. I can’t believe I haven’t done IT in like 20years but I understood everything you just explained. Last time I did this was at uni of Canberra in 2003 using Apache as a reverse proxy facing the internet and everything else was behind it. I was actually employed as a web dev and when I rocked up I was like “hey reverse proxy” and they (systems) were like “oh fuck, why didn’t we think of that” lol.

[@michaelpelley2815]

Damn - just noticed the Digital Unix box in the background. Used to have that in miy office. Looonnnnggg time since I saw that!

[@brianmccullough4578]

Man Wendell, no wonder your Linux videos take awhile! That was a work of art! Its like if Leonardo DaVinci was a computer janitor/plumber. Keep it up! Im definitely gonna try this

[@Level1Techs]

Wait till I explain how to add snort/securicata for moar awesomeness

[@sanjacobs6261]

Thanks for making this video, Wendell! You're a legend, extremely interesting stuff!

[@MartinPaoloni]

I was trying to do something like this yesterday! I'll revisit this video during the weekend. Thanks Wendell!

[@Simon8162]

This is pretty neat. I've done a similar thing for email, except rather than using a proxy, using postfix to forward via SMTP to my home server. This way I get more control over the email protocol, and messages can queue up for days rather than hours if there is a network issue.

[@autohmae]

Depends what you prefer, your VPS provider reading your incoming mail or forwarding the encrypted TCP-connection from the VPS to your home machine.

[@geogmz8277]

When Wendell comes out with a Linux video I don't even need a notification my Nerdy Sense just tingles.. 😏 By the way check Nginx Proxy Manager stupid simple to use with Docker.

[@Xantioss]

Also check the nginx-proxy-manager assistent on github 👍

[@dayvie9517]

No don't

[@MikeBohde]

I've been thinking of hosting my own email for a while this is certainly a good starting spot.

[@gardiner_bryant]

If I wanted to forward client headers through the proxy to the backend, how would I do that with HA Proxy? I'm using OPNSense rather than PFSense because PF doesn't support the NICs in my hardware.

[@mjarkk]

Great video, for a company i created a drag and drop GUI tool to create ha configs that could be directly deployed to proxy servers.

[@GardenOnTopIndia]

I use Haproxy in my day job. Pretty cool piece of work.

[@JzJad]

The amount of ACLs on my pfsense box is getting insane for haproxy, that coupled with the dynamic dns service is wonderful.

[@Crazytje123]

Been running like this for years, but imo in a easier way. Could be more secure and easy to set up if between the VPS and your home connection you use a VPN, then use SSL termination on the haproxy before forwarding. This makes things easy to manage imo, the cert used is in 1 location and the haproxy config is pretty simple. In my case there is a VM with docker containers that serve anything from web to tftp, pxe servers and more on the LAN. For the VPN, you can use pfsense or in your NAS Additionally for the letsencrypt cert, when using pfsense the renewal can be handled for that by it. A tip for this, add a post script to put the cert on your NAS. That location can then be mounted on your VPS due to having the VPN, run an automatic reload script when the cert changes. This way there is 0 down time and no intervention needed.

[@pililogan5769]

Wendell, this video ís awesome! Thank you.

[@arvindhn036]

I use a similar principle at my homelab. Instead of HA proxy you could also use docker + wireguard to do the same. My setup runs a wireguard server on linode which forwardes the traffic to port 80 on my homeserver where I've setup letsencrypt + Reverse Proxy to serve the services from my server. All the docker containers are installed on my homeserver and only packet forwarding and point to point vpn is setup on my public server.

[@ArmandoCalderon]

Amazed, great explanation. Great tutorial.

[@Maxjoker98]

I used to do something similar with SSH to make a webserver on my notebook/phone available(behind an authenticating proxy) via my VPS. Also that security by obscurity thing. I used to host a small website at home. But traffic was getting annoying. My solution? Drop ICMP pings at my router. My website was still available, but almost all scanners ping'ed before trying HTTP.

[@mjbates]

This is very similar to what Helm is doing, but they are selling a complete solution for ~$500.

[@ChrisBurnes]

Nice video Wendell, I literally just got done doing this for my home. Minus the linode server. I was just considering doing this, and was looking at AWS and Google's offerings, when this video popped up! I may give Linode a shot. I'm curious do you run a VPN as well? For access to less "securable" services/appliances? Or if you lean more towards this method of publishing what you can and staging them behind the HAproxy?

[@cptechno]

QUESTION: Why couldn't you also use NGinx as a load-balancer also? Why go to HA-Proxy?

[@damian007567]

Was just about to setup a nginx reverse Proxy to my NAS. Great timing Edit: Just watched the thing, two question remain for me: 1. Why should i not just use a standard nginx reverse Proxy, doesn't it do the same? (Except for TCP I suppose) 2. Anyway to do this with a dynamic home ip? THe only thing i can come up. Is to check on my nas as cronjob, and then automaticly update the IP in the conifs via SSH. Any better way?

[@kassim3]

In regards to question 2. Look into Dynamic dns(ddns). You can use something like duckdns or your router company might provide a free ddns because you're using their router. I used to have an Asus router and they gave a free ddns

[@damian007567]

@@kassim3 But i can only provide a IP for the hlproxy thingy right?

[@LampJustin]

@@damian007567 you can use hostnames for the services you expose ;

[@LampJustin]

As for question 1 there's a great webui for a nginx reverse proxy called nginxproxymanager.com It ticks all the boxes and yes running nginx as a rev proxy works pretty much the same. You can also forward tcp as well

[@JzJad]

In your case no point in swapping.

[@iwsfg]

Dude from the picture at 2:38 reminded me of Qain a little. Should have asked him to reenact it for you as a cameo if you still talk

[@ViniciusProvenzano]

Just gave it a try. Good stuff!

[@HANEEF95]

Hey @level1linux, Have you tried this with Cloudflare Tunnels? They're free now, and it only requires outbound connections.

[@bobruddy]

I've done a similar thing, but I don't have any ports open on my home firewall. I have wireguard setup where home is the client and my hosted virtual machine is the server. So home exstablishes the connection out to the data center and the proxy connections come back over the VPN.

[@marcpearson2913]

Just wondering why you didn't go down or at least an include an option to use DNS API-keys in-place of the http(s) challenge response? I know it's not supported by all DNS hosting services, but at least with the option, people would have a possibility of doing cert updates with 0 down time? Or the alternative is use http host based challenge responses, which is something nginx is quite good for.

[@mikeyz8775]

You will be just fine as long as decent passwords are used and not displayed in plain text config files on your remote hosting service

[@pieterrossouw8596]

Nice setup, I've been using a Raspberry Pi running the swag docker container. My AWS Route53 domain CNAME entries all point to duckDNS (DDNS) which is kept up to date by the same Pi. Ports 80 and 443 forwarded to the Pi's 180 and 1443 ,which Docker then maps back to 80 and 443 for certificate validation. I can then point any subdomain.mydomain.com registered in Route53 to something in my local network using proxy-conf files for nginx. I'm not suggesting it's better than the HAProxy-WI setup, but it's a low-effort alternative for simple home hosting.

[@thomasesr]

Linode also blocks port 25, 587 and 465 unless you send them support ticket to open it.

[@cvx10]

Great content as always...

[@roschereric]

Nice video and idea! Have you thought of protecting this with a WAF?

[@andrewwilson7169]

To solve the issue of passing root ssh credentials that allow a remote user to change your haproxy config without creating the unnerving threat of such a user doing bad stuff there is a very easy solution available - run the haproxy server in a separate container (I use LXD) - then yes, the haproxy can be changed by root but root access is restricted to an haproxy container only. Thre's nothing else to run. To break the security on that, you first have to ssh into the container (easy - but only to those who have the ssh key, i.e. your remote 'root' user) but then you need a working vulnerability/exploit that can break you out of the container into your host machine (where root access is a much more serious breach), which is very hard to do, even for experts and nation states, especially if you keep the container up to date automatically (since exploits normally gets patched very quickly, often before the vulnerability is publicly identified). My haproxy server is thus an LXD container, and it sends traffic to my different home servers. It has no other means of accessing the servers on my home network - it can't find them via root acccess of the container.

[@telenmar112]

Thinking about doing this, but using WireGuard to encrypt the traffic between my home and the VPS

[@KiinaSu]

2:04 EMBY? I'm disappointed. Use Jellyfin after Emby showed the open source community the big middle finger

[@katrinabryce]

This didn't quite work for me as published. I've posted a comment in the forum link above about the changes I had to make to get it to work.

[@BitterCynical]

This could be useful and interesting. I've used haproxy once before on a hosting service for potential ddos protection for my home network. For a dumb amateur admin (which I still am) the documentation for haproxy is overwhelming and difficult to comprehend.

[@TheMave95]

I prefer acme.sh over certbot for wildcard certificates. As the name suggests it is a shell script without any dependencies. If your nameserver isn't compatible with ACME v2 wildcard certs, you can create a subdomain which is handled by a compatible and free nameserver like zonomi.com.

[@danbrown586]

Can you explain why you're using nginx as a backend on :81, only to redirect traffic to HTTPS? Can't HAProxy do that itself?

[@TheDarkWayne]

And I am here sitting with my traefik... i like my gopher ;D

[@charlese2833]

Old school, gopher was cool

[@vgamesx1]

I'm not saying it wasn't partially on me but traefik was a huge PITA for me to get going, not a big deal once it's setup since you can more or less forget about it but that really sucked... If I had to do it again, I'd probably go with haproxy on pfsense. Also note that I used traefik v2 which isn't quite as well covered and I routed mine through cloudflare for protection which added some extra difficulty, more specifically pages not automatically being upgraded to https, so my way of solving this was going to cloudflare "page rules" and adding a wildcard for my domain and setting it to "Always use https".

[@George-zm4iu]

Would a plex server load the content through the HA proxy or straight to the client from the server isp?

[@Jr-hv1ct]

So what do you recommend to setup your own email server?

[@aaronchamberlain4698]

Suggestion: On CentOS, use certbot-auto. It’s not available through the package manager but avoids all the weird Python version issues that you may have.

[@LampJustin]

Run it in a container ;)

[@run2dos179]

MORE of these!

[@DevilDriver665]

Great Video, but you could do this similar task a lot easier with Nginx Proxy Manager as a docker service.

[@mikeyz8775]

This is the kind of kung fu I'm about. I was using digital oceans vm machines for a long time, Iptables kungfoo for the win.

[@TheDrummerSteven]

Wendell, is a GOD!

[@Arunscape]

I think I might be doing something similar with traefik is HAproxy comparable to traefik?

[@RealDids]

Traefik has more features, including Let's Encrypt support. Just be aware that Traefik v2 doesn't do scaling without the commercial version, at least not with Let's Encrypt support.

[@charlese2833]

@ 19:50 : Please do a video on setting up a user for doing haproxy config.

[@dustinkrejci6142]

But what about Docker-Kubernetes rancher install for all this stuff?

[@andljoy]

could HA Proxy load balance PCoIP and VMWare Blast ?

[@richardpayne]

Why are you forwarding port 80 to an nginx server to handle https redirect? HAProxy can do this itself.

[@180doman]

Watched briefly (i will go into details later). So external proxy just maps your subdomains to local adresses right? What if you want to expose (nextcloud) to outer World? You would need to redirect request nextcloud.wendel.com to yours networks router WAN IP. And this requires external IP (not behind ISPs nat) and static one or dyndns domain like free duckdns. I already have my nextcloud setup done like that but i dont have proxy. But i want to add one, internal.

[@Karthig1987]

No idea whats going on as usual but watched it anyway.

[@JackmeMe]

I use Traefik + Let's Encrypt with Cloudflare DNS

[@JustSomeGuy009]

My biggest issue with traefik is that, from what I could tell, it's built expecting docker. I managed to get it working on Linux container and/or VM to serve static web hosts. But it was hard and sort of hacked. Without the docker auto discovery the usefulness of traefik is questionable.

[@JackmeMe]

@@JustSomeGuy009 Yes because it was made for load-balancing docker web services lol

[@heckyes]

If you use a server in a datacenter to act as a gateway like this, isn't your home internet speed and bandwidth simply now limited to whatever the server host is offering? I get that this still gives you control of your own storage.

[@LampJustin]

True as all traffic goes through that server but not an issue since almost all providers give u simmetrical gigabit connections which is definitely fast enough.

[@heckyes]

@@LampJustin Yes, but then that server (unless metal dedicated) is prone to privacy concerns no? Even a good container vm solution like XEN/KVM can still have memory dumps and such right? Also, I can't seem to find a good price for a VPS or greater server package that has an UNMETERED gigabit port. I wish though.

[@TheOisannNetwork]

Any reason not to just run HAProxy-WI locally on your own network?

[@dustinkrejci6142]

You look tired sir. Please take rest for your body, better care of yourself. I can see your stressing. Please. Please remember to stretch and go for a 2.5 mile walk every day for basic health benefits.

[@linuxdragon57]

I've always wanted to be able to run my Nextcloud instance on my home server. Unfortunately, my ISP is Comcast and they suck with their outdated cable infrastructure, overpriced bullshit, and 1.2TB datacap.

[@TheLakeJake3]

Amazing

[@theshuz]

Why not just install a second pfsense box in the cloud instead of haproxy-wi?

[@autohmae]

9:53 actually, not 10 years, but 20 years. 14:04 4 hours ? more like 4 days you mean. 19:20 euh... can't you just use volume mounts to the host ? haproxy socket and haproxy config, etc. ? 23:46 Starting from 1 September 2020 you won't be able to get 3 year valid certs anymore, only 1 year.

[@brianmccullough4578]

Wooooo! HA proxy!

[@jonesconrad1]

Whats IE ?

[@nebadon2025]

I am doing the same thing with just nginx!

[@DantalionNl]

But can you load-balance the same service behind multiple instances of nginx running on different machines and easily see how many requests are routed to which servers and corresponding response times?

[@Fahdalrabeayah]

me too nginx proxy manager

[@geogmz8277]

Me too, using Docker Swarm and Nginx Proxy Manager..

[@Level1Techs]

nginx has a lot of cool features and that's totally fine. I used to have to use nginx plus for something.. I forget

[@kelownatechkid]

Yeah I find it to be much simpler too, takes less than 10min to set up and proxy dozens of services without any guis or anything. Config is ez.

[@jscancella]

interesting, but I would be more interesting in running this on an actual razzberry pi (along with pi-hole) locally.

[@Mr.Leeroy]

locally > pfSense

[@DeceLatina]

i mean, i just used apache to proxy all my services to 80 and 443, on one system, and then have them proxied through cloudflare

[@andljoy]

How would this deal with your home not having a static IP

[@Level1Techs]

use a ddns hostname for the proxy config, or write your own script. no waiting for dns to roll over (if not using ddns). Once you change your haproxy config, the change is immediate.

[@andljoy]

@@Level1Techs i was thinking of ddns , i guess a script that could update the ip directly would be better tho. I don't have THAT much need to host stuff internally ( well i do host lots of stuff but not much of it is external). Prob just a nextcloud so i can sync my photos from my phone without google spy shit. Hmm i wonder , could i do some janky arse crap with say a externally hosted nextclud but with the storage internal and pass just the storage over , a terrible idea i know , but a fun .... can it be done :)

[@vgamesx1]

This can be done super easy with cloudflare if you use this: github.com/oznu/docker-cloudflare-ddns You just make an API key for it to be able to update your IP address.

[@Joachim1292o032043op]

tldr: your own cloudflare

[@svettnabb]

302 redirect? Why not 301?

[@Mr.Unacceptable]

I want to build a CCTV system from an old PC. 2500K 16Gb ram. I have a couple of 8 port BNC camera cards. I tried Linux but 3 times in a row had to be rebuilt within a week due to kernel stack errors. Anyone have any better software options for a CCTV system? Or know why the kernel stack problem? Can you restore from a kernel stack error without having to rebuild the entire system from scratch?

[@MrBiky]

Some cameras (HikVision in particular) have options to save recordings to a FTP server (or SMB, or NFS). Set the cameras to record 5 to 25 MB chunks and save them to your FTP. From there you can see your recordings by using something like VLC or whatever. I would suggest you to look for another Linux distro (I'm a fan of Void Linux), but I'm not a Linux evangelist, if you don't want to use free software and free yourself from proprietary shackles, you can use Windows on your old PC and install something like FileZilla server or whatever and run it 24/7. It does the job, albeit poorly. And if you're using a version of Windows newer than 8.1 (which is the last one supported, I don't recommend running Windows 7, since it's unsupported), ie Windows 10, you may have trouble with Windows auto-restarting for updates. From what I hear, it isn't so bad as of lately, but I heard mixed feelings (for some it restarts, for some it doesn't). Not sure exactly what Linux you used, but usually there may be a way to save your old system if you live boot and repair some stuff (depending on what exactly broke).

[@kelownatechkid]

I'd just use nginx to proxy my web traffic... but HAproxy is good too

[@RealDids]

Before I would've recommended Traefik over HAProxy, but now with the v2 I'm not so sure. I still love Traefik, but the free version of v2 doesn't so scaling anymore. That said, Traefik still has too many features that HAProxy doesn't, such as built-in Let's Encrypt support.

[@DantalionNl]

The main problem with Traefik is that it is incapable of dealing with client certificates it will break as soon as it encounters one. Luckily, that is not a common problem as most things don't require client certificates anymore but if you have a service that does, Traefik won't be a solution.

[@LampJustin]

@@DantalionNl at least they're working on that aws! But yes that's a bummer. Even though they changed pretty much everything with v2 the way it's done is now much better. It's more like Kubernetes which just blew my mind at first but now after looking into it, it makes so much sense!

[@jscancella]

Wendell, why Plex and not Jellyfin?

[@LampJustin]

Because Plex just works better... I like jellyfin better myself but sadly most if not almost all viewers r running Plex.

[@kelownatechkid]

Run both in parallel IMO. Plex is a horrible company but some people can't change their client devices. Jellyfin can be at the ready for switching whenever possible.

[@cougarmain]

I do this with pfsense on cloud and vpn back home

[@cocosloan3748]

Got it guys? Good.. Coz i didnt :)

[@TheNorthRemember]

why not just use ubuntu or at least Debian?

you can setup redirect to https inside haproxy just add something like redirect scheme https code 301 if !{ ssl_fc } in frontend config and that's it. in case your nginx doesn't know how to work with that just add: http-request add-header X-Forwarded-Proto https if { ssl_fc } in backend config also no need to use acs to redirect traffic to another backend, you can simply use: use_backend cloud if { hdr(host) -i cloud.wendell.tech }

[@kaspersergej]

In this particular scenario HA Proxy is unnecessary. You can even do TCP/UDP Proxy with nginx.

[@kelownatechkid]

I agree, a lot easier to use nginx IMO.. nginx + certbot problem solved for 99% of home users in like 15min. Just gotta make sure you have ddns which is easy

[@thisrocks]

acme.sh is amazing for let’s encrypt

[@redneckrestoration9385]

woohoo linux stuff

[@davidg4512]

I use HAProxy on pfsense. Way too easy.

[@DeadlyDragon_]

I run nginx myself.

[@DrakeDealer]

The virgin maintainer made it difficult to install because he's too poor to get another job that pays.

[@chocolatebrisket3772]

ENGAGEMENT

[@PremiumGerman]

You really should explain this in more detail it's way too complicated

[@thestreamreader]

A guide without the use of the haproxy tool would be great.

[@VADemon]

Don't recommend god-ddy to anyone, they're a -terr- not a nice company!*

[@honestabe3100]

just way too long, format sucks, don"t want any more vids like this thanks

[@aarongarza4769]

I watched the whole video. To bad I don’t know Chinese.

[@sp00k1es]

Can't all this be achieved with just Nginx?

[@myownsite]

Nginx can do high availability, but HAProxy is a breeze to configure once you get the hang of it. No idea about the webUI, though.

[@sp00k1es]

@@myownsite There's been some recent UIs for Nginx, though I'm not fond of them. Personally I prefer the Nginx config syntax, that's why I asked, especially if you add the javascript plugin module to branch out into more complicated scripts for things if you need to.

[@myownsite]

@@sp00k1es I highly dislike GUI fronts for software which have robust text configs available and similarly overly complicated configurations. Nginx is a great web server and haproxy is a great load balancer. I think utilising strengths of both tools is the best approach, with as simple configs as possible. That way maintenance and deployments stay feasible.