SELF-HOSTING behind CGNAT for fun and IPv6 transition

[@cole.maxwell]

Hey man! Just found some of your older ipv6 videos yesterday. Loving this content and your presentation style! Thanks so much for sharing all this hard won knowledge!

[@mathieusplaylist]

ROFLOL! Telnet to the IPv4 address to get rickroll'd, telnet to the IPv6 address to get Star Wars. Nice one!

[@apalrdsadventures]

There’s also a sad no ipv6 animation you might randomly get

[@didotb01]

that's weird, I got the proper star wars without an ipv6. though I did get the "no ipv6" at first

[@saidsensei4862]

Thank you for your explanation, this video is fantastic

[@user-lg4le8xr4s]

You can also use a Hurricane Electric Tunnelbroker 6to4 tunnel, among some other providers - another solution that doesn't require you install any software. HE will give you a whole routed /48 block for free if you do their IPv6 certification (also free), and they even have a ton of config examples for every router OS you can think of. I have both native IPv6 and IPv4 service thankfully, but I still use a HE tunnel for public facing things just for the sake of having it on a separate IPv6 block. (+ I also like having the extra /48 to do with as I please.) I use it with both MikroTik RouterOS and a Debian router and both were easy to configure to use the tunnel.

[@apalrdsadventures]

Only problem with HE tunnels is they use IPIP and you need a public v4 for the tunnel to work (this is a protocol limitation, not policy).

[@user-lg4le8xr4s]

@@apalrdsadventures Yeah, I forgot your use case here. I got it backwards, like you had IPv4 but no proper IPv6.

[@mattisz]

I have been running a modified version of your last option using nebula instead of wireguard. Nebula has the punchy.respond option that keeps the tunnel alive so I didn't have to roll my own solution. Sadly there is still no IPv6 support in nebula but it it's robust enough for all of Slack's infrastructure it's probably good enough for me. Great video as always! Keep up the good work!

[@apalrdsadventures]

I use Nebula as well, but as a purely closed network (no routing / traffic flow outside of the Nebula subnet).

[@l4kr]

No IPv6? Just say Nebula doesn't work.

[@apalrdsadventures]

Nebula does v6 on the transport, but it's more designed as a closed network (routing individual IPs) between hosts for secure internal communications.

[@varno]

Run 464 xlat, basically a private cgnat, but you control the exit port, so you can control the port forwarding. You can use this to derive your own IPv4 exit. You can use jool to implement this see siit-dc-xlat. You can then port forward on your virtual private server gateway.

[@apalrdsadventures]

Jool is kinda a pain to work with because of how traffic bypasses a lot of the normal linux Netfilter, and doesn't work for translating packets from the local system.

[@varno]

@apalrdsadventures there is also cgnatd that can do the local traffic. But you can implement it on a router or in VMs too. The big thing is it gives you a non cgnat ipv4 tunnel over ipv6.

[@eDoc2020]

You can also use SSH port forwarding for port forwarding. A benefit is that it requires no special configuration on the public server. Just write a service on your origin server that SSHs into the public server and everything will just work even if your home network changes. Many of the solutions in your video break if your home prefix changes.

[@apalrdsadventures]

This is a good solution for SSH or other low bandwidth protocols, but not a good solution for generic hosting due to the head-of-line blocking problem when multiplexing multiple TCP streams over a single TCP socket (the SSH session).

[@jsaenzMusic]

I believe using an overlay network like Nebula or tail scale for your self hosted services and a reverse proxy back to them from the Internet, is an option. It requires you to have a public VPS as the reverse proxy for your publicly available service (i.e. - jellyfin) but you don't have to forward any ports (if you were lucky enough to have a public IP from your ISP). All traffic from the reverse proxy (which is also a node on your overlay network) back to your home network is on the overlay VPN.

[@jsaenzMusic]

Commented before reading the rest of the comments. Y'all already knew this. 🤦🏻‍♂️ 😛

[@Trainguyrom]

I have 2 different ISPs offering gigabit FTH (on separate networks even! there are reports of shenanigans of the new one "accidentally" cutting the fiber then sending a sales person and an installer behind them to sell people new service), and neither has any support for IPv6. At some point I'll have to figure out an IPv6 tunnel or something...

[@user-mo8hc6pp1v]

I (kind of) use the Wireguard method but instead of allowing the VPS acces to my whole local subnet I set up tunnels on every server and configure the port forwarding with nginx (stream directive) since I always had problems with iptables. Failover (or load balancing) come as a nice bonus, and I can use both tcp and udp, and if I ever have to route to a website, I can set up a normal http/s proxy with caching which will also allow for easier certificate management and the ability keep the incoming ip addresses in x-real-ip parameter. The configuration is not that big of a deal to do manually once a month but I may make some webui to automatically rebuild the configuration at some point.

[@DarkNightSonata]

was gonna like the video but then I got rickrolled love it

[@kkpdk]

I am currently running a more complicated version of the wireguard setup: A VPS acts as NAT for the wireguard network, but I then use policy routing combined with connection tracking on my home router, which allows me to not mangle the source addresses; It behaves as a multi-isp setup. But, debugging incorrect policy routing is annoyingly difficult; I haven't yet really discovered a good way to attack it.

[@drivetrainerYT]

besides this video, thank you for the RSS on your site, much appreciated

[@apalrdsadventures]

No problem!

[@SupremeMortal]

I have a VPS with a GRE tunnel to the IPv6 address of my home router, then enable ip_forward and set the next hop of the IPv4 address of the VPS to my router through the GRE tunnel. The advantages of this is less overhead so you can host it on a dirt cheap VPS, and it's even possible to assign the public IP directly to a server on your LAN if you use a GRETAP tunnel. All of this is configured without installing any extra software on the VPS since it only uses the "ip" command in Linux.

[@apalrdsadventures]

I did setup a prototype with IPIP (GIF on BSD) and found that OPNsense wasn't properly configuring GIF for IPv4 in IPv6 (it's normally used for the opposite, IPv6 in IPv4 for Hurricane Electric).

[@jifgif]

could you please explain a scenario, when the ISP is IPv4-only CGNAT and doesn't provide IPv6 at all?

[@patrikklippmark4213]

Hey man! been following alot of your guides for Proxmox setups, and i stumpled upon your Immich setup on your Blog, however i'm running into some issues trying to set that up do you plan on releasing any video soon to complement that blog post? I would really appreciate that!

[@Kane-5263]

Thank you for this nice list of explanations! SNID made me curious but I did not find any comparison between them, but maybe I missed something and there is none to make? For me it look a lot like the socat commande that we can easily run as a service to redirect trafic on a port to another network (v4 or v6), port, a washing machine, a file ... is there a difference between it and SNID?

[@apalrdsadventures]

snid reads the TLS SNI, then does a DNS AAAA lookup on the resulting name, and if the name is within the backend cidr range it redirects the session at layer 4 (TCP). So the big difference is pulling out the SNI so you don't have to configure snid itself. Another difference is that SNID codes the IPv6 source address with the IPv4 source of the origin, which is why it uses a /96 instead of a single address. That way, the backend server can get the IPv4 source address in their logs without needing to use PROXY protocol.

[@Kane-5263]

@@apalrdsadventures Oooh, I see, thank you for the explanation! I think I do not know enough about TLS SNI to full get the first part, but that's a good reason to learn about it! Even more since it is not the first time I heard about it. That point about know the source IP is really interesting too, in my last iteration of server and service I planned to have a host only running LXC and passing the needed 80/443 ports via socat to a container with HAProxy inside to route the request to other containers, but I never looked at the logs yet since it's still in setup, I may have ruined most of them by erasing the source IP 🤔 Maybe it's time to go back on that idea and leave HAProxy on the host instead...

[@apalrdsadventures]

TLS sends the ClientHello immediately after opening the tcp socket. This contains proposals for crypto algorithms to use, and also the domain name of the server requested. The server responds with the ServerHello which includes the servers certificate. They do this so the server knows which certificate to present to the client, but as a result, the server name is sent unencrypted.

[@brunosalezze]

I love your videos; thank you. Do you have a video showing how to make the public IPv6 devices behind the firewall accessible from outside?

[@andrew2180]

In OPNsense you just add a rule on the firewall to allow the traffic

[@apalrdsadventures]

It depends on your router. But in general it should be an “allow” rule.

[@brunosalezze]

Currently, its a UDM-Pro, but I will switch to Opnsense soon. I did create one rule (Internet IPV6) to allow incoming traffic to a device behind the firewall, but I can't ping it yet from the outside, not even the lan public ipv6 from the firewall (with another rule); I am also not able to ping it. Its a CGNAT. I was thinking maybe I need another rule elsewhere

[@apalrdsadventures]

The UDM should be able to do it, but I don't have one to test with.

[@Will-kp1iv]

I'm trying to follow the last one since that's my only option but I'm really confused, 30:04 how do you set you home network endpoint using a domain if its behind a CGNAT?

[@apalrdsadventures]

You can leave it blank.

[@caseyknolla8419]

I use a cloudflare tunnel in addition to my site SSL configured in Full (Strict) encryption mode. This means I still maintain a cert on my origin server and Cloudflare re-encrypts all traffic between them and me. In addition, I only use that tunnel for web traffic. I still port-forward for media streaming, so you can mix both.

[@AWildLeon]

But cloudflare can still read anything

[@Lenny3669]

​@@AWildLeonhow can they read everything while my server is using my own SSL cert and I haven't given my private key to anyone? (I'm also using the full (strict) setup)

[@caseyknolla8419]

@@Lenny3669 He's correct. Cloudflare still decrypts traffic in transit using their cert, then re-encrypts it using your cert before it passes along to your origin. They need to do this to enable the majority of their services, despite the privacy risk it creates. That's a decision I chose knowingly.

[@caseyknolla8419]

@@Lenny3669 not sure how/why my original reply got deleted, but @AWildLeon is correct. Cloudflare presents their cert for your domain to users. They decrypt the traffic in transit, and re-encrypt it with your cert when they proxy it to your origin server. They have to do this to enable the majority of their services despite the privacy risk. I chose to use the tunnel knowing this.

[@caseyknolla8419]

My replies keep getting instantly deleted, but AWildLeon is correct. Cloudflare presents their cert for your domain to users. They decrypt the traffic in transit, and re-encrypt it with your cert when they proxy it to your origin server. They have to do this to enable the majority of their services despite the privacy risk. I chose to use the tunnel knowing this.

[@Mr.Leeroy]

Now, how's this for a joke. My ISP has IPv6 by default, but.. If I rent IPv4, they cut off my IPv6 access. And the most hilarious thing is their excuse saying that if a client requests IPv4 address, there is a need to use it as a static IP for auth reasons in 3rd party services and if they were to leave IPv6 access routers would prefer v6 over dedicated v4 and the whole auth thing would fail. So they a preferring to break transport so that some Chad renting IPv4 for auth reasons would not bother their dysfunctional support.

[@GeoffSeeley]

Ahh ya got me!

[@thechubbypanda]

I used Wireguard in the end whem I ended up behind CGNAT. I get IPv6 but it's a dynamic /56. I dont know if/when it'll change :(

[@apalrdsadventures]

Does the dynamic /56 actually change though? Or is it effectively static?

[@nicejs8194]

by the way, on cloudflare if you only add a AAAA record (with "orange cloud" enabled), cloudflare automatically proxies ipv4 connections if you don't add an A record also I didn't have to provide a credit card to use cloudflare tunnels (though the proxy dies for an hour sometimes and I've read that happens on the free tier so maybe that's it?)

[@apalrdsadventures]

With Orange Cloud, it will not actually add either record to 'real' DNS. 'Real' DNS will point both A and AAAA to Cloudflare's CDN (if you add either an A or AAAA), the record you 'add' just tells CF where the origin is.

[@imdavidday]

Hey Apalrd, what are you using to draw that diagram? Thanks!

[@apalrdsadventures]

draw.io

[@imdavidday]

@@apalrdsadventures Thank you!

[@hsmptg]

Any of these methods would work with a MQTT self hosted broker?

[@apalrdsadventures]

MQTT uses TCP transport, so any of the TCP examples. MQTTS will also work with snid if you listen on 8883 (you can -listen multiple times).

[@designer.346]

I use Tailscale

[@elalemanpaisa]

IPv6 😢 I was so smart and left to latam and had my homelab server running behind my firewall in Germany. There it's a ds lite so same problem as you are describing here.. but in latam we have our own IPv4 but no IPv6.. it was a pain in the ass.. if anyone cares how I solved that let me know

[@nick-leffler]

Remember if using a CF tunnel for media streaming, you're breaking their ToS.

[@EthanBB]

They removed section 2.8 from their ToS some time ago, so not anymore, you just have to set a rule to not cache your streamed content if your site doesn't have it in headers already.

[@apalrdsadventures]

They made it more vague, but still don't really want to deal with media streaming.

[@Lenny3669]

Remember if you're pretending to be an expert who can teach the teacher on KZbin while you have no idea what you're talking about, you're breaking the entire world's TOS, and probably need to go outside and ride a bike or something. Godspeed.

[@theglowcloud2215]

@@Lenny3669 ok boomer

[@nick-leffler]

@@Lenny3669 I was just trying to let everyone know he careful if using with media streaming. Even before it was in their ToS years ago I got my account locked because of media streaming. Just letting everyone know. I asked them why and they said it was against their terms, but couldn't show me where.

[@ff34jmr]

..or setup headscale on your vps and be done with it :)

[@apalrdsadventures]

That would require your friends to install Tailscale though

[@lightechoes]

Headscale is a godsend.

[@PopularWebz]

I've found tailscale breaks IPv6 so that my Mac doesn't resolve AAAA records. Even with MagicDNS disabled and not connected to a trailnet. As soon as I close tailscale things start working

[@ff34jmr]

@@apalrdsadventures not necessarily you can always use your vps public ip (ipv4 and ipv6) and then use tailscale / headscale túnel tp “proxy “/ route to where you have CGNAT. Basically the same what you did with WireGuard just simple to setup and no hole punch needed on firewalls. The advantage of headscale is that you can use whatever ipv4 subnet you want for the tailscale network so doesn’t conflict with CGNAT or don’t use ipv4 at all

[@fd6216]

@@apalrdsadventures they wouldn't! I tried this as a POC, set up a headscale control server + two users (one external and one internal to your network, the internal one publishes your network routes). On the external one (which you can technically host in the same machine as the control server) you can either forward ports 80/443 to an internal nginxproxymanager or you set that up on the VPS and just pass traffic from there to your internal IPs. The second option maintains the client IP as it's passed to the backend as an x-forwarded-for, since you're terminating TLS at the VPS. Traffic into your network may be unencrypted (if you proxy_pass to http) but it's over the wireguard tunnel, so it doesn't really matter. For the jellyfin example you'd set a DNS record for jellyfin.domain.tld pointing to the VPS IP and configure that as the external domain on the jellyfin settings and that's it. Your users go to jellyfin.domain.tld and get served from your internal jellyfin server, no client needed.

[@mitchhoneysett7674]

What is your network drawing program

[@apalrdsadventures]

draw.io

[@kc8pnd]

@apalrdsadventures, I have tried to use the setup wizard on my opnsense router for wire guard but it never works. Do you have a particular video or guide you followed?

[@apalrdsadventures]

I just made a VPN video last week! It includes Wireguard.

[@kc8pnd]

@@apalrdsadventures I will be diving into that then!

[@n8lbv]

While presenting "option one" This Video completely disregards the absolute hard and so incredibly obvious FACT that pretty much EVERY ISP and especially large well known wireless or satellite providers BLOCK any/all inbound on IPV6 with absolutely no options to get around this. Not even mentioned or discussed whatsoever. I mean seriously?

[@hsmptg]

Any reason for "not trusting Oracle at all"?

[@apalrdsadventures]

Their history of being very litigious at the slightest hint of license violations

[@patrickhult]

Headscale with 2fa. Make a video about that buddie.

[@theglowcloud2215]

small pp

[@solbu-1973]

At 4:40 you say that IPv6 doesnt have NAT. Well, it does. It's called NAT66. The private IPv6 address space is «FC00::/7».

[@apalrdsadventures]

every time someone uses NAT66 a fairy dies

[@kpanic23]

There's also NPT as the lesser evil... still not great though.

[@AdrianuX1985]

+1

[@Gta5modmenu140]

how do you make your own wifi with fiber can you make a video about it sooo i want get because networking than i have fast wifi @apalrdsadventures i realy learnt much from you video s