Your ISP is lying! Monitor your Internet with a Pi

[@robertcruz7898]

I once lived in a house where the DSL would go out every day around 5 pm. I created a simple logger using ping to document this for the ISP, because they didn't believe me. When they finally sent a service person at the correct time, we discovered that one of their node boxes next to someone's lawn was being sprayed every day at 5 pm when the lawn sprinklers came on! No wonder it didn't turn up in their diagnostics!!

[@User_1795]

Omg that had to suck. That's crazy they didn't even know they're shit was shorting.

[@IDGinUkraine]

@@User_1795 If they put something dirt-cheap and simple (and they have to do this) on the "last mile" - it may go unnoticed for years. But how their equipment has survived regular showers? Miracle?

[@edumaker-alexgibson]

Now that's real-world troubleshooting and diagnostics.

[@tissuepaper9962]

@@IDGinUkraine I'm sure these things are designed to mitigate the damage from water ingress. Probably some good drain holes and passive airflow.

[@RoStepMusic]

Few years back i was on DSL and everytime somebody was using the phone the internet would stop 😂

[@mrsansiverius2083]

This channel is such a goldmine of useful content.

[@davidjwillems]

Agree, this is such a great channel.

[@conno7356]

Everything you could ever want to know about random electronics

[@t3cube]

I have to disagree with this video. Isps can also speed up your traffic when connecting to sites like speedtest.com to make it seem like they're giving you what you asked for. We've seen similar things happen with mobile phones were they perform much better when being benchmarked than they would in the real world. There has to be another way to measure the data.

[@streamx3]

My Grandma has like 900 up and down. With a fiber going all the way to the router. In a distant village. In Ukraine. For ~15 USD. So you're actually having a first world problem, with an expensive slow Internet.

[@AndersJackson]

Well, it is about the same here, in Sweden. My mother and father 80+, which live in a rural area with 250+ inhabitants, do have faster Internet then what I have in the middle of a city. :-/

[@provod_]

There are 4 available ISPs at my home in Siberia, 3 wired twisted pair etherned and one fiber (that's not counting mobile ones, which are 5 more or something). There's a promo option of 500Mbps symmetric for $8, but regular prices are more like $14 for the same speed. There are no 1gbps speeds (but you could connect to multiple ISPs and do ethernet bonding or something; i used to do that when unlimited internet was like 64kbps per ISP 13 years ago; pay-for-megabyte still was 100Mbps symmetric). One downside of siberian internet is that your latency to the nearest european datacenter is at least 100ms (50ms to Moscow). I was very surprised to see that here in Bay Area your internet is usually DOCSIS cable (a completely nonexistent thing in Russia; even ADSL is very rare, we jumped straight into 100Mbps TP at the very least with fiber coming a few years later) with 40Mbps up being the max that you could get at home for any amount money.

[@streamx3]

@@provod_ you are absolutely right. I'll tell you more: I've been to several places abroad, and the Bay Area was THE ONLY DAMN PLACE where staying in roaming and using 4G in roaming was actually cheaper than buying anything from a local carier.

[@wowslayer521]

Same in svk

[@IraeCarvalho]

US has some serious problem (and that's my guess) with legislation and taxes around infrastructure. Even Brazil, which is tax hell, we have 300Mb down, 150Mb up fiber everywhere, for about 40 USD. You would need to go into the mountains and where there is no paved roads to find places without decent internet.

[@niyaziugur]

hahahahh as a turkish citizen, I just laughed. expensive internet: checked monitored internet: checked nigthmare of CGNAT: checked website blockage: checked

[@JeffGeerling]

At least I get a stable IP address!

[@Levent_Ergun]

Let's not forget "fair usage policy" and lack of ipv6 connection of any sort.

[@paulmaydaynight9925]

if you have the wan connections anyway then it might be worth using 'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab' as a starting point to increase the personal upload rate to a cheap remote diy vpn etc. "We build a Virtual Network in Proxmox for the MPTCP or OpenMPTCPRouter Test lab. In this video we do A LOT OF STUFF ! We create Proxmox network virtual networks, we use the tc netem QoS Filter of Linux for shaping of network lines, we add latency to the network in order to simulate a slow network, we install OpenWrt in Proxmox but also OpenMPTCPRouter. We will run a MATE desktop on a Linux machine in an LXC container and run the X2GO Server in the container plus the X2GO client Windows. Furthermore there is a nodejs client that shows ssh remote exexution integration. The three shaper machines are in fact linux LXCE containers which we turn into routers and dhcp servers. The full description with ALL commands, URLs and instructions etc is here (it is longer than 5000 characters)"

[@paulmaydaynight9925]

oc integrating Webtop in to the mix is also good, although i usually like vm's with 'nomachine' for no fuss single install better real time remote video streaming to a local android/linux/win gui terminal 'Techno Tim Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop.'

[@fazlymawlarafi9331]

Website blockages when the government's involved for censorship reasons then that's great and is a reality for some countries

[@robertcruz7898]

When your ISP promises "UP TO", and you get 3/4 of that promise, consider yourself lucky!

[@JeffGeerling]

"Count your blessings"

[@mewtwo064]

@@JeffGeerling "Pray we don't alter the deal further" lol

[@tissuepaper9962]

@@mewtwo064 "HERE IS A UNICYCLE"

[@phinix250]

In Australia with our National Broadband Network (NBN), a lot of ISPs were charging people for higher speeds than actually possible for their connection (e.g. charging 100mbs for a line that could only support 40mbs) this resulted in action by the Australian Consumer Commission. As a result, now, if you can prove you do not get at least greater than the next tier down in speed the ISP has to refund you the difference for all the months it has been charging you the higher rate. This tool could be quite useful for that.

[@james10739]

Mine rarely doesn't give me full speed I get it's supposed to be 105mbps and like 12 or 15 up I don't remember but I have QOS set up but last time I turned it off and got like 135 I had not really heard of getting more before but was like cool

[@smmb4818]

"My cable internet connection which is supposed to be 1Gbps, is only about 700Mbps on average..." Me: **cries in 10Mbps ADSL**

[@tormaid42]

ISPs in the US sell gigabit plans with a 1TB/mo data cap, which you could literally burn through in a few hours 😂

[@abousono1]

I don't think they all do, I have Verizon Fios and I don't have a cap. However, Xfinity is available in my area also and I believe Xfinity has a data cap.

[@JeffGeerling]

I'm lucky to not have a data cap. I can't imagine living under those conditions.

[@tormaid42]

@@abousono1 It’s mostly cable providers that do this, that’s true.

[@DeadlyDragon_]

Yup comcast tried to pull this shit in the middle of the pandemic. They quickly pulled back and “graciously” decided to wait on fully implementing it.

[@RyanTheGenreWanderer]

I have Suddenlink with 400 down and 40 upload with unlimited data and for our 4-30-21 billing period my house used a whopping 3.2tb of data

[@The-Nil-By-Mouth]

This is why they say "Speeds *up to* 1 Gbps". Speed isn't everything either - one time I was getting reasonable speeds but crippling packet loss due to a local fault.

[@heliumlabs]

Found this channel when looking for Raspberry pi. Had no Pi back then but still loved to watch this valuable content. Now, Thanks to God I got a Pi 4 4GB and I love watching to your videos. Keep up the good work.

[@zRunes]

hearing $150 a month for internet is giving me a heart attack

[@j.p.ijsblok5304]

Yeah... i pay 40 euros for 1gb fiberglass a month. It's also giving me 950+/950+ mbps too, which is reasonable I think. But then again, I don't live in a third world country. It's a good thing that starlink comes to the rescue for those in need.

[@j.p.ijsblok5304]

@@MrMediator24 Very true in the USA, but i think/hope Starlink is going to change that.

[@draconightwalker4964]

go look at business internet, you have a stroke and heart attack at the same

[@31redorange08]

Workers get overpaid there so it evens out, I guess.

[@roowut]

for like $80 we get 50mbps in australia, and the average cost for a 1gbps is $321.50 all in aud

[@AzaB2C]

Thanks Jeff! Xfinity/Comcast have a monopoly in our area. Feels like an abusive relationship. Being able to keep them honest, and know what our devices are doing is super useful. Cheers!

[@morbidmanmusic]

same.

[@armyofninjas9055]

Of course they do. Time Warner won't operate in cities/states they are in. And they don't operate in states TW operates in... It's collusion. They need to be broken up. Wtf is congress even doing?

[@awesomeyoutubehandlewowitslong]

i just wanna say, i set this up today as a linux noob. your tutorial was so easy to follow, but i did have pi knowledge. i had a big fight with errors, but fixed them by giving 'pi' docker perms [thanks issue tracker!] this tool is so nice. i have pi hole just to track queries, as i have two ad blockers already. the internet-tracker is super nice. im gonna gather data on this as i love seeing data. cheers!

[@PATRIK67KALLBACK]

I would like to see Jeff Gerling setting up his own ISP and show how to be an internet rebell.

[@JeffGeerling]

Oh, believe me I have considered it often. But it's a lot lot LOT more than just the technical problems you have to deal with. I'm not sure if I'm up for the political and social maneuvering that's involved in the local community (which can often lead to a strong division and a lot of foul stares).

[@theundertaker5963]

@@JeffGeerling well you don't really need to allow that stop you from starting the business. There are always others who can take care of the political and social maneuvering while you take care of the technological aspect of the business.

[@JosephArata]

@@theundertaker5963 North America is literally the worst place to start up an internet service provider fresh from ground zero. You need millions in capital and political leverage to shoo out the 3rd party monopoly providers from your region. It's a financial risk that literally no one will give you money for, unless you have the cooperation of every single seat of power in local government.

[@paulmaydaynight9925]

@@JosephArata well, dont think so much "start up an internet service provider" more private diy Far bigger pipe than any single isp WAN... without the 3rd party isp hassle/continuous cost starting with how jeff Can make a long distance private diy big pipe to/from his sister ( how many miles line of site) with pi's, old cheap/free 5v-12v wireless routers + pv+ reclaimed batteries, several collectively routed with m.kzbin.info/www/bejne/iV67npafnJaXmqM 'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab' at both ends & potentially every pi relay in-between with cheap integrated m.kzbin.info/www/bejne/rXbXnZWEntdrgac LoRa LoRaWAN Range World Record Attempt (Re-published No. 120) Andreas Spiess for collective instrumentation & small quick txt, alerts etc ..collect up the sw tools and shell script it to make a working one off on demand clean usb Debian super light 'slax' slax org customize .php deployable from pi-KVM etc

[@suchathit]

In Thailand, many ISP can let's customer to set up Internet speed by their owners from the ISP website, for example, customer package is 500/500 Mbps, customer can set speed 500/500 or 600/400 or 800/200 or 900/100 Mbps.

[@natebo72]

Thank you Jeff! I have played around with a friends Raspberry Pi several years ago but didn't have any good use for it until now. Went out and bought my first Pi because of this video :)

[@JeffGeerling]

Have fun with it!

[@PlanetEleethal]

Sooo considering speedtest-cli uses 1-1.3 GB every test, every 30 minutes would equal to around 50 GB a day or more. * 30, that's 1.5 TB a month, even with my gigabit internet through Comcast that would exceed the 1.2 TB/month quota and I would end up paying for extra bandwidth even if I never used my internet during those 30 days lol.

[@JeffGeerling]

Good point-I didn't realize data caps were that low, OUCH! I might change the default to every 60 minutes at most, to help protect people from ISP stupidity. I also plan on making it easier to configure the interval so people could choose once a day or something more for long-term monitoring.

[@PlanetEleethal]

@@JeffGeerling I still really like this project, very impressive, I think I'm going to order a couple Shelly plugs! Thanks again for another awesome video! Long live the Pi generation. :)

[@jerryschull2122]

I left Xfinity as soon as AT&T Fiber arrived. They actually threatened to BAN me for a year for using over 1 TB (back when 250MB was the limit). I had to move over to Comcast Business and sign a contract to get unlimited data. Then when they lifted the cap on their Xfinity (Residential) side they wouldn't let me move over. And then when the contract expired and AT&T Fiber arrived I said good bye and good riddance.

[@driodeiros]

Thank you for the video Jeff. What about the impact to the other processes in your network while the test is happening? Have you noticed it? Also, the other way around, does the traffic in your network affects the result of the tests?

[@jimkozma]

Verizon FIOS supposedly doesn't have a data cap, but I would not want a speedtest going off while trying to game online. Good video though. Maybe someone could make a much shorter version of speedtest.

[@zyghom]

so I started with 1 Pi as my DAC - now I am at 12 (doing many things - not music only), and now I need a dashboard to my Pis ;-)

[@JeffGeerling]

A monitor to monitor all your Pis!

[@armyofninjas9055]

One monitor to monitor them all

[@realityveil6151]

Dude, get a hypervisor and run VMs, lol. 12 tiny physical boxes replaced with one big physical box running infinity tiny boxes

[@zyghom]

@@realityveil6151 Dude, you cannot put VM into active speaker to be working as LMS client and DAC. And this is only one example - actually 5 of my Pi are LMS clients and DACs. Everything has its purpose

[@dot_boi]

Im thankful a smaller local ISP started up near me. I'm a few miles outside the ST Louis metro area and I just went from only having satellite to a fiber connection which is faster by 100 fold. Spectrum wanted 20k 6 months before Gateway fiber just ran the line down our street for free if we payed for a subscription. I pay for a gigabit connection. I still see what you get most of the time like 700-900 down depending on the time of day but my uploads are always right around 900+. Still very happy with them, I only spend about 90$ a month on it.

[@gril.gasper]

It should be noted that if you have internet speed greater than 300 Mb/s, you MUST use a R Pi 4 as the 3B+ only has a 300 Mb/s NIC. Otherwise great video as always!

[@EvenTheDogAgrees]

If you want the automated speed tests, then yes. If you're just using it as a pi-hole, then no. The traffic doesn't flow through the Pi; it just resolves DNS queries. But good point nonetheless.

[@qwertybg333]

Nice video as usual 🙂 Well, I think that this problem is relatively extended in every country of the world I guess. Until 20 years ago in Italy there was a single telephone and internet operator but, at the end of this, nothing changed to much because the entire national infrastructure still was owned by that company, so the other "new companies" had to rent part of the network. To be short, this caused a lot of issues as there were more ISPs on the same infrastructure that wasn't upgraded as needed up during the years mainly for the high costs: here if you want to "put down" a new line/fiber you have to broke up streets as the old pipes are too small and full of old cables. Anyway, until 5 years ago I was paying 45€ monthly for a 6 Mbps ADSL, then the fiber (FTTC 100/20) finally came and, as I'm not so far to the cabinet I'm not losing too much signal. I still paying likely 50€ monthly but at least now all it's working as intended. Ah, phone calls are included 😆 Sorry for the poem and thanks for your work here.

[@Hyperion60]

0:11 Ah, for 1 Gb/s down and 400 Mb/s up it's 20€/month in France.

[@expresseducator1511]

In Mumbai we have FTTH , 1gb up and down with 3tb cap at around 56€/mo plus voip and a bunch of ott services .

[@x5Deadmeat5x]

I live in a rural area of Northern California, USA. $130/month for cellular based service… 10 down/3 up. Still wait for my Starlink service.

[@expresseducator1511]

@@x5Deadmeat5x hope you get it soon👍

[@HappyBeezerStudios]

gbit down and 50 mbit up is 50€ in germany

[@warny1978]

In france, I pay 38€ for a sold _up to_ 8Gb down/1Gb up which is, in fact, a measured 5Gb down/1Gb up connection per month with no data cap. I don't complain with the difference, i was never able to use my connection at its full speed.

[@TanelM]

Where I live, when you have high latency or slower than promised speeds, you call the ISP and it’s fixed in a few hours. ToS go both ways. Greetings from Estonia!

[@CilusseYT]

Grafana is the only service that I wanted to have for a long time, but still very confused about how to start…. Video series Jeff ? ;)

[@thomasjohanns7661]

Grafana on itself does nothing, you need some data to display in Grafana. Typically this would be in a database (in this video, it's Prometheus, but it could also be MySQL, SQL Server, etc). But then you also need something to put in that database in the first place, which in this video are the speed test results and the website checks.

[@freezingCode]

Thanks for this. I already have a Pi running PiHole connected to the router, but I've been using a scheduled task on a windows machine to run an hourly speedtest to track my internet speeds. This looks like a much better setup.

[@MarcusPHagen]

Thanks Jeff! Once again you provide an excellent set of tools for us. I'll be using a Pi 4B in my cluster for a dedicated instance of this project.

[@tygi]

I did run your docker compose and within minutes it works flawlessly ! THX

[@seshpenguin]

I feel you with cable upload speeds. Here in Ontario I can get 1000/30, which definitely hurts to pay for when all I really care about is upload!

[@JeffGeerling]

You know the pain!!!

[@ashtentheplatypus]

You're lucky having 30. Here I can only get 1000/10. It's almost always saturated, and I have to host a lot of stuff on the cloud simply because my connection would get choked as soon as someone remotely starts a download.

[@Commander_Wolf32]

Cry’s in Australia internet getting 40/10 which is concerned good for my area

[@AaronMichaelLong]

They're not lying. It says right on the ad "Up to...". What's more, the rates are for the last mile, ISPs make no guarantee of your throughput over the public internet, over which they have no control, and they don't even guarantee your throughput across their own backbone, because the internet is an "as-able" service. All you're really doing is generating pointless traffic for the express purpose of documenting what you already should know: the internet is not reliable. It's not designed to be reliable, in fact, the whole reason the ARPAnet was originally built was to try and create a communication network which could survive a nuclear war. If you want to buy a synchronous gigabit connection, you're welcome to, just be prepared to pay about a grand a month, because the ISP's cost model is built around the understanding that you'll be a client, not a host, if you're using residential internet. From a former ISP engineer: Stop doing what you're doing. You're making the problem worse by spamming out bogus internet traffic on a cron job. If you want to monitor the quality of your network connection, install ElasticSearch Beats or Cacti or something, and monitor your traffic sent and received, and set up Smokeping to monitor your latency. Don't be a douche-weasel and DDOS your upstream just to prove you're getting less than the marketing team printed on the box.

[@jaimerosariojusticia]

I found out years ago that the offered bandwidth is only guaranteed from the demarc point to your nearest ISP first hop.

[@spacescout87]

So meaning from my home to ISP but not my home to ISP to Google for instance? So the advertised bandwidth means literally nothing if my ISP's connection to google is slow in that example?

[@AndersJackson]

@@spacescout87 exactly. But hen again, they have no control of the internet between their point and Google (or whatever you connect to).

[@DeadlyDragon_]

So there are peering agreements for ISPs, they have to peer with someone else to get access to the internet. Literal ISPs for the ISPs. These are expensive contracts for multi gigabit connections. Not too mention BGP ASN’s and the constant battle to obtain more v4 addresses from ARIN or other addressing entities. They do have their own battles.

[@CFSworks]

@@DeadlyDragon_ In a past job, I was the one to have to fight those battles - can 100% confirm they're tough. We were able to operate with only one ASN (as many nets are) by using the same routing policy everywhere, and we addressed (pun not intended) the IPv4 shortage problem by going the DS-Lite route (pun still not intended). But all of that was a walk in the park compared to the challenge of purchasing enough transit to meet customer demand. We kept having to readjust our oversell ratios as we updated our traffic engineering practices, and getting a good handle on what "peak demand" looks like is a surprisingly complicated question. We weren't yet big enough to get caching appliances from Akamai, Netflix, and friends... not that I wasn't checking constantly. But we always strove to do right by our customers - small ISPs have to or they don't survive. (So be a customer of a small ISP! Support local businesses that actually give a crap about you!) Glad I wasn't still working there for the COVID pandemic. I would hate to be the one to have to figure out how to handle a bunch of WFH traffic overnight! :)

[@Diviance]

@@CFSworks Oh, if only I lived somewhere a small isp existed.

[@Yaegermeister163]

Almost every ISP shares bandwidth. Unless you are paying for it you are sharing bandwidth with all the customers that are in your area. The analogy I use is a water hose. There is only so much water that can be pushed thru the hose at any given time. Bandwidth is the same. Great video as always!

[@fate2022]

Really great videos, and information on new and upcoming hardware, the only problem I see, is your github instructions on this are not the greatest. For someone like myself that is still learning about linux systems by utilizing a raspberry Pi, again for someone like myself hat knows enough to be dangerous, can really get frustrated with install instructions that are lacking, and would make someone want to just throw in the towel all together and stop learning more about raspberry pi's. This is by no means a dig on you, people like you are what made me want to get into buying my 1st raspberry Pi, but again with the github install instructions, its really hard to even tell what exactly it is that I am installing, or how specifically. Am I installing Ansible, or Pi Hole with docker, or do I have to have docker already installed, is it installing ansible? it's really hard to understand what all the instructions are having me do.

[@morcom01]

Your channel and this video specifically has helped me catch Cox throttling my speeds and get them to give me my actual internet speed back. Went from 200Mbps to ~700-940Mbps with no hardware changes on my end. Thank you!

[@ryanj4274]

Hey, you're sitting in the city with your 1 Gbps plan, out here in St. Charles County we get 25/5 and even then AT&T can't deliver it without some spottiness!

[@sontjer]

Really appreciated your explanation, from which I learned that the telecommunication infrastructure of the U.S. is a bit outdated...

[@pd8559]

One missing feature. With everyone monitoring their ISP 7/24 there needs to be an opt in option for users to send anonymized data with ISP, Location down to the city level to a central site on the internet. Then this can be a public service that anyone can search their location/city and see the averages that users monitoring 7/24 in that city are getting from each ISP including outages. This will allow everyone to know before hand which ISP is better to go with and turn this project into the next level.

[@nathanadhitya]

I'd like to also see a crowd-sourced ISP statistics! This way, no more pesky ISPs claiming "no our internet is not the problem, go restart your router."

[@erictyrrell8692]

This is a really cool idea, but I can't think of a way to stop ISPs from poisoning the data. As the service gets more popular, it's more likely that ISPs would send fake data to it

[@TheAnoniemo]

Speedtest dot net already collects this kind of info, not 24/7 but it does save the speeds and providers for analytics.

[@kimiko2547]

Samknows

[@husher5142]

8 minute AD for starlink. Keep in mind the RF to starlink will congest the same as any other wifi signal with the more people using the service. Also 1 gige ports, or maxed out bw will always consume overhead to be stable. Typically ~5-20% - it will vary depending on last mile. So that's normal.

[@aaryavtech]

CBI:- Lets track em Jeff:- U know na I got y'all

[@ScottPlude]

I am so glad you posted this. I have "suffered" through years of service via charter at only 200mbps down and 11mbps up. I too care only about the upload speed since 200mbps down was totally adequate. I upgraded to gigabit just so I could get the 40mbps up. It's ALMOST adequate now. Symmetrical would make everything right in the world!

[@JeffGeerling]

Even if the asymmetric were more like 500/100, that would be leagues better than it is today.

[@condemned77]

It would be awesome to have a real time traffic monitoring in place. As the pi hole is already recording DNS requests call, it would be pretty nice to see what clients are currently doing on the Internet, since the pihole frontend it’s basically just a list of requests that were resolved. This data may probably be relevant to show such a real time traffic, but it itself doesn’t record the traffic volume for instance.

[@laurencearcon]

Hi, thanks for this! Your tutorial was very helpful. It has a steep learning curve and took me about a day to set it up on two orangepi boards. Now, I can finally monitor my two ISPs.

[@menno763]

Hi Jeff, it would be great if u did a video about backing up the sd card on a pi to a nas. I am kinda worried it will fail and having to set everything up again.

[@XTrumpet63X]

Finally someone else noticed this problem! Every time I had an internet problem and called my ISP, I couldn't believe how little monitoring they do. ISPs should ENCOURAGE their customers to set up monitoring like this if they're not going to.

[@ryanfortner1868]

Man, this channel is a gem for the open source community. Love your content, keep it up 👍

[@meosh930]

I did something similar but in my laptop and found that my ISP was slowing it down by more than two-thirds at times. When confronted, they doubled down saying that data consumption was "too high" and that it could only mean I was getting the speeds I was promised. Ofcourse, when you calculated the average speed it was a little over this throttled speed. The data consumption was just because of the speedtests running every 2 minutes in the background. I switched my ISP as soon as I found an alternative. Best decision ever.

[@pavan13]

In India internet is pretty cheap it's about 54 dollars for 1Gbps connection with no data cap and 10 dollars for 100Mbps connection with no data cap

[@riccardosacchetti]

In Italy we pay about 25/30 dollars for a stable gigabit connection. BUT, we pay the same for 100/50/20 or 7 Mbps ADSL!

[@pavan13]

@Utkarsh Amitabh Srivastava ya you are right but 3.3tb more than enough 1 tb is enough because generally we don't reach that limit

[@RaduRadonys]

@@pavan13 If that's cheap, what about 1Gbps down and 500Mbps up, unlimited data, for $7?

[@miles_thomas]

There is a company in the UK (and I think also the US), who does internet performance monitoring for homes/small businesses. The company is called Samknows, and has been involved in broadband information and internet monitoring for some years. Their commercial proposition is to give selected customers a monitoring box for free, and then sell the data (only performance data, not content) to ISPs and governments who want to understand and regulate ISP performance. Some ISP routers comes with the SamKnows monitoring software built in. The benefit to the consumer: free reports (online, near real time, and monthly summary). The monitoring box has been through several generations. It was originally based on a flashable internet router (e.g. DDWRT core), but is now a custom developed device (still based on router-type tech). It hard wire connects to the router, and any hardware connections need to route through the monitoring box (for accuracy). It also listens to the router's WiFi traffic to make an estimate of home much traffic has flowed by WiFi. No assembly required, simple to install, invisible in use. Yes, it does not offer any additional privacy protections, ability to monitor other network connected devices etc. but it does mean any measurement is likely to be accurate and unimpacted by other activities. More details on the SamKnows website.

[@TheTechieScientist]

Cool Project ! looks like my pi 3b+ is getting some more work to do than just host a discord bot and a local chat server :)

[@paraniodify]

@The Techie Scientist what kind of chat server do you host? I also want to try that out but I cant decide what application I should choose^^

[@TheTechieScientist]

@@paraniodify nodejs socket.io and express

[@paraniodify]

@@TheTechieScientist thx mate :)

[@iguanac6466]

When I tried to switch to an LTE ISP, I had to talk to the CenturyLink customer retention specialist. They pretty much gave up when I told them "Look, I'll pay you $200/mo if you will give me 100Mbps down." They act like they don't understand how 3Mbps is not sufficient for anything except for email/browsing nowadays. Without ad block, my wife's PC consumes the full 3Mbps down when she opens her forums due to all the auto starting videos/ads. I currently pay $90/mo for two 3Mbps dry DSL circuits. CenturyLink/Frontier/Hughes/Dishnet are going to be in a world of hurt when Starlink gets rolled out in earnest...

[@GrimSpec]

I wonder when you will move to Home Assistant ...slowly :D

[@JeffGeerling]

Heh, it may happen, we'll see. I think there's already a drop-in plugin for the Shelly Plug there.

[@GrimSpec]

@@JeffGeerling Yea there is an integration for Shellies among many others :)

[@hypurrfpv9483]

And that is why they say "up to" in their ads. I have the opposite issue. My Xfinity connection is "up to" 400mbs down and 20mbs up. I consistently get 480mbs down and 24mbs up so I can't complain. Nice way to monitor your connection, very interesting.

[@juststeve5542]

Friend in Hungary pays €10 a month and gets 600Mb/s down and 200Mb/s up... I'm not impressed!

[@84GDi]

Yup, DIGI FTTH is 1Gbps/300Mbps GPON for 3100HUF (approx 8-10EUR)

[@juststeve5542]

@@84GDi I'll have to tell my friend she was stitched up! 🤣

[@pgp-acc]

@@84GDi In Spain Digi offers 1Gbps/1Gbps for 20€. In Spain, there are only symmetric connections.

[@HappyBeezerStudios]

@@pgp-acc for 30€ you could get a 50/5 over cable here in germany. Or 100/10 for 35€

[@84GDi]

​@@pgp-acc Most connection in HUN are asymetric dl/ul (even business packages other than VDSL), but for 7200HUF/mo (approx 20-22EUR) Telekom offers 2000/1000 on fiber here too (for residential consumers). Not anywhere of course, we have areas where ADSL on crap-old phone copper still cost like 9k HUF (30EUR) and barely reach 10M/500kbps up.

[@benjackson11111]

THanks for this vid. I finally got it installed (I'm the guy that commented about 15 times over the last day or so in the 'issues' section!). Great package, even though it was a nightmare to get it installed lol. Thank you!

[@sagegeas9205]

Symmetrical. You would prefer 150mbps of symmetrical. Asymmetrical means exactly what you already have with the 1gbps down and 40mbps up. Don't sweat it Jeff, I made this mistake too.

[@JeffGeerling]

Yes, oops!

[@MarkusOrt]

If you download at 1gbps your upload is almost maxed out sending ACKs which is probably part of the reason you „only“ get 930 Mbit/s

[@mirage7436]

They beat me to it, but yeah...I much preferred my 75M synchronous/symmetric FiOS at my old address over the 1G asynchronous/asymmetric Spectrum Business at my current one. Unfortunately I have no other options besides cable here, the phone lines are too old to support more than 128k (per AT&T) and there is no fiber in the area.

[@daveo1346]

​@@MarkusOrt It's mostly TCP overhead which if you had a gig would come out to around 949Mbps max. There's just no way to get a 1000Mbps on a 1000Mbps link negotiation.

[@JohnPMiller]

I called Xfinity because my Internet had been down from around midnight to 7 AM. X: "No, your Internet connection is currently up." Me: "I'm talking about last night. X: "Yes, it's currently up." ...minutes later... X: "Yes, it was down, and we'll credit you $10 on your next bill." (was never credited, but I gave up) Ever since, I have been running a ping test from a Pi every 5 minutes to track outages. Nice job on your tool and video!👍

[@FabioSpelta]

Well they clearly state "up to" 1Gb per second.

[@jong2359]

This makes it right? No.

[@PeterEvans-english-adventure]

I note that this video is quite dated, however, for an update from where I live in Bangkok, I have fibreoptic cable to my home and I get 223 Mbps download and 216 Mbps upload speeds locally. Using a server in Singapore I get 95 Mbps download and 110 Mbps upload. I am paying about USD22 per month. I am so happy with all my Internet experiences.

[@shivsankermondal]

I get 990Mbps max min 950Mbps on my gigabit fiber ping under 5ms and 530Mbps upload for 31 euro

[@JeffGeerling]

/me is jealous!

[@pd8559]

None of the European comparisons are apples to apples. The large geography, fragmented regulation etc. of large land mass countries will always allow pockets of monopolies and more expensive operations than small densely populated land areas. You get gigabit or more for less than the price of a pizza in countries like South Korea et al. The better measure is the quality of service and uptime from any ISP.

[@RaduRadonys]

@@pd8559 Well I pay $7 for 1 gigabit down and 500Mbps up. Real speeds are constant 940Mbps down, 510-520Mbps up. And as for the uptime, I am pretty much always online (me and my servers) and in 10 years I had 1 single time about 2 hours of Internet outage. SO I would say the uptime is good. I live in Romania.

[@Manguitom]

I really appreciate the work you put in your projects. Yesterday I was wondering if I could use my Pi to check which of all the ISPs would provide the best internet quality and here I have an entire git repo to test! I will be testing 3 providers connected to my Pi via usb eth adapters. Thanks a lot!

[@Manguitom]

Now I'm wondering how can I run multiple Pi-hole and the Grafana dashboards for each network interface.

[@fristysOriginal]

*Laughs in European*

[@BenReese]

4:50 - YES!! There is little to no reason ISPs couldn't offer a 150 symmetrical service. DOCSIS 3.0 supports 200Mbps upstream, 3.1 is over 1Gbps, and DOCSIS 4.0 is 6Gbps. They should make that the standard just to properly compete with fiber ISPs.

[@RichBurgundy]

Timely as always. I was just this afternoon looking for a home network bandwidth monitoring solution! Cheers Jeff.

[@ttomkins4867]

My options are 120/20 Cable or 7/1 DSL both costing $80. The DSL is only 3/0.8 and become unstable any time it rains. (Its known to be the neighborhood feeder line, but they wont replace two blocks of wire) Cable is rather solid with the exception of media being throttled drastically (240 KZbin needs to buffer) in the evenings but a VPN works around that.

[@johngermain5146]

What to monitor with the PI? Well, water cooling. You see, Corsair, EK, AquaComputer, etc. make temperature and flow tools to monitor your custom water loops, fans, and pumps. The problem is that they are all internal to the PC, and I don't want to stand on my head to read the meters. I wanted to mount a Pi to a bracket and get power from the pcie slot. In the pi, I would monitor fans, pumps, flow, and temperatures of various points in the custom water loop and display the data (and even set curves) on a web page. Yes, Several companies are starting to get the idea that we want the data displayed on the monitor instead of inside the pc, but while they are making some of the parts they still haven't completed their projects.

[@lmamakos]

I have a rural cable ISP which works really well... except for those times that it does not. For me, it's usually an upstream packet loss problem cause by all sorts of random faults. AGC in the trunk amplifier on the pole going nuts when it gets hot outside, busted laser in the backhaul from the remote node I'm served from to the head-end. Noise ingress "somewhere" in the system. But most people don't noticed, because downstream still works great, and small TCP ACK segments have better odds getting through a bad S/NR upstream link. So a couple of years ago, I hacked up a python script that wraps around fping and I continuously ping 4 destinations, each with 3 (full size) packets per second. One of those is the cable modem's management address to ensure my pfSense router hasn't gone stupid, and the others are the ISP's default router, 8.8.4.4 and one of my servers at a remote colo. Every 100 seconds a packet loss and latency min/max/mean measurement get pushed into influxdb (and grafana dashboards). Using this, I diagnosed the first temperature-related issue. The packet loss data graphed against outside temperature measurements coming from Home Assistant really made this painfully obvious. The local cable ISP tech (and it's always the same/only guy out in the sticks where I live) and I have a great relationship. He believes me when I tell him stuff, and we skip entirely over the whole "reboot your router" and "plug your laptop into the cable modem" dance. He doesn't even bother to come out to my house when it's plain that it's an outside plant problem. Being able to get them good data is helpful, and I can call in a ticket much sooner when looking at the performance data. Thanks for the video, great stuff!

[@AdamChiaravalle]

In Chattanooga we get municipal gigabit fiber (1000 up & 1000 down) for $70/month. Pretty sweet. Thanks for the video! 👌

[@Markste-in]

you could also monitor the power grid frequency. This gives indication about the stability of the grid and whether any power provider had issues with their power plants or maybe even an outage

[@SadmanBishal]

Thanks for this awesome project. I have already installed it and it's working great.

[@dirtyvinyl8817]

Helpful tips: If you are getting less than 900Mpbs from a Spectrum internet connection then you have a actual problem with your connection and you should have a technician visit your home to run a docsis throughput test to insure docsis itself is getting no less than 1gbs off the coax. Before you call for a technician, you should honestly try using the spectrum provided modem and insure its their new 2.5gb modem and that it is not made by HITRON try asking for a Technicolor/ubee/ or sercomm modem. it costs you nothing and puts the responsibility on spectrum to insure your wired connection is no less that 900mbps. I know this might be sacrilegious for IT professionals but their new modems do provide consistent internet connection and they cost you nothing. If you do call for a technician, don't be shy and ask them to show you a DOCSIS speedtest from their Signal Level Meter, and do trust that machine, its a modem in itself and tests the throughput of the coax cable. All homes that are given the 1 gig option should be getting about 1.15gbs out of the coax and if its less then that then its either a cable issue unless the problem persists to the street connection in which its a plant issue. you could have neighbors causing signal leakage thus affecting your internet connection but because nobody reports it, you are all living with it.

[@Fostravel]

Wow! As a "stats junkie" and a Pihole user, I really like that project's results! The bit I would maybe mention/change would be the automatic speedtest. I understand there is no other way to get "real" data for this, but I would fear that the sporadic tests would impact my internet experience. If someone in the house is playing a game for instance, they'll get lag spikes every half hour for "no reason". Other than that, I might try this out on a raspberry pi 3 I have laying around and if it works well, have it replace my current pi4 config. Thanks for the awesome content as usual!

[@devotee]

Samknows fixed this issue by monitoring the network activity. If there is any significant Internet usage, the device would do the tests later. I'd love to try this project (I'm on a 1Gbps connection) but with the current prices and stock shortage for Raspberry Pis 4 and 5 it seems it will take a while.

[@all.day.day-dreamer]

@Jeff Geerling ... I'm in Kansas City, the first home of Google Fiber. I was lucky enough to have it starting in the Spring / early Summer of 2014. We specifically moved from Nebraska to Kansas City for this amazing service if you can believe that. Sadly, I had to move a few years ago and lost Google Fiber. I still consider myself lucky with Spectrum, formally Time Warner Road Runner. I have the 400mbit down, 20mbit up package. In reality, it's about 470mbit or ( 57MB/s ) down. I've been on the Internet since Dec of 93 using Windows 3.11 and Trumpet Winsock and a USRobotics HST Dual Standard if memory serves me right. Jeff, there are ways to max your bandwidth if you know what you're doing to make the most out of your internet connection, along with some great software tools to help with packet loss, ping times, etc. Maybe you've done videos on this subject before but if you haven't, it is worthwhile information you can research and share with your audience. cFosSpeed is one such great tool and at $10, it's a steal. This is some incredible software engineering out of Germany and it really is a worthwhile investment. Another great tool is SpeedGuide.net's TCP Optimizer. Other improvments one can make to increase ping times and stability is replacing the cable modem supplied to you. Many ISP's are still proving DOCSIS 3.0 modems to their customers. A very much improved cable modem experience can be had off Amazon with a purchase of a $80 to $120 Motorola DOCSIS 3.1 modem. For those geeks out there, you can also bond 2 different internet connections and or, use USENET with concurrent connections to really get some amazing speeds. We now have 2gbit internet in Kansas City for $99 a month or, 240+ MB/s a second. Can you get those speeds? Absolutely. While I do not currently have this service myself, many friends local to me do and they routinely get these speeds ( 200+ MB/s ) off Steam, EA, Amazon and many of the bigger brands on the internet. Again, if you like downloading movies and music from USENET, you can easily hit 240 - 250MB/s. An example of this would be a 10GB file in about 2 minutes. Jeff, you have some valid points but to be fair to your audience, I would provide them with the same information I've provided you with ( you will have to do some research ) so they can get the most out of their internet speeds. Lastly, a good VPN will circumvent ISP throttling. I've tried them all and I highly recommend NordVPN or ExpressVPN. VPN's can vastly increase download speeds and ping times. NordVPN gives me 25 - 30ms ping times in online gaming as where Spectrum will often give me 35 - 55ms ping times.

[@fearisan]

I just set it up and submitted my first pull request for it, thanks!

[@BillinSD]

I just learned the speedtest run from my Ubiquiti USG3 router caps out around 200Mb/s and is unreliable. Having another test is worth it!

[@jamesaikens]

An OpenVPN GUI install and a DDNS updater on this box and it becomes something I can bill for, as opposed to a rabbit hole of hours spent behind my keyboard. Great work!

[@paulmichaelfreedman8334]

If you really want to know what the practical speed is of your internet connection, download a torrent with max 3000 open connections. That really stresses the router (NAT table is maintained by the router/firewall) AND completely saturates the available bandwidth. Provided of course you choose a torrent that has sufficient seeds. My 500/500 connection falls just short of 62.5 MB /sec, so there's no overhead in the carrier that's for sure. Figures as FttH only requires a Fibre to ethernet media converter - at least here in Netherlands the signal going over the fibre is EOF (Ethernet-over-Fibre)

[@thatguyontheright1]

I did this. Used a raspberry pi to automatically take speed tests, and log the data into a google docs spreadsheet. I did this because I was paying $40 a month for 3 megabit internet (in 2015-16), but they oversold the area, too many people on a single DSL node so my speed was 1/10th of that on a good day....1/100th usually. My ISP, Frontier Communications had a total monopoly in the town I lived in. Not even satellite was offered. They had no reason to fix it, so they blamed my equipment. It was because I wasn't using Windows, it was because I wasn't using MacOS. It was because I had Cortana turned off. It was because I wasn't using Wifi because that's faster than wired. It was because my computer was too old. After years of dealing with this, I took the spreadsheet and sent it to the FCC, and started setting up a class action suit for false advertisement. As soon as the FCC started investigating, my speeds went up to where they should be. As soon as the investigation was closed, my speed dropped to slower than dial up...and it became unreliable. Around this time, things switched to the FTC and they did nothing. Was so glad when I canceled my internet service and I told them off. No I could not use cellular internet since I lived in a literal dead zone. a couple carriers worked, but it was always spotty. During this time, they rolled out an update to my router...and the service cut during the update and it bricked my router. I had to pay for it, because connection issues were my fault. The biggest shit they took was when a town across the state line got a municipal fiber connection and they wanted to run the fiber connection up my way because it was only a couple miles. They talked to the town and frontier held their roll out plans in court, accused them of price dumping as they were offering 1Gb speeds for $30 a month. They held them up in local courts till the fiber company canceled their plans. I moved out and into a house where the previous owner had specrtum. Daily I get ads from them to switch in the mail. One time I had a very rude salesman tell me I was stupid for not going with spectrum.

[@frankbohnen2347]

Jeff, I thought here in Germany we'd it bad. I live in a very small village, not much bigger than Nutbush on Highway 19. I get 250Mbit down- and 50Mbit upload (all max of course) with a Phone flat for local and long distances calls included about 60 Bucks a month. I never complain about that again.

[@lis6502]

what's amazing about channels like Jeff's is that you can set up all this opensource stuff on recycled x86 terminals - for instance i do have hp t610 running proxmox as super-cheap lxc containters' "pod", yet it does serve pihole, gitea, redmine and occasionaly my own stuff. > why bother when rpi costs $35 well, from time to time you can buy two complete terminals for price of 1 rpi, also you help reduce ewaste and are giving second life to "obsolete" stuff. given a fact that these can be happily used as a base for retroarch or other retro gaming solutions and are expandable in terms of memory, storage and sometime even peripherals like pci(e) cards, they might fit better in one's needs.

[@JeffGeerling]

Definitely recommend reducing e-waste and reusing good older computers, as long as you're not using something that also eats up tons of energy (some old servers for example can use 100+W at idle!). But things like light small form factor computers can be perfect for this, or just old office PCs that can be had for cheap (or free!).

[@lis6502]

@@JeffGeerling seems like fun topic for one of upcoming episodes, huh? ;)

[@Well_Edumacated]

It’s mind blowing all the stuff you can do with a raspberry pi

[@TomAtkinson]

I like this cow shot at the end.

[@kane587mad]

Some ISP in Germany have policies that prohibit to test connection speed too often. Every 45 minutes can be considered as "too often" as it can have very big impact on the ISP network if there are many people doing this.

[@grmasdfII]

Care to share which and where they state this as a policy? I've never heard of it and would like to look into it, but couldn't find anything on my preferred search engine.

[@kane587mad]

@@grmasdfII it's stated in the "Allgemeine Geschäftsbedingungen (AGB)". don't know the English term for that. I think 1&1 is/was one of the few who actually enforced that some times. I think my cable TV and internet provider (pyur) has this policy too but i didn't mind to look it up as I'm not planning to do connection speed tests that often. It's mostly formulated very "open". Like "any behavior that can cause unnecessary load to the ISP network".

[@grmasdfII]

@@kane587mad Ah, the "don't be a dick" clause. I doubt a check once every 30 minutes will be a problem.

[@sheevBL]

great video. I think you should mantion that some cables cannot handle 1gb speed. example: 1gb requires cat6 cable

[@andrew2004sydney]

Monitoring solar power production and household power consumption would be cool.

[@cgsnascar]

Can't wait to get home from work and get this set up on my Pi, amazing!

[@prashanthb6521]

Your content is always interesting. Thanks.

[@nathanisip]

Interesting set-up. I stopped using PiHole a few months back when a KZbin update broke the block lists and I couldn’t make exemptions for all the weird Google ad domains and ended up not able to watch any YT for a week until I just gave up.

[@JeffGeerling]

The cat-and-mouse game with KZbin ads finally convinced me to pay for Premium. Not advertising for it, but since I watch a lot of KZbin it was worth it for me.

[@mjc0961]

You have *_TWO_* options for internet?! In the United States?! You're a god among men.

[@faux_grey]

I wish I had the bandwidth or data cap to leave a device plugged in running speedtests every so often. Yours is truly a first world problem Jeff. xD

[@i_am_macgyver84]

Thanks, looks like I can set this up on my Unraid server. Also in my opinion anytime I hear anyone talk about a raspberry pi saying it's only $35 is being a little dishonest. You need micro sd card, usb cable, power supply etc. Yeah a lot of people have spare cards, and cables laying around. But if you don't it's about $80 for one of those kits that come with everything including a case.

[@PterippiGaming]

Here in Finland it is common for mobile network operators to sell you internet subscription that will offer "up to" 150/300/600/XXX Mbit/s but in reality not only this is limited by the type of the base station that you are connected to (those high speed access points have way lower coverage and are mostly limited to large settlements), but also out of what should be on paper a 300 Mbit/s (based on the speed of the base station) you get maybe 100 Mbit/s. And it is not a problem of reception strength or device type. Just other people sharing the bandwidth with you. And yes, you can chose more expensive plan. And no, this has no real effect because operator simply does not aim to provide these limits by having more access points because they are expensive. The advertising says "up to" and so the "up to" you get. 0 kbit/s is also in the range of "up to" xD.

[@crimson750]

I'm on AT&T with the 1 gig plan at $60/m. Works fine for me, 1GBPS download and upload.

[@CrayonIRL]

I have the same plan same price with Spectrum, no other options. It goes out often and one time it took them 11 months to figure out a packet loss issue smh. I currently have a Pi 3b+ setup to email me ping results every 6 hours via a cron. Unfortunately the Pi 3b+ caps out at ~330Mbps in a speed test due to I believe the USB controller iirc so I am thinking of upgrading it to a 4. It’s also currently my Pihole. Great vid!

[@JeffGeerling]

You know the pain!

[@Nexalian_Gamer]

One day this man will land a rover on Mars using only Raspberry Pi's

[@jonmayer]

Home assistant has been monitoring my internet for years. It's actually quite consistent, but the 10mbps upload is a real drag. I'm waiting for AT&T fiber to finally get installed in my neighborhood.

[@Hugocraft]

I'm already using pi hole on a dedicated pi wired to my router so now I'll look into adding the other things you listed but my main goal is to see downtime when I have no internet at all. A report showing date, time, and duration of outages is what I was really wanting until spectrum came out and replaced the cable from my box to my router since it was testing bad. My parents are signed up to get star link since their only choice is hughes net satellite so a pi doing the same stuff as yours will be nice to monitor how good its doing.

[@pedro_8240]

In Portugal, for, if I'm not mistaken, less than 50 euros you can get gigabit that is actually gigabit, or at least very close to it, at around 950Mbit consistently and 200Mbit upload. And that 50 euros includes cable TV and a landline.

[@avs-forum]

I watched to the very end and I appreciate the artistry of the final cow mooing. It worked, dude. And yeah, Pi-Holes kick ass.

[@Derpuwolf]

100% gonna add this next week to my pihole settup. Our ISP rarely ever gives me my full upload speed and it’s horrible because upload is all I want.

[@ecotts]

This guy is awesome, I really enjoy his videos.

[@JensIhnow]

Jeff, you should get or add an Ripe Atlas Probe! Contributing to the global data research as well as having a great additional monitoring for your ISP as well as connectivity and routing aka peering.

[@DeadlyDragon_]

Gig is 940 down 40 up. This is due to overheads in networking. It should also be noted that when multiple devices are connected at once you wont be able to get a true test of bandwidth as some bandwidth is actively being used up by the various devices on a network.