Can you trust your smartwatch?

[@TheQuantifiedScientist]

Great video and explanations ❤

[@DrAndrewSteele]

Oh wow, thank you! The ultimate KZbin wearable testing endorsement! Anyone watching this who hasn’t already checked out @TheQuantifiedScientist, do so now and you’ll see why. :)

[@TheQuantifiedScientist]

@@DrAndrewSteele Thanks Andrew! I got some testing ideas from your video :-). Well explained, and I appreciate you pointing out the N=1 problem most videos (including my own) have. It's so tricky, because scientific literature is usually generations behind the current market (and/or sponsored by the company being tested).

[@DrAndrewSteele]

Haha, well _I_ got a few testing ideas from _your_ videos, so I guess we’re even? :) Totally agree about the frustrating issues with studies and KZbin reviews though… I have a few ideas but they’d need some resources to do! Let me know if you’d like to have a chat sometime, it would be great to at least make it _n_ = 2!

[@mhd230289]

I put off my apple watch purchase. After this test I see none are accurate and acceptable

[@fudgesauce]

At 9:00, you said that during sleep is when the HR measurements are at their most accurate. I would like to point out that is also when their step count is at their best as well.

[@nathanpinnock9541]

I love that 😂

[@DrAndrewSteele]

LOL

[@the13thguardian98]

But do they count the steps i do when i run in a dream 🤔

[@fredfarnackle5455]

🤣🤣🤣

[@ja.martini]

imagine you waking up and discovering you made 600 steps during sleep lol

[@notenoughtech]

A couple of things to consider in your tests. Most of tracks are relatively sensitive to the location on the wrist, how tight it the tracker is wrapped around it and any ability to slip and move around. In my personal test I also found out that metricks were much closer to my chest strap (consistency) when I shaved my wrist. Optical sensors that have the ability to glide on skin don't play well with hair moving across it's optics. It's hard to conduct parallel tests without disadvantaging a brand. Perhaps a row of 3 tests with each watch in primary position would compensate for that. I appreciated the collam with Steve explaining how actual sensor tracks the pulse, as it explained to me the early miband controversy over recording heartrate of inanimate objects.

[@lukearts2954]

If you don't want to shave your wrist and look like a dork in the swimming pool, you can also just wear the watch on the inside of your wrist. I tested the differences and found the watch (Garmin Forerunner45) to be more reliable and accurate when worn on the inside. It wasn't the purpose of my test to find that it worked better, so I didn't try to find out why. I guess you just explained why =) My reasons for testing were dual: I have a habit of damaging lots of stuff by getting stuck behind them with my watch, and the reciprocal damage to the watch and sleeves hanging over the watch is also relevant, so I needed to find out a better way of using the watch. The other reason was to settle a discussion =) I asked my brother who is an active athlete and avid user of tech gadgets, and he immediately said he assumed that it would not work or not be reliable at all, given that it was designed for the blood vessels on the outside of the wrist... Luckily I know more about anatomy than he does, so his remark actually made me think that it could definitely work. To prove me right, all I had to find out was if the data was consistent and usable (I'm only interested in observing my own evolution, so it wouldn't matter if the numbers weren't absolutely correct, just as long as they were consistent). I had also considered wearing it on my ankle/calf if the inside of the wrist didn't work. Just anything to not have to wear it on the outside of my wrist. So I tested, using heartrate monitor and blood pressure devices for the norm values and found out that not only does it work perfectly, but the actual values measured matched the more bulky old-school devices' values much closer and much more consistently. On the outside of the wrist there seems to be much more variance in the signal (both compared to itself as well as compared to the norm values), while on the inside the difference between the measurement and the norm values was always the same.

[@angrypotato_fz]

@@lukearts2954 Interesting, thanks for sharing that! I also tend to wear my watch on the inside of my wrist because of practical reasons, similar to yours (and a habit now) and was often curious if a sport watch measurements could be wrong because of that. Unfortunately my watch is a total garbage, so I can't have any consistent results, but your experience assures me that I can wear it in a convenient way.

[@DrAndrewSteele]

Thanks for the comment and sorry I missed it for a few days! I’ve read this too but interestingly didn’t find the effect very strong… I wore the watches on the same and different wrists, in different orders etc so they were in lots of different positions, but it didn’t seem to make a great deal of difference to their relative accuracy. That said, I’m white and have thin pale hair on my arms so the watches are definitely playing on easy mode! I did manage to get some strange effects which may be due to the watch sliding up and down the wrist but, as I said in the video, these didn’t appear on the test run I did with all the devices. D’oh. We discuss it a bit on the extra content for Steve’s Patreons, and I’ll probably talk a bit about it in a future video too! This one was already quite long…

[@veggiet2009]

​@@DrAndrewSteeleeasy mode! 😂 But seriously this brings up an interesting question in my mind of might these watches suffer from a hidden racial bias? Since they use optical sensors.... It would be interesting to see if the designers/programmers could greatly improve accuracy if the added sensors at different points along the watch band.

[@izzieb]

I feel something that has to be pointed out is how cheap the Xioami Mi Band is compared to the others in this comparison. While it may not be the most accurate, it is definitely more than enough for people to get a good picture of their fitness level and make changes accordingly.

[@DrAndrewSteele]

Very true, I did mention it briefly and I am impressed for the price!

[@altioar]

10:27 mi band price mentioned here

[@valleyshrew]

Another thing to point out is that Xioami is owned by the Communist Party of China so if you dont want them spying on you or dont want to support a genocidal dictatorship then you might want to consider a different brand.

[@AnimilesYT]

While I was paying attention to the video, I also didn't notice the moment he mentioned the price. So I think it's good that you pointed it out here :)

[@estebanod]

Yep, the Apple Watch can be 10 to 30+ times as expensive

[@MedlifeCrisis]

The point about measuring some of these only at night is very important one as people get a false impression of their resting heart rate, HRV etc (everything appears better when you’re asleep!)

[@DrAndrewSteele]

Feels appropriate to heart your comment on this subject (Also it’s a good point!)

[@pseudophd1073]

Like you said the absolute numbers might be off but the relative trends day to day (assuming decent precision in measurements) is the aspect that can help us improve our lives. They’ve come a long way though! I remember getting thousands of steps on the original fitbit in ca. 2013 when all I was doing was mixing samples on a vortex mixer. At the end of the day if someone wants a device that is cheap and encourages them to stay fit or sleep better, maybe accuracy isn’t the be all and end all. On the flip side, if you’re into tech like me, you’ll end up watching the quantified scientist and looking like a cyborg wearing a whoop, oura ring, apple watch, polar etc in order to get ALL the data! Question is, how many years until implanted trackers? Scary thought.

[@DrAndrewSteele]

Haha yeah I found The Quantified Scientist while researching this video! Serious dedication! I admit reading all about these made me a bit cyborg-esque/tracking obsessed too…

[@Crowski]

Apple watch saved my Grandmothers life. I advise anybody who's older or has CoPD, sleep apnea, etc to have one!!! Her oxygen was in the 70s and we had no idea....I grabbed my watch and put it on her... called an ambulance. She was in the hospital for 3 weeks... now she's on oxygen, has a CPAP and I got her her own apple watch to help track her oxygen levels.

[@AlexMkd1984]

biggest lies 🤥

[@Crowski]

@@AlexMkd1984 I’m her caretaker. It 100% is true. If I didn’t have that watch she could be dead. -_- Low oxygen isn’t something you’d notice till it’s too late or you have a way to measure it. She still has her Apple Watch 6. Which I got for her on EBay for $165. It’s a life saver and well worth the $$.

[@EjayT06]

@@AlexMkd1984????

[@owen-trombone]

@@AlexMkd1984we get it, you work for Fitbit

[@donoru0612]

I had a MI about 7 years ago and my FitBit captured my heart rate which was extremely high at 2am which I was supposed to be at rest. At the end of the day these are NOT medical devices but tools to assist us. Do your research, some are better than others. My Fitbit probably saved my life.

[@ibimslawa]

I wonder how a Garmin watch would have performed in your tests since I heard that their sensors and measurements are one of the best.

[@TheS4ndm4n]

I have a Vivosmart 5 and sadly, last time it did horribly. I was trying to do some cardio and once I felt reasonably taxed I repeatedly checked the wach, but in never went over 120 bpm. After a lot of wheezing I checked on the machine I was on and it said I was over 160 most of the time. Maybe it would have helped if I told it I was on a cross-trainer. Possibly the relatively slow movement threw it off.

[@tams805]

They are among the best, though Apple tend to be the most accurate. But really, using optical sensors to measure heartrate just isn't very accurate. *None of them* are great. They are a trade-off between comfort and convenience, and accuracy.

[@JoshuaTootell]

When I do a bicycle workout, my optical sensor is near dead on perfect compared to wearing a chest heart rate monitor. First few minutes are iffy, but after that (still in my warm up), it will be within a couple BPM. This is using a Garmin Fenix 6. Accurate enough that I don't worry about wearing a monitor for planned workouts lack that.

[@dagger6467]

I have the VEVO Smart 2 and it's dead on the money. I check every Friday at IViG treatment. I do make sure I keep the app updated as well with weight etc...

[@Gazzab6]

@@PTXDDI’ve just watched that too and the testing was very accurate and he ruled out anything that could cause inconsistencies. The Garmins didn’t do that well compared to the much cheaper Apple Watch SE.

[@MedlifeCrisis]

It’s good to know I can continue fancy walking without upsetting the numbers too much. Not sure why you call it silly, it’s rather classy if you ask me

[@Lampe2020]

9:44 That shock is also what allows me to jump out of the bed in the same second I wake up, sometimes I start the jump while sleeping and land on my feet, awake. But I don't jump every day, only if I have slept good and something I classify as important is happening soon after my alarmclock goes off.

[@LadyCatFelineTheSeventh]

I do a lot of crochet and while winding a rather large ball of yarn I realized my smartwatch was reading it as steps. I guess I got a lot of exercise that day.... I now take it off when I crochet or wind yarn.

[@lighthealerastrid1465]

I cleaned my windows once and at the end of the day was extremely impressed with my step count while doing so.

[@scaredyfish]

I got my first fitness tracker because I was diagnosed with sleep apnea, but the CPAP wasn’t making me feel any better, so I wanted to measure my overnight oxygen saturation. I got a Garmin watch, which at the time was the cheapest one I could find with that feature. It gave me some very alarmingly low results which sent me back to the doctor for another sleep study. The clinical pulseox machine showed the results from the watch were complete garbage, and had been stressing me out for no reason for months. I now have a FitBit which doesn’t claim to measure an absolute O2 number, just variability, and seems to be far more consistent. (Not saying Fitbit is necessarily better than Garmin in general, as it was also a higher price point) I still don’t know why I’m so tired. The Fitbit says I don’t get much deep sleep, but as you point out, this may not be very accurate. Thank you for doing these kinds of tests. These devices affect people’s lives and health decisions, and we need to know how far to trust the results.

[@DrAndrewSteele]

Sorry to hear about the low oxygen readings…but I’m afraid I think it’s quite typical of these devices. I didn’t test these in the video because it’s quite hard (I’d need to either go to high altitude or do a test on a plane to change my sats and, even worse, a lot of these watches don’t allow a one-off measurement, they only do it when you sleep or whatever…the reason being, spoiler alert, they’re not very accurate!) They all seem to give readings that are too low for one reason or another, and in a way that’s obvious because most readings are 95-99% with a ceiling of 100% so any error is going to be in the downward direction! It’s not even clear based on my reading around that there’s much correlation between the readings and the actual values… Interesting, and glad the Fitbit is a bit more consistent. I hope they continue to improve. :)

[@baguskusumaloka]

As MI Band 7 user actually satisfied with the performance. Actually my use case is more like smartwatch than smartband, with small size and cheap prices actually it's a good device. Thanks for reviewing and compare.

[@ifacedownworship]

I'm using it too! The Quantified Scientist said Mi Band 7 has great potential. The new Band 8 is worse.

[@baguskusumaloka]

​@@ifacedownworship yes good band. But in May 24 i sell it for Galaxy Fit 3, but after some month without change of bug i trade for Huawei Band 9 now. Galaxy fit 3 have bug with outdoor workout.

[@Oort1900]

The amount of hard work done for this video is commendable. Loved your work

[@majortom4543]

I come from my long time friend Steve Mould's channel, i really liked your collab and this video and you have earned my humble subscription. Great work

[@vidhoard]

Such an underrated video! I've wanted to know all of this for so long! I really want to know if the "sleep quality" and/or "stress" measurements are accurate. Mostly because I don't think they are!

[@DrAndrewSteele]

I’ve collected a bit of data to check out sleep quality, but stress is a nightmare…what can I compare it against? Anyone got a good-standard stressometer I can borrow?!

[@Sekir80]

@@DrAndrewSteele Your alarm clock. :D I still don't understand why people like to wake for a heart attack.

[@frankhooper7871]

I've found with my Garmin watch that it's pretty accurate on step counting under _most_ scenarios. It gets it pretty close (tested by actually counting 1000 steps) when walking normally, when carrying a shopping bag, when walking with my hands in pockets. The one time it absolutely fails to count any steps has been when I've been pushing my cycle (puncture) and keeping my hands on the handlebar. On the flip side, it absolutely fails on stair climbing LOL, either registering 0 flights or stupid numbers like 50 flights when walking alongside a river, which definitely isn't on a slope.

[@insidejazzguitar8112]

I appreciate how thorough and clear this is. Answered every question I was having. Great editing and use of graphics.

[@VintageJunior]

Literally got my first smartwatch yesterday and was wondering how accurate those measurements were. Now thanks to you and for once, the YT algorithm, I instantly got what I was looking for. That alone is reason enough for a sub.

[@mahamoudtraore2339]

is it Apple or Garmin?

[@tristanwegner]

Came here from Steve Mould and it was totally worth it.

[@DrAndrewSteele]

Welcome!

[@PiPerson7472]

The Oura ring would also be nice to check out. I don't think there are too many ring wearables out there. But even to see how they perform compared to the big bulky watches and other medical grade equipment could be very interesting. Especially since they are about the same price or more than something like the Apple watch, and they claim to track just about all of the same data points if not more!

[@DrAndrewSteele]

Yeah I’d love to get hold of a few more devices and try them out too! The Oura Ring and the Whoop Strap are both interesting for not being wrist-mounted…

[@Nilguiri]

​@@DrAndrewSteele Whoop Strap? Ooer, sounds a bit rude. What do you strap that to, dare I ask?

[@DrAndrewSteele]

@@Nilguiri Your, er, upper arm, disappointingly!

[@Nilguiri]

@@DrAndrewSteele Oh well, to each his own! haha.

[@PiPerson7472]

@@Nilguiri They also have sports bras, underwear, swimsuits, thongs, and shorts that you can put the Whoop sensor into!

[@analuisaadornodelima9503]

A follow up with Oura, Samsung Galaxy watch and Garmin would be amazing

[@CookerSeven4]

I gotta say, I 100% vouch for the underestimation of the fitbits heart rate monitor during high intensity workouts. I typically have a pretty high heart rate during exercise, and i can say with certainty that I’ve never seen my Fitbit actually track my peak heart rate accurately. It’s quite frustrating.

[@jorgeherrera9003]

I love the way you present de data on these videos, really helps understand everything

[@DrAndrewSteele]

Thank you!

[@eduardog3000]

It would be interesting to see a comparison between modern smart watches and Nintendo's Pokewalker (and its later iteration, the Wii U Fit Meter), which at the time was one of the most accurate pedometers. Also just comparing arm based watches to hip or leg based pedometers in general would be cool too.

[@randyhavard6084]

I've always felt like I could trust the pulse rate and 02 percentage but the step counter has been very optimistic on every smartwatch watch I've ever had

[@PaulBrassington_flutter_expert]

As a techie and health enthusiast this is a great video, I totally agree that changes of the data over time is the prize, hence the app is important and paywalls are also significant. Thanks

[@wakeupuk3860]

As once teaching Science, then an IT Trainer and now an old man prescribed Sleep Apnoe who has gone down the getting 'super fit' path in the hope of being not having to use CPAP machines, which I hate and have problems wearing - I am currently using such skills looking at how well, accurate and cost of getting a smartwatch would be to me - so thank you, a very good, informative and far better than anything else I have seen on here

[@evanbarnes9984]

I can't believe I didn't know about your channel before you teamed up with Steve Mould! It's right up my alley. I'm going to show this to my high school stats class. I'm trying to show them that there are lots of ways that a basic familiarity with statistics and the methods of collecting data can help answer really interesting questions. I really want to help them see stats, and math in general, as a powerful tool they know how to wield, not just an esoteric puzzle system that math teachers make you learn.

[@DrAndrewSteele]

Nice! Hope they enjoy it! I really wish I’d done more stats at school. My A-level maths teacher forced us to do S1-3 (no idea if that still exists!) instead of mechanics M4-6, and as a bunch of physics nerds we weren’t happy about it…but I’m very grateful for it now! Even some fairly simple stats plus a bit of thinking can get you a long way. :)

[@banksarenotyourfriends]

I've had several Garmin watches and found that the different price brackets of their watches all give slightly different results too. I'd imagine a lot of the hardware is identical between the watches, but Garmin are notoriously bad at releasing timely software updates so I'd always felt the differences must be down to the various software versions. Really interesting video, cheers.

[@ibimslawa]

I am however surprised how frequently updates are pushed to my one generation old Fenix 6. But maybe it's not that old after all..

[@banksarenotyourfriends]

​@@ibimslawa Yeah, updates seem to be ok for Fenix and MARQ watches, but their 'cheaper' watches are often left for months with bugs that need fixing. My wife's had quite a few of the sub-£400 ones and all of them have had issues at one time or another.

[@muvidz]

Most algorithm done by Firstbeat. Other brand implements Firstbeat as well, not limited to Garmin.

[@konanamegakure4342]

Very informative. It saddens me educational videos tend to get lower views vs garbage content that doesn't improves ones self grow. Thank you very much.

[@dudib94]

Great video! it would nice to see compression between garmin, galaxy and pixel watches too

[@DrAndrewSteele]

Really would. I’m going to need a bigger budget. (And a longer arm…)

[@huawafabe]

I think Pixel Watches should use the same tech as Fitbits :)

[@IceBergGeo]

@@huawafabe they (might) use some of the same, but not all of it, at least according to the Fitbit website, and the range of functions each tracker/watch can and cannot do.

[@Ricky_B_2453]

Great video - keep up the good work Andrew👏.I agree with you that there should be more open test results available especially given the increasing use of smartwaches. Also, I have a Garmin instinct 2 and it measures this thing called HRV (Heart rate variability). I was wondering if you knew, given your research whether it's actually an important metric?

[@DrAndrewSteele]

Great question! And one I’ll be discussing with a cardiologist in the next part of this series… :)

[@snazzypazzy]

Interesting! I own a Mi Band, that I mostly got because my heart rate was doing funny things and for €30 I figured it was worth the risk to see if it would help with my anxiety about that. (Turns out I was super anemic.) And it does help with that, because I don't have to actively count my heartrate when I'm dealing with a panic. The seem pretty accurate for that, compared to my counts. Only thing it does weirdly is occasionally it drops to 45 when I can count my pulse and it's higher then that. I don't use the fitness tracker, and even try not to use the step counter, because it's not good for my mental health at all. Which I wonder how that is influenced for other people too, since these trackers became so common.

[@Debraj1978]

You should include battery consumption. In most of portable/ wearable device, the infrequent measurement is made to conserve battery.

[@markstinson4434]

Very enjoyable and timely, too, since my Fitbit is on its last legs. A couple of chuckles from its algorithm is that (1) pushing a shopping cart counts for nothing while (2) pushing a lawn mower counts as time on a stationary bike (a neat trick since that isn't one of its exercise modes). Thanks for putting this out there.

[@DrAndrewSteele]

Haha yeah the autodetection is something else! Whenever I do a talk/TV interview it ALWAYS comes up as ‘outdoor cycle’!

[@mrjoelpt]

I feel like price comparisons should be done, also would be interesting adding other brands to compare, specially Samsung and Garmin

[@davidmorton8170]

A problem I've had with step counters is similar to your water test - walking when pushing a shopping cart or pulling a disc golf cart. My Garmin watch seems to handle that a bit better than the fitbit.

[@boomish69]

There is a setting for Apple Watch heart rate, you have to switch the tracking on & check the app background refresh, plus there are a ton of extra things the heart tracking does that you didn’t mention.. like phoning your doctor if it thinks your having a heart attack, it’s not really comparable for those extra but I think even the cheap ones are worth having for basic health tracking.

[@SarahKchannel]

Interesting - I just wrote firmware for some comparable device. The 'secret' sauce, or reason behind the difference, lays in the power management (frequency of measurements), smoothing / averaging of measurements. Normally there needs to be some averaging and low pass filtering to eliminate random values. Depending on how many measurements are taking to do the smoothing, will lead to how reactive the device is to changes. Higher power sample rates -> higher power consumption -> more accurate / responsive readings. So its all an trade-off between power and how many different activities such a device can track.

[@therabbithat]

Batteries only need to last a week if you are using is to track sleep. Otherwise you can just charge it at night so it would be good to have a high performance mode that was more accurate but only lasted say, 15 hours. The thing is I know that gui people don't like options and marketing people don't like it when you admit that your other mode isn't maximally accurate

[@vpnconsult]

Thats why i would choose a Garmin or Apple above the Chinese watches.

[@signalshift6676]

I noticed a problem with the heart rate measuring when doing sports. When I sweat a lot under the watch the reading can be wrong and I mean completely .. like suddenly I had 20 beats. So maybe thats another factor to include. Nevertheless my Mi band has certainly the best performance for that price, because I just want to know the rough amount of steps and distance when I go for a run. Lastly I think the different brands try to stretch their battery time with messureing the heart beat less often. But nice channel, will watch more videos :).

[@ga5712]

Interesting vid. Would have been interesting to see the samsung Galaxy watch included on this.

[@sirmyself]

I see a bit of a methodology problem in your tests. I noticed that the accuracy of the measurement of my FitBit varies depends on where I wear it on my arm. so for more consistent Data for the three fitness-trackers/smart-watches, I think it would be relevant to perform those tests 3 times, each time moving the FT/SW in a different position on the same arm. Very interesting video nonetheless. Shoutout to Steve Mould for making me discover your channel :D

[@DrAndrewSteele]

Off-camera, I did try the watches in various different orders/on both my arms and I found it made surprisingly little difference! I’ve also done the max heart rate test with my Fitbit REALLY FIRMLY attached slightly higher up my arm as you’re meant to and…still nothing. More data are always good though! It would be very cool to do this systematically with a bunch of devices/testers/arm positions/etc…

[@DivyanshBalchandani]

Loved it! While I'm not in the market for a smartwatch now, when I eventually will be in the future, I'll be sure to check for the accuracy of data as much as the features - something I might have overlooked had I not watched this video. However, for consumer devices not posing to be a replacement medical equipment, I believe there is a point after which these devices can get away with being inaccurate. As long as the data they provide is actionable for normies like me and the methods they use to gather that data are not very intrusive, I can allow them to be inaccurate.

[@DrAndrewSteele]

Definitely, there’s a trade-off when choosing between accuracy, convenience and other functions when choosing which one to buy. Fingers crossed they all keep improving-and most of them seem to be good enough to be actionable now!

[@PJTierney]

I recognise that outdoor location, that’s Windsor’s Long Walk 🙂 Lived in the town for 6 years, that’s a great location for walking/running, zero cars for hours 😎

[@Magikarp_With_Dragonrage]

Most of the problems with heart rate monitoring disappears during exercise if you set the proper exercise for the fitbit(in my experience)

[@devluz]

6:35 I used a fitbit and samsung galaxy watch 3 in the past. The fitbit is very inaccurate if you jog up hill. I suspect they get very noisy data from the heat-rate monitor and use the step count to help compensate for it. If the heart-rate goes up but the step rate goes down they get confused. The samsung galaxy watch was a lot better in this but it often stops recording entirely because it doesn't get a well enough reading.

[@davidmccarthy6061]

It boils down to the owners needs. I've gone many decades not knowing my what my heart rate was at any given time, but I do want to check texts, a call, and Google pay without needing to dig my phone out of my pocket.

[@noinghenah2764]

For the sprint test its very difficult for the high battery life band because they lower sample rate to extend the battery life so they cant catch up short and high intensive movement.

[@TinusBruins]

That the Mi Band is missing out on the heartbeat on intense run part doesn't surprise me. I have been using a third-party tool for it from version 2 till 4 and it gave interesting insight. The sensor used sometimes misreads. For example if you have a stable heartbeat of 70 a reading of 160 or 40 once every 30 minutes can happen. Mi Band 3 (the first with heartbeat reading) didn't filter out these. With the Mi Band 4 they added support for continuous reading (once every minute) but they also filter out possible misreadings. This off-course has a side effect, a stable heartbeat below 48 in my sleep can't be registered, and the same for the extreme high values. But even though I still think it's the best value option

[@soy_leche]

Andrew, love the video! Would be very interested to know how accurate these measure total calories throughout a day. A common use case is calorie tracking for weight loss and this would be helpful to know.

[@eric13hill]

I loved your insight on Steve's video!

[@DrAndrewSteele]

Glad you enjoyed it!

[@MrJbenn]

These are some fascinating insights to how smartwatches work. I hope Andrew will do more on fitness trackers, both smartwatches and chest strap wearables. I think people would be very interested to know about caloric burn calculations also. Great video, Andrew!

[@DrAndrewSteele]

Definitely, I’m going to make a video about calories so watch this space!

[@tpaddock891]

Great video. I wear a Garmin Vivoactive 3 while running and have found the built-in HRM to be quite reliable for me. I've verified that over the past couple of weeks where I've been tracking my treadmill runs with a chest strap and the Garmin. The Garmin is usually within 1-2 beats at all times, even up to 160BPM. Don't see much sun in the Northern hemisphere where I'm located, so you could say that I'm pretty white. 🙂 In terms of the sleep tracking, it's total rubbish, but that's not important to me.

[@joecornelius4334]

This was fascinating and helpful, but, as someone with CENTRAL apnea whose O2 can drop during sleep and who's burned out 3 all-night pulse oximeters, I was hoping you were going to test watches for that, too.

[@Kitulous]

how did you burn them out?

[@DougHolmes]

I'd be interested to see a follow-up to this with a few different watches (I own a Withings Scanwatch, and I'd like to see how it compares to others).

[@emmaodom7201]

Amazing video!!! How were you able to download the data from your Apple Watch ? Also are any of apples algorithms public or all protected / hidden?

[@Baitingmaster8D]

In my opinion, it’s best for it to run a little short, it doesn’t hurt to do a little “extra”

[@polarbear128]

Informative video! Was going to mention The Quantified Scientist's channel, but see you're already familiar with him. Can I ask where the filming was done for the green screen and your run? Looks like a good place for a walk.

[@DrAndrewSteele]

Thanks! And yes, I’m a Quantified fan thanks to the research for this video! It’s the Long Walk in Windsor. Highly recommended, it’s lovely!

[@victorwan7346]

I really appreciate your hard work to get these data keep the good work going.

[@therabbithat]

It's cool you can see visibly with tech that you are getting healthier without having to pay attention to how you feel

[@holdintheaces7468]

I seem to remember reading that the apple watch and a paired iphone will work together to get steps. The watch actually has a lot of battery considerations beyond being a fitness tracker. The fitbit and Mi band only display time and track fitness. The relatively infrequent checks by the apple watch (when not specifically tracking an exercise) is very likely a design choice to allow people to get better battery life while using the other features of the watch.

[@DrAndrewSteele]

Agreed, if you want the smartwatch features (and are an Apple user!) then the Apple Watch will give you a lot more than a Fitbit-but I think it is important to point out the price you pay in health tracking. And the price in battery life is pretty extreme too! I’d love it if Apple released a tracker…

[@Patrick94GSR]

When I switched from a Fitbit to Apple Watch, I think I read that the Apple takes less frequent HR measurements to help with battery life. But I was glad for the improvement in step counts when driving. I was getting some crazy step counts and even calorie burn numbers while driving with my Fitbit which was really frustrating.

[@prakhar9473]

Very well made video, I just wish you used a samsung watch as well. Found you through Steve.

[@Lizlodude]

I totally agree that we need more testing on these things. I've had the same issue with safety equipment; it's great that it's got pretty colors and nice foam, but if it's going to be 18% less likely to prevent me from getting a concussion, I care about that a whole lot more. I think my favorite weird issue with fitness trackers has been that my Fitbit will report garbage but seemingly valid heart rate data...while sitting on my bedside. Or left in my pocket on a hanger. It seems like if the algorithm can pull data from literal white noise, that might not bode well for its accuracy heh.

[@DrAndrewSteele]

Totally agree about safety equipment! Though I may leave testing that to the experts… And good point about garbage data when not on…I’ve got a great day of garbage data from a time my Fitbit strap broke and it spent the day in my jeans pocket which I’ve been meaning to get around to looking at!

[@eTwisted]

Thanks! My Garmin watch is great for speed and distance on my racing bike but if it's

[@DrAndrewSteele]

Interesting! I’ve seen temperature talked of as an issue (less blood flow near the surface of the skin when it’s cold) but I’ve never tested it! Only data point I’ve got is, it’s 0°C in Berlin at the moment and my Fitbit still seems to be working OK…

[@Fulminin]

I assumed that the step count was super accurate as there would be multiple ways to cross reference steps. although I always wondered how they actually determined the steps from general movement, I always feel like just accelerometer alone would be super inaccurate. I would have expected them to use as many sensors available and do something like below. In theory you can: - determine steps and steps per time interval from the accelerometer - determine steps and steps per time interval from the gyroscope (most accelerometers these days are 6-axis or 9-axis so they do both accelerometer and gyroscope) - Use GPS to determine distance traveled per step - if the heart rate tracker picks up step as well get steps from that as well (which I'm not sure about but with heart rate detecting steps sometimes would suggest that the steps is some kind of noise in the heart rate sensor) - then if you have a phone in your pocket you have another source of information for accelerometer, gyro and GPS Then you can put that through a filter algorithm like a Kalman Filter (which all GPS units use already use to determine location) using both input data from the 7 inputs and prior state data to filter out noise in the signal. Which should be super easy for a device with as much processing power as a Smart Watch or a Smart Phone to do. While all data will be not be accurate like GPS when you're on a treadmill or not carrying your phone those sensors would probably be excluded by the filter (with the downside of being slightly less accurate) when everything else is telling your device you are taking steps but the more sensors you can use the more noise the device should be able filter out making the steps determined more accurate.

[@garreswe]

I have a Galaxy Watch and a Mi Band 6 and I use them for a rough estimation of my steps though the day and for registering workouts. It can be fun to see how often I exercise and how often I move more than average but I don't care much for the other "health" features. I really don't understand why people are so interested in tracking their sleep, most aren't going to use that data for anything anyway. Also checking heart rate during the day is useless for most people. These are features the companies use for selling more products and apparently those features can be wildly inaccurate. To me the best things about smartwatches is to be able to change watch faces and to see notifications with my phone on silent. Almost all other features are just "nice to have".

[@marcradermacher6244]

The fun thing is that I actually wear smartwatches for all the reasons but fitness tracking. It's kinda strange that there are pretty much no options without these features.

[@harrisonpope7172]

So, I got a Pixel watch and one thing that a lot of people don't like is that the battery life is low and that's typically attributed to the watch taking a measurement of your heart rate every second. I'd be curious to see how that data matches with other watches and see if it really helps get better measurements, making it worth the hit to battery life. Also, I once went on a walk with my child, so my hands were on the stroller the whole time and my watch ended up tracking very few steps. So, I unintentionally did a similar experiment and learned that step tracking isn't the most accurate in some situations.

[@B9Robot]

Thanks @Steve Mould for referring you. God Bless.

[@Slakass55]

What a superb video. Thanks for putting this together, very nice production value (and extra points for not adding annoying background music like every other utube video). This helped reassure me that the incredible number for my heart rate while shoveling snow was accurate. Last week I was pushing large chunks of ice and snow with a prowler shovel some distance. Stuff is heavy and I kept a constant eye on the Apple Watch's heart rate monitor. At one point it hit an incredible 196 bpm. Wasn't sure if that was a fluke but thanks to your comparisons, I now realize it probably was accurate. This is kinda scary. Could explain why so many people have heart attacks while shoveling snow. Let this be a cautionary tale for anyone that has to deal with the white stuff. Your bpm can spike hugely without you realizing how much strain you're putting on your clunker.

[@blueckaym]

I've been using the Huawei gt2pro for about year and a half. Haven't tested its accuracy. Regarding sleep, it's ok I guess, but you shouldn't trust it much, more like a guideline, and average thru longer periods of time. I do have periods of irregularity sleeping, so there've been some edge cases. Regarding the quantity of sleep it's normally accurate, with the following exceptions: - if I'm relaxed, like watching a movie or waiting for my the eye drops to expand my pupils it often decides that I sleep :) On the other hand during the night it sometimes (not too often) decides that I've been awake in the night for no reason. I do remember when I awake at night (you know ... I don't remember when I don't ;)), because it happens sometimes. If there's a reason for discomfort (like being too cold/hot or irritated by something else) I do awake, and usually I fix the issue (if it's fixable) and continue. But sometimes the gt2pro decides I've been awake in the middle of my sleep. It's usually very short "awakenings" so I guess that I simply have been rolling in the bed (which I do regularly). - Regarding the quality. I'm not very convinced it's very accurate. New I usually have issue with the amount of deep sleep (according to the watch), which might be true - I don't know - but what I definitely noticed is that my score and some of my other issues with the quality of sleep have been mostly fixed by a previous update of the software ... apparently I sleep much better now that I know my watch has updated :D GPS tracking when outdoors is nice feature, though you actually have to track it - in the sense that if you care more about the recorded track you might want to disable the pause-activity function (or NEVER forget to resume it when you're done resting). But if you're more interested in the time-aspect (how fast you've been, what pace etc...) and not interested in the time for rests then you probably want to get the strict habit to pause activity when resting and resume it when ... well resuming activity :) It has functionality to pause/resume, but it's unreliable ... it works most of the time, but it takes very long (and different) time to detect the changes. But by far the most useful feature I've found yet is the vibration mainly for the gentle alarm - it's much, much more comfortable way to wake up rather than any other alarm I've used (incl. gentle songs/sounds with the low-to-high volume progression ON). I normally hear well when asleep, and in the past have heard my phone alarm (quiet gentle song) even over ear-plugs, but since last year I've lost by right-ear hearing and while it have the unexpected bonus to "mute" the external sounds when I sleep on my good ear :) I couldn't rely anymore on hearing my phone, not unless I put some major and loud song/sound. However the watch vibration works perfectly, and I pretty much have forgotten the panic I get when I hear my phone in my sleep :) Also before the smart-watch I relied on the sound of my phone to get notifications, and even then it has happened to missed some, because the environment was too noisy to hear it or I've been moving to much to recognize reliably its vibration. But with the smart-watch my phone is in silent mode permanently and I always see the notifications on my wrist (the ones I consider important enough to not block them of course). The screen is too small (and too much of it is spent unwisely imo) to see actual long messages directly on the watch, but at least I get who-&-where info, which gives me idea how important it is and I check it on my phone if it is.

[@bendeguzvarga816]

If someone is not an athlete and considers walking and counting steps as workout or a contribution to their health they don't need accurate data anyway. It's a rough estimate, if you had 20-25k steps a day you walked a lot which helps contributes to your health and shows u were pretty active or hiking. If you just have 5-6k a day you should probably be more active.

[@edwardledin3929]

I appreciate the insightful information. Over the past 3-4 years, I've been using a Whoop on one wrist and a Fitbit on the other. Have you ever compared the Whoop to the Fitbit, despite the fact that the Whoop doesn't track steps?

[@Alov4]

Amazing content! It's a pity you did not try a so called "sport watch", as well. It would have been interesting to see how it compares with other devices during your run...

[@noahwagner999]

Many Smartwatches today use Gps to track distance and steps more accurately, BUT they usually only do that if a workout is started first. So that should also go into consideration. Great video though!👍🏻

[@vpnconsult]

Happy with Garmin.

[@biancaboswell6501]

For me I find that it is a great starting point to a fitness journey. I found within the 4 years of having a smartwatch I’ve managed to have more motivation to workout and move my body. Even if the accuracy is there or not 100% at times I’ve gotten into the habit of a better lifestyle.

[@Ninon]

This video was super interesting... It would be dope to see you compare the gold standard heart rate monitor to the new Pixel watch! it apparently checks your bpm every second...

[@ggs4989]

I wish you'd tested a Garmin. I have the Fenix 6x pro and subjectively it seems like it's impressively accurate. However a friend with a different model said it didn't do very well for them. Maybe it's a difference in hardware, or them having thicker wrists than I. Would be nice to see a video that tests more variables like these.

[@iamdedlok]

Thanks Andrew! This was informative , great to see the affordable Mi band not doing too shabbily! I use Apple Watch Series 4 myself, and as you said, instead of depending on the absolute measurements of steps, sleep, activity I try to focus on week by week improvements based on relative numbers. However a part of me always keeps thinking about the absolute numbers 😂🙈. Question for you: How accurate do you think is the latest Apple Watch when it comes to tracking Calories ( Resting Energy, Active Energy - both during normal movement, during exercise / workouts ). And for someone interested in tracking these health metrics, does it make sense to upgrade the latest Apple series from series 4?

[@DrAndrewSteele]

Hello! Great question, and I think a complex enough one that I’ll probably try to answer it in a future video…but to mildly spoil the anticipated conclusion, I don’t think any of these devices are very good at calculating calories expended, not least because their estimates vary pretty widely! Annoyingly the gold standard against which to test them involves going to a lab and being strapped up with a mask to measure exhaled CO2, or something called ‘doubly labelled water’ where you drink some water which has had its isotopes replaced and then go back to the lab a few days later to get a measure of calories burn over that time, and these are both complex and expensive, so I’ll have to have a think about how to make that work. From what I’ve read, watches that measure heart rate are significantly better than those that don’t (though it’s quite hard to find one that doesn’t these days…). I don’t know if there are many tests of the Series 4 but my guess would be that there is an upgrade in accuracy with more recent watches: The Quantified Scientist reviews of Apple Watches seem to show a slight improvement between Series 6 and Series 8 (to the point where they’re almost perfect!): kzbin.info/www/bejne/ZobFaKSjj99sm8U but my only reservation with the Apples is how infrequently they take measurements when not exercising. I’m not entirely sure if/how long-term heart rate tracking could be useful, but I do wonder if in a few years’ time we might wish we had access to it more than every five minutes! (More on that in the next video…) That said, if you’ve got an iPhone and use the smartwatch features, I think an Apple is going to be hard to beat, and they do seem to have the most accurate sensor at the moment, so… :) Hope that helps, and many thanks for the Super Thanks!

[@iamdedlok]

@@DrAndrewSteele thanks for the detailed reply! Yea I follow The Quantified Scientist reviews too. He does a great job! Ok, I understand what you are saying. Can’t trust these watches to track our calorie spent accurately. Sigh! Cheers, keep producing great content like this!

[@DrAndrewSteele]

@@iamdedlok Thanks! And yeah alas I think we’ll need to wait for the perfect calorie-counting watch…never say never though!

[@traisjames2]

Can you also check differences in counting steps on dominate vs non-dominant hand?

[@Deathboredem]

I came over from the Collab with Steve. As a cs nerd I recognize fellow nerd right away. Instant subscribe.

[@PerryScanlon]

I've found with extreme sweating my chest strap actually stops tracking accurately but before that seems to peak briefly in sprints.

[@spiderbite7849]

Great work. I was a bit distracted in the conclusion with the no blinking and little movement. Teleprompter issue maybe

[@PasminicaL]

Great video, loved the depth you go in. Waiting for the next part! Subscribed.

[@E.Mulchi]

7:00 I had the same problem with my Garmin watch. Really frustrating making a great effort like sprinting or riding up a hill, just for the watch to not record any real increase in heart rate.

[@2810Mad]

You should have included the galaxy watch 5. Anyway you've won a new subscriber. The video quality is very high for the amount of subs you have

[@MrShakla1]

These are some good quality comparisions. The tests, visualization and conclusions are well thought. Great work :)

[@zafiralpstv8004]

Your channel is my best Discovery in this year

[@jordanwoods462]

Ever since having kids a couple years ago and I started pushing strollers and carrying babies/accessories every day, I’ve grown pretty upset at my Apple Watch for not validating how far I’ve walked. It’s become so inaccurate during my most active times that I hardly use it anymore. Thanks for the video!

[@BigKingJohn]

Thanks for the informative video. Which model treadmill are you using?

[@ragmanintx]

I have a Polar H10 chest strap, Apple watch series 6 and an Oura ring. The series 6 correlates strongly with with the Polar Re heart rate. The Oura not so much. They all use different algorithms for calorie burn. At the end of the day, I think the apple watch is the best bet when it comes to accuracy Vs convenience of wear.

[@PabloTBrave]

For me it's more about checking trends over weeks rather than accuracy on a particular day . ( Ha that's what you said at the end)

[@KimLind]

Good video. Do u update them and will compair if some is more correct?

[@abidmorshed5980]

Excellent demonstration, loved the overall quality of the content. Enjoyed very much

[@6681096]

The Apple watch also comes out on top with the quantified scientist. He has done many tests of about 50 different devices. The Apple sleep analysis was also number one and even better than whoop and aura.

[@chrisogrady28]

Would have liked to see more serious devices like a Garmin Fenix/Epix, the 3 devices tested are all for very casual use, not serious tracking

[@Scatter_Guy]

A great series idea! Great video production and info!