Backups: You're doing 'em wrong!

[@MrWachtus]

Hey that's me 😂 Thanks for mentioning gickup

[@JeffGeerling]

Thanks for making it, and for your responsiveness in getting all my nit-picky issues sorted ;)

[@joonasfi]

Hey Buddy

[@gautamkrishnar]

Great job buddy

[@MrWachtus]

@@joonasfi Hi Joonas 👋

[@MrWachtus]

@@gautamkrishnar thanks 😄

[@codemonkeyhacks3973]

That smile on your face after the nail gun - priceless!

[@DarrylAdams]

No. That was Redshirt Jeff. Normal Jeff is a totally different person.

[@iScherma]

"You can almost always do better than you are right now." This is applied to pretty much everything.

[@mtargetproduction]

"If my entire network rack got Thanos snapped... I'd be okay" Don't lie Jeff, we know you'd be sad

[@JeffGeerling]

This is true. A lot of blood, sweat, and tears goes into a nice network rack.

[@mtargetproduction]

@@JeffGeerling the way you talk about some of the pis in that rack, there is some definite emotional investment.

[@JeffGeerling]

​@@mtargetproduction They say cattle, not pets... but sometimes it's nice to give your servers a little TLC.

[@chuckcrizer]

The most important and most useful video you have ever made. Backups are absolutely vital.

[@JeffGeerling]

I've been saved by good backups a few more times than I'd like to admit!

[@chuckcrizer]

@@JeffGeerling I'm a Senior Network Engineer and to me, backups are a religious activity! Sadly, they are also the last thing management wants to spend time and money on.

[@Wordsnwood]

@@JeffGeerling Psst, Jeff, I suggest you add the "kissy lips" emoji to your blocked words list in YT studio -- it will cut all these porn comments that are popping into comments these days.

[@JeffGeerling]

@@Wordsnwood Ugh... will do.

[@JeffGeerling]

@@chuckcrizer The last thing they spend time and money on, then the first thing they'll yell at you about when poop hits the fan!

[@fairbanksFUMC]

This is so true. I have one 10TB HDD that holds everything for the videos I make. A couple months ago, the folder holding hundreds of raw files that hadn't been used yet vanished. Two days and $100 for a copy of Disk Drill later, I got it back. I need to implement a proper backup that involves more than just dragging and dropping folders when I remember to, and this video reminded me of why.

[@YeOldeTraveller]

I have a friend who says the following: You don't want a Backup Plan. You want a Restore Plan with Backup as but the first step. And if it has not been tested, you don't have a plan.

[@SeanFisher]

New video idea: How to Properly Test your Restore Plan.

You won me when you said “the cloud is just someone’s else computer.” Great video. And yeah, I have a NAS, but gotta make an offsite backup fast

[@kamikazilucas]

implying cloud servers are just 1 computer and not tons of computers backed up

[@baldpolnareff7224]

For stuff like photos that you really care about, another extra backup could be a collection of archive grade blu rays, they're not that expensive when you factor in how long they last compared to HDDs or SSDs, as long as you store them properly. Just an extra level of redundancy that I see suitable for personal family photos and things of that great importance that aren't as heavy as videos.

[@defipunk]

And a bank safe deposit box is < 100 bucks around here. Also a good place for a lot of non-digital documents in case of a fire or similar at home

[@vijfsnippervijf]

Or maybe even a paper photo book! They are also lots of fun to show to family and friends!

[@questionmark576]

Red shirt Jeff is definitely the most safety conscious agent of chaos I've come across.

[@test40323]

Excellent tips on backups, recovery and testing in current ransomware age...I would add encryption to sensitive data and don't forget backing up your private keys!

[@JeffGeerling]

Good points-in my repo, I do have a note to explain encryption a bit more. I also need to work on a couple parts of my backup plan where things _aren't_ yet encrypted...

[@autohmae]

I've been using restic. What do you use ?

[@MfBPhone]

One extra thing with ransomware, I would advise to try and have one of the 3 copies also be a offline backup (or another way protected against unwanted encryption). You don't want to sync the encryption to all your copies.

[@charleshines1553]

To protect from ransomware, keep one copy almost never connected to the PC in any way. Ransomware can't hurt it if it can't find it. If you find yourself to be the victim of ransomware, make sure other computers are not affected also. I would also keep a USB stick some place that I can boot from and install a clean copy of Windows. That would also give you a chance to erase all partitions from a drive (it doesn't wipe them but it should kill the ransomware).

[@autohmae]

@@charleshines1553 yeah, best to have a 'write-only backup' service somewhere which allows uploading, but only deletes old backups with a specify number of days.

[@IsmaelLa]

OMG I need one of those "The Cloud is someone else's computer" shirts!😀👋

[@JeffGeerling]

I can't even remember where I got mine! ☁️

[@lescarneiro]

You really nailed your backup!

[@JeffGeerling]

I understood that reference!

[@bummers]

Back in R&D in the 90s, one of the director would bring a physical tape backup of the source code offsite on top of the onsite tape backups. Then one day the senior software engineers responsible for maintaining scripts for the backups realised that the backups were not working. They never quite test the restoration part.

[@BillinSD]

BackBlaze will mail you drives with all your data if an emergency happens. This video is really good at bringing up the reality of loss to people and how to manage it without being overwhelmed.

[@midjetville]

You missed the most important part of this: testing your backups! If you don't try to restore from your backups (e.g. the stuff in Glacier) until after you have a disaster, you may be in for serious pain when you discover the backups weren't working like you thought they were. Another issue is versioning data - what happens when your data gets cryptolocked, and you automated systems back up the cryptolocked files on top of your good backups? ZFS is great for this... :)

[@JeffGeerling]

I mentioned the testing-and how I don't do it right now, heh ;) In terms of cryptolockers, a good solution is an offline backup-which I handily get for at least some of my data via AWS Glacier, since it puts the files in cold storage after a day or two.

[@danielsmullen3223]

@@JeffGeerling Testing is critical! See my other comment -- especially with "unlimited" cloud storage providers, sometimes the time you have to recover your backups before the provider's retention policy kicks in and deletes the "old" backups isn't enough time for you to actually recover everything. You must be able to test the backup and recovery strategy so that a back of the envelope calculation can be made about whether it's actually feasible to get back up and running again (let alone within the time frame needed to keep your affairs in order).

[@deusexaethera]

No, the most important part is not testing the backups, but just having the backups at all. If you have multiple backups performed using different software then the need to test each of them is diminished.

[@perwestermark8920]

Even backup testing can fail. I once relied on Windows Backup for some data. I did test I could restore files. No issue there. I did not test if I could restore the full disk. Oops. When the ssd failed, I learned the hard way that Microsoft forgot to keep track of deleted files. So it restored the union of all previous backups. When it had multiple copies of the same file, I got the newest. But how well do program version 3.16 like to find files from version 2.9, 2.7, 2.2, 1.14 - files that the installer had removed when updating? It was pure chaos and a totally unusable backup.

[@ThylineTheGay]

@@JeffGeerling red shirt Jeff is always an option

[@GRBtutorials]

On AWS Glacier restores, it’s worth noting they’re expensive mainly because of the outrageous transfer fees, as in $90/TB. In your specific case, it’d cost over $500 to restore everything if you ever need to! However, it’s still much cheaper than other cloud services for large amounts of data, if you consider the fact you most likely won’t need to do that frequently, as you mentioned. In fact, with a good local backup system, you might never need to! On the other hand, you might be able to roll your own off-site NAS for less money if you have such a location, depending on the amount of data you have, especially if you use cheap low-power hardware (like an SBC, the Odroid HC4 and Rock Pi SATA look like good options), and you only power it for a few hours a day.

[@timt7940]

Another great video Jeff. I find myself mostly watching your Raspberry Pi videos but it's interesting to see how others approach backups. One approach is not specifically better than another as multiple parameters need to be considered (e.g. cost vs. benefit, complexity, maintainability, Recovery Point Objective, Recovery Time Objective, and many others) but, as you stated, it is most important that you actually have a backup and a strategy to do so. At my work we use a backup strategy consisting of 4 levels with each succeeding level more distant from the working copy. L0 - Local device copy and backups. L1 - SAN, NAS, or DAS backup. These also serve as a data aggregator. L2 - Rotating backup of L1 (e.g. external HD/SSD/Tapes). One set is ALWAYS offline. L3 - Offsite or Cloud backup of L1 and/or L2. Other non-backup technologies to consider are RAID, snapshots, and network distributed file systems (for Linux GlusterFS and CethFS come to mind) for additional redundancy. If your data is your source of income, strongly consider having a Disaster Recovery Plan and Business Continuity Plan in place.

[@JamesBos]

Re: router backups. Having recently restored my pfSense box from backup after catastrophic failure, I can attest that their backup configuration mechanism is pretty amazing.

[@chrisb.2609]

Im pretty happy with my solution. - PC's using Active Backup for Bussiness -> Synology NAS - Servers using Proxmox Backup Server -> Synology NAS - Synology NAS -> external HDD - Synology NAS -> Synology C2 Cloud So I always got 3 copies

[@JeffGeerling]

NASes can be extremely useful. Nice to have them as a go-between for important data too.

[@strandvaskeren]

Excellent video. An added bonus of two local nas boxes that sync is, that you don't have to waste disk space on raid. If a drive dies, no worries, it's still running on the other box. A quick dns hostname adjustment and your family are back to watching movies on the secondary nas while you mess around with the primary one. If lightning takes out both nas boxes at the same time, it's time to restore from the cloud.

[@earthling_parth]

My first experience with data loss was when my Google Nexus 5 got stolen and I lost a year's worth of photos and videos of my friends and family. Currently I just backup all the photos to Google photos and important documents to Google Drive but thanks for reminding me Jeff, I really need to have a NAS or secondary backup other than Google's cloud for all my important data.

[@JeffGeerling]

`rclone` also works with Google Drive too, so just getting a NAS of some form and having a script use rclone to back up the entire drive should get you covered!

[@earthling_parth]

@@JeffGeerling Thanks Jeff. Any beginner level NAS suggestions/recommendations? I have two Raspberry Pis and some SSDs to play with. Maybe you can create a video on it. I'm sure me and my friends would appreciate it 😬

[@JeffGeerling]

@@earthling_parth My series on ASUSTOR vs Pi NAS from earlier this year should help a little. I may do another video on a simple and cheaper ASUSTOR NAS soon, though!

[@earthling_parth]

@@JeffGeerling I didn't know about that, will definitely check it out. Highly appreciate you sir. Also, really enjoying your book on Ansible. Thank you for that too ♥️

[@johnwinters4201]

I missed the bit where you explained how your backup versioning worked. As in - when you discover that you deleted a crucial file a fortnight ago and now need it back. One other important point is that backups must be automatic. Backups which involve plugging a removable drive as and when you remember will very quickly cease to happen.

[@papersnowman]

Thus, so much! Good backups should be more than just a synchronized copy of data

[@_Miner]

Not to forget about encryption, keys, passwords etc.. documenting what software and tools you used to backup that data, having backups of those too, and locations. Documentation can be just as critical as the data in some circumstances. So so much goes into a full backup and DR solution well at least for enterprise. EDIT: Immutable / offline backups are a must now for cryptolockers, ransomeware etc.. who can actively seek out to destroy all backups, so having them offsite but online (even if in the cloud) might not be enough.

[@Rem1xDave]

Time Machine is handling most of his versioning needs.

[@johndoughto]

very well articulated- in a short concise way!!! and for most, just going through this and thinking about WHAT you have and WHERE it's at - will be a great first step!

[@JeffGeerling]

Sometimes you see something like this and you're just like "oh... I forgot I don't even have a backup of XYZ at all!" And that's a good reason I finally decided to post this video before I felt like I had a 'perfect' system in place :)

[@applesushi]

I need to send this video to all my friends. My backup plan right now is: 1a) Data on my Mac, 1b) Data on NAS, 2a) Mac Data on a TimeMachine disk (separate from the NAS), 2b) NAS backs up internally to Dropbox, 3a) Mac backs up to Backblaze, 3b) NAS also backs up to Backblaze by virtue of it being an iSCSI drive on my Mac. I also pay for iCloud, so Images, app backups, etc. exist in there as well. This means I pay for three cloud storage providers, but when my good friends had their house broken into and their Mac AND TimeMachine drive stolen, I went a little overkill.

[@CreateTeen]

I can couch to this, I lost a year's worth of data this week, BUT I was able to keep most of it, because I had an external archive drive disconnected from my computer, and in the cloud. Was a pain to transfer and lose the data not backed, but at least I still had my import stuff, like photos and such

[@JeffGeerling]

Better a pain and still having the data than losing the data, for sure!

[@CocoaEm]

@@JeffGeerling accidentally formatted the wrong drive. All the data survived though. Likely it was the largely unimportant stuff.

[@deusexaethera]

vouch, not couch.

[@CreateTeen]

@@deusexaethera yes but I'm still sitting here, working or recovering data (remaking files and redownloading)

[@darrenvail8726]

@@CreateTeen I'll couch for that!

[@kevinmcaleer28]

The nail through the hard drive WAS cool!

[@deeplightstudio]

This helped me figure out the last piece to my data recovery puzzle. I needed offsite disaster recovery and rclone to aws glacier seems like it'll fit the bill. Thanks, great video!

[@likilike501]

True. I'm building my own NAS right now and i plan to store HUGE media library and this is exactly something i would need. I did not had an idea that sollution like this exists. Backing up relativelly small stuff like documents or pictures is a piece of cake and cheap(of course it depends what you do) but backing up multiple TB of data is kinda pain in the ass and definitelly not cheap.

[@DavidJones-pi8rl]

I first watched this video the day after it was published. I struck me, that I had spent the last week documenting the process and drafting a project plan for my employer's clients Business Continuity Plans (BCP) and Disaster Recovery Plans (DRP). Not once before watching this video, had I been prompted to think whether I was adequate to meet my needs (no it doesn't) or even check if my backup software, which is the Open-Source product called URBackup, was working (and no it hadn't worked for over 2 months. Given I already fit into the category of someone who has lost data, you'd think I have it all sorted. But I didn't - Thanks for wakeup call Jeff!

[@larrywilliams8010]

Thanks for this. Backups have saved me at countless times over the last 4 decades, even from myself.

[@apcyberax]

i have the advantage of being in IT and having a fast connection. My NAS 30TB is backed up. Mirrored every 6 hours using snapshot to a remote server in my office (offsite) and that office is then backed up to Crashplan. i have over 15TB of video and photos as well. I also have a 10Gbe internal network and nas so for my everyday documents I have a iSCSI mapped to my PC so the data is never on the PC. Its always directly on the nas so its always backed up even if the PC is off at the backup times

[@zakpappnase]

When you test your recovery plan, make sure you can do it without your main 2fa solution because that may well have been in the house when it burned down. (I have all my 2fa stuff backed up to a second phone which I leave at my dads house).

[@MaxPrehl]

Oh my GOD. I'm DYING over the stock footage of the airplane guy looking out the window thinking, "did i remember to turn off the stove?"

[@whette_fahrtz]

CrashPlan just backs up to AWS anyways, but it does some nice encryption and de-dupe, all in a nice little GUI. We use it at work and it's basically set and forget. Had it save my ass multiple times.

[@JeffGeerling]

And I think I said "crashplane", oops!

[@markgarrett8963]

backups don’t exist until a successful restore has happened. also multi versions or sha validation needs to happen lest over time you may be replacing a good version of a file with crap. good video

[@TechnologyGeek862]

I mainly use rclone and my computers files but never thought to automate my nas photo backups from my computer as well. Shall make it now :D Thanks for the reminder. Been using rclone for many different style of backups (mainly video) and it has been awesome so far. Rclone mount has a little progress to be made perfect but non the less it still works as it should too :D

[@ferd352]

Personally, I backup all my data with Duplicati 2 on a mix of Windows, Fedora and RPi machines, once to a local NAS, a second backup to Wasabi (which is tons better than AWS). I also have adhoc bare metal backups using Veritas System Recovery of the OS drives of some machines just in case.

[@frauseo]

The smile at the END of RedShirtJeff.... Priceless!!!

[@MiniLuv-1984]

I've been preaching a very similiar backup and recovery approach for 20 years - I built a business on it...no one listens - until its too late and then, well its too late.

[@KameraShy]

Backups are like computers - you can never have too many. Backups have saved me a number of times.

[@dannyarnold9823]

If it all goes south I'll ask the NSA. they probably have had all my data for years.

[@doq]

Jeff backup strategy: *9 minute video* My backup strategy: "yolo"

[@kcvv]

Great Video Jeff. Can you also make a video about how you keep your different data organized? My different types of data is either all over the place or in on big dump of a folder!

[@colinwatt9387]

My photos are backed up on Cloud, on an external drive and on 2 Pcs. Same with my project folder. Fungible data like books, comics and audiobooks are stored on my laptop and duplicated on the external backup; It would be annoying to lose that stuff but not heartbreaking. I had a hard drive fail about 10 years ago and it had my friends photos as well as mine - the only copies. I put the drive in a triple-sealed bag and then into the freezer overnight. Astoundingly it worked and I learned a valuable lesson.

[@DigitalJedi]

I'm pretty happy with my current setup. PC has redundant drives, so I can lose any drive and still have everything there. My PC backs up to my One Drive daily and to my NAS weekly. The NAS also has redundant drives, which has actually saved my ass once already when a WD NAS drive I bought second hand crapped out last month.

[@ads1035]

Ya know what I really, really wish I had? Tape backups. Most of my backup wants (not necessarily needs) would be archival storage... If I leave a hard drive in a safe for a decade, how do I know that drive will spin up in a decade?

[@kiptonm]

Use a SSD if that is your concern. Hard Drives are pretty good. If the hard drive is not old (like running constantly for 5 years), it will probably start up in 10 years. Tapes are not as reliable as hard drives. Hard Drives are sealed. Tapes are not. Tape is fragile. It can break. It can get twisted. it can get damaged easily. A hard drive is not (compared to a tape). Hard Drives are cheap. And tapes can not hold very much compared to a hard drive. You could back up to a DVD which holds a lot more than a tape, but that is still a lot of DVDs. 7 GB per DVD, versus 18 or 20 TB for a single disk. When I was starting back in the 1970's our computer center got a 14" removable disk. It held an amazing 5 GB of data! But the manager still backed it up to paper cards, because he can physically read the cards, he cannot physically read the hard drive. I do not want to go back to those days.

[@Slada1]

@@kiptonm you need to power on SSDsrl regularly to prevent bit rot

[@barkovurur3043]

I followed Jeff's advice, setup a backup system and tested it by burning down my house. It worked! Thanks, Jeff!

[@JeffGeerling]

Umm... 😱

[@iamalittler]

I’m lucky that way, I’m a writer, so my backups don’t take much room on a 20GB Dropbox, so I have it backed up there, my Mac, my PC (intermittently, when it turns on), my NAS, Time Machine, but you did remind me I need to set up a backup on my (Orange)Pi. Mostly for the hell of it. (And that isn’t counting the 30 days history Dropbox has, the few months of TM, and the copies downloaded onto my iPad whenever I sync Scrivener.)

[@HepauDK]

I have a NAS that holds pretty much everything besides my OS and installed games. It is backed up (1-way sync) to a second in-house NAS once a week. Every 3 months or so, the primary NAS is synced with a 2nd. backup NAS that I keep stored at my little brothers house. Not the perfect solution, but at least I sleep well at night, knowing that the vast majority of my data is safe.

[@rpavlik1]

I know it's synchronization, not backup, but I've felt a lot better about my day to day family files since I started keeping most of them (that aren't on Google drive) in a folder set up with SyncThing. Open source, I get encrypted, automatic folder sync across machines and across platforms. (Even has a public hosted peer discovery and Nat traversal server, though you can disable it or run your own if you prefer) It's almost made me not really need a file server... I should probably send a computer to my parents house set up with a special SyncThing folder configured more as a backup: there's a lot of ways to configure it, but I've been basically using it like Dropbox. Highly recommended.

[@alexanderhall-hognason7576]

As far as backing up network gear - consider using pfsense software/hardware. Their configurations are easily exportable through their GUI (you can optionally encrypt these too, which might be worth mentioning for remote backups of data).

[@SaltyNotSweat]

In 2020, I lost all my data photos, world backups, phone backups from all the phones I had, current video projects(my archive videos were safe), and much more. How I lost it? Well, I used the same local drive as the destination for all my drives. So after I backed up my user folder, and starting backing up my video archive drive. It overwrote my user folder, because of a bug in the backup software. After the backups were done, I looked and saw my user folder was there and wiped my OS. Went to restore, and found the folder was empty. Lots of pain.

[@JeffGeerling]

F

[@Heirl00m]

Do you mind sharing the name of the backup software in question?

[@SaltyNotSweat]

@@Heirl00m I think it was Back in Time, but it is my fault for not using another destination(didn't have money tho). Ideally, it's better to use a different program for cloud than locally. That way, if one program has a bug, the other hopefully does not.

[@nwhaynes]

I'd be interested to see a video about your backup pi setup.

[@Azteca_X]

I had two drive failures (one physical, one filesystem) in a week after 5+ year of no issues. Definitely had me scared straight. Still replacing a handful of things that were lost but nothing irreparable.

[@bobwong8268]

👍👍👍👍👍Dear Jeff, THANK YOU VERY MUCH for sharing your backup system & plan. Could you also come up with a Jeff Data Restore system, please? You reminded me of my lectuer: 1. Your backups is only as good as your ability to "restore". 2. If you can't restore, you don't have a backup. So, i don't have a good backup plan until you shared yours. Awaiting your "restore" plan & system. Once again, thanks! Perhaps bullet-proofed or even "Red-shirt-Jeff"-proofed; only you can do that ;)!

[@MikeSwanberg]

Nice video... I have to add that backup is WAY more than just this. Versioning is a large part of it (which was sort-of mentioned in the code part). And accidental deletion is also a big thing to worry about. I have had video files that I don't check on very often (maybe I watch that movie every 5 years or so) that were suddenly (over the last 5 years) gone. No idea where to or why... but such things also need to be covered. It is also difficult to revamp a backup strategy because you could lose a lot of things. Think about that movie that disappeared. Well it was in that backup I used to do, but I changed it to a new method and deleted the old backups. Whoops. Let us also not forget about system backups, for getting a system back up and running when it crashes seemingly for good. All in all, backing up data and systems is a VERY deep topic. Entire books have been written about it and they even only scratch the surface. They also get obsolete quite quickly.

[@awesomearizona-dino]

Thanks Jeff, I recently learned this lesson the HARD WAY. fortunately, i was able to restore all my docs and files. had to rebuild both W7 and W10 computers.

[@jenniferprime]

I'm doing backups?

[@JBoy340a]

Another really useful and timely video. We just had our Synology NAS say it could not access the data on the 24 TB of drives. Luckily we got it working, and I did have a local hard drive backup. But, I now need to move the offsite solution in case of the apocalypse up the priority list. I was thinking about using Synology's backup solution, but it is kind of expensive. So I am leaning toward Glacier and your video is very timely. Hopefully, I never need to restore, but if the house burns down we are in trouble with the current setup

[@timsievers2067]

Glad your house didn't actually burn down! I thought red shirt Jeff got ahold of the rework station again.

[@JeffGeerling]

Haha, he's just playing around with the mail gun right now. Hopefully he doesn't figure out any other uses for the gunpowder...

[@arshamskrenes]

I have two old workstation with 4 drives in ZFS RAIDZ each. One is at my parent’s home in another province. Every night they send snapshots to each other.

[@bikerchrisukk]

Nice short vid, I changed my backup method after a backed up file was corrupted - no media or O/S told me there was a problem. So TrueNAS it is, with a separate TrueNAS Snapshot machine and a 3rd server in a separate building but on the same site. It's something I've slowly built and learned over the last few years. I mainly do this because many of the files are business related and have worth to them (Building/Architectural Design)...using the same system for my personal files is just an added bonus 🙂

[@popcorny007]

Excellent advice, but complete cloud backups are only realistic for the lucky few who have decent upload speeds.

[@JeffGeerling]

Yes and no-I'm stuck on a 35 mbps connection, which is anemic compared to some (but lightspeed compared to others). In any case, 'slow and steady wins the race'. You'd be surprised what 5, 10, or 20 mbps could do over a long period of time (usually just days) for that initial backup.

[@popcorny007]

@@JeffGeerling I'm considering shipping a hard drive to a provider, instead of uploading ~7TB over 200+ days, lol

[@JeffGeerling]

@@popcorny007 Good use case for AWS Snowball! aws.amazon.com/snowball/ :)

[@hammer86_]

@@popcorny007 Sneakernet has always been a good option. Never underestimate the bandwidth of a station wagon full of hard disks.

[@TecSanento]

great to hear that you did not test the deasaster recovery eigther ;)

[@JeffGeerling]

Heh, you test it when you need it, right? Trial by fire!

[@sayvilletech9135]

I agree 100% with your take on backups. Years ago I backed up servers using RAID 1 in the server itself. Then I went to external hard drive with offsite storage (what a pain). Then I had several inhouse NAS devices, with backups of backups. Finally the administration decided they wanted cloud backups, and we ended up with DATTO. I still kept my NAS devices, very handy for a quick restore when someone did something silly. And some DVD's for a few small directories that didn't change very often. You can't have too many backups. It also helps to do test restores occasionally. We did not fall under Sarbanes Oxley, I think those techs had to prove they could restore an entire network, hardware and software, in a couple of days, I might be wrong on that.

[@scotts1138]

I like the use of the Thanos snap, nice touch

[@minecraftchest1]

I can't believe Jeff let Red Shirt Jeff put a nail through is hard drive. I hope it comes out of Red Shirt's budget.

[@waynewilliamson4212]

thanks for the info about glacier, did not know it was that cheap. and yes, I sync to a nas and have that rsync to another both in raid 5 so I can lose one drive and keep working.

[@AnsgarWollnik]

Jeff, that’s a great video. Only thing I always wonder about is, what happens if it is you that got seriously ill. Will your wife be able to access the data in case of need? I believe, in most cases that is going to be tough for our loved ones that do not have technical skills like we do. I do not have an answer to that yet. In general, I guess we need to try to keep things simple. What’s your take?

[@JeffGeerling]

In my will (joint one with my wife), there's a section covering website / data continuity, and there is a short list of tech friends and acquaintances who can be contacted and contracted to help manage everything. Since all passwords and accounts are managed through a password manager, as long as someone gets access to that, they should be able to pick up where I someday will leave off :)

[@Dad-mode-locked-on]

Similar setup here. Primary and secondary nas are both asustor. Third is google cloud. Fourth is external drives only connected for backup purposes. I have been doing backup and disaster recovery since 2002 for a living.

[@LordHonkInc]

Once a year, I burn a DVD with all my important docs (mostly financial, medical and education records) and leave it at my parents house. On the one hand, it's an offsite backup (not as convenient and accessible as some alternatives, admittedly), and on the other in the case of an emergency they have access to it in case, say, I get into an accident and they have to provide somebody with my medical history. It's something I learned from my mom a long while back, she does it physically in what she calls a "drop-dead book". That's just one of my storage options, but this one I think has a very important premise: Some data you don't want to keep just to yourself; if you encrypt and lock up all your stuff, you make it impossible for anybody else to get at that data, and while in many cases that's a commendable thing, you might in fact not what that in _all_ cases.

[@MarcoGPUtuber]

0:44 That's why I use my mum's house as my offsite backup. It's still my NAS/Computer and nobody's gonna touch it cause they don't know what it is.

[@JeffGeerling]

"Don't touch that thing, it makes a racket and has a lot of blinking lights. It has to be important!

[@MarcoGPUtuber]

@@JeffGeerling It's more of Marco's computer stuff.

[@Jakob-o8f]

Trick 17: Start to not care, billions of years have gone by without a single backup. And there are trillions of planets in this universe where they dont matter, too.

[@adyanth]

Hey Jeff, awesome content! For the routers, most of them have APIs that can download the backup files (like openWRT) Can you make one or provide details on how you backup your VMs, and more importantly how you restore them? something like an ubuntu server for example.

[@gorinator]

I can't focus on this video because I see that you're covering your air vent again! If you won't ditch it, could you prop up the top or bottom edge of the cover? No reason the visual cover should be air tight.

[@JeffGeerling]

I'm working on a much better solution, but I now have a CO2 alarm set up so I'm reminded to pop it off a lot quicker now :)

[@gorinator]

@@JeffGeerling Yay! You are the singular maintainer of Jeff, a resource that is important to many of us. I appreciate your prompt response to my issue submission. My predictions about the "much better solution": - Googly eyes to make vent look like angry robot - SCUBA gear - New house - Move office to living room for Winter as part of an attempt to heat the home with nothing but Raspberry Pi's - A 3D printed thingy

[@JeffGeerling]

@@gorinator Ooh... "Needs more googly eyes" just like Device Orchestra!

[@bjoernalbers]

Hey Jeff, do you count the primary data on your Mac mini as one copy or is there an additional local backup of those files? PS: Great Video! Björn

[@JeffGeerling]

I count the mac as one copy.

[@IsmaelLa]

Man that ending smile! Made my day!

[@Andrea-Zerg]

back when I used to work for AMEX banks, I recalled the back was 135 to tape drive A and 246 to tape drive B, and sundays get back up to a off site safe deposit box. right now I have 3 NAS, 2 of which are at home and the 3rd NAS not only is it off site, but also off country. i dont do 246 but i do the 135 and the 7 goes to off country NAS.

I have two NAS (Raid 1) syncing with each other over the internet; one at the office and another one at home. Photos are also in iCloud 😊

[@computersales]

I'm slowly getting there on backups. The goal is to have 3 local copies and one offsite of all my files. I'm still torn about cloud backups, but I am considering it.

[@peterkambasis]

I get a little backed up when i think about my backups. Great video man!

[@fattomandeibu]

Funny you post this. The Pi for my NAS arrived in the mail this morning. I had been just going around every device is the house lugging around an 8TB external 3.5" HD to back everyone's stuff up manually, but I'm gonna combine this with the Pi and OpenMediaVault, then automated monthly disk imaging for all devices that support it, then do a manual backup of all the other stuff. For offsite, once every six months, I currently back up my C:(1TB SSD OS, programs and documents) and F:(2TB HDD media store), leaving out my other "gaming" drives I can recover from Steam, and use spanned images over multiple optical discs, old school style. Difference being I use Blurays for capacity reasons now, but I still end up with an entire spindle and an entire day used near enough.

[@Alpha8713]

What happens if you lose one Blu Ray in a set, though? Does it make the entire backup unrecoverable?

[@fattomandeibu]

@@Alpha8713 Yes, but it is literally the cheapest option by far that isn't something as unreliable over time as say, a USB memory card that can die if stored too long. Something I could leave for months(sometimes even years) and know it'll still be readable.

[@bits2646]

NAS + cloud + external (cold) storage is what I'm using for over a decade besides a "local copy" P.S.: My external disks are not permanently connected and don't rest on top of computer case :)

[@Patrick_McFadin]

Old Ops guy here. You get 1 point for a backup, 99 points for a restore. In 30 years of doing this, I've never had to restore a backup from a hardware failure. It's always been some sort of human error "Oops I rm -rf ed everything!" Make sure you know how to get back to where you were before hooman caused all that trouble and make sure your backups are staggered so you have a window to restore data that was mistakenly deleted.

[@classicrockonly]

This reminded me I need to rotate my offsite drive this weekend lol

[@anthonymudge9768]

If you want to encrypt your online backups, where do you store the encryption keys so they aren't lost in a disaster?

[@anonvoila4883]

That's crazy I just recently brought the extender for my asustor and was thinking about my 3TB Plex Library and what I would do if something happened. This was vid was heaven-sent.

[@kozmaz87]

I also learnt a valuable but essential lesson in backing up. Learn to delete. It works wonders :D

[@revstar3165]

You are so right…i keep backups on my macbook , my pc and my nas, all on different hard drives…

[@alvarovelasco29]

I'm currently using Syncthing to synchronize my files between multiple devices, and Mega to backup my most important files to the cloud, my current problem is having an off-site backup. 😅 Great video and very helpful!

[@alexlee949]

Great Content! As always. Look forward to the next video!

[@olivier2553]

The WiFi or router config are just a file somewhere on a computer (provided you export the config after any modification). It is up to you to make that "somewhere on a computer" part of your automatic backup.

[@bpbrainiak]

thanks for the hint... I want to do my backups better and this is a good start

[@SmokeytheBeer]

Gickup, that's awesome! I'm going to start using that for my repos.

[@redace001]

Oh that grin on red-shirt Jeff at the end of the vid! 🤣😂😁. We've all wanted to do something like that to some drive that up -n- dies, ruining your day.😈

[@enissay9950]

Thanks, this topic always kept me awake at night but I never managed to fix the whole chain... I will certainly steal yours and maybe even improve on it... Thanks again ♥

[@NiyaKouya]

When I saw the video on recommended my first thought was: "He'll probably talk about the 3-2-1 rule" xD And about backing up the config of your switch (hooray Mikrotik!), that should be fairly simple. Exporting the config to a file is a single command (export file=filename), and then you can either actively push the file out via FTP, or pull it from some other device via FTP/SCP etc. The config exports (in .rsc format) are simple text files with the console commands to replicate your current settings. You could also create dedicated backup files, but AFAIK those are in some binary format and can/should only be restored to the same device, so IMHO the exports are the better choice.

[@JeffGeerling]

ideally that would work, but it seems SwitchOS doesn't allow that type of use.

[@NiyaKouya]

@@JeffGeerling Ya, SwOS is IMHO cut down in features a bit too much. But hey, you should be able to easily switch (back) to RouterOS, though I'm not entirely sure if any config you already did would carry over. But ROS is definitely the better choice even for their switches.

[@dw.imaging]

I'm going to have to find a way to use the term "GeerliNet" at work! LOL

[@EbayDK2K]

Nice video, but you missed the malware factor. If your backups are always online, how do you retain versions of files or make sure corrupt files dont get spread across your backups? Enterprise solutions retain a number of versions per file. Sure this approch uses way more storage, but helps you against these types of scenarios.

[@tpobrienjr]

Reminds me of the guy in my office who was told to get rid of all the junk on his desk. He boxed it up and sent it to himself through company mail. Problem "solved".