I lost my ASTROPHOTOS in the past 😓 THIS could have prevented it - Data hygiene best practices

[@CuivTheLazyGeek]

My Patreon: www.patreon.com/cuivlazygeek My Merch Store: cuiv.myspreadshop.com/ Important note: this is just ONE step towards good data management! The ideal scenario would be to have physical backups in TWO different places AND a cloud backup - and a RAID setup is cherry on top to ensure the integrity of a data source! I personally use Google Drive to backup my favortie masters to the cloud. Still, I've used this drive a LOT after this video, and it was great to have access to all my astrophoto data even when I was in France :) Synology DS923+ : tinyurl.com/jmydny4h or amzn.to/4d7R6sJ (Ideal for RAID5, RAID6, RAID10) Synology DS223j: tinyurl.com/hzmafk73 or amzn.to/3MODBU2 (ideal for RAID1) Sample 4TB Hard Drive: amzn.to/3MOfYuW Sample 8TB Hard Drive: amzn.to/3XqB87k Synology BeeStation 4TB: bee.synology.com/ (no data redundancy/RAID) Amazon affiliate: amzn.to/49XTx01 Agena affiliate: bit.ly/3Om0hNG High Point Scientific affiliate: bit.ly/3lReu8R First Light Optics affiliate: tinyurl.com/yxd2jkr2 All-Star Telescope affiliate: bit.ly/3SCgVbV Astroshop eu Affiliate: tinyurl.com/2vafkax8

[@AstroIsland]

Synology is indeed one of the best NAS drive systems on the market these days. I use them both at work and at home. I am so jealous right now that you got the dream NAS I have been eyeing for so long. LOL. Enjoy Cuiv. You will love it!

[@briantheprimateastro]

The timing of this video is impeccable... quite literally the project I am getting ready to start!

[@chippymm22]

It was almost perfect for me... I had just bought a Synology NAS for astro image backups a couple of weeks before this video!

[@CuivTheLazyGeek]

Glad to hear it! Cheers!

[@revolting887]

Love the topic @Cuiv, when I started this hobby I already had a home built TrueNAS (formerly FreeNAS - open source) server out of old PC spare parts and have slowly upgraded it over time too. Plenty of modules (plex, fileshare, automatic backups/remote-sync, MFA/OTP, remote access, etc). The Synology products sure have a come a long way since I last had to support one. Recommendation for you (and those reading/watching this), if you are serious about keeping this data safe long term, backing up to an offsite service (for example) AWS S3 Glacier or Deep Archive bucket (inexpensive, very durable cloud storage). There are modules for your Synology that simply need AWS account credentials and will manage syncing your Synology data to an S3 bucket so that if something happens to the drive enclosure (heaven forbid, fire, theft, power surge, etc) you still can pull your astrophography photos and other backups from your S3 bucket to a new replacement unit for minimal cost.

[@CuivTheLazyGeek]

I'll look into those, thank you so much!!

[@d1rksm33ts]

Hi Cuiv (et al.), when using a RAID system, it’s a good idea to keep spare disks stored in a safe place. This is especially important when replacement disks might no longer be available after a year or two. If a disk fails in RAID5, your system becomes vulnerable until you replace the failed disk. Therefore, it’s crucial to have spares on hand at all times. Personally, I use RAID6 to buy extra time for replacing failed disks :)

[@CuivTheLazyGeek]

I'll think about reformating to RAID6, or even RAID10!

[@shubinternet]

I have a Synology NAS that I use to backup all my data. My friend and former co-worker Curtis W. Preston (author of the O'Reilly book "Backup and Restore") has said for about thirty years now that you need at least three copies of your data. One is the primary data you're actually using, one is a backup stored locally, and the third is a backup stored remotely somewhere that you trust. This is the 3-2-1 model. So, I hope you've got a good remote backup solution that you're also working on!

[@shubinternet]

In the case of a four or five bay Synology NAS, you could easily fill those with 16TB drives, so that you locally have something like 48-64TB of storage. But RAID is not a backup solution. You’d need to use something like Backblaze, Amazon S3, or something else like that for your remote backups.

[@CuivTheLazyGeek]

Thanks so much for this feedback! Yep, I have a second local backup, and then a cloud backup of the most important stuff!

[@DanielSchweinert]

@@CuivTheLazyGeek you can also use the Hyperbackup app within Synology and connect a big external drive via a USB Dockingstation. It will do automatic backups too.

[@ColeDedhand]

I designed NAS back in 2000. Our max storage at that time was 2GB for a 4 drive system, which cost $3,500 ($6,500 in 2024 dollars).

[@CuivTheLazyGeek]

Wow! Storage has gotten cheap!

[@AsKyOdA]

What is the life expectancy of each drive? I thought these were few years too.. with $spent per year this is quite steep ! Isn’t online could storage cheaper ?

[@snowflakeete]

After working many years with lots of big images and data managment, I can share my experiences. Simple keep the raw files (single subs) on 2 or 3 external hard drives. Buy them with discounts (prime day or at the store) and store 2/3 copies of your raw images. On internal hard drive keep the final or processed images which can be cleaned from time to time. In the worse case of a local damage , you can easily go back to the subs and process them again. I have 20 years old external hard drives and they work fine. Replace them after 10 years if you want. This is a very cheap and straight forward solution.

[@CuivTheLazyGeek]

That makes sense as long as you remember to keep all your drives in sync :) thanks for the feedback!

[@mykol68]

Hi Cuiv, I built my self a server using Unraid about 4 years ago. The beauty of that is I can add drives as I get them, usually purples off Ebay or Shuking them from WD desktop drives, rip it apart and take the drive out of the casing(used to be cheaper to do that way). Currently have 20Tb total with only 204Gb astro data. This is just the raw captures as I then work them in Pixinsight on my PC , another 900Gb with blinked, Calib, Coscor, Debayer, Measured, Aligned, then the intergrations Ive currently settled on a file setup for raw data from N.i.n.a. of Z:\Astronomy ew\Object\Telescope\Date\time-exposure-temp-camera-filter Its all a growing arrangement lol

[@EricTheCat]

I ordered a Synology NAS several months ago and have been very happy with it. I have many TBs data. I periodically back it up to an external USB hard drive which I have 2 of so I can alternate to keep the previous backup. Now I need larger drives to back up to. So much data and it is always growing. Especially this year with eclipses, aurora, comets, meteors and deep space.

[@egar4767]

Im running a Netgear RAID 5 with 16 terabyte drives. I bought a lifetime license of Plex on a media server. I use it for all my data and also record over-the air channels. I’ve had it for about 8 years. Three drives have failed, easy to swap them out. Love it. The only thing is that every once in a while Plex/Netgear takes away a feature. I think Synology is better. Great vid. Thanks for sharing.

[@ktydeck]

I was watching another video on NAS drive recommendations. He recommended to always get the 4 bay and if you can't afford to get 4 drives then you can always just use 2 drives and it will work fine. And you can always add more drives in the future. :)

[@40Sec]

Really depends on the user. I run my NAS in my bedroom, and I don't need the additional ambient noise of two additional drives. I think a lot of folks who aren't hoarding movies and TV show underestimate just how much space something like an 8TB drive has. I AM a movie and video hoarder (although in 1080p), and also use my NAS for PC, phone, and Steam Deck backups, and four years in a mirrored 8TB setup is doing me just fine.

[@margaretanddave]

I just went through the same thing a week ago, moving my astro data from a cheap 2TB USB drive over to a new 4x4GB QNAP NAS. This was a good review and and I agree that these NAS devices are really cool. I just set up a Jellyfin DVR/streamer for my TV antenna so can watch TV from my phone/tablet/computer/Roku. The only thing I might add is that if you really want to be safe, do a periodic backup from the NAS to an external USB drive and store that drive at another location (e.g. a friend's house). This saves you in the case of house fire/flood or other catastrophic damage to your NAS. You can also pay to store backups on the cloud but you'll have to pay a subscription fee. For example, Backblaze will allow unlimited backup storage for $100/year which is a pretty good deal considering the value of your data.

[@CuivTheLazyGeek]

Thanks for the feedback and the suggestion! Backblaze sounds amazing, I'll check them out!

[@stephenthexton7056]

Hi Cuiv, another great video thanks. Just a word of caution though. I had this idea about 7 or 8 years ago back and bought a 4 bay QNAPsystem and set it up in a very similar way. My kids worked in France and Switzerland and we had loads of work we all wanted to potentially back up and share. We lost everything after being hacked with ransomwear. This was pre 2 level security logins etc. I am sure my system with greater knowledge may have been better protected but my internet virus protection etc has worked fine on all pc and laptops etc but didn’t on these. Fortunately I stored all the old discs which I had copied on to the NAS of family growing up but alas a lot of intermediate stuff and early Astro files are all locked. SO just be careful that you may want to think about a separate un-networked library of your life’s work archived elsewhere where!!!! Good luck and thanks again

[@CuivTheLazyGeek]

Very good point Stephen, thank you very much! I'll probably update the drives in my "bad" enclosure and use those as additional offline backup!

[@jeffbrennan5394]

Thanks for covering this overlooked topic in astrophotography. Storage is a constant struggle for me. This system is definitely a solution to this problem and appears to be user friendly. Will add it to a future purchase. 👍

[@CuivTheLazyGeek]

Glad this was helpful Jeff!

[@WestwoodAstro]

I've used Synology NAS's for years, both at work and home. Between file versioning and automated off-site backups, I haven't lost anything since I started using them.

[@CuivTheLazyGeek]

Thanks for the feedback, good to know!

[@psyire9913]

I use a 2bay Synology NAS and highly recommend. Great product. Also use it for backing up all the family phones so if anyone loses one they have all their data safe.

[@CuivTheLazyGeek]

Good use-case, thanks for sharing!

[@RobertMercer-s1f]

Been using Synology NAS units for a while now, I’m now on my third generation of units. I use 4 x 4TB SSD drives (10Tb+ usable) for my main storage - that is a shed load for astrophotography. The Hard drives are useful but slow so I use those for backup on another Synology NAS. You will also need to harden the NAS against cyber attack (plenty of videos on KZbin about that) especially if you want to access it from France or anywhere off your network. You should also look into enabling snapshots since you are using BTRFS. You can also sync files between computers I.e. your imaging computer and desktop which you might find useful.

[@CuivTheLazyGeek]

Thanks for the tip, looking into hardening techniques and snapshots!

[@shubinternet]

So, the thing we have learned about RAID-5 is that if you have one drive failure, there’s a good chance that you’ll have a second drive failure - but before you can finish resilvering the array and recovering from the first drive failure. So, RAID-6 is highly recommended for larger capacity arrays.

[@CuivTheLazyGeek]

I think I need to reformat and go for RAID6....!

[@_TriffiD_]

I love Synology. I use them at home and at work. It is a great device but I dont need to have my old astro-data available at all times. Large 4 bay Synology-NAS use qute a bit of power (35 to 40W - more than a fridge or even a good freezer) and need to be set up correctly to be reasonably failsafe. Besides I highly recommend using special NAS-Harddiscs that are made for permanent operation which are - guess what - quite expensive, too. I personally put my raw data and normally even project-files on M-Discs (25, 50 or even 100GB each) which are a special kind of blueray discs. Not cheap either, but safe for a 1000 years, storable on my shelf and no maintanace needed.

[@CuivTheLazyGeek]

I had never heard of M-Discs, I need to check them out!

[@_TriffiD_]

@@CuivTheLazyGeek It is basically a Blueray Disc, readable with any BD-drive. But the special coating needs a writer with a suitable laser. Not overly expensive and you can use the devices with normal CDs, DVDs and BDs. Perfect for my liking.

[@rechsteinerreto]

I store my most important data in Google Drive Cloud. 2TB cost 100 swiss francs per year, 5 TB cost 250, and so on, mirroring to my mac and one external Hardrive. I just destroyed some 20 old external drives and this method works totally automatically and stressfree.

[@CuivTheLazyGeek]

While it's good, you absolutely should have a local backup as well! It depends on how much you trust Google!

[@rechsteinerreto]

@@CuivTheLazyGeek By having the datas in my Mac/PC and in one external HD, I have TWO lokal backups

[@BiscuitAdvisory]

I bought a synology NAS that does RAID1 with my astro data. when you map the drive from the network on your PC, its absolutely seamless! but do keep in mind every NAS should have a UPS or equivalent power outage protection.

[@davanti]

I have owned the DS920+ for a few years now and use it as my work NAS, plus a media server for plex and other docker-run applications. I've bumped the memory to 20GB and have utilised the two NVMe bays for a cache and it runs great! I think perhaps having the actual plex server itself on another, more beefy computer would be better (and simply accessing your media from the NAS) but for everyday use it's been awesome. 10/10 product.

[@CuivTheLazyGeek]

Now I want to add NVMe to mine for movie streaming :)

[@davanti]

@@CuivTheLazyGeek the ONE annoying thing is the NVMe bays are controlled as a cache by the OS only and you can't use them as regular storage volumes. But I've seen a noticeable difference in speed with them in place!

[@dukrous]

Don't forget to also back them up offsite. I use Glacier Backup on my Synology NAS to put backups of all my astrophotos in Amazon S3 Glacier Storage. Total cost to me is about USD $4 per month and I know that if anything ever happens to the NAS itself that my photos are further backed up.

[@CuivTheLazyGeek]

I need to also check Backblaze! Seems a lot use it!

[@joakimastro]

I do the same-ish thing, but my folder structure is like this Year (eg 2024) - Targetname - Name of session/filter/rig - Light - Flat Naming scheme: SESSION 17-10-2024 M31 S135 ASI1600MM R Explanation of the above: SESSION (just the phrase I use for pixinsight to separate sessions in WBPP) - DATE - TARGET - SCOPE SHORTNAME (in this example, for Samyang 135mm f/2) - CAMERA (ZWO ASI1600MM Pro) - FILTER (R for Red)

[@CuivTheLazyGeek]

That's better organized than mine :)

[@ktydeck]

Great video as always Cuiv...

[@CuivTheLazyGeek]

Thank you !

[@Chiclets1]

Very clear and through video. I made the mistake of not taking storage too seriously when building my post processing machine. Now my pc looking like a porcupine with 5 additional drives wired to the motherboard here and there, with no redundancy. I do have an old NAS I need to dust off and incorporate into this machine. Thank you for another great video!

[@CuivTheLazyGeek]

Thanks for the feedback! Yeah storage is an easy thing to overlook for sure...!

[@Chiclets1]

Your hearts make me feel all warm and gooey inside. Just like lukamatiko 😊

[@chrisastrophoto2904]

Thanks for this video. Yes, it looks expensive, but if you take into account all the time you need to recover crashed files and all the frustration when loosing data … it is worth it. I use the predecessor 920+ with 8TB disk for years now, in combination with online (slow) storage for my very old data and in combination with Drive Sync, which syncs all my files during acquisition from my mini-PCs directly onto the server. This makes life so much easier. It just recently saved my life again, when the NAS told me, that one disc had rapidly increasing number of bad sectors, so that I could even change it (hot swap) while the RAID kept the NAS alive. It is so convenient, once you have set it up nicely.

[@CuivTheLazyGeek]

That is really good to hear, have an "advance warning system" like that, thank you!

[@olafbaeyens8955]

RAID 6 is important when you have massive data, and one of your hard disk fails. When it takes days to repair a failed hard-disk to a new hard-disk there is a bigger chance that a second disk will fail because of the unusual stress. Also when you have massive files, once ever few months you need to go trough it bit by bit to detect bit-rot. Bits that flipped but your hard-disk does not realize it. And also my NAS drive failed 2 times, everything gone, so I use an older NAS with too old drives I recycled from the new NAS as backup of the new NAS.

[@CuivTheLazyGeek]

Very interesting, thanks for the feedback! I'm wondering if I should actually have gone with RAID 10 for complete mirroring....

[@olafbaeyens8955]

@@CuivTheLazyGeek Not really, but realize that the NAS itself can fail or get hacked. It is possible to have an external hard disk connected to it and it backs up the most important data just in case. NAS drives need an UPS connected to it. The good news is that Synology never failed on me. NETGEAR did fail on me twice.

[@TechnoBillyD]

hehe I have two. One is on all the time and is my main network drive I use to store everything, then once a month I turn the other one on and duplicate the changes to the second unit that I leave offsite. Yes you can use use cloud storage, but both my units have 24TB which would be expensive cloud storage

[@jongreen2362]

I've just seen the header of your latest video and it's given me chills. I literally lost all of my astro images only yesterday while trying to transfer them to an external HDD!!

[@CuivTheLazyGeek]

Oh noooo I'm so sorry to hear that!! Have you tried data recovery software?

[@jongreen2362]

@CuivTheLazyGeek I've tried multiple data recovery software and also a system restore with no luck. Your NAS would be the perfect solution, unfortunately out of my price range. 😪

[@FrancoGrimoldi]

Hi Cuiv, nice video! I use a NAS at home (four bays populated with three drives on RAID-5, room to expand with the addition of a fourth drive) with two extra security features: . The NAS (and my network equipment) are powered by an UPS, so they don't lose power and/or get powered off un-gracefully. . I use a backup app within the NAS OS (QNAP QTS in my case) to run a daily backup into a cloud backup system. This way, my data is secure even if the NAS dies or I lose access to it. I truly enjoy the comfort of knowing that all my files are being automatically backed up behind scenes... for the first time in my computer life I feel that I'm safe. It's not cheap but definitely worth the monthly fee.

[@sianikolaou5440]

Definitely something to consider, especially when I only leave a little bit of space that keeps my laptop functioning while I fill the rest of the space with a single solar imaging session.

[@anata5127]

This is great video! I store all my raw files in calibrated form. Otherwise, they will be useless without flats and darks.

[@CuivTheLazyGeek]

Yep that's what I end up doing as well!

[@kevinn9jkp839]

Great video! I got a two bay Synology NAS two years ago. Using mirroring so I only have 4TB of memory and I will need more likely later this year. For me your video validated my decision to have a Synology NAS for astrophotography. My telescopes computers are in buildings separate from the house. I worry about writing directly to the NAS given my network speed and if that could affect the raw camera data download. Maybe the computers buffer this, if it does then that would be simpler than what I'm doing now which is saving the images on the telescope's computers (NUC near the mount) and then manually transferring the next morning to my NAS in the house. Also, I need to have better data directory naming/hygiene. Your method looks good to me. Thanks for making the video.

[@afd33]

I've been using a NAS since I got in to the hobby basically. After 3 or 4 different sessions I realized how much storage they take up. Luckily I had a bunch of 4tb hard drives laying around so it was pretty cheap for me to get started. It's so easy to go back to old data as my processing gets better. I basically only keep 1 or 2 multi night projects on my PC at a time now and put everything else on there.

[@CuivTheLazyGeek]

That's really good! Now I need to add an additional NAS as backup somewhere...!

[@afd33]

@@CuivTheLazyGeek I don't know how well it would work for you since you get to try so many scope and camera combos, but my file structure is B:\astrophotography\camera\telescope\target\date\ and in that date folder I still keep my lights and darks separate instead of the calibrated files like you do. And then masters just go in the date folder for each night. Masters that include multiple nights just go in that \target] folder. If I would modify it at all, I would say instead of \camera\ I would put \sensor\ but I've stuck with ZWO cameras so far so it's worked pretty well for me. The biggest trouble I have is when I frame something differently I don't really have a way to denote that. Hope that makes sense.

[@old_photons]

3-2-1 strategy is what "they" say. My first step was to use an old NAS 4-bay 6TB drives RAID yields me 16TB. Next I added a thunder-bolt connected DAS, an eight bay Terramaster. Only 4-10TB drives for now, As the NAS maxes out, I'll likely populate the other 4 bays with something bigger. My cloud backup is Backblaze which backs up the DAS (it won't backup NAS) for something under $100 / year.

[@old_photons]

Another thing I do is I trade space for time - while I keep the originals and calibration frames the mid-processed files (calibrated/cosmetized/registered) are "safe" to delete as once I have masters and "final' processing - I can always reproduce these if needed.

[@CuivTheLazyGeek]

That's awesome! Now I need to look into Back blaze pricing!

[@Hilmi12]

I remember when 1gb singl hdd was the size of that NAS drive. I am talking about drives without enclosure

[@CuivTheLazyGeek]

Dang! Good thing it's no longer the case :) it reminds me of Multivac from the Isaac Asimov short stories :)

[@Eoraph]

😅 i sit next to my own synology nas while watching this and didn't pay attention for a minute. The startup beep had me totally wonder if something is wrong with my device and only after a few seconds remembered that youtube is on and a nas is being set up😂

[@xxlocobassistxx]

Raid 5 is great if you're using 3+ similar drives and don't plan on adding to it, but UnRaid might be a better option if you're just starting on building a NAS and plan to add more drives later. It uses at least 1 parity drive as well but you can add to them at any time without having to reformat every drive.

[@xxlocobassistxx]

I built a Raid Z NAS before i knew about UnRaid and keep meaning to redo it, but it'll take a lot of time and storage to backup while reformating. One day though lol

[@CuivTheLazyGeek]

To be honest I think I'll copy the data out again and go for RAID10... The complete mirroring might be better for my paranoia

[@xxlocobassistxx]

@@CuivTheLazyGeekthat's definitely safer. But if you really want to be nearly 100% safe from data loss, you should have backups in 2+ locations as well. That way if there was something that destroyed multiple drives (or all) at once, like a flood or fire, you'd still have a backup to copy from.

[@flappah]

I'm doing my main processing on a 1TB Mac that has 2 additional 1TB drives attached to it. That is my direct storage. But when I'm done with processing I move all my raw data to my DS414j Synology NAS that has 11TB storage. For now that should be enough but you're right, my camera (Nikon D780) generates 50-ish MB of data each frame and I also take 30s - 1m frames. One night of capture usually gives 20-22GB of data. I should have enough space in the foreseeable future (and since I'm about to switch to a ZWO ASI533MC-pro camera my data-needs reduce a bit) but eventually I have to decide on an archive solution to move even older data to.

[@CuivTheLazyGeek]

It's really interesting how many Synology users there are in the comments! Thanks for this feedback, and yeah the long term planning for data growth is always hard...

[@hanimehrez9423]

Raid 5 disk array is the best and most reliable. These disk arrays were available over 25 years ago but were super expensive. Its a lot cheaper today. Of course, it would be best to use a disk array and cloud storage for redundancy.

[@stevepecket2535]

Between you and Luke...you're gonna bankrupt me 😂😂😂

[@CuivTheLazyGeek]

Hahaha I'm sorry! But in all seriousness please do not bankrupt yourself :) you don't always need the latest and greatest :)

[@tibbs4gaming]

Funny... or much more ironic. This is exactly what happened to me a few weeks ago, where I not only lost ALL my astrophotos, but also a lot of other files (including my complete archive of 3D files). Now I'm saving up for a NAS so that this doesn't happen to me again.

[@CuivTheLazyGeek]

Ooh no, I'm so sorry to hear that! Have you tried asking for professional help? Your data may be recoverable? Make sure to have additional backups even after getting a NAS...!

[@Design_no]

Keep as many backups as you can afford. And remember to regularly test them.

[@CuivTheLazyGeek]

That's the right way to do things :)

[@paulyoung4817]

Thanks Cuiv, I would have gone with 10, for redundancy sacrificing capacity and performance, it is a more costly solution though, I understand your choice.

[@CuivTheLazyGeek]

Yeah I'm thinking I made the wrong choice, I'll probably reconfigure to RAID 10 at some point!

[@tigithomas]

You can connect your old hard drive via usb to the front usb of the Synology and transfer you files faster.. Also you can use your old drive as addational back up..

[@CuivTheLazyGeek]

Oh right of course!! And yes I'll use my old drive as backup, thanks for the tip!

[@AlfredNitsch]

Would be interrested in the noise level of the 4 bay Synology (when disks are in standby). I run several 2 bay synologies and the fans are okish, how loud are the 2 fans in the 4 bay?

[@CuivTheLazyGeek]

You can hear it but it's not so bad, at least for me!

[@calimark7448]

I love my Synology NAS. 15Tb total capacity. RAID5. Also back up to separate HDD. After one year of astrophotography I have over 2 TB of data. ASI2600MC and ASI294MM add lots of data...

[@CuivTheLazyGeek]

Hahaha I think you were right to get that much capacity!

[@calimark7448]

@@CuivTheLazyGeek Yes, I also us it for Plex! :-)

[@sHuRuLuNi]

I store the data for future use by having huge-ass photos (on PAPER) created. Because every data will inevitably either disappear (server die, wars happen, politics) or the HDDs will stopp working after a few years. Good old photographs on paper is the way to go.

[@40Sec]

The data doesn't disappear when you use a NAS. If one drive dies, the mirrored data is kept and you replace the faulty hard drive and nothing is lost. Data scrubbing also corrects any flipped bits that appear over time from things like cosmic rays. Paper is not safer, although everyone is free to use the methods they like best.

[@CuivTheLazyGeek]

I love the analog way of storing things although even paper will degrade and photons will hurt the pictures...!

[@billbeauchemin8692]

I use a net gear 4 drive Nas. I installed 4 6tb drives in raid 5. Fou d a really neat plugin for ninacalled robocopy that while your imaging it will look at your imaging directory every minute and if there is a change it will take that image and copy it o er to my NAS. When I get up in the morning all my images are on my NAS available to my processing computer. I don't have to copy everything over manually during the day.

[@CuivTheLazyGeek]

That's such a cool flow, I have something very similar :) nothing to do, everything is ready! I love it :)

[@KitHein]

Timely topic. I recently discovered that my 2TB hard drive with my astrophotography files was at 95%. I'd like to see a brief video on which files you need to keep and which you can delete to free up storage. In my case, K deleted the debayered, calibrated and registered folders from each project figuring I could just restack from the raw files if I needed to. Your filing system does seem to be better than what I'm using. I've just been putting things in folders based on the imaging date. I'll probably convert to a filing system similar to yours.

[@CuivTheLazyGeek]

I do have a section on this topic in this video you can use the chapters to find it!

[@SteveHookEm]

@cuivthelazygeek Question: which DS923+? They have a "diskless" and one with a portable SSD. Are they both the same RAM and Processor? One is $598 and the other is $835. You don't mention an SSD.

[@CuivTheLazyGeek]

Mine doesn't have the SSD! As far as I understand for our application we don't need SSDs - the RAM and processor should be the same, so I would go for (and have) the cheaper model!

[@slmdaddy1]

interesting I just bought external SSD drive and fast one. after learning the hard way about read write speeds matter when processing huge amounts of data. I also take external drive out to my observatory and bring in each time. This NAS system could be helpful and thank you for posting the video. One question is if your using Pixinsight and processing hundreds of images at a time - does the processing become slower because its on a network?

[@CuivTheLazyGeek]

For processing, I would bring over the files locally first - so my NAS is more of a backup than a primary storage for my astrophotos (although it is a primary storage for regular photos)

[@muayaddrdentist2313]

It's very very useful and more than enough for astrophotographer but for those with limited budget they cane benefit from raspberry pi 4 or 5 or an old PC that I did before 15 years with freenas OS and Ubuntu server and it was amazing and even with heavy tasks (it's needs gaming PC and a lot of RAMs abd it was cost me less but for lazy users 😊 users Synology (unix based) I have terramaster (Linux based) and Netgear servers and they bot work fine, thank you for this interesting subject and good luck.

[@CuivTheLazyGeek]

I love that you're finding creative ways to make things work with limited budgets - thanks for sharing your experience!

[@KevinRudd-w8s]

I don't really generate that much data due to the limited amount of time I get for imaging ( two or three nights a month, and that may not even be a full night ) and the relatively small sensors in the cameras I currently own (183, 533 and 585) though I am thinking of getting a camera with a 571 sensor. Anyway, at the moment I just use an external hard drive or for short term storage a 128 or 256GB flash drive, If things change and I begin acquiring more data then of course I would consider a more robust solution along the lines of the set up you have described here. Thanks for the info on this anyway.

[@CuivTheLazyGeek]

That makes sense! Just make sure to have the data copied on multiple such external devices so you have backups!

[@geomark8851]

@13:04 How much total storage? 😚 I make this error all the time! I remember the old VAX 780 days with a roomful of washing machine sized disk storage devices that totaled less than a gig. Meg - gig - now terra; I wonder how long it will take for the next prefix to take over? To put it another way, my grandchildren are the Terra generation - it's their common currency...

[@babybalrog]

A question for future updates after you use it for awhile. How do you manage transferring your files from your PC to the NAS? is it an automated backup or manually periodically? Is it fast enough just to store the raws and use Pixinsght from the NAS, or do you need to move files back and fourth to your PC to work on them?

[@CuivTheLazyGeek]

This ends up being up to you - I'm choosing to do it manually after an imaging night!

[@richfavinger]

A NAS alone is NOT a backup!... 3 copies: Keep the original data and at least two copies 2 different media: Store the data on two different types of media 1 copy offsite: Store one copy in a separate location, off-site from your primary data and on-site backups That being said, I do love Synology units...

[@CuivTheLazyGeek]

Very good points - I added similar subtext to both the description and pinned comment!

[@LearningAstrophotography-jj9en]

Hi Cuiv, I joined your patrion about a month ago, and got my membership card, but I am still not showing on the list at the end of the video, any ideas why? Great video again by the way. Membership is Roy Hornyak

[@CuivTheLazyGeek]

Thanks for checking this with me! I actually edited this video a while back so that was before you were a member - I've double checked my upcoming video about bandpass shift has your name in it! Thanks so much for your support!

[@LearningAstrophotography-jj9en]

@@CuivTheLazyGeek :)

[@robertcarroll5036]

Been running a NAS for a several years but I need to update my drives as they are a bit on the small side now.

[@CuivTheLazyGeek]

Might just add a new NAS and use the old one as an additional backup!

[@0815mkl]

You should blur out the QR codes showing you 2FA code and the access code for your NAS. Maybe also resetup everything with a different name. Please be also aware that a RAID setup is not a replacement for a backup. the RAID setup protects you from hardware failure, but not from files delete by accident or a complete "death" of the whole NAS (like from a power spike). The best practice for backup is the 3-2-1 rule. 3 copies of your data on two different media and one of them in a different place.

[@CuivTheLazyGeek]

Thank you! I've just redone the setup with a different name, and KZbin is now adding a blur as you suggested (will take some time though) thanks for the tip! As for backup you are right - I also stressed that in the video description and the pinned comment... Data is hard. Thank you so much for your feedback!

[@Jcastleinfo]

Backblaze is your friend! I don’t mess with backups in the house in Japan! Get the one year plan and you only have to plug in once a year to sync and ensure backups.

[@CuivTheLazyGeek]

I definitely need to get Backblaze!!

[@AsKyOdA]

Each ssd has life expectancy of 5 to 10 years.. these are expensive to replace, isn’t online storage a cheaper solution & more sustainable for the long run?

[@revolting887]

Hey @Cuiv, please edit this video and blur out those QR codes (and generate new ones!), with a simple freeze-frame anyone can read your quickconnect URL and scan that QR code for your OTP! Many unsavory folks out there who are looking to cause trouble, I want to save you from those folks!!! 🤞

[@CuivTheLazyGeek]

Thanks! I can just regenerate my quickconnect URL for that?

[@revolting887]

@@CuivTheLazyGeek I think it's just easier to regenerate a new OTP token. The URL, I don't know if you can change that (as synology hosts the DNS record, etc). But I was more worried about some script-kiddie using your URL, username and OTP from this video and iterating through passwords until it's cracked.

[@RobertMercer-s1f]

SpaceRex is your KZbin friend for greats tips and techniques (I don’t follow or support him in anyway), he’s one of by gotos though.

[@CuivTheLazyGeek]

Thanks, I'm checking it out!

[@rossmartin2868]

When buying drives buy them from different suppliers or find one that will check the batches so you don’t get drives from the same batch. Also use a hot spare!

[@CuivTheLazyGeek]

Oh wow I hadn't thought of the batch/lot thing good idea!

[@publiux]

I just use cloud storage. It's much more versatile and feature rich, not to mention more affordable than buying a NAS and 4 hard drives.

[@jortor2932]

But we have to renew subscription after its over and if we don't then we will lose all that data

[@publiux]

@@jortor2932 true. And those hard drives don’t last forever. I got tired of replacing them time after time. Cheaper to use the cloud for me.

[@vijayravisekar394]

I find nas to be a pain, it shows down the wifi considerably and the spores are nowhere close when it comes to data copy speeds

[@CuivTheLazyGeek]

How much data do you have on cloud storage?? For masters I don't think it's a problem, but terabytes of cloud storage sounds very expensive!

[@publiux]

@@CuivTheLazyGeek I have about 2TB. Any old data gets put into Amazon Glacier storage, which is super cheap.

[@Prometey77777]

I built my own truenas based 38TB storage. Jonsbo N1 case 5x12TB ironwolf pro in raid5 + N5000 or N6000 cpu. works like a charm. But remember (that very wisdom rule that everybody knows) RAID/NAS it is not a backup and loosing data is not a question "how", it is the question "when".

[@CuivTheLazyGeek]

TrueNAS sounds amazing! And you're right on the backup part - I stress this in the video description and pinned comment!

[@AnomalousAstro]

You should edit your video and blank out your MAC address and QR codes. A 'talented' hacker could use that to find your device and cause problems for you. I also had a NAS fail last summer (20yr collection of high quality music files >25TB) because 2 drives failed at the same time. Nothing to do but cry...

[@CuivTheLazyGeek]

I've blurred the QR and resetup my credentials! When was my MAC address displayed? Thanks so much for having my back!

[@AnomalousAstro]

@@CuivTheLazyGeek Your Synology MAC address is still visible at 16:19. Long time fan of your work, hate to see you have any problems, wish my retired bank account could support you more!🙃

[@CuivTheLazyGeek]

@@AnomalousAstro Thanks again! I've added a blur, hopefully KZbin will reflect it soon - don't worry about financial support, your help like this is more than I could ask for!

[@SleepyAlice895]

3-2-1 backup rule! 3 copies, 2 medium, 1 off-site! And RAID is not a backup!

[@EugeneRu777]

👍👍👍👍👍👍👍

[@Astroduck44]

Wowie

[@Kris-jk9mq]

Been using my home built NAS out of a Single Board Computer and USB spinners for a decade. I bought a cheap 80$ 2TB SSD to work with my files, and once a week/month I will "winrar" compress my SSD archeive and copy to my NAS as a backup. Compressing these is great before backup! Takes a while but 1 file vs a million small files is easier to work with and faster to transfer over the network.. AND NO you do NOT want to work any POST editing off of your NAS..

[@glennn.3464]

It wasn’t "out of the blue". They were tracking your online activity and jumped in to offer a solution. 😱 Also, I’m not saying that you don’t give completely independent and honest reviews - I believe you do - but how is it any less potentially influential to receive fairly pricey gear for free compared to having a sponsored video? If you wanted to show that you’re being completely independent and not influenced in any way you would refuse to keep the gear just as adamantly as not doing sponsored videos. Many channels do the exact same thing but that doesn’t make it any better because of having to disclose that they received some form of compensation. Free stuff is the same as money when you would otherwise have to pay for it.

[@CuivTheLazyGeek]

From what I've seen, "real" sponsored video prices are much higher than what I ended up getting - you're right I could insist on sending it back though.

[@egar4767]

Im running a Netgear RAID 5 with 16 terabyte drives. I bought a lifetime license of Plex on a media server. I use it for all my data and also record over-the air channels. I’ve had it for about 8 years. Three drives have failed, easy to swap them out. Love it. The only thing is that every once in a while Plex/Netgear takes away a feature. I think Synology is better. Great vid. Thanks for sharing.