How to Use the rsync Command to Transfer Files (Linux Crash Course Series)

[@Wandering_Horse]

I have a server in LA, and when I am at school working on my coding, I can upload my work, go home download it, work on and upload it and have fresh code available all the time. Also prevents me from having to use the schools horrifically slow vpn. Also while the other kids are struggling to get there homework open in eclipse, I am already rocking my code in Vim on a remote server in LA!

[@fullstackwithsantosh]

Wondering why are not already using git.

[@adreto2978]

@@fullstackwithsantoshgit is good for code but if you have many blob files like images of notes / graphs from textbooks then it quickly becomes inefficient space wise to use .git for tracking. Can sometimes nearly double the dir size

[@cristoforopontikas]

i would like to let you know something very useful to your guide. When you using the -a archive option you must be aware that using this include and preserve all those options (-r recursively, -p permissions, -t modifcation time, -l symlinks, -g group, -D devices) so you don't need to use the -r option as in your example when using the -a option. Hope this make the rysnc -a option more clear... thank you

[@KILLERTX95]

I always use: Rsync -Pavh --dry-run blah blah Progress Archive Verbose Human readable Super useful

[@jim7smith]

Thanks so much for your linux series. I especially like the way you provide links in the ...more under the video so that we can have a TOC to help us find the desired content. Blessings on you and yours from the banks of the Suwannee River in North Florida.

[@jebucaro]

¡Gracias!

[@rcdenis1]

Rsync will resume tranfers if the link drops, with scp you'll have to start the transfer over in that case. Also, the OpenSSH folks declared scp deprecated a few years ago. Rsync -avP is how I use it.

[@AustroPower]

Man Jay i am forever grateful and thanful for all the knlowledge and tuts you are dropping every week, i learned so much from you when it comes to Linux, i am now looking forward to buy your books :)🙏💯

[@fvgoya]

16:50 Hey Jay, and how we can do a real sync? Is there a command for that?

[@PhilipBonev]

Thanks for the tutorial. I am working with linux for a long time now, but still I watch your basic tutorials there is always some little bit that surprise me. Little comment on the tutorial - I think 'command' is bash specific. Better use 'which' in my opinion.

[@dono42]

FYI When using the -a argument you no longer need -r as it includes it (and some others).

[@mojoblues66]

8:54 What do you mean with "and DNS is often a problem" ???

[@ronaldronald8819]

rsync is an excellent tool and worth your while spending some time learning it's options.

[@PaPaTheGMan1215]

Thanks for the video. it is very informative. One question, how do you add a password to the destination address? Many of my systems have a password for access and don't have ssh keys. Thanks.

[@michaelcooper5490]

Hello Jay, Very good video thank you for this. I have a question How do I only sync the files that are different is that possible with a command syntax?

[@lawman2112]

Loving the content here!! Thx!!!

[@matteocassino3172]

Easy to understand and very helpful. Thx!

[@gabriele8332]

Hello, can you also compare rsync with unison for file sync? Thank you.

[@jasond.valentine5931]

Thank you very much for this explanation. Concise and with examples. This is a great way to teach.

[@davocc2405]

In a practical sense RSYNC is sometimes over-used by some sysadmins - I remember it being used for large-group transfers (entire database filesets across 100mbit, we're talking instances of 40gig) and they were creating a huge overhead on the disk packs on both sides in one chunk. The database files were a mixture of some which changed heavily a a lot which didn't but every time you ran RSYNC with the checksum calculation the overhead on the disk was kinda wild. A later admin saw this and changed to a process where he created an MD5 checksum separately and worked out the files which had actually changed that needed to be replicated to the target; he then used RSYNC on only those files. To boot he had a checksum file which he could also transfer to the target to do a validation offline without involving the source system. This was on a shared SAN so disk utilisation spikes had to be considered, this method reduced peak demand quite a bit; the way they were using RSYNC before it was initially creating its internal block checksums on every single file every time (I think they didn't use the change date/time metric). This guy got me into checksum files as a standard practice and I've done them for everything - I archived data to several thousand DVD-R's and produced a checksum for each file set (burnt with the target files to disc) so when retrieved we could work out if there'd been any degradation or loss of files on the optical media. It's a bit of due diligence which can really save your arse especially if your processes are audited.

[@denalimike8159]

GREAT VIDEO! YOUR THE BEST! FROM DETROIT MI!

[@QuimChaos]

Isn't the -a option (archive) the same as doing -rlptgoD ? if so the -r (recursive) option is not needed?

[@CaptZenPetabyte]

Thanks for this tutorial, it will become very useful I think ... I am ditching Wordpress and going back to static websites, and have decided I will simply update the files locally and rsync them to the website. Though having said that there are a few people that argue the ssh vs ftp and reliability issues, so I am still working some things out

[@PswACC]

Excellent explanation. Thanks

[@tigreonice2339]

I've a question related with linux (not this vid): How to install fingerprint reader: U.ARE.U 4500 fingerprint reader Digital Persona, inc (crossmatch patented) on Linux mint? 😢 Any way that works and is safe?

[@jonathanchevallier7046]

Very nice explanations. Thank you Jay. Do you use a program in particular for your markdown notes, or just a text editor ?

[@Handelsbilanzdefizit]

If you do PXE Diskless-Boot, you have to transfer the entire Root-Directory to a remote system. What's the best way, to do this? So that no files go lost, because of access rights and other stuff. The only reliable way I found, is: Create a complete partitionimage "rootpartition.img" , and then mount it on the remote system. That's the only way that works for me.

[@paullee107]

Oh cool- I'm glad you covered rsync. I'm an intermediate user, but you touched on an 'issue' that I just experienced on 16TB of backup data. I was using the --delete flag (and -av) for backing up my NAS drives... at some point the NAS went down, but my backup server kept diligently running its daily rsync. GASP, when I found all backup folders were EMPTY. --delete did what it does, and... poof. Any solution here? My local NAS mounts weren't present and I lost backup data. Is there a solution you might suggest while still using rsync? Second; I've been looking at borg-backup since that issue... I don't love it. Any other suggestions? Thanks!!

[@shanent5793]

You have to be careful with the design, it sounds like what you were doing wasn't really a backup at all. Any accidentally deleted file would have had at most one day of protection. There's no way around keeping multiple copies. For example you could rsync to a different destination directory each time. If bandwidth is limited, first make a copy of the destination, then rsync. Those are the safest ways, consuming a lot of redundant storage. Unfortunately doing anything too clever to save space is a risk. Filesystem-level deduplication or snapshotting are probably the lowest risk, or you could triage the data and reduce the retention accordingly. The risks are a bug in the filesystem or putting valuable data in the wrong retention level. Define some requirements and how you're going to evaluate them, choose your level of risk, keep some offline copies and test thoroughly

[@paullee107]

@@shanent5793 I don't understand what yer saying.

[@OXIDE777-is6gs]

Awesome video, thanks man!

[@nathan445]

Awesome video! I'm going to try and use this to back up my Ubuntu Minecraft server to my WSL

[@cbmeeks]

Really loving your videos. I would like to see you do one on rclone as well.

[@readwriter]

I have studied a lot about rsync recently! My simple use case is to maintain backups of files to an external attached drive. I was disappointed to see that in backing up from macOS to exFat, the metadata/resource fork stuff created by Mac (or some other cause) causes MANY unchanged files to be transferred again unnecessarily. I just saw a Reddit post that said, now, in macOS Ventura, inclusion of the standard “--modify-window=1” flag fails to be useful due to a kernel flaw. This topic is probably not substantial enough for a separate video, but I look forward to your comment.

[@ChadsHobies]

Gentoo's Portage package manager uses rsync to synchronize your package list to the current in the repo. It also uses it when issuing commands with emerge to install packages. Rsync is an intragal part of the initial tool kit required for the installation of that distro. Rsync is really cool.

[@wyfyj]

Another Gentoo enjoyer I see.

[@CDE.Hacker]

Weird, everything I've read about backing up says to use rsync. But Syncthing seems to work so much better. I use it as a live backup or clone of my work files and do an off site backup periodically. It's mostly to protect against system or drive failures. If something goes wrong, a full up to date copy is always ready. What do you think?

[@ultimatedjX]

It's because in reality rsync has old architecture, operates on a signle thread etc. It's a very popular tool and it's been there since... Always? When it comes down to syncing large amount of files or big ones, syncthing is going to be better in a lot of cases. There is a modernized version of rsync called rclone that is much more efficient and ready for nowadays workloads. PS: I use both syncthing and rclone (and rsync in the past). People tend to not write all over the Internet about their problems with rsync. They rather move to something else instead, like rclone which is very similar. This is how the legend of mighty rsync goes on and on :-)

[@gwojcieszczuk]

One limitation with (-a) archive option is that it won't preserve ownership/group unless you're syncing to target as user "root".

[@QuimChaos]

or super user (SU/SUDO). -a is a group of flags that has the -o option that means: preserve owner (super-user only)

[@onknight]

Is there a way to transfer files from an open media vault server to a Synology server?

[@littlenewton6]

Another rsync problem with multi hosts: I CANNOT use auto completion when I type the full path of destination! Each time I have to ssh to the destination host and pwd the full path, then copy and paste.

[@cybersekkin]

the -a option includes the -r so when you added it, you can remove the -r as no longer necessary. The -avh --delete is so ingrained in my fingers (muscle memory) I have to pause when I want anything else and think about it.

[@hansphung]

May you share what Linux distro you’re using and the theme? Maybe not

[@SlideRSB]

I'm guessing Debian.

[@marioandresheviacavieres1923]

Thanks for another great one

[@wiepcorbier]

You can skip to 2:14 😅

[@GooogleGoglee]

How and where do you set that ssh banner message for every ssh connection to the server?

[@jamesbaxter2812]

I got my RSYNC working. I am thinking. the cache file. I dont know if deleting it is good. But when RSYNC is done. Can it leave the cache alone.

[@user-ji1rq6tq2p]

Really need a simple program that mirrors 2 directories so I can have a backup and separate backup of the samething on my external drive! Please help

[@vPeteWalker]

Thanks for the video! I gotta ask, where'd you get that shirt! I want one for my daughter and I! :) Much appreciated!

[@peterjansen4826]

I used rsync to clone my Arch-install from one SSD to another (faster). It was easy, just looking up the exact parameters on the wiki (I needed an extra one for VM's) and everything went well. All I had to do after that is install the bootloader again and I could boot on it.

[@mitchellpayne3674]

Can you share your .md note files? Or is there somewhere you've already published them so we can look at them?

[@nairtana]

Is it possible to use rsync to transfer files to Google Drive if I am signed in on Ubuntu?

[@websolutiontechnical]

helpful :) what if you have a file of a different user(other than joy) and having 700 permission. it will give permission denied even you run rsync using root. any solution for that?

[@spasha90]

$ command -v rsync -->> what is command in this?? $ which rsync --->> usually this was used to check if it is installed.

[@lawman2112]

I assume SAMBA is installed on both the source and target machines?

[@deadlinerhorus]

smb client always is - at most distros while the server part has to be installed, that is correct.

[@ultimatedjX]

Rclone is so much better than rsync in any way... So many times I've been having problems with rsync's low transfer rates due to unknown reasons. It's an old software, guys. Rsync on the other hand does the same, is breeze to use, and didn't fail me even once! I imagine people still use rsync due to the habit mostly (like I did). An example of functional superiority of rclone: it has a buil-in mechanism for hashing ssh passwords of the hosts you want to sync with. Very handy.

[@sandeepvk]

I learned that I can *ls* the subdirectory using: ls/backup/notes. Never knew that.

[@luderetoplayistolive9824]

thank you for this!

[@RinksRides]

CONTENT starts at 2:30

[@camaycama7479]

The link for Akamai is missing

[@littlenewton6]

谢谢！Thank you for your tutorial. Now I use Rsync to do replication between two NAS. The only pity is that Rsync has no windows version.

[@ironfist7789]

maybe you could use Windows Subsystem for Linux (WSL) but it might be a pain

[@-aexc-]

​@@ironfist7789wsl has extremely poor disk performance iirc

[@ironfist7789]

@@-aexc- ah ok, that sucks

[@hmbird47]

waiting for an episode about kill command 😁

[@JohnnieWalkerGreen]

I hate it whenever I forget to put a slash "/" at the end of the folder name.

[@absmustang]

Is that a rhel or Debian server? sorry, I didn’t catch that the first 20 times…😂 Great tool, great video.

[@sussusamogus7831]

great video

[@squalazzo]

are you sharing that notes folder somewhere? interesting

[@roadkill9173]

Are your notes published to the public?

[@ChrisJackson-js8rd]

rsyncd.conf should be covered next time you return to this topic

[@rhb.digital]

Thanks.

[@k.chriscaldwell4141]

Thank you.

[@davidmutua6971]

I use scp command for the same including the -r flag. Works same way. Don't see any difference Jay

[@readwriter]

Concerning the single-directional nature of rsync (without ‘delete’ flag), the *best* wording I saw to describe this is, “Rsync is ADDITIVE”

[@jameswhite1910]

Yes, seems an important distinction if one is looking for a .photoslibrary backup, for instance.

[@mariozamora9684]

For backup, yeah. But if you are migrating data from OS version 6 to OS version 8, you may want to preserve some of the new files generated on version 8.

[@sureshkrishna3425]

😊

[@sureshkrishna3425]

😊

[@NuruislamMiah]

BB❤ VP❤❤b❤❤​@@jameswhite1910

[@zinkzxd2891]

Cheers!

[@TheSecurityAgency]

Wow, a dry run, I can't even imagine what a "wet" would do, but it certainly gonna be a liquidfall?

[@RAYJOS]

I haven't seen the difference between rsync and scp.

[@kazialireja262]

Kisuka dil jeet ka tum tumra hak adaya karna joga❤❤

[@GilbertJane-oq8nc]

6492 Misty Square

[@WildeSelena-m1r]

Schimmel Expressway

[@HoltHugh-i1t]

Schowalter Road

[@AngelaWalker-h4i]

Willard Shoals

[@ArifKamaruzaman]

Eyy Im fast af boi.

[@BarbaraLewis-q8i]

Simonis Lock

[@AntwanArzabala-u3y]

Demetrius Corner

[@AngelaGilkey-o7e]

Hudson Street

[@FloydCeder]

7911 Osinski Fall

[@UlyssesTwomey-u8s]

Keven Highway

[@AnthonyJacob-y5q]

Rogelio Lodge

[@KrisThornhill-l5h]

Prosacco Estates

[@kazialireja262]

Tumhumra pas sa kaya sikhsa Mila tumko etni jara bat ka liya Tum gio sisa ka umra gana sunna abhisa etna lachar aga baro dakho kaya hota hai❤❤

[@gauravkalpasi]

First viewer

[@BHUPENDRA._.JOGI9]

😅😅9m 😅9😅😅 Nn KN nHistory HiKstory oion0lpm9 9nnn 899mm Nn Nnnnn😅.lmko

[@ZubairKhan-vs8fe]

Why do you need command lines in linux. That is so old days. Like using dos when you have windows

[@SlideRSB]

Because command lines are super leet

[@johnbaines9580]

If you are allergic to using the terminal there are GUI front-ends. It's linux - so you can choose!

[@hpalvz]

Also... even Windows have a tool like this one! (robocopy)

[@ZubairKhan-vs8fe]

Thanks thanks for explaining

[@christianemden7637]

Also for the purpose of explaining rsync, using it on the terminal it becomes completely independent on what distribution you are running, the commands are always the same, and the output looks the same.

[@Jorokusaki]

Which emulator do you use?

[@FossWolf]

Hey this is a great video. rsync is a great tool and with argument -a -a, --archive This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything. :)