Rclone SMB Mount on Ubuntu/LXC - 💪The Power of Caching

[@AnandsLab]

15:55 If the Rclone mount fails, check Rclone log file (as in the SystemD script) for errors. I should have mentioned this in the video. Relevant Files: github.com/htpcBeginner/docker-traefik/blob/master/scripts/systemd/rclone-ds918.service.example github.com/htpcBeginner/docker-traefik/blob/master/scripts/hs/start-media-after-boot.sh.example

[@FloTexaTenn]

Another excellent video, Anand. From a Auto Traefik + supporter :-), a video on the use of your script to delay spinning up of the media apps would be PHENOMENAL! I have a method now that works without scripts, but I would love to try your script.

[@AnandsLab]

Hey thanks! Here you go: github.com/htpcBeginner/docker-traefik/blob/master/scripts/hs/start-media-after-boot.sh.example

[@MarkConstable]

Good video, thanks. What would be the difference between using rclone to mount a smb share compared to using smbnetfs? Ah... spoke to soon, I got to the end of the video and it's the cache which is makes the difference.

[@AnandsLab]

Thanks. Glad it was informative.

[@vitusyu9583]

Great work, very interesting,

[@jongyrocka]

This is a great concept and I implemented it, but ran into one major issue. Polling-interval is not supported by this remote. So any changes / new files added on the host are not reflected on the rclone mount unless you restart the service. Are you experiencing the same?

[@AnandsLab]

Thank you! When you say "new files added on the host", do you mean the machine running rclone? Or the SMB server? Please explain, I may have experienced this couple of times. Take a look at this to see if it addresses your problem: forum.rclone.org/t/can-you-refresh-a-mounted-remote-without-polling/22144 Let me know how it goes.

[@jongyrocka]

@@AnandsLab New files added on the SMB server. File's added to the machine running rclone, will upload to the SMB server instantly, but files added directly to the SMB server, rclone will not pick this up without restarting. Setting the log level to info instead of errors, indicates that polling interval is not supported by this remote.

[@Ripichip1]

@@jongyrocka Greetings, I hope you are well, I also realized this problem, the solution is to change the value of ¨--dir-cache-time¨, for example, 10s. Restart the server and it should resolve.

[@la3135]

Hi Anand, great instruction video! This was very useful and I like the idea of using a cache of media consumption and keep the "backend storage" on "low power mode". However, what are the options if you don't want to use SMB? Might a S3 backend using Minio f.i. better for faster access and transfer times from rclone to S3 Minio? And what about connecting rclone to NFS server? This last one seems not to be possible, since I don't see a storage backend for NFS in the rclone config. Please let me know what your thoughts are. Have a great day and looking forward to your other posts on this channel!!

[@AnandsLab]

Rclone does not support NFS unfortunately. If SMB does not work there is SFTP support, which is an FTP over SSH connection. Unfortunately, I have no experience with S3 Minio.

[@la3135]

@@AnandsLab Thanks Anand, SFTP works fine and is less complex than Minio

[@evanmarshall9498]

Awesome video! I have both my docker media server and media files in one bare metal unit (arch linux os + docker on nvme and media files in a storage pool made up of four hdd). You wouldn't be able to point me in the right direction on how to get my docker instance to recognize and mount my storage pool (locally on same machine) to use for my media download and other app's storage? Can rclone do this? I would rather not have to build a NAS or purchase one external and connect to it remotely when I already have the hardware setup. My goal is to have everything on one powerhouse machine (gaming, workstation and home server), but no one seems to do this.

[@AnandsLab]

your setup is then simple. YOu do not need any kind of mount mounts. You can mount your media folders in Docker containers directly if they are accessible from the host machine. Have you checked my Docker video? kzbin.info/www/bejne/innYfZp9rNZjrMk. Reach out on my discord you need help. There several people who can help you out as well.

[@evanmarshall9498]

@@AnandsLab will do! Thank you. I think what's causing me issues is that I am transferring from Unraid so I have four HDD formatted into a ZFS pool which I do not think Linux likes. Might have to reformat them into a storage pool which Linux likes.

[@geekcruz]

can you make a small clip showing this process in action?

[@reyastaroth]

Hello! I have the shared folders of the HP gen8 with NFS (multimedia) and also the downloads folder (nas synology DS218+), until now I have it in a VM with DietPi and mounted everything with NFS to the virtual machine. What advantages does rclone offer versus NFS? I understand that if it is an LXC container it is an rclone option. It could also be mounted via NFS on the LXC, right? Thanks for your videos!!!

[@AnandsLab]

Hi! Rclone offers the advantage of caching. But Rclone does not support NFS mounts. Only SMB/CIFS. Caching can be beneficial but sometimes might require manual (or automated via cron/script) refresh of caching for new files to show up immediately. Only privileged LXC can mount NFS. I use (as in the video), unprivileged LXC, which cannot support NFS or SMB mounts. So Rclone is the best option or use Proxmox bind mounts.

[@reyastaroth]

@@AnandsLab I understand that you have to empty the cache directory to fill it again using cron/script, do you have this step documented?

[@AnandsLab]

Actually it is simpler than that. You do not need to empty the cache. You just need to refresh/force a scan of new files. Should only take seconds. I lost that command. I will update here when i find that command.

[@reyastaroth]

​@@AnandsLab Okey!!! Thank you for your trouble

[@fanofc-adecoulomb8477]

Great work. I was getting an error when I attempted to force-recreate jellyfin after successfully enabling SMB_NFS mount...I had to comment out the following two lines from your original yml - $DATADIR/data/media:/data/media1 - $DATADIR/data2/media:/data2/media2. The error was: Container jellyfin Starting 78.4s Error response from daemon: error while creating mount source path '/media/rclone/MYSMB/data/media': mkdir /media/rclone/MYSMB/data/media: input/output error Any ideas??

[@Thiccalus]

Thanks for your videos as always Anand... I am going to try to use rclone on my recently setup LXC Ubuntu Host before moving on to step 2 of your renewed guide. I currently have 4 8tb HDDs in a single system (the same one that proxmox is installed on) in a ZFS pool running Raidz.. hoping rclone will enable my unprivileged host to communicate with the ZFS pool, which is where I will store all of media for my media server stack. Probably not going to spring for the auto-traefik script just because I am really doing this to learn, what is the best other way to help you out, a subscription on your website?

[@AnandsLab]

Hey Thanks for the kind words. Setting up Rclone and moving to the next guide sounds like a plan. But remember, you will have to setup SMB share on proxmox host. If you are familiar with bind mounting, that might be an easier way to as well if all your media is on the same system as the proxmox host. Give me a week or so I will try to make video on bind mounting. No worries on the script. In fact, I recommend doing it the hard way to learn. But some of the membership levels do provide free access to the script: www.smarthomebeginner.com/membership-account/memberships-products-services/

[@Thiccalus]

@@AnandsLab thanks anand, you are the best! I followed another guide that took me through setting up an smb share, and confirmed it was working by uploading a file to the smb share (also running in an LXC with Debian 11 and cockpit for Identity access management) and such. To do this we did bind mounting, so it would make sense that I could use bind mounting to make the ZFS pool visible to the Ubuntu LXC, I was a bit concerned I may run into permission issues with the unprivileged Ubuntu host trying to access that bind mounted drive, but maybe I am over thinking it. Anyhow, really appreciate all you do. Thanks again.

[@mithubopensourcelab482]

Good job indeed. This is the easiest way to take backup. Can Rclone preserves versions ?

[@AnandsLab]

Keep in mind that Rclone is not a backup system and does not do versioning.

[@GothicGame]

I have following problem: I use veracrypt for encryption. I always have to restart my smb-server to get the new content in my smb-client. (This means for example, if a new file gets created in my samba-origin-folder, it won't be displayed on my share. I need to restart the smb-server to get the new content. What could be the reason for that? (Until now I used bind mounts, mp0, passing through the host)

[@AnandsLab]

This is a good point. I run the following command using a daily cron job (replace remote-name). This refreshes the cache. sudo kill -HUP $(ps aux | grep REMOTE-NAME: | grep -v grep | awk '{print $2}')

[@GothicGame]

@@AnandsLab thanks. But what's the reason for that? I really don't understand. Is it because of veracrypt? My other smb shares work like intended.

[@ravndude]

Which script are you editing? under systemd, is it rclone-drive?

[@AnandsLab]

Ping me on Discord, I will send you latest one. I made some improvements since publishing the video. I haven't pushed it to GitHub yet.

[@GothicGame]

I want my smb-share to be read only. How to I config this in rclone to get it ro only?

[@AnandsLab]

You can add the following to the mount service in /etc/systemd/system: --read-only I do this for my media server.

[@mithubopensourcelab482]

Can you create a video tutorial on Minio Server ? I have installed minio server, but could not use it....

[@AnandsLab]

I would love to at some point. But honestly it is going to take a while until i finish all my basic docker videos.

[@Jarery2000]

in the video of the systemd script, there is a line that is --user-agent home-server \ but in the scripts on your github its --user-agent cDoc \ . is this an important variabkle and what should it be ?

[@AnandsLab]

It’s not important. You can pick any name

[@jayparton5584]

I am getting this response after the chown command. I have double checked the typing and its the same as yours except my username of course. chown: cannot read directory 'cache/lost+found': Permission denied

[@AnandsLab]

Why is there a lost+found inside your cache folder? Are you mounting a whole disk as your cache folder? If so, I would recommend against it. Create a folder called cache inside the disk. Alternatively, the lost+found is not so important. You can try with sudo.

[@ravndude]

@@AnandsLab i missed the options of deselecting backup and selecting skip replication. My mistake. I successfully completed this rclone method. thank you!!!!!

[@nadpul]

How about sshfs?

[@AnandsLab]

Yes. Rclone supports SFTP protocol and there sshfs should work with FUSE as described in the video.

[@nadpul]

@@AnandsLab the nice thing about sshfs is that you don’t need to deal with samba and add shares and users, it is same as ssh, you only need the credentials and you have access to the user’s directory. This is to clarify to your followers here. As for performance that might not be the best choice, but I am no expert.