This is why I like docker-compose, because it's easy to recreate containers and is practically self-documenting
@AwesomeOpenSource2 жыл бұрын
Absolutely! Compose is a tremendous tool.
@default_youtube_profile2 жыл бұрын
But docker-compose is not docker swarm
@froid_san2 жыл бұрын
As someone still learning the in's and out's of docker, this is a very great tip. Luckily since I'm still a newbie, I've been doing all things docker using docker compose and always saving it as I'm afraid to mess up. Now I must keep in mind the data files of the containers to backup. Great vid!
@AwesomeOpenSource2 жыл бұрын
Thank you, and it sounds like you’re on the right track.
@microcisco Жыл бұрын
Great channel and love your videos! I know most people don't have the infrastructure to be able to run a hyper-converged setup, but if you have 3 physical machines and SSD disks to run a Ceph storage (storage as a service - StaaS) on 10Gb interfaces. I run a Proxmox 3 physical server setup with the 3 in a cluster and Ceph Storage to form the HCI. I run the Proxmox backups nightly and it has saved my ass a few times and easily restore the backups to any of the 3 in the cluster with all attached to the Ceph storage. If you can setup the HCI environment it will give you peace of mind and the functionality to easily restore failed VM's and containers. (As a side note, I used to run LXC containers but I find it much easier to run VM's with docker on those VM's.)
@AwesomeOpenSource Жыл бұрын
Great information and advice. Thanks for sharing.
@MrNoBSgiven2 жыл бұрын
Great topic. 1. Backups is a must. Snapshots of VM would have helped you recover in no time at all 2. . Portainer has a backup function,mwhich backs up the entire setup and allows recovery to a new machine in no time at all. Finally, have you tought of automation of the backup and recovery/seting up using terraform (proxmox VMs) and ansible for the "fresh rebuild" of your VM environments? THESE WITH distributed file system storage could help. I am in the process of setting something like that. I am thinking about using docker swarm to take away the headache of manual splitting of docker apps.
@AwesomeOpenSource2 жыл бұрын
Agree 100% on the backups. And i do, now, have a backup of my VM, and it's automated as well. The docker part is an interesting one. I did find the Protainer backup (after the fact), but will start using it too...yay! As for Swarm, I'm not there yet, but was also thinking this would be a great reason to learn and start using kubernetes at some point. As it would have provided the no-down-time reliance I might want some day, while I'm re-building one server if needed.
@gravyflex2 жыл бұрын
I'm the middle of setting up a docker swarm built with terraform and Ansible. Working on NFS persistent storage with a docker plugins. Right now I am figuring out traefik reverse proxy.
@MmMm-oh5ct2 жыл бұрын
Its Awesome, I am on my way to Setup kind of something with ansible at the moment. Great Video
@AwesomeOpenSource2 жыл бұрын
Thank you.
@JPEaglesandKatz2 жыл бұрын
Also the way I've set it up is bind mount every single data folder from any container I use to a cental data folder... I backup this folder with borg backup and also create periodic toal VM backups, including syncing them to an external drive as oposed to only my zfs pool.
@AwesomeOpenSource2 жыл бұрын
Nice. Great tip!
@JPEaglesandKatz2 жыл бұрын
checkout borg backup.. its great.. easy to put into a cron job, backup up root owned folders, incrementalized backups, compression, protection, etc... set and forget once setup correctly
@AwesomeOpenSource2 жыл бұрын
I've looked at Borg a few times, just haven't tried it yet. It's on my list though.
@JPEaglesandKatz2 жыл бұрын
@@AwesomeOpenSource Cool... Yeh it is pretty good... Keep on rocking!! I love your channel, came across some really cool pieces of software that way... I was a linux noob 8 months ago... now it is starting to feel more like home than windows!
@rockettpc22 жыл бұрын
Great info, funny, I kinda intuitively started like this, but I find myself doing the same in my individual devices, so it was kinda the norm for me. But this is very important to do, stay organized and have backups etc. I am playing with docker swarm on a 6 Node Odroid C2 cluster and I'm using NFS for the data, I really need to build a good NAS ideally more than one for redundancy of course, but it's all allot of fun self hosting and pushing the edge of home computing.
@AwesomeOpenSource2 жыл бұрын
Keep it going, and glad you are already on track with organization!
@artedwards717 Жыл бұрын
Hey brain awesome I wish you could make a drive with that all on it I am learning wow on your videos
@AwesomeOpenSource Жыл бұрын
What do you mean by a drive?
@artedwards717 Жыл бұрын
@@AwesomeOpenSource oh brain I mean like a CD to load on my main computer
@CaptZenPetabyte2 жыл бұрын
Great idea as Im about to start a new project & setting it up well like this from scratch will be so much better. Could prob even write a bash.sh to copy & bring everything back up again to automate
@AwesomeOpenSource2 жыл бұрын
Better to start off organized, but worth going back and re-organizing as well!
@kronos34282 жыл бұрын
Thanks for the tutorial. This has always been a big concern of mine, I like to do things just once. I recently started using TrueNas Scale which is debian based and has docker support and a very interesting thing I found out is that you can change docker root folder, so I have this disk that when attaching it to any other instance, the whole date the containers are there and also the containers, they even start automatically (following the rules previously assigned).
@AwesomeOpenSource2 жыл бұрын
Nice setup.
@majidkhoram24512 жыл бұрын
Did you know you are brilliant? Or should I mention it?😉👍🏻
@AwesomeOpenSource2 жыл бұрын
I appreciate the compliment, thank you!
@kevinhughes98012 жыл бұрын
Great stuff thanks
@AwesomeOpenSource2 жыл бұрын
My pleasure
@50_Pence Жыл бұрын
Did that mariadb backup work OK? Or did it corrupt? Did you have to stop the container first?
@AwesomeOpenSource Жыл бұрын
If you have a system that writes to MariaDB a lot, then it is better to stop the container, then do the backup, and start it back up. I've been lucky so far, and my containers are generally low usage, so when the automatic backups run, no one in my home is up and using anything.
@SaroxDuPoney2 жыл бұрын
Hello thanks for your tips. What is your container for your dashboard @16:07. I like it a lot.
@AwesomeOpenSource2 жыл бұрын
Currently I'm using Homer, but just recorded a new video on Dashy, and I'm thinking about switching to it. I need to test it for another week or so before I decide. Video coming soon!
@SaroxDuPoney2 жыл бұрын
@@AwesomeOpenSource ok thanks. I will be looking for your next video. Thank you very much for all your info.
@markstanchin16922 жыл бұрын
Ok i’m in trouble. So I installed a bunch of docker containers using different methods portainer, docker compose etc. When I ls under my home directory I only see two dirs , Heimdall and guacamole soo I thought when you installed a docker container it installed and created the Paths like in your example automatically. There installed and working perfectly but I have no idea where the dockers Containers and data is installed by default. So without redoing like 15 containers is there a way I can find out we organize and regroup the way that you’re showing in this tutorial? Thanks
@AwesomeOpenSource2 жыл бұрын
When I look at my file system, I find Portainer stuff under /portainer/Files/appdata/ , so you may start there. There is also stuff under /data/compose. You can use the Portainer UI to backup your docker stuff as well. But if you want to organize it, you'll need to move stuff from the file paths above, and then start cleaning it up. Do one at a time, take your time, and make sure each app comes up as you expect.
@markstanchin16922 жыл бұрын
@@AwesomeOpenSource Thanks! Are you going to create a video on how you actually do this and how you back up your folders zip up script etc? Also I was thinking of moving data from the containers over to a Nas device not sure if it makes sense or how it would work have you ever done anything like this? Also off-topic I host a couple of websites not WordPress on site ground. Well the renewal is coming up and it’s four times my original price. So I’m thinking of moving to self hosting my sites but I like the way site ground works with the control panel. Is there anything that can be installed on a Ubuntu vm server that can act as a control portal like cPanel? I’m thinking of creating a separate VM just to host my websites does that make sense?
@lezz272 жыл бұрын
Nice video. I just had a crash and I know the pain in getting everything up and running. I have a mic of Windows and Linux machine. My Docker stuff runs on Ubuntu 20.04 where as I have shares on my Windows machine. I was wondering whats the best way to backup the docker files onto the Windows share. You mentioned something abt the script you use to zip and upload to a share. May you please share the script? I tried using Duplicati and Duplicacy on my Windows machine and then mounted the Docker volume on Windows machine using "Mountain Duck" application. The issue that I am facing is that there are some parts of the Docker folder that I am not able to backup using these backup apps due to permission issue. I am not sure why I have that. I tried to grant my local user on Ubuntu a root access, that didn't work either. Any help/suggestion in this matter will be appreciated. Note: All my shared folders are on my Windows machine.
@AwesomeOpenSource2 жыл бұрын
Couple of things to look into. 1. make sure your user on ubuntu is part of the docker group. sudo usermod -aG docker {$USER} 2. Setup a Samba share, and make sure you mount it on your Ubuntu server. 3. Now you can zip your docker folder ( you may have to use sudo ), then copy that folder to your mounted Samba share. When doing this, you'll want to make sure and delete versions over time so you don't fill up your drive on the Ubuntu machine. The script i used, 1. created a gzip file of my docker folder, then 2. rsync'd the file over to my backup server, and then 3. deleted the gzip file. So each day the files were zipped, backed up, and then the zip was removed. It was a very simple script, but effective.
@lezz272 жыл бұрын
@@AwesomeOpenSource This is the quickest response I have every seen in YT. Thank you! I am bad with writing scripts. Would you mind sharing your script? I did setup my shares on windows with required permissions and currently trying to figure out how to mount the Windows SMB shares on Ubuntu server.
@markstanchin16922 жыл бұрын
Hi, I just ran your latest script, how cool! I’m rebuilding my system in a new VM and docker etc. trying to get organized the way you show in this video. The script helped. A Couple of things I noticed it does not create a docker folder for the containers it installs. Also it says it installed Portainer agent but it’s not listed under the containers with port 9001. Also the stacks that are created say they have Limited management under Portainer. Any thoughts? Will you be adding other applications to the script? Also when I click on a port from a container it takes me 0.0.0 local host I remember somewhere how to set this to the IP but I can’t find it. Thanks ! now I’m back to work .
@AwesomeOpenSource2 жыл бұрын
Yeah, I need to check for or create the Docker folder, not am not doing it today. Not sure why the Portainer container isn’t shown if it installed properly. The IP is set from your main Portainer Home, then I believe under Environments. I’m considering more options for containers, but at some point I’m just duplicating what Portainer can do in the UI. As for the limited management, I thinks that’s what it always says for Docker-compose created outside of the Portainer UI.
@dimitristsoutsouras27122 жыл бұрын
10.45 Pleaseeeeeeeeeeeeeeeeee pass the link of that procedure. If it is what I think, I am looking for this for months. My use case scenario is..... in a win2019 vm which acts as n A.D and have been installed on a raid10 zfs storage (storage derived from the same server the VM is installed)I have attached an extra portion of another raid10 zfs (derived from the same server as well).....what I left thinking afterwars is... if I migrate the VM what happens with that extra disk I have attached to that VM since it is not a shared folder or iscsi... it was just a block device that windows format and used. I m stuck there for ages!!!
@AwesomeOpenSource2 жыл бұрын
Instead of a script, I've found a great container called Duplicati and just recorded a video on it, as well as how to use it to backup your docker folders (or anything else) on a nightly schedule. It's super great. Coming soon!
@dimitristsoutsouras27122 жыл бұрын
@@AwesomeOpenSource I already know about Duplicati and UrBackup and other programs as well. I have to wait for your video and see if it's use case scenario can be applied to me too. PS By the way for me it would be useful a program that can take a block level backup and not file level.
@AwesomeOpenSource2 жыл бұрын
@@dimitristsoutsouras2712 Timeshift on Linux can do Rsync backups / snapshots, and does diffs between the files, instead of complete files. Not block level exactly, but may help. I believe clonezilla can do block level, but you have to boot into Clonezilla to do it, or run it as a VM.
@dimitristsoutsouras27122 жыл бұрын
@@AwesomeOpenSource Thank you for your answers. ..... and that is exactly what I do. I use rescuezilla (even better than Clonezilla) and backup each disk (it is a mirror), do the upgrades or whatever I need to and then if anything goes very wrong, I revert back. Problem is (as you mentioned) I have to shutdown the server boot from usb stick make 2 backups (25min each on a 120gb disk - for the OS) and takes time for a production system.
@AwesomeOpenSource2 жыл бұрын
@@dimitristsoutsouras2712 is this for Windows or Linux?