Redis: How to setup a cluster - for beginners

[@Redisinc]

Fantastic work, couldn't have put it better myself!

[@zinobi]

This is a really good overview-video for someoone like me who is kind of used to redis and Docker and does not want a 2 hour introduction on how to install Docker and write a hello-world in Qbasic before getting to the relevant stuff.

[@alfa-being]

Thank you so much always learning something from this channel...waiting for "kafka Cluster on Kubernetes".

[@elibukin4840]

my dude.... you are officially awesome!

[@mahmoudalaskalany]

most valuable video on clustering for redis thanks but i have a tiny question how to connect automatically to the new elected master if the main one fails

[@Dobby_zuul]

This video was absolutely stellar

[@et4493]

Outstanding job man! I'm all in

[@devtech7830]

Very Good content. How can I do this as a service on Docker swarm?

[@dila2219]

+++

[@ragnadrok7]

I have a doubt at this point, how to configure client to switch to new master on the fly without manual config ?

[@MrApalazuelos]

Thank you so much for this video! I learned a lot. Just one question how do you know what's the new master after the failover?? Thanks

[@ApodyktycznyCzlek]

12:00 if you came here for Redis Sentinel :P

[@rajeevkalangi2472]

Wow... Superb bro... Very crisp and clear... Thank You...

[@andresantana9148]

Hey, great video, but I have a question. If services provides loadbalancer, how can I ensure my externall app will always hit the master for RW operations?

[@linovieira9739]

Great work! Thank you very much. I just have a quick question: How do you suggest running one container of Redis Cluster and Redis Sentinel across three different hosts?

[@wisdom514]

Thanks for the information video

[@harshakkrishnaa2014]

Hey great video, I tried this for the new versions, and when I delete the redis-0, my sentienl get this error "Failed to resolve hostname 'redis-0'". Is there anything I need to change in the sentinel config? I also tried announcing the ips

[@samuelvicent5034]

Thank you so much, really helpful for understanding the fundamentals! just to mention that I added to the redis-1 and redis-2 configuration the following parameters for those replicas to work as "master" (writable): replicaof redis-0 6379 and replica-read-only no. Very appreciated for your tutorial!

[@NeilMcFadyen-r2z]

how would this work to run the redis containers on separate machines, do you need to use docker swarm?

[@RahulSharmaSingularity]

Brilliant !

[@swarajlaha6778]

How can we implement cross kuberantes cluster replication in the sentinel mode?

[@moma2189]

Awesome 👌. I think it is time for ELK and Graylog :)

[@techbeauty205]

- so how will our application detect the failover and change the connection to new master automatically? - and what if one of sentinels go down?

[@brianhernandez2919]

Great video!!.. question for you:i see you shut down master, there is anyway to switch in the app to new master without manual modification?, imagine you have a web app and you don't want this process be transparent!! thx

[@jadeedstoresupport8916]

Sir, you have this gift of communicating in a way that I find very easy to understand. Many thanks. Would be great if you could produce content pertaining to MySQL as well (like replication, sharding etc.) Thanks

[@sijaanhallak5279]

Too bad I didn't discover this channel a long time ago. Can I ask about your background?

[@nirmaln]

Thank you so much, as always i find lot of information in your videos. Quick question - In a production env, how do you prefer the client application to detect the current master? Do you recommend connecting to any one of the Sentinel to query the current master or put a proxy server in front of the Redis cluster to make the master switch transparent to the client application. Thanks

[@MarcelDempers]

Thanks for the kind words 💪🏽Check your client library and ensure it has sentinel support. Your client library should allow you to query the sentinel to discover the current master

[@varunmb3310]

Very good tutorial. Able understand easily with the github code.

[@attrib75]

Great video, thanks for doing this work. It would be great if you could create a video that talks about partitioning data across Redis instances, and the implications of doing so. Particularly insofar as, can you achieve clustering for redundancy as you show here, but also for partitioning?

[@MarcelDempers]

Thanks! Sure you can definitely do sharding as well as high availability. Sharding is very powerful in Redis. I have not had a chance to play with it yet 💪🏽

[@tornoutlaw]

I have a hard time setting up a daemonized/dockerized redis instance with the sysctl-param vm.overcommit=1. As far as I understand, sysctl-commands have to be executed on the host-system, therefore cannot be put into the dockerfile. Does anyone have a good example how to do it with a docker-compose.yml, if possible? I would like to avoid additional .sh files.

[@raphaelquivel6399]

I had to use in a newer version replicaof instead of salveof and I had to remove the bind directive from the redis config so it will bind to all ips

[@budi0580]

Your video really help me. I am on mission to setup redis cluster, rabbit cluster and postgres cluster for production, would be very nice if you can have video about postgres cluster as well 🙏

[@Shubham__Saroj]

Tremendous!😃

[@shashankshekhar7839]

Thank you for sharing such a great video. Waiting for more such videos... :)

[@predcr]

How to get all this content into a vs code editor. Pls explain that

[@GruffGuy]

Thanks. It was useful for me. Does anyone know, If I dont use docker then I need to install a new redis instances manually on a different ports (ofc If it's local). So for example, my master will be 6379 redis and slaves 6380, 6381.

[@zoom0211]

Thanks a lot!

[@loftkey]

Thank you so much, your explanation really helped me!!

[@lokmanhossen7354]

Thank you so much for your video, it's really helped a lot. But I have a question that is you changed the REDIS_HOST manually. Is there any way to find the new master automatically without restarting the application ?

[@MarcelDempers]

Yes, you need a sentinel aware library. Checkout one if my new uploads on Go with Redis

[@sberthu]

thank you for your explanation, but don't work for me with your scripts. I have "Running in standalone mode" and no cluster feature seems to be available. What about cluster-enabled yes, cluster create, etc... ? Thank you for your reply. Sylvain

[@akashshukla177]

What if we Set appendonly yes And we have 1 cluster of 3 master and 3 slave that mean we've 3 shard. After adding data into one shard that shard got down. . . I've tried this and my hole cluster stop working, I want to know why that happened Why consistent hashing won't works or hash slot won't shift

[@Stanoweb]

Hi, I thank you for your tutorial very very very helpful :-) A quick question, is it bad to use Redis Sentinel on the same Redis host (for each instance, let's say I have 3 redis servers)?

[@MarcelDempers]

Thanks! 🙏🏽 It's quite common for folks to pair a sentinel with a redis host on the same server, no problem with that, as long as you maintain quorum among the sentinels (I.E majority to vote on leader election), you're good 💪🏽

[@seven-life]

Thank you for sharing such a great video. I have a question here, why you use slaveof but no replacof ? The redis will use replaceof to take replace of slaveof after 5.0.0 version, is no right?

[@MarcelDempers]

Thank you💪🏽 As far as I am aware, you initially set the `slaveof` value, and when the sentinels take over, they will automatically set `replicaof` fields and manage it based on availability and status I am not aware of >5.0.0 and would need to check the docs to confirm it

[@seven-life]

@@MarcelDempers You are welcome. You can check this doc: redis.io/commands/slaveof

[@kumaranil181]

Could you please make one on Redis ACL and TLS ?

[@martinzen]

Great vid. How would this look in a production deployment though? Surely you wouldn't dedicate a separate box for each sentinel?

[@MarcelDempers]

Thank you 💪🏽This really depends, the sentinel can technically run anywhere as long as it has network connectivity and running as per availability zones and meets quorum requirements. you want to run more than 3 and you want to ensure they are distributed in case of host failure. As per docker best practices you always want to run only 1 process per container. I would not run redis + sentinel in the same container, but definitely consider running 1 redis container + 1 sentinel container per box as it may help having 1 sentinel local to the redis instance. And then distribute this across a few boxes. Hope that helps

[@martinzen]

That DevOps Guy thank you, that’s what I figured, looking forward to the follow up vid!

[@Oyedeloy]

I am a little bit confused about the choice of port 5000 in the sentinel.conf file. i thought sentinel will run on 26379.

[@MarcelDempers]

That's the defaults correct, you can override the ports for sentinel to use any desired ports

[@AskewTarantula]

So this is probably not the place to post a question for you but not sure where else. I followed these instructions and when I go to info replication on a replica it says master_link_status:down. Any ideas?

[@AskewTarantula]

Figured it out. Commented out bind 127.0.0.1 and now status is connected. This wasn't mentioned in the video.

[@itguymichal]

Like the video very informative. :)

[@thebluesclues2012]

great vid, redis clustering on kubernetes is a steep learning curve and Redis Enterprise is not viable for us, so it looks to me Sentinel is to be avoided and using an operator in K8s is the way to go but there seems to be too few mature OSS operators, are you going to do the K8S vid with an operator?

[@AnilRoyalJSp]

useful 🙂

[@amirbakz]

Can you make a video about the Amadeus operator and how to spin up a redis cluster with master/slaves on kubernetes using Amadeus operator ?

[@eriktomcsyk8483]

Does anyone know why I can't change my password? When I try to modify the "a-very-complex-password-here" section in the configuration files unfortunately the slaves can't connect to the master.

[@MarcelDempers]

Yes! A kind contributer has made a PR with explanation for the fix here github.com/marcel-dempers/docker-development-youtube-series/pull/62 Hope that helps 💪🏽

[@eriktomcsyk8483]

@@MarcelDempers Thank you very much your suggestion solved the issue. Greetings from Hungary!

[@MrVnelis]

As usual, you are my go-to channel for any topic related to kubernetes and devops. Thank you! PS: Can you make a tutorial on kafka?

[@geoffmahugu8983]

Quick question, is there an instance you would prefer running the Redis application on docker rather than a Kubernetes cluster? Awesome Tutorial Marcel🔥 👍🏾

[@MarcelDempers]

My first choice would be neither. I would choose a hosted offering first which avoids having to deal with all the HA engineering overhead. Very rarely would I use docker over k8s. Possibly if there is an existing infrastructure constraint that makes running k8s more difficult, like PCI compliance. I'll also use docker for learning, it helps me peel away k8s layers and just get to understanding Redis.

[@geoffmahugu8983]

@@MarcelDempers, well understood i'll keep an eye on the upcoming Redis Kubernetes deployment tutorial.

[@tkk554]

what if Redis run on different machine?

[@siulian]

Nice video regarding redis but unfortunately is not covering one major aspect. Application needs to know automatically who is master and use it instead of changing the code manually. How do you achieve this ?

[@MarcelDempers]

Your application needs to use an SDK that is sentinel aware. An example in Go is covered here kzbin.info/www/bejne/bJ2tdKyBrNSEl7M

[@studio99webagencysanmarino48]

I followed your "tutorial" and everything went smooth. Redis master replicates over the 2 replicas, and the 3 sentinels can see correctly the master and the 2 replicas. But something strange happens to me. If I connect with redis-cli on one of each sentinel and I run the command "SENTINEL FAILOVER mymaster" the failover procedure works correctly and a new master is elected, but if I put down the master redis server service, the 3 sentinels start the voting procedure but they vote just for themselves withour reaching the quorum. I even tried lower the quorum to 1 but they keep voting for just themselves and no slave is elected to master. So sentinel1 votes for redis1 (even if it's down), sentinel2 votes for redis2 and sentinel 3 votes for redis3. Why???? I'm losing sleep over this pal.

[@foobar1963]

You have massive arms :P

[@cdev4040]

My sentinel pod false after 1 or 2 days. I deleted it and it recreate and work perfect for the next 1-2 day and false again @MarcelDempers can you help me with this? 2023-05-29T09:07:03.359Z | 2023-05-29T09:07:03.359Z | *** FATAL CONFIG FILE ERROR (Redis 7.0.11) *** 2023-05-29T09:07:03.359Z | Reading the configuration file, at line 4 2023-05-29T09:07:03.359Z | >>> 'sentinel monitor mymaster 6379 2' 2023-05-29T09:07:03.359Z | Unrecognized sentinel configuration statement.