[ Kube 73 ] Elastic Cloud on Kubernetes - Part 1

[@shrinidhiym6499]

finally, found the best one, the best, thank you Venkat🤩

[@justmeandopensource]

Hi Shri, Thanks for watching. Glad you found it useful.

[@Ak0tnik]

This guide helped me a lot. I was so impatient on why ECK wasn't working (besides the fact I needed to create the PersistentVolume before starting) and seeing your steps on at least getting Elasticsearch working, it worked. Now I feel confident in going forward on the other steps.

[@justmeandopensource]

Glad to hear that. Thanks for watching.

[@Carlos1979Mad]

Subscribed in 3, 2 1 Great content and great teacher. Thanks a lot.

[@justmeandopensource]

Thanks for watching and subscribing Carlos.

[@sunils5834]

Thank you bro! as always clear to the point. was very helpful to understand the basics

[@justmeandopensource]

Hi Sunil, thanks for watching.

[@umartariq8365]

Good topic and very well demonstrated.

[@justmeandopensource]

Hi Umar, thanks for watching.

[@JoshSmeda]

Really great video. Thanks for uploading!

[@justmeandopensource]

Hi Joshua, thanks for watching.

[@BiohaZd5]

Brilliant video as always, thanks very much.

[@justmeandopensource]

Hi Andy, thanks for watching.

[@jayanitatiparthi9228]

Hi Venkat, nice explanation, more informative

[@justmeandopensource]

Hi Jayani, thanks for watching. Cheers.

[@omishagupta1428]

This was helpful, thanks. Would really appreciate if you can help me understand: 1 - Versioning in the files being used for deploying 2 - Adding more advanced options in the manifest 3 - Adding custom plugins in this kinda setup.

[@ManojKumar-je6er]

Hey Venkat, I unblocked my adblocker, watched all the Ads before starting the video (usually i hit the skip button) just to support your channel. btw, thanks for this video, i am just thinking to install new operator

[@justmeandopensource]

Hi Manoj, many thanks for watching this video and so kind of you. Much appreciated.

[@madeeshafernando8496]

very informative video. Thanks a lot.

[@justmeandopensource]

Hi Madeesha, thanks for watching.

[@EthanLR]

helpful stuff, thanks

[@justmeandopensource]

Hi Ethan, thanks for watching. Cheers.

[@KamranJarayev]

Hello, this is very useful video i have ever met for elc. I am trying to install elc on my own k8s cluster. unfortunately i can not use any persistent volume for this setup. Temporarily i have created quickstart with emptyDir and it worked. Could you please share more detail information how to use local storage (PVC or StroageClass) is more suitable?

[@anilg7915]

Good one Venkat👍Tq.

[@justmeandopensource]

Hi Anil, thanks for watching. Cheers.

[@trixiemp890]

thank you sir

[@justmeandopensource]

You are welcome Archelle. Thanks for watching.

[@jaispiritco.ltd.490]

Thank's for that guide! I would like to ask how you specified the persistent volume? ECK provides a persistent volume claim without a storageClassName. And that setup results with the error: 'pod has unbound immediate PersistentVolumeClaims'. I tried a local storage and an NFS storage without success. I would be very pleased to receive an answer.

[@justmeandopensource]

Hi, thanks for watching. In this video, I used dynamic nfs provisioning as storage provisioner. And the storage class I have is set to default storage class. So any persistent volume claim will use my default storage class (nfs provisioning) to request a persistent volume. Cheers.

[@anupraj762]

Would you be able to show case Apache solar search engine instead of Elastic search engine? Thank you for show casing Elastic search on Kubernetes.

[@justmeandopensource]

Hi Anup, thanks for watching. Currently mu focus is on Kubernetes/AWS/Elastic Stack. I don't think I will have time to explore Apache Solar. Unless I have explored and understood a product, I won't normally do a video just for the sake of publishing video. I am sorry. But if I get some time to learn about it, I will definitely do a video on it. Thanks for your interest in my videos.

[@anupraj762]

@@justmeandopensource Appreciated your time and valuable response . Thank you sharing your knowledge and Appreciated your efforts involved in making videos . Thumbs up .

[@justmeandopensource]

No worries. You are welcome.

[@N-et6gb]

My curretn ELK(6.8v) setup is on VM's and now i am willing to migrate the setup to ECK. Can you help me with the document/video links for data and pipelines migration from old setup to ECK setup?

[@kirankumareai]

Great stuff! Can you point to any reference architecture or if you have anything that you can share and any best practices from your experience?

[@justmeandopensource]

HI Kiran, thanks for watching. I haven't been much into this lately. Just followed the official documentation.

[@automationlearner2253]

can i run all these operations inside lxc master node or i would need to redirect apiserver of k8s to my host machine ...and if i would need to redirect apiserver then please tell me how can i access kubectl commands from host machine

[@Ajay44461]

Hi Venkat- where can I find the helm charts for these kind: elasticsearch , kibana and APM server? Could you please help me finding this?

[@therus000]

what the way to use self signed certs? with this elastic

[@santroproin6969]

Hey Venkat Is this setup is good for production. And also the files of this cloud are too huge. And is it possible to create helm chart for this elastic cloud or not? Because i can see the files are too large

[@justmeandopensource]

Hi Sandeep, thanks for watching. Which files are you referring to that you believe are huge?

[@santroproin6969]

@@justmeandopensource i mean official files which we can edit via kubectl edit cmd. Which you mentioned in your videos For operator, service, deployment etc

[@justmeandopensource]

So huge in the sense too many lines and not in size

[@santroproin6969]

@@justmeandopensource yes in the sense of line

[@justmeandopensource]

You can't do anything about it. Those are official manifests that elastic.co maintains. There should be helm charts as well maintained by them. They will equally be big.

[@Martin-pd6mc]

Venkat, one question please what shell are you using? Fish? Thanks!

[@justmeandopensource]

Hi Martin, thanks for watching. I use Zsh and oh-my-zsh on top of it with zsh-autosuggestions plugin for auto command completion from my history. Just search in KZbin for "just me terminal setup" and you will find a video in my channel for that. Cheers.

[@Martin-pd6mc]

@@justmeandopensource Nice video!! I copied your setup :) One question though. All this works locally, but what when you connect to other servers using ssh? you get the setup that the user has on the server you connected. How you deal with that?

[@Ashokkumar-uz1st]

Hi Venkat, thanks for this videos. Its more useful. Is the elastic cloud an opensource? is there any pricing for setting up this elastic cloud in my cluster ?

[@justmeandopensource]

Hi Ashok, Yes, it is.. But if you want some advanced features like security and other stuff you can go for paid version.

[@Ashokkumar-uz1st]

@@justmeandopensource thanks for ur valuable reply Venkat. So without xpack security features or TLS certificate I can use elastic cloud for free?

[@ashwathmendan732]

Thank you sir, I want to know how can I use my own SSL certificate instead of self signed? Any docs

[@justmeandopensource]

Hi Ashwath, thanks for watching. Its definitely possible to use your own SSL certificates. I haven't tried it. Please follow the documentation from Elastic cloud. I will let you know if get a chance to try this in my environment. Cheers.

[@alixak4304]

It is possible to disable tls for kibana in the custom ressource if you are doing tls over the ingress controller : spec: http: tls: selfSignedCertificate: disabled: true

[@nagendrareddybandi1710]

HI Sir, Nice stuff... Any news on Openshift from you? :)

[@justmeandopensource]

Hi Nagendra, thanks for watching. I wish to start Openshift series but not having enough time. I will try my best. Thanks for your interest.

[@adonaik8s]

Congrats Venkat.... very nice and didact.... but lack APMServer... what do you think teach us? :-)

[@justmeandopensource]

Hi Adonai, thanks for watching. I will look into it. Cheers.

[@blackpearl1903]

Hi bro, how can I get the elastic search endpoint here? To pass it to my application.

[@justmeandopensource]

Hi, thanks for watching. Starting at 11:00 after creating the elasticsearch deployment, you can see the service quickstart-es-http which is of type clusterip. Just edit this service and change it to loadbalancer if you have load balancing implemented in your cluster or just use nodeport. Then you will be able to access it outside of the cluster through that nodeport on any of the worker node. If you want to access the elasticsearch endpoint from an application within your kubernetes cluster, you could just use internal dns name for the elasticsearch service quickstart-es-http:9200

[@naganaga3731]

Thanks venkat, when part2 will be release

[@justmeandopensource]

Hi Naga, thanks for watching. Part 2 will be released next Monday. Cheers.

[@naganaga3731]

@@justmeandopensource we have to wait one week can u release as early as possible waiting to see

[@justmeandopensource]

@@naganaga3731 My release schedule is one per week. I can share the video privately with you if you can't wait and don't mind sharing your email address.

[@naganaga3731]

@@justmeandopensource [e-mail]

[@justmeandopensource]

@@naganaga3731 Shared. See if you can access it.

[@MrDjegsi]

Nice video , but l am stuck in the beginning :) pod/quickstart-es-default-0 0/1 Pending 0 6m28s ??? any help

[@justmeandopensource]

Hi Lulzim, thanks for watching. Can you investigate why the elasticsearch pod is in pending state? You can check the output of kubectl describe deploy or kubectl get events. May be you don't have enough resources (memory/cpu) in your kubernetes cluster.

[@MrDjegsi]

@@justmeandopensource l have this from description: Warning FailedScheduling 4m27s (x36 over 50m) default-scheduler 0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims.

[@justmeandopensource]

@@MrDjegsi Ah okay so your pods are waiting for persistent volumes. Don't you have dynamic volume provisioning in your cluster? I have used dynamic nfs volume provisioning in this video.

[@MrDjegsi]

@@justmeandopensource no i did not , l just created some pv and pvc: task-pv-claim Bound task-pv-volume 10Gi RWO manual 10m task-pv-volume 10Gi RWO Retain Bound default/task-pv-claim manual 11m

[@justmeandopensource]

@@MrDjegsi When the elasticsearch statefulset pods are in pending state, can you paste the output of $ kubectl get pv,pvc

[@amleshkumar8150]

Thanks for this awesome video...but i am unable to assign persistent volume and claim in Operator deployment.. can you please help me to identify in which file do i need to make changes... Please help.. i have stuck in this part your support in this will be really helpful..

[@vigneshk5520]

Hi Amlesh, Can you able to identify how to overcome this?

[@KapilKumar-mk1zc]

Hi Amlesh, Are you able to through with the same, as I am also stuck in the same

[@spawar1841]

bro it's not working i tried so many times

[@justmeandopensource]

Thanks for watching and reporting. I will give it a try soon and if its really broken, I will do a follow up video. Please follow my channel to get notification. I might forget to let you know about the follow up video as I am inundated with comments on various videos.

[@drggfish9482]

Hello - I see that most of the comments are old but I just started working through this and am having a similar problem that many others had and I don't see any resolution. I set up dynamic volume provisioning in my cluster based on one of your previous videos. I am seeing the same errors on the pod by using the describe command "Warning FailedScheduling 4m40s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims." I am following the Elastic Cloud on Kubernetes site but it is not too clear. It says the name of the volume claim must always be elasticsearch-data. apiVersion: v1 kind: PersistentVolumeClaim metadata: name: elasticsearch-data spec: storageClassName: managed-nfs-storage accessModes: - ReadWriteMany resources: requests: storage: 1Gi root@kmaster:/play/kubernetes/yamls/nfs-provisioner# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE elasticsearch-data Pending managed-nfs-storage 7s What am I missing? Thanks!