ELK using Docker Compose | Elasticsearch Logstash Kibana Tutorial

[@iranigroup7909]

best yml elk on youtube thank u brother !

[@Thetips4you]

You are welcome brother.

[@ahmedfayez]

Excellent, but how can you do it using TLS?

[@muaz-memis]

Thank you very much. Very helpful...

[@Thetips4you]

You are welcome. Happy to know.

[@imposter4778]

Sir how can we add host or remote machine like EC2 to our elk stack which is hosted on ec2 instance

[@VitaliKiSS]

my container of logstash keeps restarting... after i started docker-compose up command also when i enter the 5601 from the browser logstash says that it's missing security requirements...

[@Thetips4you]

Please check the container logs to see the reason and take action based on it.

[@dineshgupta01]

I have updated the logstash.conf file but that file was not found logstash container. I have .log & simple-logstashfile.conf along with logstash/logstash.conf. Plz help me out.

[@Thetips4you]

Did you started Logstash with the config you want to apply?

[@dineshgupta01]

@@Thetips4you Yes, all 3 docker started, but mounted directory not visible on the docker logstash server

[@moonchild23945]

@@dineshgupta01 Hello, do u find the answer?

[@moonchild23945]

I kind of find the solution to it First mount the volume in /usr/share/logstash/pipeline instead of randomly create directory. So can remove the command in the yaml file Second, in logstash.conf, change the hosts to {container-name}}:9200 and also add ssl_certificate_verification = false sample output in logstash.conf output { elasticsearch { hosts => ["my-elasticsearch:9200"] index => "index-name" user => "elastic" password => "password" ssl_certificate_verification => false } } ps: I am using 8.4.1 version

[@trollingday7124]

Great vídeo dude, if I want to send logs with other server to elk I only need to install filebeat and point to logstash and elastic for this docker right?

[@Thetips4you]

Yes. That is correct.

[@vladgursky149]

Thanks for good simple test example of elasticsearch in containers. ElasticSearch server started and accessed by browser without ssl cert. Logstash didn't accept log file even moved to logstash.

[@Thetips4you]

Can you check if the config is set correctly to send logstash to elastic search container.

[@hicnar]

I run into something similar, but in my case I tried the latest version which is 8.4.0. When I reverted to 7.16.2 as in the video tutorial... taaa daaa logstash can read the log file.

[@pastoralopez9284]

Thank you. I have tested in an Ubuntu VM and it works fine. I would like to collect Netflow v5 data through port 2055 for example. May I ask you for orientation of how to modify your docker compose file to add elastiflow?

[@Thetips4you]

You are welcome. Need to re-look on your request.

[@mohamadsaffarini4788]

Thank you for sharing..... if you're looking for the most recent version, typing ImageName:latest instead of version number.

[@Thetips4you]

It may not work with Elastic search, kibana . Try it and let me know. You will have to specify the version.

[@mehdimojaradi]

Thanks indeed for your efforts and sharing great contents. I am learning a lot from your channel.

[@Thetips4you]

Wow!. I am super happy to see your feedback. Thank you and keep learning.

[@agilebarsfromtimebarsltd.4918]

Thankyou this is awesome.

[@Thetips4you]

Thank you

[@lorenzozuluaga4309]

thank you for this awesome tutorial, maybe anyone knows how I can solve an issue where logstash is not authorized to reach the elasticsearch host? I was reading it's something related to the logstash output configuration, but I was wondering if I did something wrong, since on the video it works smoothly. Thanks in advance

[@Thetips4you]

You are going. Do you want to share the error message about authentication?

[@azzabenabid2669]

thank u for the video. i have to run the docker-compose file everytime i want to change the Logstash conf file?

[@Thetips4you]

No. If you have any changes in conf file you can update it, so by default the docker compose will take it.

[@azzabenabid2669]

@@Thetips4you i didn't found the index for Logstash in the index management. Can u help me ?

[@doublee7417]

@@azzabenabid2669 Me too

[@azzabenabid2669]

@@doublee7417 just add the elasticsearch url in logstash service part and the kibana service part

[@mohamedmazlin218]

@@azzabenabid2669 where ? in the yml file? can you give me details pleaaase

[@diegopalo4073]

Great tnx ! Can you maybe do it also for the latest one, elasticsearch:8.9.0 ? Because it is not working like this for that.

[@Thetips4you]

I will check

[@jeevan9057]

I am having issues. I am using windows and I followed the above steps in the video, But I don't see the logs getting reflected on Kibana

[@Thetips4you]

An error message?

[@jeevan9057]

@@Thetips4you i don't see any error message.

[@gc-104viri4]

thanks for your help

[@Thetips4you]

You are welcome

[@hicnar]

Good tutorial! Have you tried to upgrade the version of the components to the latest and run it? For me, when I tried the 8.4.0 it will all started up, but no index gets created almost as if the logstash chooses not to read from the log file (it detects the log though)

[@Thetips4you]

Not tried. Thanks for the information.

[@BandarAlmarri-zm2th]

good man keep going

[@Thetips4you]

Thank you man

[@hendisantika]

Can you update it into ELK Stack 8.14.1 (latest) vesion?

[@roya2045]

Hi Bro is this a cloud version of elastic stack you are showing?

[@Thetips4you]

No. Local installation.

[@roya2045]

@@Thetips4you Bro you have knowledge on how to setup snmp in logstash for monitoring network device on kibana/elastic search

[@eliumoraes]

Can you provide an example of how to setup logstash to receive from UDP or HTTP?

[@Thetips4you]

Let me look in to this.

[@Olegt0rr]

Please update this video to 8th version of ELK. Also your github link in description is not working :(

[@Thetips4you]

Sure. The github link is working for me. Please check it again.

[@chetanpatel1464]

Great tutorial. I was getting some memory issues in my machine which had 8gb but I'll use this to see if I can restrict it

[@Thetips4you]

Thank you.

[@angnguyen-vn2on]

I want to use opensearch logstash and opensearch-dashboard. is it similar?

[@Thetips4you]

Not sure. Opensearch is forked from elastic search as per my understanding.

[@angnguyen-vn2on]

​@@Thetips4you in the example, input file log must created inside container logstash or at the marchine host will also work

[@shalomig4411]

Great content, please where is the link to copy the content (compose file) from github?

[@Thetips4you]

Please check the video description.

[@ryaddraou851]

hi thank you for the video, I have a question pls if I get between 190 and 210GB of logs (firewalls and AD) how much CPU, storage and RAM do I need? and also, how can I setup my Logstash to receive logs from my Active directory Thank you for your help

[@Thetips4you]

Kindly check the elastic search hardware requirements and system requirements

[@adityendrapratapsingh7999]

What does version: '3.6' represent here?

[@Thetips4you]

Do you mean the docker compose version?

[@ImranKhan-fr8bn]

I have run the compose file its running from last 30 minutes, not sure when it will end , can you please let me know if this is the normal behavior, also my internet bandwidth is good and system has desired resources for setup.

[@Thetips4you]

Did you used -d with docker compose up command?

[@ImranKhan-fr8bn]

@@Thetips4you thanks its working now, could you please send me the link of any vide where the data is send to elasticsearch and visualize in kibana

[@p.hariprasadreddy6528]

@@ImranKhan-fr8bn is the content written in log file is showing in port

[@pellax]

You docker-compose doesn't work, logstash exited with code 0

[@Thetips4you]

Check the logs for exact error.

[@lesetjamaribe389]

super easy to follow, i like it!

[@Thetips4you]

Thank you

[@arnavsaraf6404]

Thanks for this video. Very helpful! :)

[@Thetips4you]

You are welcome.

[@EnitinEnitin]

Doesn't work. Kibana requires me to enter an "Enrollment token".

[@kennedyngare3224]

why is the index empty and the log file has content

[@mohamedmazlin218]

did u find a solution to that?

[@shohrabuddin9562]

@@mohamedmazlin218 I am having the same issue. The content of the index is 0. did you find a solution?

[@anilsaipinnelli1866]

where can i find the elasticsearch.yml file

[@Thetips4you]

It will be under elasticsearch home path, config. You can login to the container to check that or map it as a volume to control from outside m

[@p.hariprasadreddy6528]

its showing kibana is not yet ready

[@Thetips4you]

Check the docker logs on why it is not ready.

[@p.hariprasadreddy6528]

@@Thetips4you i checked them localahost:9200 and localhost:9600 showing output but this kibana is not showing output

[@haroldpepete]

why not use elk-docker image, it already implement all three technologies

[@Thetips4you]

Do you mean official elk docker image?

[@haroldpepete]

@@Thetips4you the same

[@FinancialIndependence2022]

Tried ok. Unless you have other tasks use up too many resources, it should be fine.

[@Thetips4you]

Ok

[@konstantinkaiser2383]

This doesn’t work on Mac M1 though?

[@Thetips4you]

I didn't tried in MaC

[@dancansikuku6760]

Great tutorial

[@Thetips4you]

Thank you

[@shahrzadkhorrami8542]

Thank you ❤

[@Thetips4you]

You are welcome

[@p.hariprasadreddy6528]

logstash | [2023-06-15T04:57:19,855][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"} I am getting error like this

[@Thetips4you]

Check if the ports are not blocked and configurations are correct.

[@p.hariprasadreddy6528]

@@Thetips4you every thing is fine but it not displaying the result sir

[@ivancalisaya4218]

Buen video 👍

[@Thetips4you]

Gracias

[@tailarsen6961]

Excuse me wheres part 2?

[@Thetips4you]

Please check the playlist

[@bhaskarreddy-bd7hz]

fake slang

[@Thetips4you]

You are the expert here. If you say so.

[@bhaskarreddy-bd7hz]

@@Thetips4you I can easily identify who is faking ! no need to fake slang ! Our own accent is beautiful and understood by everyone

[@Thetips4you]

Do you know me? Have you ever spoken to me directly to know who is faking? Personally I don't like to spend time talking to people like you, who don't respect others or always negative mindset.

[@bhaskarreddy-bd7hz]

@@Thetips4you your heart knows you are faking accent ! chill bro 😎

[@Thetips4you]

I think you need to grow up and think before giving feedback to someone. Everyone is having their freedom to share their view and this is yours. I don't care about it, as far as I think, it is not valuable feedback.