Installing and Configuring Logstash to Ingest Fortinet Syslogs

#elasticsearch #kibana #logstash #fortigate
In this video, we install and configure Logstash to receive Syslogs from FortiGate, parse them, and send them to Elasticsearch.
Watch how I installed and configured Elasticsearch and Kibana:
• Installing and Configu...
Thank you for watching!
Follow my Twitter: / ayounes9
Follow my Blog: www.thelionping.com/

[@chokfulla]

This helped a LOT. Thank you, sir.

[@youk1824]

High quality content :) Keep up the good work +1 sub

[@davidraymond7420]

Nice one, continue good work

[@pmorenos]

Muy buen trabajo, te felicito 👏

[@tobiashelbing1233]

Great! Many thanks for this Video

[@MrSalFav]

This is very helpful thank you !!

[@AliYounesGo4IT]

Glad it was helpful!

[@moinakbhattacharya878]

Great video

[@TeenaKohli]

Thank you for the information video, can you suggest if in case i have multiple pods running and i am capturing logs of all application pods on one common persistent storage mounted /var/log what could be possible input string

[@tiagoolv5115]

Perfect. Thanks!

[@moe-bash]

thank u. it was Grate and useful

[@bilelbenzerafa253]

كل الشكر و التقدير على هدا العمل الجميل شكرا اخي

[@seniortaco100]

Hello Ali - @ 9:18 - how do you know that is the format? can you explain a bit more how you identify such a format?

[@zaylinhtun3423]

What can I do if I don’t want to define the hosts for fortinet? because I want to send the logs from multiple fortinet FW Can I remove hosts line?

[@user-rv4ib2qs5g]

HI Ali , i have FortiAnalyzer and want to ingest the log to security onion , could i send it directly or should i have syslog vm between FAZ and security onion . what is the best scenario to do this . thanks for your help

[@user-og8hr7vr9h]

I would like to see data from DB being pipelined with logstash and loaded into Elasticsearch 8.2.2

[@muhammedajsal1816]

Thank you Can you please do cisco routers and switches integration

[@silentreader8426]

which one is better depends on performance, send log via filebeat or via logstash?

[@ati43888]

Very nice. Thank you. Esselamu Aleykum.

[@aminazgol3918]

Wonderful, thank you sir very informative

[@iswariafala898]

how we can get the log from more thaan 1 network device forti ?

[@samiyamusthafa]

logs are not coming to elastic search from syslog server. we done as "enroll in fleet" way? can u pls help me with this issue ?

[@anatchaisamretsin8770]

thanks a lot bro. it very useful. Can you explain more about create index for firewall log?

[@AliYounesGo4IT]

Will work on that, thanks!

[@UntungRaharjo]

Hi Ali, thank you for this video. Right now, we are facing issue to integrate log of Fortigate using Logstash+Elasticsearch+Kibana. We have an issue with the grok from Logstash to create indexing in Elasticsearch.

[@AliYounesGo4IT]

What is the issue?

[@seniortaco100]

Nice Vid Ali Can you make this Elastic, Kibana, Logstash Videos for Windows Server 2019/2022 or Windows 10?; and perhaps ingest data from an SQL Database server; if at all possible. Thank You.

[@SnakeFredy]

Hi Ali. Thank´s for the video. At this moment I am getting an error which does not allow me to continue. The port is occupied by java, therefore when starting the logstash it tells me "The address is already being used (bind) port: 5144". (Address already in use - bind)

[@AliYounesGo4IT]

I think that happened to me once, Logstash was already running, maybe try to stop all Logstash processes and try again (run: ps -aux | grep logstash)

[@seniortaco100]

Nice Vid Ali - very helpful - Question - can Logstash conect securely to Elasticsearch using an API key instead of sending a username and password thru the .conf file? perhaps configuring such API key in the logstash.yml or placing it in the filexyz.conf file used to connect to elasticsearch. Generate the API key in Kibana with logstash-user role and index permissions to publish such logs from logstash into Elasticsearch. Thanks for your previous response to making such video series in Windows servers. - Cheers

[@dv7045]

Hi, it is possible to write SNMP input plugin or it is hard work?

[@AliYounesGo4IT]

I will try that and make a video!

[@Hunti21]

i have the error: Failed to perform request {:message=>"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"

[@deepti_priya_panigrahy]

is the issue fixed? I am also getting the same error

[@Wolfhound_81]

Why is that required? I'm new to elastic but I understand from their documents that they have Fortinet integration so I would expect them to be able to parse that without me doing all that stuff manually?

[@AliYounesGo4IT]

That's right, you can use the Fortinet integration and it does a great job. This is a demo on how to install Logstash, you can also use it to ingest and enrich the Fortinet logs with other data from other sources.

[@sleba96]

Can you pls help whats the rest of the filter in the fortigate.conf? I cant see the whole code. mutate{ remove_field => ["@timestamp", "host", "@version",................................

[@abdenourguellati3124]

you can use this remove_field => ["@timestamp", "path", "host", "@version", "log", "event"]

[@sleba96]

@@abdenourguellati3124 thank you so much!

[@user-nz2tg7bb4b]

Hi, This is very helpful thank you but It would be much better if you could upload the config file as well.

[@dinhhoangvietminh]

Could you demo how to use fortigate agent on ELK?

[@AliYounesGo4IT]

I will do that soon!

[@ToadRash-mu3ln]

Great video, tnx! But why you use this primitive Putty instead let's say... MobaXTerm or another handy client?

[@AliYounesGo4IT]

I'm glad you like the video! Putty works Ok, I will look into MobaXTerm!

[@jameseduard2092]

can you check and work with opnsense as well?

[@AliYounesGo4IT]

I will look into it!

[@jameseduard2092]

@@AliYounesGo4IT ok great thanks sir, do you have linkedin'?

[@LeyviGarcia]

Hello, thank you for the tutorial, it was fine until I had to start the logstash service, from then on I no longer received the logs, in my case from a Cisco switch, even though I also gave permissions to the certificate and I still do not receive the logs. I runt again the command /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/cisco.conf -r and there I receive the logs , I don't know what I did wrong, but it doesn't work normally for me

[@LeyviGarcia]

Sorry I had port 514, I watched the video again to listen to your comments and I solved it, thanks

[@jamaaberdihna1076]

When I run logstash I get this error no configuration found in the configured source

[@AliYounesGo4IT]

I can think of the path to the .conf file, make sure that it is the full path, example: /etc/logstash/conf.d/example.conf

[@jamaaberdihna1076]

I resolve this problem how u generate logs with firewall my firewall is empty

[@AliYounesGo4IT]

@@jamaaberdihna1076 I setup Syslog on the firewall to send to a specific UDP port on Logstash server, and on Logstash I open that port. It should start sending

[@ryaddraou851]

Hi man, thanks for the video it's excellent but can you pls give me more explanation about the command in min 7:30 I didn't get it

[@AliYounesGo4IT]

Thank you! The command is starting Logstash from the source binary /usr/share/logstash/bin/logstash and then specifying the configuration file with -f that tells Logstash where to get the data from (input plugin), filters the events (filter plugin) and send to an Elasticsearch output (output plugin)

[@eytest1555]

match => {"message" => "%{SYSLOG5424PRI}%{GREEDY$} is truncated from the video, the window size is hide the rest.

[@AliYounesGo4IT]

match => {"message" => "%{SYSLOG5424PRI}%{GREEDYDATA:message}" }

[@Lukaszorr]

@@AliYounesGo4IT Could You please tell me what is in the mutate{ remove_field as it is not shown on video? ( ["@timestamp","host","@version"......... Thanks!

[@abdenourguellati3124]

@@Lukaszorr remove_field => ["@timestamp", "path", "host", "@version", "log", "event"]

[@RobertoRamirez-wh3zm]

Hello Ali, I'm new to ELK and wanted to ask if you've ran into the problem I am encountering or if you can provide guidance. I am experiencing the issue you encountered at the end of the video where the logstash service stops and stops presenting data on Kibana. I've attempted to restart the logstash services and modify permissions for logstash, but I continue to encounter an issue. I ran a journalctl with logstash.service and notice the error below. Would you be able to provide guidance on this? [INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"/etc/logstash/conf.d/*"} [ERROR][logstash.config.sourceloader] No configuration found in the configured sources.

[@AliYounesGo4IT]

I never encountered this error, but can you double check the pipelines.yml file if it has the path to the conf file, and that there are no syntax errors in the file as well Hope that helps

[@RobertoRamirez-wh3zm]

@@AliYounesGo4IT Thank you! I was missing the ca path and fingerprint for Logstash. All good now!

[@AhmedEid-lr6xe]

hi Roberto i face the same issue could you provide me how to solved it