Everything you Always Wanted to Know about Filebeat * But Were Afraid to Ask

[@sriveralopez]

What a good speaker, explanations were clear and concise.

[@KleinKwakuFHouzin]

one of the greatest presentations I have seen. BRAVO

[@scottmccarthy3354]

I thought this was great, Until you missed that Nicholas Cage was Ghost Rider in a 2007 movie. (Or did you block that out because it was so bad?)

[@brijeshwani101]

Really great and detailed presentation. Very nice...

[@milequinze]

Excelente! Não só explicou muito bem, mas também explicou o que era fundamental. Grande aula.

[@pkhler4438]

can you please make a video on filebeat-cloudfoundry to logastash video

[@OfficialElasticCommunity]

Hey Pratik! I'd recommend asking about that on discuss.elastic.co or in our slack workspace-ela.st/slack

[@scottza]

Very great talk thank you!

[@richanigam1]

Hi Team, Can you please let me know how filebeat decides that under which Index , the particular document should go in Elastic Search. I am not able to find this answer.

[@bhaveshkunbi2164]

How to define path in filebeat yml if i want to read data (realtime csv file) from another machine in network?

[@apexvalan4369]

If we add new changes every time need to run filebeat setup?

[@rajrana1206]

Hi Does filebeat work for Logstash as output?

[@georgelza]

... Hoping there is similar AuditBeat, PacketBeat, MetricBeat, WinLogBeat videos... if YES, please update the video text with links to them

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@georgelza]

with heroes 04 ... you pulled the config into a separate filebeats.yml file. this imply you will run 2 processes, or can you pull this into the main file, with this file still going to it's own idex/pipeline, and the other /var/log/*.log's index... just thinking, you might have multiple files in the same directory, and you want each to go into it's own index, some single line, some multi line, some structured etc, ... expanding on this... i might want to have a single filebeat.yml processing running, but push each source log onto it's own kafka topic, to be then pushed via a Kafka Connector to it's own index.

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@georgelza]

... with one filebeat process running,I see we can specify the topic, based on a "when" clause, and I noticed to you can include a kafka message key (helping make sure all messages for a key (maybe message per file) is in same order on a topic (localised to a partition), question, in a scenario where I say don't want to use a kafka key, can I then split the output to different topics (or even indexes) based on the originating input file,

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@georgelza]

question, when shipping via kafka, how can you execute the kibana configuration, thinking you might have a setup where the sources (*beats) then only have access to the kafka brokers and not the elasticsearch or Kibana server.

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@wilmarneto6145]

Question: I have several fortinet firewalls, and I want to create a single filebeat server with several indexes, one index for each fortigate device, how do I do this type of configuration?

[@OfficialElasticCommunity]

Hi Wilmar! Please check out discuss.elastic.co/ for technical questions or ela.st/slack to connect with other Elasticsearch users who might be able to help. Thanks!

[@christinaredmond3203]

How can I have access to the sample logs?

[@georgelza]

a technical question, the prospector's look for new files, is this based on name or a inode.. as with file rotation todays file is compressed and renamed tonight and a new file is then created with the same name, which implies the registry entry needs to be reset to line 0.

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@forabraham1]

how would you control filebeat to ship the log/data from the current timestamp or the day prior? is there an option to control this? or in other words, whenever I stop and restart the filebeat it should take the current timestamp or a predefined config value like day - 1 or so to parse and ship it and not the entire file. is it possible?

[@OfficialElasticCommunity]

Hi Abraham! Please ask this question on discuss.elastic.co or ela.st/slack

[@shubhampatel7277]

Can I download slides from somewhere?

[@riferrei]

Hi Shubham. The slides shown in this video are not part of any larger slide deck that contains the content shared. They were add-hoc slides used during the recording to help with the explanation. Therefore, nothing to be shared exactly, I am afraid. But all the content from the slides were taken from the Elastic documentation if you care to search for any specific content about Filebeat: www.elastic.co/guide/en/beats/filebeat/current/index.html

[@georgelza]

... for structured events, if the start and end includes a event id, can they be associated with each other, in the current form of your example it plays to a batch process starting and ending, not to many transactions that can end being interlaced ?

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@jesuisfootballeur4673]

Bravo thanks. But I have question Please which software do you use for making courses

[@OfficialElasticCommunity]

Hi there! Ricardo uses: OBS (To capture the video and audio stream) DaVince Resolve Studio (To edit things and apply effects)

[@joseph9mm]

Thank you for the explanations.

[@vasusp4842]

Awesome presentation. Thank you

[@georgelza]

hhehee, apologies for all the questions, noticed you also on a MAC, noticed you not doing a sudo on each command, did you change the ownership, allowing filebeat to operate, or did you do a sudo su - when were not lookin, as when you created new files you also never modified permissions.

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@milequinze]

He isn't running a Mac.

[@georgelza]

adding to the structured event, #1 you want to extract the main start and end as a event, what if the main "loop" include sub loops that you want to show thenself. thin a large batch starting with a start and end, but inside the large batch you have multiple looping processes that you want to show as they cycle, (and not wait for the main batch start/end ) to complete.

[@OfficialElasticCommunity]

Thank you for taking your question to discuss.elastic.co, George! If you're looking for the answer, you can find it here: discuss.elastic.co/t/miscellaneous-questions-on-the-back-of-ricardos-all-you-want-too-know-about-filebeat/283528

[@011azr]

Thank you :)

[@richardmarques7468]

Amazing content!!! Great instructor!! Congrats!!!

[@chinedudimonyeka2856]

A nice, concise presentation. Thanks