Azure Sentinel Lab Series | Setup Syslog Collector and install Azure Sentinel Agent | EP1

Azure Sentinel Lab Series
Join me as we will lab and do exercises on a journey to become azure sentinel ninjas.
This lab will focus on first understanding syslog. We will setup a syslog collector and I will show you how it works so that you properly understand it prior to jumping into Azure Sentinel. Then once you understand syslog, we will then install Azure Sentinel and generate some sample logs and understand how it works.
Azure Sentinel Lab Series Playlist
• Azure Sentinel Lab Series
Become an Azure Sentinel Ninja: The complete level 400 training
techcommunity.microsoft.com/t...
It is not required, but please watch the KQL tutorial series prior to doing this lab.
• KQL Tutorial Series
00:00:00 - Intro
00:05:00 - Install and configure Rsyslog
00:07:45 - Generate synthetic syslog logs with the logger command
00:12:00 - Understanding RSyslog Configuration rules
00:19:24 - Configuring Facility/Severity for OMS Agent
00:21:07 - Where are the OMS Agent Rsyslog configuration files stored
00:24:22 - Generating more logs to understand facility/severity
00:29:00 - Installing the OMS Agent
00:31:20 - Understanding how CEF logs are sent to Sentinel
Connect with me!
Twitter - / teachjing
LinkedIn - / teachjing

[@nathanwebb2800]

Also, the reason you may not have seen your test message come through when you listened on the OMS agent port (around 24:02) is that the tcpdump command by default listened to eth0 but the traffic to the OMS agent probably went through the loopback interface (lo) because I imagine the OMS agent is listening on localhost only :)

[@TeachJing]

Your exactly right. This ensures the OMS agents don’t listen remotely to those ports so makes complete sense later on after I posted the video lol. Good catch! Also if your interested. You can also program additional ports on the OMS agent to send logs directly to a custom log table! Like 25227 to goto a named custom table and use Rsyslog to convert the message to match that new custom table schema similar to how we handle CEF on 25226. Ubiquiti connector does just this.

[@nathanwebb2800]

@@TeachJing that is super awesome to know! One thing that is bugging me is how long Sentinel takes to process a new source and see it as connected. Syslog via OMS has taken over an hour then magically appeared. Do you know how you can use the OMS config file in /etc/rsyslogd/ to send all messages except when they contain a phrase? I’ve seen your videos on exclusive includes but wondered about excludes. Currently I’ve put a file in /etc/rsyslod.d/10-exclusions.conf which has an if statement with a stop when the condition matches, however. I’m worried this will stop the system locally logging it, instead of just not sending it to the OMS agent. I think Sentinel could do with kind of configuration on the Syslog Workspace configuration instead of just syslog facility and priority level!

[@TeachJing]

If your worried about local logging. Locate the entry that logs it locally which is typically the Rsyslog.conf. Move that over to your 10 file or put a file in front of it that logs it all and Rsyslog knows to handle it first. Then your 10 file excludes and then your OMS gets the leftover. So again 5 file to do local logging 10 file excludes OMS file handles the logs that have been filtered If you look at Rsyslog.conf you will notice the include Rsyslog.d gets brought it AFTER local logging so I think your ok, but try it out and come back to me with sample logger commands and let me know. I’m actually almost done building a fully containerized docker container with all kinds of scenarios with Logstash, kafka, file logging to mess with and see how it flows. Stay on the lookout for that in the near future.

[@nathanwebb2800]

@@TeachJing ha! I feel so stupid now because that makes perfect sense. Log to disk and then have a code block after with an if statement. Awesome, thanks for your help. I’ve come across your content in the past few days and you’ve helped my get Sentinel spun up with logging in my dev e5 tenant! Looking forward to more content from you!

[@TeachJing]

@@nathanwebb2800 Yes! All this I learned bro through trial and error. If you really want to be a fancy, Add a few variables like host to that log path and boom, all your logs from each host goes into their own file into a auth folder. Like /var/log/auth/.log Then you don't have to tail auth but just tail the respective host file. Throw in some log rotation with wildcard and ensures those disks never get full but you can still log locally. That is also how I troubleshoot as if I don't see a file... then that host isn't coming in :D

[@khurramwzd]

Thanks a lot. A much awaited series.

[@anwarullahsyed3058]

love your videos!!! keep it up

[@Panda-bd6gc]

Thanks a ton for this series! I've subscribed & liked and hopefully you'll publish more useful content like this!

[@TeachJing]

Thanks for taking the time to comment and subscribe!

[@trendyniro]

Thank you so much Teachjing! you gave me loads of info!...

[@riadoszh6616]

nice video! it was very useful and very interesting :) your content is very informative. thank you for your valuable contribution! please keep going!

[@jagadeeshg90]

Thanks a lot for Lab Explaination, It helps to understand deeper, Keep Rocking @TeachJing

[@muhammadamin6759]

Thank you so much for making this video. kudos to you

[@TeachJing]

Glad you enjoyed it!

[@sergiot3143]

Love the content! A lot was over my head, but I'll work on my Linux skills to get a better understanding of what you were doing. I don't know VI or many of the commands you used but will work on learning them. I'm also working on learning jupyter notebooks, so anything that incorporates the notebooks with Sentinel would be awesome. I am trying to become our SME for Sentinel but I am just getting started with SIEM's so anything covering Sentinel is a bonus. Your video has been the best coverage I have seen and I've seen a lot. I'm working through the Ninja course but what you covered is some much more useful then anything I've done in that course. Either way, continue the great work and I'm looking forward to future videos. I'm going to watch your P2 on CEF and your KQL course. Cheers!

[@TeachJing]

Keep practicing. I am going alot deeper than the other content, but I'm glad your able to keep up. Eventually it will come together!

[@rmp5s]

Great vid, my dude. Would love to see an updated video with the new AMA agent. OMS is going away. For some reason.

[@RafaelOliveira-vg8gq]

Thank you so much for saving my life everyday \o/ \o/ \o/ I really appreciate. Keep doing your things.

[@TeachJing]

Happy to help!

[@darrensmith5544]

Good vid!

[@dirkl9652]

Good stuff bro.

[@TeachJing]

Appreciate it

[@ciarahoulihan8734]

Great video! Would love to see azure sentinel and terraform together

[@TeachJing]

I will add it to the list!

[@usarkar2006]

You rocks

[@mr.cmoorecrypto]

Amazing video Jing! My one question is how do you stop CEF logs from feeding into the Syslog table? I am currently working on that, and it's causing me some trouble.

[@TeachJing]

You gotta stop the CEF events prior to before it gets sent to 25224. Check out /etc/Rsyslog.d/ folder and you prob don’t got a stopper.

[@sreepuli2311]

quite confusing between syslog and cef logs, if you can show how to forward CEF logs to sentinel will be helpful

[@nathanwebb2800]

just a heads up, I use tcpdump -X to show the body of the packet. Super helpful when you're setting up SIEM solutions

[@TeachJing]

Very Nice!! I’ll definitely remember that

[@zyeuh2565]

A fucking god send. Thank you for the videos !

[@TeachJing]

Your welcome! Thanks for commenting

[@charlesquansah4847]

in order to set this up I would need to buy Azure Sentinnel right? And if so what pricing would be appropriate to complete lab exercises?

[@harrier113]

Great video, are your VMs virtualized on a local hypervisor or cloud, both?

[@TeachJing]

Both!

[@harrier113]

@@TeachJing Thanks for the reply, which linux distro are you using in the Sentinel series? CentOS?

[@TeachJing]

@@harrier113 Ubuntu

[@issamnaouali1574]

Very advanced xd Plz is it possible to show us how to collect lors from asa cisco to sentinel ?

[@simple-security]

great video but it's now obsolete. consider a new on on the AMA agent.

[@martinmatacek2990]

Thank for this great Syslog tutorial. I have a question to Configuration of OMS agent. I am trying to configure 2 connectors, where one connector "Eset Security Management Center (Preview)" needs data in API format. My problem is that other connector "Cisco ASA" stops working if I update section "" and change value "type out_oms" to "type out_oms_api" in /etc/opt/microsoft/omsagent/{yourworkspaceid}/conf/omsagent.conf. Can you please give me some hint how to configure this two connectors to live together?🙂 Many thanks Martin

[@mikeslates353]

Great video! I'm just starting to look into Sentinel as a possible SIEM for our company. Can the syslog/collector be setup on a Windows server? We're not a Linux shop, and thus have zero experience with Linux and would rather not bring in an OS we know little about. All new maintenance, updates, and security concerns. Thanks.

[@jayjoshi3853]

Hi Mike, have you got your answers yet? I am looking for the safe stuff.

[@oipoip3935]

No it has to be Linux

[@user-wg2ij7lk6d]

What is the headset you are using? 🙂

[@kasarlasrividya5383]

If I have to configure log collector on cloud but my machines are on-prem (in my lab) . How do I tell my machines it has to forward lags to machine in cloud

[@jimtaylor4938]

What about the AMA agent ?

[@uriel4292]

Hi TeachJing, I learned a lot on your videos. I'm new to KQL and Sentinel. I have a question by the way. I work in a small company any we have M365 and Azure Identity Protection. I noticed that Sentinel will display duplicate alerts coming from Azure Identity Protection and M365. What could be the reason for that?

[@TeachJing]

Check if your actually receiving two events. If you are generating two alerts then you need to check if you are grouping similar alerts together so only one incident is generated. I’ll make a video they explains it next week that explains it in detail along with other things things

[@uriel4292]

@@TeachJing Thank for the reply. If I may, can you also create a video about threat hunting in sentinel? Like hunting IOCs such as (Hash values, services running, backdoors, etc.). Thank you.

[@kasarlasrividya5383]

How does one machine know to which machine it has to forward the logs? I mean how does the machine know where is the syslog log collector

[@user-bd2vp6mk5l]

Why i don't have message file in /var/log ?

[@yusareba]

Can this be done for free? I'm interesting in doing this but assume it has costs associated

[@yashwantbikaner]

just wanted to add, i think you are running tcpdump on port 25224 which uses eth 0 default , it should be interface lo. That could be the reason why you didnt received the test msg.

[@bcyz1000]

You are 100% correct. tcpdump -i lo port 25224 would have caught it. Smart man.

[@RIYADMURAD]

Awesome tutorial, but how to configure the OMS Agent to send specific Logfile to a specific(new) custom table instead of standard syslog table?

[@TeachJing]

Did you figure this out. That can be achieved with simple Rsyslog/Syslog NG Parsing or you can use logstash.

[@RIYADMURAD]

Not yet, i'm working on several usecases

[@TeachJing]

@@RIYADMURAD docs.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-custom-logs. This site explains a little, but you will want to first figure out how you can parse those logs. If they are syslog format where you can read it in Rsyslog, you can simply save logs to a new file and then setup a custom connector that will basically listen on another port but is directed to another table. Customlog --> specialOMSAgentPort (ex:25229), This special connector will point to your custom table you created and your done. Have rsyslog then forward traffic matching that message to this new port you setup. Or you can use logstash to read from a custom log file you have parsed out and upload that to sentinel.

[@TeachJing]

@@RIYADMURAD Logstash link here docs.microsoft.com/en-us/azure/sentinel/connect-logstash

[@RIYADMURAD]

Thanks a lot.🙏🙏🙏

[@ssad-yl8nd]

This is all cool, but now Microsoft says that workspace agent configuration method is legacy. Would love to see if there is a solution to push this to Sentinel that is future proof beyond August 2024

[@adamzachary6947]

Have anyone deployed Sentinel SAP Connector ?

[@zakecysec]

tail: cannot open 'messages' for reading: No such file or directory help

[@nitinmaurya6835]

Hey Hi, I want to know where the linux systems are hosted? Is it VMWare or Azure?

[@TeachJing]

My home lab behind me most of the time, but sometimes it’s azure in some demos. I don’t use VMware too often, but do have a cluster I use sometimes. Any reason for that question I could help with?

[@nitinmaurya6835]

@@TeachJing Yeah, I was just curious about sending logs of guest machines to azure. If you know then please make a video on that also because most of my attack defend setup is on VMWare.

[@TheTCPTalk]

man how did the log just show up on sentinel? when did you even do the config and set the sentinel ip, how does your logger know where is the sentinel ip? its like these videos just assume that people watching know most of the stuff lol

[@mr.cmoorecrypto]

The OMS agent sends the log to Sentinel. It already knows where to point the data for Sentinel to receive it.

[@OurCloudSchool-Hindi]

How make changes to your cursor?

[@TeachJing]

That is with the ZoomItTool on Sysinternals docs.microsoft.com/en-us/sysinternals/downloads/zoomit

[@bbrendon]

audio quality is bad. the clipping hurts my hears.

[@TeachJing]

Sorry to hear that! The audio gets better. I didn’t have a good setup. Ping me on LinkedIn and I can show you the ropes!

[@Guest-gy9vp]

Your video is good but MS product are terrible anyone use apple/linux/AWS would know that.

[@syscabmcommunity3368]

OMG, I really thought this kind of stupid comments passed away when the novices grown up,, but I was wrong

[@learnergaa]

@@syscabmcommunity3368 true, why did they even come here in the first place

[@ashishhingmire123]

hello, I have followed the steps mentioned in this webinar. But unfortunately all the PA FW and Cisco Meraki events are getting forwarded as normal syslog messages to Azure sentinel via the local oms agent running on syslog collector/RHEL server. Any idea what should be changes in the 95-omsagent.conf or rsyslog.conf file?