Detecting Exploits - OMIGod (Linux Logging with Auditd)

[@daneilyan6419]

Three videos in 2 days? Thanks a ton Ippsec, you material is always amazing and appreciated

[@armandkruger911]

Love this approach! When I pop a box I always run "whoami / whoami /all". We created a Azure KQL query to look for those command and additional arguments across the organization and run a custom detection rule when it finds such possible post exploitation activities on any endpoint. Dropping ATP EDR on Linux machine with custom KQL queries can help a lot , especially when one sees the offensive arguments being executed. We look for argument that contain "/dev/tcp/" for example. Very helpful. Every time I watch your videos I adjust the queries lol One can always limit egress traffic on Azure NSG to just drop the revere shell if it gets popped and limit inbound traffic on NSG to limit the overall attack surface Working with Microsoft a lot. , they are not fixing it because it does "not affect back-end infrastructure"

[@onkarkoli8621]

Preparing for OSCP your video's are helping so much ❤️❤️❤️❤️❤️

[@garnettk]

More Blue team stuffs please!! Big up ippsec

[@jacobwarners887]

100% agree this was great!

[@SmoothestOperator]

Thanks for the video, very nice to see blue side here, but at the same time I feel like it shows how hard it is to create an universal auditd policy. Even with basic set of best practice rules and reasonably standard fresh Linux install, auditd still flooded logs by misbehaving package manager

[@ippsec]

Well you could just exclude dpkg/apt and then all those alerts are gone. They have minimal chance for abuse.

[@SmoothestOperator]

@@ippsec that is true, but dpkg is just an example here that you came by in a 1h video, I'm afraid many more like that will come. I guess it comes down to managing policies in a larger environment and that is not what the video was meant to be about.

[@ippsec]

​@@SmoothestOperator Yeah at the end of the day the noise isn't great but it doesn't increase the difficulty all that much if you are just using it for threat hunting. Running some searches like show me EXECVE calls in path's that contain TMP, and suddenly all that dpkg noise goes away. Or you make a broad hunt and go show me all the syscalls that failed and group by process then show me all the unique and reverse sort... Then you're just looking at the one off's which are generally more interesting than the ones that happen 24/7. At the end of the day the main disadvantage to the noise is it is just eating up money, but at the end of the day its not much money and you can archive it off.

[@Rienck]

Really insightful, thanks!!

[@SALTINBANK]

Thanks so usefull you are the one IPPsec love you :)

[@zaferb]

Great video! I started using Laurel after your tweet, then watched the video. Unfortunately, wazuh does not have any rules for Laurel. And translating audit rules to laurel versions takes time. I gave up, for instance. Do you know if any other SIEMs can consume Laurel log easily?

[@wkppp4732]

Thanks for the vid!

[@berndeckenfels]

Thanks for defender content

[@FrancescoM-]

thanks for the A/D content from Twitter :)

[@SomeGuyInSandy]

This is really good.

[@berndeckenfels]

Would be good to log the pathname for the mknod syscall

[@berndeckenfels]

Echo is a builtin, it does not do any exec syscall (only write(1))

[@patrickbland422]

Great video!!! Question for you - how could we send this information to Kafka? I saw an Rsyslog plugin to output to Kafka but do you have any recommendations? Maybe use something like Fluent and scrape the laurel audit.log?

[@boltthrower3215]

Fun fact: You sound like the host of the LGR youtube channel 😊

[@buhaytza2005]

21:00 I know asking is lazy but I assume doing a `chmod 4000 /bin/bash` would be recorded in the audit log

[@ippsec]

Yes that’s an attribute change. I think all attribute changes in /bin/ (and normal directories for $path) was logged. If not easy to add.

[@buhaytza2005]

@@ippsec thanks for the reply. I installed auditd on my homeserver - had an old tower kicking about and trying to understand more around protecting an environment rather than breaking into it.

[@shadow_absorber]

how does this affect the logging of something like apparmor profiles in complain or enforce mode???

[@ippsec]

I'm not exactly sure what you mean by this, and honestly I've never really touched AppArmor. But I'll take a stab at trying to answer, let me know if this helps. AppArmor does do some syscall monitoring, but I don't think it monitors everything. If you are just concerned about Process Execution/File Monitoring it may be fine... But AuditD could also go into network monitoring which. At the end of the day, AppArmor/SELinux were built to prevent applications from performing actions, so they are tailored at that. When AuditD is specifically at discovering what your system is doing. So they data you'd get from the two are wildly different.

[@boltthrower3215]

Question: I‘m watching your video on my ipad while in bed…so, I apologize for my laziness to try it out by myself. If a user deletes 1000000 files…what happens to the auditd log? Will it explode or insane IO?

[@jacksoncremean1664]

Am I the only one that can't get logging with apparmor to work when auditd is installed?

[@luqmanhamdan9285]

Can you make video about selinux to prevent system exploitation.

[@boltthrower3215]

Dude…dnf is not a redhat firewall…it is the successor of the yum package manager AND the key also states software_mgmt. 😂😊

[@amerramamni9461]

Can you please let me know if I can combine the following rules: -a always,exit -F arch=b32 -S chmod -F auid>=1000 -F auid!=-1 -k perm_mod -a always,exit -F arch=b32 -S chown -F auid>=1000 -F auid!=-1 -k perm_mod -a always,exit -F arch=b32 -S fchmod -F auid>=1000 -F auid!=-1 -k perm_mod -a always,exit -F arch=b32 -S fchmodat -F auid>=1000 -F auid!=-1 -k perm_mod As: -a always,exit -F arch=b32 -S chmod -S chown -S fchmod -S fchmodat -F auid>=1000 -F auid!=-1 -k perm_mod OR -a always,exit -F arch=b32 -S chmod,chown,fchmod,fchmodat -F auid>=1000 -F auid!=-1 -k perm_mod Is this valid? thanks

[@kovacs-andras]

RTFM! This video is so inaccurate that Steve Grubb is rolling in his grave - and the fella is not even dead yet...

[@ippsec]

Be helpful if you said what was inaccurate. Presume some auditd stuff based upon Steve Grubb. But yes, like all videos it's just my understanding of how it works. Alot of why I put videos out is to learn where my understanding is wrong.

[@kovacs-andras]

@@ippsec I couldn't see your UMASK but... - you created a new audit.rules with (probably) 0644 while originally it was 0640 - the same goes to your audit.log as you could ausearch in it locally with the ippsec (non-root) user. It's something what shouldn't happen. These logs can be really sensitive.

[@kovacs-andras]

@@ippsec You mentioned the "cost of storage" which is ofc. a tipical security/logging team point of view while more the rules mean bigger the impact on the performance. It will mostly hurt before you would run out of log storage. So I would mention the performance impact as the real cost instead.

[@kovacs-andras]

@@ippsec You jumped back and forth in the rules and the explanations were quite inaccurate, but comments are there so... maybe just a few notes: - the rules at the beginning are also important. - the purpose of opasswd is to store the "old passwords" and keep track of changes. (man pwhistory_helper) - auid is the original ID the user logged in with. So even if you sudo/su/whatever to an another user, it will keep track of your user id. (man auditctl) - who is moving towards ufw? It's a Canonical/Ubuntu "thingy". Linux distros are moving towards nftables for a while (which is quite painless because of iptables-nft) or eBPF maybe... ufw is a simple tool which works with iptables(legacy) on Ubuntu as they couldn't change to nftables yet for some messed up lxc-compatibility issue. But maybe I'm wrong. - 0x06 was SIGABRT

[@ippsec]

The ippsec user is a member of adm, which has read access to the logs. It’s also a member of lxd, as that’s the default configuration when installing with vmware, and there’s a PrivEsc there too. The point of that section was to demo the logs you can get out of linux and show why logging is important. Not perform perfect hardening, which I agree *should* be done. However, that takes actual time to prep so something I did start to finish on a Sunday afternoon, would take multiple days and multiple eyes to get it right. That’s just not worth it when at the end of the day the video will earn me $30-40 over the course of a year. Which I put 6-7 hours into so my hourly rate for that was around $5/h. So at that rate I stick to shitty demos and hope people use it for inspiration and not to be a SME