Finding 0day in Apache APISIX During CTF (CVE-2022-24112)

In this video we perform a code audit of Api6 and discover a default configuration that can be escalated to remote code execution.
CVE-2022-24112: seclists.org/oss-sec/2022/q1/133
GitLab: liveoverflow.com/gitlab-11-4-...
Challenge files: github.com/chaitin/Real-World...
Chapters:
00:00 - Intro
01:09 - Initial Application Overview
02:15 - Discussing Approaches
03:56 - Reading Documentation
04:57 - Initial Attack Idea
06:15 - Identifying Attack Surface
08:46 - Discovering Batch Requests
09:18 - Bypassing X-Real-IP Header
10:15 - Testing the Exploit
11:11 - Reporting the Issue
12:16 - Outro
=[ ❤️ Support ]=
→ per Video: / liveoverflow
→ per Month: / @liveoverflow
=[ 🐕 Social ]=
→ Twitter: / liveoverflow
→ Instagram: / liveoverflow
→ Blog: liveoverflow.com/
→ Subreddit: / liveoverflow
→ Facebook: / liveoverflow

[@quaternaryyy]

to me, this is a story about how timed competition and a trusted source saying "yes, it's still possible" leads to tons of people independently discovering a real 0 day, just like that.

[@KitsuneAlex]

Strongly aggree x3

[@LB_]

You'd think they would have reported it after the competition 😬

[@Mr_Yeah]

Yeah, I'm worried that the other teams might've found a different exploit and didn't report it yet.

[@henke37]

The headers only working "sometimes" is a classic sign of an unsorted hashmap. Enumerating the key/value pairs will return the entries in an unpredictable order. My guess is that the code responsible for handling the header enumerates the hashmap entries and uses a switch statement to figure out what to do. The end result is that a random header ends up enumerated last and overwrites the work of the previous headers.

[@almightyhydra]

Yea, would be interested to know what the fix is. I doubt this is the only header manipulation code that might be vulnerable.

[@guiorgy]

@@almightyhydra Is there any time when detecting more than one header is ok? What if you just terminate if more than one is found?

[@yScribblezHD]

@@guiorgy Couldn't the injected header still just be read first? I feel like the real issue is that batch requests is relying on a supplied IP address that can be forged as localhost.

[@emptylog933]

Idk, why would the plugin need proxy support if the host expects requests only from localhost?

[@sknt]

​@@almightyhydra Here's the pull request for the fix: github.com/apache/apisix/pull/6251/files All they did was call str_lower() on the "x-real-ip" header field name. If I understand it correctly overwriting the header happens in nginx. Likely due to the already mentioned reason of enumerating a hashmap in a random order. RFC 7230 ( www.rfc-editor.org/rfc/rfc7230#section-3.2 ): Each header field consists of a case-insensitive field name followed by a colon (":"), optional leading whitespace, the field value, and optional trailing whitespace.

[@ThePowerRanger]

This is literally the dream.

[@monad_tcp]

Super useful when you want to root that pesky device. There's always a fucking webserver and it's apache running PHP. I love shitty technologies, it means I can always POWN my hardware.

[@FrozenFire1997]

@@monad_tcp what kinds of devices are you talking about?

[@monad_tcp]

@@FrozenFire1997 smart TVs for example

[@otherkrabs]

@@monad_tcp This isn't the apache web server though. It's APISIX, which is not the same software (and in the video it's running on nginx anyway)

[@Stopinvadingmyhardware]

Nope

[@saketsrv9068]

This man is a gem and super talented guy.

[@perceptoshmegington3371]

it's a case of hard work over talent

[@ibrahimkalantn4072]

Man i love your channel great video as usual

[@jandalfDerNice]

Great video as always! Thank you for making this awesome content for aspiring InfoSec students

[@kevinwydler4405]

So simple yet ingenious!

[@larditard]

Excellent video. Thank you for making!

[@JaspreetSingh-qg2xp]

Thank you so much and Congrats on solving and idetifying the issue . You're really making a valuable content and please , I request you to keep posting such a good informative as well as interesting things. You're full of knowledge and a motivation for me.

[@MisterL2_yt]

Very interesting video, but how did this situation happen? Did the RealWorldCTF organisers find (or purchase) this vulnerability some time ago and just decided to keep it secret for the CTF and then not even report it afterwards? This seems very strange :o

[@ThisIsTheInternet]

Yeah that's very questionable

[@ahmedifhaam7266]

Man.. this got me thinking, should I stack the vulnerabilities I find before reporting, and just create a prized challenge? lol. Maybe if the service is containerised I can report it first and use the old containers for the challenge, but then hmm that's impossible since people will just look for changes in the patch. How are real world CTFs actually done?

[@siddharthchhetry4218]

You and your team are awesome

[@Aquriez]

This is really cool, great video

[@randomguy3784]

Superb content!

[@LukasSMF]

I really love these videos

[@mikflores]

This is amazing. Great.

[@seif-allahhomrani2169]

Crazy how you make it looks like it easy to find a 0-day ! great video @liveoverflow

[@RahulSinghInfosec]

Thank you for sharing!

[@flopana5762]

What bothers me is that you had to report it and that just due to the fact that you wanted to make a video about it. I can understand that the organiser didn't just want you to read a couple of commits to find the vulnerability that wouldn't be a good challenge. But I think it was a bit unresponsible from the organiser to not report this issue immediately after the ctf has ended or contact the apache foundation in some way. They basically just led multiple hacker groups to a remote code execution without caring about fixing it.

[@dennydravis8758]

Yeah it does violate the spirit of the ethical hacking CTFs

[@damiannowak3811]

@@dennydravis8758 yeah just did a masscan and there are a lot of those not updated yet. executing cross-compiled botnet binary on them for monero mining.

[@aescling]

@@damiannowak3811 i hope you're capping because otherwise you just admitted to a crime in public?

[@Sina-rw3bl]

@@aescling "in public" settle down buddy, nobody is catching him 💀

[@The_One_0_0]

@@damiannowak3811 already took skid lol

[@SkippyDa]

I liked your outro.

[@JuanBotes]

thank you for sharing your knowledge \o/

[@zekiz774]

Finally: a video I understand

[@aakashadhikari3752]

Dream boiiz dream..but congrats comrad for the CVE

[@monad_tcp]

This is such a good news!

[@mynameisrezza]

Just saw this and WOW!

[@allezvenga7617]

Well done 👍

[@atraps7882]

this shows me that being a "hacker" isnt just about using the popular tools, they got to have a lot of deep background knowledge in systems, web technologies, networking, bit manipulation, scripting, cryptography, containerization, virtualization and much more.. Im just an average software engineer focusing on backend development but man, these guys are just levels above and beyond

[@hovnocuc4551]

that's the difference between a hacker and a scriptkiddie.

[@generallyunimportant]

i find it funny that no one actually reported the vuln to apache lol-

[@1vader]

Funny but also pretty sad. It's honestly pretty shocking and irresponsible that the organizers didn't do it themselves.

[@theairaccumulator7144]

@@1vader the organizers probably had a different vulnerability in mind, there's a chance that these guys found something entirely new.

[@bigl9527]

Another video of Ed Sheeran explaining about security in detail

[@31redorange08]

This isn't Ed Sheeran.

[@nztpill]

@@31redorange08 thats literally him check his instagram

[@patrick1020000]

Did you hit up the other challenge solvers to make sure they found the same bug you did?

[@faizalqorni7969]

this is the dream man

[@johnpathe]

Such a great video. Really well explained. Doing amazing work as usual LO :) I had to playback some parts and ended up watching it at .75x speed :) Gratz on the first blood! :D

[@Reichstaubenminister]

I only listened to the video while doing something else, and the entire time I though you said "bad requests plugin" and that the name was quite ironic. Turns out it was batch-requests.

[@thepenguin9]

I feel like one of the organisers shares my mentality on chaos and it's current reign including a 0day

[@kirdow]

Great video, haven't watched in a while but this title got me hooked. Will definitely watch some of your other videos to catch up :D Also at 12:15 in the report message, should you really have "1. " twice in Mitigation? :P Anyways, you surely have improved your editing and video style since the day I became a member, keep up the great work man

[@sodiboo]

Isn't that markdown? in source files you often find numbered lists with all the points as 1. for easier reordering, because markdown rendering does NOT use the numbers in the document for the resulting list, it's simply that you have a number in front of every line, and then the marker starts at 1 and counts up for each entry. This can be somewhat confusing when viewing the document as plaintext, but it also isn't plaintext and shouldn't be viewed like that, so it's not a huge issue for most people.

[@andyelgangster5320]

nice video 😎

[@Najumulsaqib]

Very engaging stuff.

[@hyperdrone900]

nice :D

[@EER0000]

A bit odd that it was not reported yet, but very nice find. HTTP header capitalization can be a nightmare sometimes, not just in LUA 😅

[@meh.7539]

If you check out the HTTP request smuggling attack preso from, i want to say 2019, he explains what's going on in it's most basic form. What you're showing here looks pretty similar to what he presents.

[@yy6u]

that kind of ctfs are really great, it's all about expanding knowledge of someone's work and educating everyone else

[@alwan7777]

Yeyyyy

[@casperes0912]

That was a weird Minecraft Let's Play, but I liked it

[@joaokoritar2141]

Very cool! Btw, which VSCode theme do you use, it looks nice!

[@aescling]

looks like Solarized Dark

[@AbdelrahmanRashed]

if it didn't work for you the first time what would you have done ?

[@chiragartani]

Incredible. Do you think that the servers are using APISIX? And are vulnerable? I mean I want to see in the real life, If I can find this vulnerability in the live servers.

[@xB-yg2iw]

The ending had me rolling hahahaha

[@bobsmithy3103]

lmao i love it when you run the same piece of code but get different results

[@HritikV]

Checkout pateron and stuff. Lol, best ending ever

[@anion21]

Well done. So, was your solution the "correct" solution expected by the creatores of this CTF-challenge or is there any other solution which does not contain 0days?

[@kebien6020]

I think in this setting any solution would qualify as a 0day, since the challenge involved RCE and was meant to work on the latest version.

[@ahmedifhaam7266]

pretty sure there was another exploit.

[@odessairenikute6961]

So it is all about just to learn how to ask smart questions. Not a rocket science but it is still tricky thing :)

[@EduardVasile5]

Ah, yes. Of course.

[@D0Samp]

I guess it's finally time to set or change some localhost-only admin passwords/tokens.

[@DarkOverFlowOverflow]

i've never seen you with glasses before, congrats on your 700k followers tho

[@LiveOverflow]

why didn't you watch the previous video 🤡 haha

[@DarkOverFlowOverflow]

@@LiveOverflow damn i got hacked lol

[@MTRNord]

This makes me wonder: Are there other services with this exact bug or a close variation? As it seems like a fairly normal pattern to have for things like this.

[@FlorianWendelborn]

Most security vulnerabilities aren’t unique. A lot of them even make it to OWASP top 10 list :) I wouldn’t be surprised at all if there’s 100 different pieces of software out there somewhere that have this exact _kind_ of vulnerability.

[@ThisIsTheInternet]

Do you know of other stupid gateways that let you dynamically create remote code execution endpoints? lol

[@MTRNord]

@@ThisIsTheInternet there are countless of these api gateways yeah. It is pretty common in stuff like cloud. Serverless also kinda is a framework doing something like this. So is aws lambda kinda. Both not exactly like this but similar goals of having dynamic customer provisioned api endpoints

[@ahmedifhaam7266]

literally found something similar in a local community SAS

[@mr.guljaan7175]

🆂︎🅾︎🅾︎🅾︎🅾︎🅿︎🅴︎🆁︎

[@ahmedifhaam7266]

was a very engaging and fun video. I am just surprised how this Flo guy writes Lua code so fast.. I got the gist of it but I couldn't understand the Lua script that well, anyone care to explain? would appreciate, thanks

[@ArnaudMEURET]

Huh, I’m frustrated that you did not present the actual piece of faulty code and its fix ! 😒 …I’ll look it up.

[@jonathan-._.-]

🤔 halfway through : maybe we can set the host header to lcoalhost

[@jaopredoramires]

what's the `(base)` at the top-left of the terminal prompt?

[@sadhlife]

it could be a python virtual environment, or the name of their ssh sever / docker container, or anything really.

[@necroowl3953]

Bro, pls look into golang, I have a fast recursive hasher that you guys could write down in minutes

[@_AN203]

Are you working in another CTF in the time of recording ???????

[@ChrisBigBad]

LoL. SSRP as a Service :D

[@mikena8519]

yes that was a good punch line i thought too!

[@_Slaze]

After watching some of your videos I feel like I should quit learning pentesting. If you call this "not a hard challenge" what am I doing all the time? ^^

[@nobodynoone2500]

I mean, it's not super advanced stuff. I think the most technical thing was the proof of work code.

[@leesalmon7672]

how to bruteforce hash 26bits

[@alexanderwences6600]

So Are you gonna help with the Cyber war?

[@sookmaideek]

zerday guyz

[@bibasbajgain1434]

🧐🧐

[@awesomesauce804]

SSRS as a service, lol.

[@oeerturk]

IS IT JUST ME OR DOES HE LOOK LIKE MR ROBOTS FATHER GUY WITH THE GLASSES??????????????? thx for allllll the incredible content

[@CentigradeMind]

Yup

[@dev__004]

Now, what was the real solution or were the organizers too expecting you guys to come up with a 0day😆😆

[@yeetyeet7070]

he w i d e

[@v2nd2tt44]

69k 😶😌 lucky

[@raass9316]

the main take away , if you want to hack it just try it ! why all new bug is like this ?

[@ndm13]

Why, Apache? Why do you keep doing this?

[@stef9019]

Is it me or did you recently switch to reading from a script? If it was the case already before it's a bit more obvious rn IMO.

[@LiveOverflow]

I have read from a script since I started this channel hahha. There are only a handful of non-scripted videos. But this was the first time wearing glasses while reading of the teleprompter. Maybe I struggled a bit here? 😅

[@stef9019]

@@LiveOverflow Ahah could be the case, I've never noticed before!

[@cbruegg]

@@LiveOverflow Completely fine IMO :)

[@creepychris420]

ayylmao123 😂

[@random_guy1024]

Can you make a video on how a script-kiddie like me can be a hacker like you... or at least try to be......

[@tomysshadow]

Check out his video "the secret hidden guide to hacking."

[@casperes0912]

Rule number 1. Have fun.

[@gowthamanks3654]

You posses lots of knowledge. Why dont you make an udemy course. Or beginner friendly youtube course's

[@karanb2067]

Very cool, but like realworldctf people just decided to not expose this vulnerability? kinda sketchy....

[@svet3804]

How to register for RealWorldCTF?

[@aziztcf]

Hey, you might want to calm down your body language a bit while explaining stuff. It can get kinda distracting, especially to people who rely on subtitles :) Other than that, great job once again!

[@shinkurt]

First

[@codywohlers2059]

I like the videos better when you were doing it as you filmed. I don't like these videos where someone talks over what they did after the they did it.

[@LiveOverflow]

This is literally how I made CTF channel videos always. Only very very few are in a different style. Which videos were you thinking about?

[@codywohlers2059]

I don't know what I mean lol. I guess when there was less full screen cam and more code. Don't get me wrong I love your videos!

[@espero_dev]

Bro there is a new 0-day hack lol it’s secret because it’s just my company that found it but it’s pretty secret no one else knows about the one we do because it works with mobile and desktops and laptops

[@evanjsx]

> Rust ... > redstarosx ... *subscribes*

[@nhazittas]

ayyylmao1234

[@nobodynoone2500]

thats the password on my luggage!

[@echoo200]

First