Learning Awk Is Essential For Linux Users

One of my favorite command line utilities is "awk" which is a text-processing program. It is mostly used for pattern scanning and processing. In this video, I will give examples of some of the basic awk commands, and show you some of the ways that I often use awk.
REFERENCED:
► www.gnu.org/software/gawk/man... - GNU Awk
WANT TO SUPPORT THE CHANNEL?
💰 Patreon: / distrotube
💳 Paypal: www.paypal.com/cgi-bin/webscr...
🛍️ Amazon: amzn.to/2RotFFi
👕 Teespring: teespring.com/stores/distrotube
DONATE CRYPTO:
💰 Bitcoin: 1Mp6ebz5bNcjNFW7XWHVht36SkiLoxPKoX
🐶 Dogecoin: D5fpRD1JRoBFPDXSBocRTp8W9uKzfwLFAu
📕 LBC: bMfA2c3zmcLxPCpyPcrykLvMhZ7A5mQuhJ
SOCIAL PLATFORMS:
🗨️ Mastodon: distrotoot.com/@derek
👫 Reddit: / distrotube
📽️ LBRY/Odysee: odysee.com/$/invite/@DistroTu...
DT ON THE WEB:
🕸️ Website: distrotube.com/
🐿️ Gemini Capsule: gemini://distro.tube
📁 GitLab: gitlab.com/dwt1
FREE AND OPEN SOURCE SOFTWARE THAT I USE:
🌐 Brave Browser - brave.com/dis872
📽️ Open Broadcaster Software: obsproject.com/
🎬 Kdenlive: kdenlive.org
🎨 GIMP: www.gimp.org/
🎵 Audacity: www.audacityteam.org/
💻 VirtualBox: www.virtualbox.org/
🗒️ Doom Emacs: github.com/hlissner/doom-emacs
Your support is very much appreciated. Thanks, guys!

[@TecnocraciaLTDA]

AWK is not just a text processing utility, awk is a complete programming language!!

[@TheSulross]

yes, remove Perl, Python, and Node JS (notice no mention of Ruby - ah how far they do fall) - awk and bash is all you need

[@ziggyspaz]

@@TheSulross you need Perl to keep your sanity, you can remove the rest

[@gordongoodman8342]

Calm down. It's not that exciting.

[@TecnocraciaLTDA]

@@gordongoodman8342 it is, a lot!

[@tekvax01]

yup! I was going to comment the same thing. you could even use it as your shell.

[@erics7004]

DT: AWK is important for linux users Luke Smith 1 hour later: Why AWK is useless for linux users

[@charlessmith5465]

_Awk is the predictable result of misguided Enlightenment philosophies._ 🤭

[@PS3PCDJ]

Brodie 2 hours later: Boomers fighting over usefulness of linux utilities

[@TarebossT]

_Why learning awk when you can learn grep_

[@blogattacker]

Luke Smith 1 hour later: "You can replace Awk with Emacs" 10hs later: "You can replace your wife with Emacs"

[@elderberry-hamster]

@@blogattacker 10.5 hours later...I did. 😂

[@SPSHOfficial]

At 8min, when you pipe uniq, note that 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without 'uniq'. so it would be | sort | uniq OR | sort -u

[@fabrice9848]

indeed

[@joedunder4439]

quite

[@BrendoIsGaming]

rather

[@OscarFURIOUS]

I just came to say that I can see that you listened to the comments and now you are sharing your knowledge on simple but technical stuff that's really helpfull. I apreciate that and also can feel that effort

[@JosephSaintClair]

awk is a must. So powerful. Btw. You actually did just square. Not square root :)

[@itsalivevideo]

Could have done the square root of i*i is i

[@baruchben-david4196]

I tried it with sqrt(i) and that worked. I kind of wonder how much math you can do with awk... Yes, exp(var), log(var), cos(var), sin(var) - but not tan(var); var**x gave exponent, as did var^x. Lots of different functions...

[@sachindeshpande1]

I have poorest knowledge of maths but I paused and rushed to comment section to check I am not wrong. But tutorial is very good.

[@ecavero1]

In the square root example, very time DT said square root, he means square! Great video, DT!

[@antoninperonnet6138]

Just swap the 2 fields and pretend you didn't see that: "10 is the square root of 100" is a completely valid statement 😃

[@ecavero1]

@@antoninperonnet6138 I could do that! 😉

[@KeithPeters]

It's AWK math

[@pleaseenteraname1215]

@@KeithPeters 🤣🤣🤣🤣

[@BrutusHiatus]

This is a great tutorial about awk. I've learned quite a bit. The only thing that red flagged me was that NONE of those were square roots. They were squares.

[@edwardonsax9919]

I just posted this same thing, before seeing you posted it.

[@danieldevito414]

He just got the order backwards. "The square root of i*i is i". Anyone could have made that mistake.

[@jeffcauhape6880]

That is the most clear demonstration of AWK I have ever seen. Amazing.

[@intrepidca80]

14:00 - /^[b,c]/ matches anything that begins with a "b" or a "c" *or a comma*.

[@Joe3D]

neat

[@InfiniteQuest86]

So true.

[@emeraldmasta7752]

Regex yeet

[@driden1987]

Had to process a 120k line csv at work the other day. Awk was a god send

[@milo_andrs]

Nice, thank you. I am beginning my journey on bash and this is awesome.

[@tarekrahman1]

Glad you made this video. I was stumbling upon awk in everywhere and wanted to learn it but didn't have time to do so. Thanks

[@VexisMorlock]

Great TUT. I ran into an issue because I don't use uniq and sort often enough; uniq doesn't work if dupes are not adjacent so you need to sort first.

[@KeepItTechie]

Great tutorial DT! AWK is very powerful!

[@DistroTube]

Thanks, Josh!

[@KeepItTechie]

@Earthling-Z3R0 Thanks man. You're right.

[@psadi_]

I bet someone is already rewriting awk in rust for the memes.

[@alessandroferrari4699]

you mean for speed and safety? /s

[@rishirajsaikia1323]

@@alessandroferrari4699 people use rust because it's fast and easier to solve errors and even rivals c in speed while being easier.

[@TheSulross]

yes because obviously all these 40 year old Unix tools are buggy and broken after thousands upon thousands of users applying them daily over many decades

[@cgme7076]

@@TheSulross exactly 😂😂

[@arnoaldloukasscirpionis1558]

Well. People have already rewritten it in languages much less speedy than Rust - Go and even Common LISP. Knowing this, rewriting it in Rust is a much better idea !

[@gwgux]

This is one of the best explanations of awk I've seen! I'll save this one for some people I know that are still just learning Linux.

[@4lc0h0l]

This is great, this video arrived just in time since currently I am on problem automating a lot of stuff with bash scripting and using awk for text-processing. Thanks for the content!

[@OpusFocus1968]

Thank you Derek I appreciate the time and effort you put into making these videos. You help us "middle users" take things up a notch. Middle as in we are no longer beginners but won't ever be sysadmins. We do however want more out of their Linux system.

[@rochakgupta6116]

DT, you are amazing man. Yours are the only videos I have been watching on a daily basis. Thanks for sharing all the knowledge!

[@RobertGuilman]

Sir, your a wizard with awk, i only knew half of this 😂

[@pnavais]

Straight to the point & excellent as always. Keep up the good work DT ! 👏

[@DaveRogersEsq]

Needed a quick refresher on awk as I had no used it in years. This was a great video, covering the essentials. Cheers.

[@ourallabrador4023]

I have been using linux for years living on a few commands. I had only a faint knowledge of the command/programs such as awk or sed. Mainly because I had no ideas of their capabilities and use cases. Now I can see how useful they can be. I usually recommend your channel to my PhD students and observed a real improvement in their computer skills. IMO your channel is vital for linux wannabe and "familiar but not expert" users. So thank for your work.

[@ghodawalamoamanzahirabbas8996]

He also inspires me to learn about this powerful utilities.

[@skirnir393]

Good introduction to awk. Back in 2013 i had to parse huge log files that had no field separator, even worst each line could be completely different depending on certain conditions. Awk saved my life.

[@fabrice9848]

The simple fact that fish is your default shell deserves a like and a subscription.

[@h0ck3y66]

I love you man, you're the only one who was able to explain awk in a simple yet effective way. I finally understand how awk works!

[@davidruedeman9990]

Learned more about awk in 20 minutes than in the previous 40 years

[@fargoflagrant7796]

This is really helpful and practical, thank you DT!

[@abarocio80]

Good video. Some of these, I didn't knew. An error in the video: it wasn't the "square root", but just the "square", or "power of two". The square root of 4 is 2, the square root of 9 is 3, the square root of 25 is 5, and so on.

[@christopherdavies7403]

i came to the comments expecting every comment to be this. I am highly disappointed in the YT comments shit posting today.

[@gordonbai5320]

@@christopherdavies7403 And this guy says 'I didn't knew', grammar mistake.

[@charlieoocharlie9466]

@@gordonbai5320 ok boomer

[@weege5.45]

@@charlieoocharlie9466 FLAME WAR! And you live in your mother's basement.

[@Hexalyse]

awk is awesome, but if you just want the number of lines, using wc is less characters to type :)

[@TheSulross]

yeah, that was one example that did not resonate at all because wc will also provide totals when applied against multiple files - much less to type to just use wc

[@egor.okhterov]

The point is to show what you can do with awk =)

[@zainjadoon759]

I have been putting this off for a long time thanks for encouraging me to finally learn it.

[@jeffg4686]

Thanks for the share. I used use text editors like Textpad to do these things with regex, but this is so much better since not just find/replace, but perform operations on each result, and being able to easy have in a shell pipeline of commands. Always too lazy to learn it for some reason, but now I'm motivated.

[@mrdanymotion1903]

Hell yeah!!! AWK arrived at the right time!! Thank you Derek!

[@JoshBenore]

I have always had trouble with awk because of all the syntax involved, this helped sooo much. When you pressed enter on the square root part then said that the output was correct, this is the task list I had to perform to get back on track: 1. Facepalm 2. Pull fire alarm 3. Crying session 4. Shower with clothes on 5. Call Mom 6. Eat some sweets 7. Finish the video

[@wowgingermobile2123]

This is truly amazing, hope to have another new Awk video in the furure!

[@TheBlueThird]

DT, this is an excellent video! Thanks!

[@baruchben-david4196]

Thanks for this video. It helps me understand how to use awk better.

[@bradleyeverson697]

Hey DT. Great review of the power of "awk". You cover all the basic functions for a new user. I use "awk" in a shell script to sort through my datalog's results that i need. Then dump it to a csv file.

[@rickbhattacharya2334]

Thank you professor DT for the lecture .

[@balochcode]

Excellent presentation!! to the point and straight forward! great job!

[@Desi-qw9fc]

Great video. An accessibility note: Your console’s active line is at the very bottom of the screen, which is where KZbin on Smart TVs puts the subtitles, so the subs block the thing you’re typing. It would be better if your terminal was sized so that the bottom 10-20% of the screen isn’t used.

[@name1355_0ne]

Very nice introduction to awk features. Thanks for sharing this useful knowledge!

[@jacksystems]

Very useful info DT! I only knew how to print columns in awk.

[@keebler6591]

Great video DT! Thanks for the awesome awk tutorial Except: s/square\ root/square 😀

[@dragonsage6909]

That was some awesome awk-fu bro, thank you!

[@addaline9645]

I learned so much more here in 20 mins then I did in my last two classes about this

[@Jerry-fy2gc]

Thanks buddy for such clear explanations. Now I'm comfortable to start write some crazy scripts.

[@Lucas-md8gg]

My payment will be available in 2 weeks, then I'll be a patreon. Your content is amazing.

[@vn1384]

This is a great video on AWk..thanks

[@zeppelinmexicano]

This is a great review of some practical usages.

[@al1234]

This is the kind of stuff I love to watch

[@elad3958]

Great tutorial. I learned more in 20 mins w/ awk then in 7 years in IT.

[@datturns9947]

What did you do then in 7 years of IT? :))

[@eritert]

Awk is awesome, great vid I learned a lot

[@kirk0831]

Thank you for doing this video. Very helpful.

[@davidgomez79]

Great vid DT!

[@anthienvo]

Heaviest 20 minutes in my life. Worth it

[@ismoiljonabdumajidov9260]

i really appreciate your effort , thank you for the great lesson MAESTRO .

[@rogernevez5187]

Awk ... very meaningful name !!! Looks like an assembly instruction.

[@drzmuhammed]

This was helpful, thank you 😊

[@amanchauhan4356]

This was helpful…. Thanks to u and ur team

[@elderberry-hamster]

Great tutorial! I do prefer the WTF command. It covers all my confusion when trying to navigate the command line. 😜

[@Soupie62]

A little bit of Awk can be useful. My standard example: I have a list of shares I track. The website I use lets me download data, but holds more data (open, close, max, min,volume ) than I want. Awk is perfect for filtering a single column (close values).

[@korleone72]

that is a very cool podcast, very useful. thank you

[@umop3plsdn]

love both Sed and Awk

[@dumitrus.6925]

I enjoyed your presentation on awk and I subscribed to this channel. I have a question: Can awk to the following job -- send the first line of a file1, delimited by default space, to another file2, file2 that would take its name from the first few fields of the line 1 just transferred from the file1? Thank you

[@c434rdd410]

thanks awk command that help me a lot in my project after watching this video ^_^

[@bitti1975]

The problem with awk is that most people know it but all seem to think its whole purpose is just to extract columns. This Video gives many practical example that it's so much more.

[@seansretroverse9082]

Nice Luther insult at 8:20! I for real laughed out loud on that one. I'm surprised no one called that out in another comment...😁

[@christianalvarez8606]

Your audio sound 1000x better then before

[@user-ug4gy4bn1p]

Great video, helpful comments and even better resourceful and corrective people in the comments

[@mehdimahdavi6619]

Hi, thx for ur great videos reagarding Linux! I have a question: r u still using trackballs?

[@Winnetou17]

I think a very good example of how good & useful is awk is the example of a log like the apache access log (in the default format) and properly filter the requests that responded with a 400 or 503 response, given that you can find 400 and 503 in the request address, or in the size or the time it took to complete. Also to filter all requests that are longer than a second for example and many other things like that. It's basically Excel powers in CLI over column-structured data/files (which are common in commands outputs or log files)

[@calvinproject]

Amazing! Basically I can programming using awk. Thanks

[@DistroTube]

Yea, you can actually use "awk" as a scripting language. Might have been useful in past decades. But now, I'm not sure why anyone would script with awk when things like bash and python exist.

[@raushonmunni2633]

@@DistroTube maybe awk faster than python and more specific programming language bash or posix shell

[@sureshsurendran737]

Awsome presentation 👍

[@ToallpointsWest]

USEFUL!! Thank you!

[@akya]

Never needed awk in last 6-8 years or so... but thanks!

[@ghdbdeeeeeaqqq]

Excellent, thank you very much!

[@senthil_the_analyst]

Quality content 💯

[@Learnerofthings]

Great channel, great information.

[@chang112x]

Very Powerfull. Thx for the video.

[@SeekerOfSouls]

Awesome... good to know how versitile awk is.

[@vkg2958]

Good Morning Sir, I just wanted to know which Distro and terminal you're using here?.. it's so dope! 🔥 Also that fun message after entering wrong syntax LOL that's where I lost it..

[@adriancostin9138]

i always found it easy to use because i'm AWKward.

[@AWriterWandering]

I really need to push myself to use awk more. A lot of this can be done by just combining grep, sed, and cut, but it’s kinda nice having it in one process.

[@necronlord52]

As I found out, you have to sort _before_ running the _uniq_ , otherwise it wouldn't work.

[@flesz__]

Thanks! What mic are you using? Also 7:40 , you could use awk -F "/" '{print $NF}' /etc/shells for the same result Your df command gives output equivalent to df -h , I suppose you configured an alias?

[@aldeezy1]

small suggestion, bring the bottom of your terminal window up a bit so the youtube progress bar doesn't cover up the command you're entering when the video is paused. Thank you for making this!

[@warhawk_yt]

I know in most window managers you can change the size of a window from left or right but not on the top or bottom at least from what I have seen unless you are in floating mode than you can change it to any size. Most of the space around each window is configured hard coded in your window manager config. You could change it but than would have to change it back after a video.

[@aldeezy1]

@@warhawk_yt clearly he just needs to increase the edge padding to something crazy like 48-64 pixels for the [ M A X I M U M R I C E ]

[@Merigold83]

Your example for the line count can be reproduced with the command "wc -l /foo/bar/file_a /foo/bar/file_b"

[@magefreak9356]

Always great content DT! Is there a way around the case where the field separator is inside the entries? Like the ps -ef command output has spaces inside the content as well as spaces being the field separator.

[@locatemarbles]

I'm sure there is an awky way to do it, but in this case i would be lazy and choose sed.

[@TheJacov]

Yup, awk is cool. Just as an aside that was squares, not square roots. For square roosts you would need something like (n^(1/2)). AWK does have ^ which is raise to the power function. Raising a number to a fractional power is taking the root.

[@rolcarislew7761]

Yup, it's a complete language, as it is taught in CS degree courses (when taught). However this guy, otherwise, is a far more interesting teacher. Thanks, DT! This app/language confusion here is, like with Python, there is the language, AND there is ERIC, a free IDE that comes with Python. ERIC executes Python, but it's NOT Python. The AWK IDE is more invisible, but there. You can write independently, then execute.

[@braelinmichelus]

A couple months ago I wrote a little shell script to gather system information such as CPU model, memory, computer model, shell, WM (using lscpu, dmidecode, etc) and I had to learn a little bit of Awk, Sed, and Grep to clean it up and make it look pretty. This video would have came in handy back then, instead of spending 30 minutes reading StackExchange articles on them!

[@ghodawalamoamanzahirabbas8996]

Could you share your script? I would like to use it!

[@hendrikbonthuys9190]

Useful video thanks 🙏

[@AkashAgrawal03]

Thanks for making this video! I've tried a couple of times to add AWK to my arsenal, but gave up halfway. This has inspired me to give it a more serious go. I'm generally pretty comfortable with grep and sed but looks like I can make this work for replacing those two for many use cases.

[@pasej5]

like you said i should have learned this earlier. Good tutorial..

[@DavonAllen92]

David Draiman loves this program.

[@trido3815]

Thanks for sharing.

[@jacksongodsey]

This couldn't have come at a better time! I was seeing this command everyone in people's dwm configs and had no clue what it is.

[@janekmachnicki2593]

My favorite tutor

[@zur4203]

hey DT! i like your videos. they're very informative. now this is an older video and maybe you fixed it but when you have the terminal on thescreen the youtube navigation bar is covering the field and it is hard when pausing to see the commands.