I wrote an ebook on GNU awk with hundreds of examples and exercises

[learnbyexample@programming.dev]

Hello!

I am pleased to announce a new version of my CLI text processing with GNU awk ebook. This book will dive deep into field processing, show examples for filtering features, multiple file processing, how to construct solutions that depend on multiple records, how to compare records and fields between two or more files, how to identify duplicates while maintaining input order and so on. Regular expressions will also be discussed in detail.

Book links

To celebrate the new release, you can download the PDF/EPUB versions for free till 06-April-2025.

• https://learnbyexample.gumroad.com/l/gnu_awk
• https://leanpub.com/gnu_awk/c/free

Or, you can read it online at https://learnbyexample.github.io/learn_gnuawk/

Interactive TUI apps

• 80+ exercises: https://github.com/learnbyexample/TUI-apps/blob/main/AwkExercises
• Brief tutorial: https://github.com/learnbyexample/TUI-apps/blob/main/AwkTutorial

Feedback

I would highly appreciate it if you'd let me know how you felt about this book. It could be anything from a simple thank you, pointing out a typo, mistakes in code snippets, which aspects of the book worked for you (or didn't!) and so on.

Happy learning

[jimbabwe@lemmy.world]

Awesome! Thanks so much for doing this and sharing!

[eldavi@lemmy.ml]

i'm in awe everytime people do this.

i learned how to do this before code sharing sites like github existed and it forced me to turn everything i've learned into muscle memory and i think that this next generation of greybeards are going to be so much better than my generation's greybeards because of it.

[magister@lemmy.world]

I like awk, I started using it in the 90s in university, I'm not a pro, but this is so powerful!

[swab148@lemm.ee]

Finally, time to learn how to use awk! Sed, you're next.

[enkers@sh.itjust.works]

Could someone perhaps explain the major use cases or give a real life example of a time you've needed to use awk? I've been using Linux casually for quite a long time now, and although I learned the basics of the tool, I can't recall having ever felt I had a need for it. If I want to glue a bunch of cli stuff together and need to do some text processing, it generally seems like it'd be easier to just use a simple python script.

Is it more for situations that need to be compatible with most *nix systems and you might not necessarily have access to a higher level scripting language?

[learnbyexample@programming.dev]

Well, if you are comfortable with Python scripts, there's not much reason to switch to awk. Unless perhaps you are equating awk to Python as scripting languages instead of CLI usage (like grep, sed, cut, etc) as my ebook focuses on. For example, if you have space separated columns of data, awk '{print $2}' will give you just the second column (no need to write a script when a simple one-liner will do). This of course also allows you to integrate with shell features (like globs).

As a practical example, I use awk to filter and process particular entries from financial data (which is in csv format). Just a case of easily arriving at a solution in a single line of code (which I then save it for future use).

[commander@lemmy.world]

I'm not expert in sed or awk. I always have to Google. For me though, it's generally that you can do a great deal in just one line of awk or sed. They're standard on any Linux distribution I've ever used. When building out pipelines, scripts that you want run from an installer you built post install and when removing, sed and awk rather than needing python.

All really nice when you have strict configuration management and versioning and there's something deployed but it doesn't have the python packages installed that would make it easy in python and you can't just pip install it on hundreds+ of computers without going through a process of approval and building a new tagged version release but sed/awk/etc can do the job. If it's hard enough, python and whatever packages you can install. If simple enough to do in a small bash script, no python just what's standard in your Linux distro

[balder1993@programming.dev]

Also AWK is made to be fast, right? I suppose doing something in Python in a non efficient way might not be noticeable with a bit of text, but would show up with a large enough data stream.

[silverhand@reddthat.com]

Beautiful, thank you so much!

[danitos@reddthat.com]

I manage some servers and awk can be useful to filter data. If you use commands like grep, and use the pipe operator (the " | ") command, awk can be very handy.

Sure, a Python script can do that as well, but doing a one-liner in Bash is waaay faster to program.

[anamericanpotato@programming.dev]

Is it more for situations that need to be compatible with most *nix systems and you might not necessarily have access to a higher level scripting language?

Yes, and also because integrating Python one-liners into shell pipelines is awkward in general. I'm more likely to write my entire script in Python than to use it just for text processing, and a lot of the time that's just a pain. Python isn't really designed for one-liners or for use as a shell. You can twist it into working in those use cases, but then I'd ask the reverse question: why would you do that when you could "just" use awk?

On macOS, Python is not installed by default. So if you are writing scripts that you want to be portable across platforms, or for general Mac administration, using Python is a burden.

This is also true when working with some embedded devices. IIRC I can ssh into my router and use awk (thanks to it being included in Busybox), but I'm definitely not going to install an entire Python environment there. I'm not sure there'd even be enough storage space for that.

[n0x0n@lemmy.ml]

I'm not expert in sed or awk. I always have to Google. For me though, it's generally that you can do a great deal in just one line of awk or sed.

Same here ! I recently used a one liner awk piped into sed, piped into another command to find duplicated lines and merge both files.

Writing a python script would have taken an unknown amount of time !

[harsh3466@lemmy.ml]

Perfect. I'm feeling comfortable enough with bash that next on my list is AWK. Gonna download this when I get home!

[tuckerm@feddit.online]

Looking forward to reading it! awk has been a huge blind spot for me for a long time now.

[jagged_circle@feddit.nl]

Thanks but omg no. I'm happy to use awk like cut.

[vfsh@lemmy.blahaj.zone]

I thought your comment was going in a totally different direction, it's nice to hear appreciation of improved teaching methods instead of the old "well I figured it out myself so everyone else should too"

[monkdervierte@lemmy.ml]

Btw, there's asciidoctor-epub3.

[dino@discuss.tchncs.de]

You misstyped fish.

[harsh3466@lemmy.ml]

Heh. I haven't started exploring other shells yet.