Revisit Perl programming after 20 years - using Perl to code a generative language model

20 years ago, I started shifting development to Python. In this video, I revisit the Perl programming after 20 years and use it to build a simple generative text model based on Markov chains. I have forgotten many things about Perl, so a Perl guru will note many issues as I'm developing the code. The end product worked remarkably well, and I enjoyed getting reacquainted with Perl and its community.
I'll post a link to the github repo in the comments so you can take a closer look.

[@RandalLSchwartz]

At 06:47: the hash is initialized with the stringification of a newly created empty hash as a key, and undef as a value. It definitely is not making an empty hash.

[@wrongdogreckons-cp2sy]

You are correct, but you wrote the book that I learned Perl from, so I didn't even have to check to know you would be :) What happened here was the same as in the video script - I got switch confusion with Perl's hash and Python's dictionary notation. The line parses and executes but doesn't create an empty hash. Thanks for catching this - I'll put a note in the video description.

[@johnnapiorkowski4539]

Welcome back to Perl, we need more people like you!

[@wrongdogreckons-cp2sy]

Thanks - I'm super happy I revisited it!

[@user-vy5on4ju5b]

Thanks for the interesting lesson on the Perl 5 programming language

[@masteryoda0047]

a hidden gem on youtube

[@Pattelito]

I really enjoyed this video. Keep up the good work :)

[@wrongdogreckons-cp2sy]

Thank you! Will do! I am working on one on Crystal-Lang right now - its a lot of fun.

[@a_tabaza]

Never used perl, but this is very interesting to see as someone who's into LLMs, video production value is also remarkable, great work for real, kudos!

[@wrongdogreckons-cp2sy]

I really appreciate hearing that! My hope for this channel is to build out my video skills while I show how much I have to learn :)

[@brtastic4565]

Good job, amazing how much you can do with Perl even without expert knowledge of it.

[@wrongdogreckons-cp2sy]

That's what drew me to it in the first place - first year college and was able to follow along in the book enough to start making stuff happen right away. It really is an effective language.

[@IrredeemableGhost]

Perl will always have a special place for me

[@wrongdogreckons-cp2sy]

Thanks for sharing that - I agree, and I discuss this frequently with my partner: discovering Perl was like my breakout moment in using code to solve problems. I think the nature of it lets you think about problems and solution spaces in new ways.

[@IrredeemableGhost]

@@wrongdogreckons-cp2sy Imagine how freeing it was for someone like me who had only known cmd batch scripts until I was told about perl! Microsoft made a nightmare.

[@lorensims4846]

Perl is still the lead in Regular Expressions. I still judge programming languages based on how close their ease of use for regular expressions comes to that of Perl. So many programs and programming languages depend on the pcre (Perl-compatible regular expressions) library. The syntax of Perl is based on that of the Unix utilities grep, sed, and awk, just some of the tools Perl was designed to replace.

[@wrongdogreckons-cp2sy]

Absolutely - Perl has set the gold standard for regular expressions.

[@fplove]

Latest Perl _has_ signatures for subs, anyway. You can avoid the use of @_ and $_ at all.

[@wrongdogreckons-cp2sy]

Thanks for that! I did see that now and should have said 'don't require signatures' - it plays in to the 'There's more than one way to do it' :) You can do these things if you choose, and some do choose, but a lot of the code I've looked at do not use signatures and do use special variables a lot.

[@snepmilian]

@@wrongdogreckons-cp2sy Also, there are prototypes and signatures, which are different things. Libraries are usually made to run even on ancient Perls, so they rarely use those new features. Production code, however, does and it makes all the difference. I consider the lack of signatures a bug, for contemporary code, if there isn't some very exceptional use case for it.

[@jasonchesshir9551]

Don't forget "Pathologically Eclectic Rubbish Lister" as an acronym for Perl.

[@wrongdogreckons-cp2sy]

Good point! I love accuracy in naming :)

[@heroslippy6666]

Learned Perl recently for one of my courses which has since transitioned to python. I tend to use it for scripting on windows (instead of batch and whatever powershell uses). Its pretty good.

[@wrongdogreckons-cp2sy]

It makes you productive really quickly.

[@alanclaughan362]

Wow, this video brought back so many fond memories. Perl was my secret weapon for normalising data and repetitive task automation. It took little to no time to throw something together that made the C guru's turn green with envy - yes it was slower. Yet it got to the results much faster. Like yourself, I finally succumbed to the python flashy lights. I just blew the dust of some of my 30+ year old perl apps. Took a minute to get them running, but most still function perfectly. Had to look up quite a few short codes 🙂 Love your casual but informative style of video. I get the feeling this channel is going to explode!

[@wrongdogreckons-cp2sy]

Thanks! In many (most?) cases, development time is more important than run-time and Perl excels at rapid development. I absolutely geek out when I find some old code I wrote - and hats off to the Perl folks for staying so backwards compatible. Thanks for the note!

[@flyinglack]

I think readability is why I prefer Python over many languages. Of course, it is up to the developer to make a readable code.

[@wrongdogreckons-cp2sy]

Agree! I think that a lot of design decisions were made in Python specifically for that purpose. Remember PEP-20 (The Zen of Python) - there should be one obviously correct way to do it - I think that helps.

[@snepmilian]

Just for the bookkeeping: Perl nowadays *has* function arguments. The author kept his coding style from the 2000s, so unfortunately this video does not really reflect on what has changed since then, rather it shows, how much he still remembers. Still, great video!

[@Dcp404]

One of the design decisions in Perl is exactly why he was able to take a coding style he half-remembered from 20+ years ago and have it "just work" today. I think it's slightly unfortunate that the last five years of changes are hidden somewhat by the first 20 years of information on the web.

[@wrongdogreckons-cp2sy]

This thread has several good points: the video narrative (and code itself) is VERY biased by what I remember from Perl and what the first examples I found to searches like 'declare Perl hash' returned. Now I am interested in what first-time experience today might look like - it could be a considerably different view.

[@twenty-fifth420]

If I do learn Perl, I expect my transition to be somewhat like when I did Ruby then went to more modern Crystal. I think dynamic languages have uses for simplifying the productivity process and a series of scripts can do marvelous on data. I am on Linux now, I heard people tell me I can run Perl scripts like Bash ones, but like I implied, if I did learn Perl, I would like to have features of compilation. I also love the concept of Raku, so my level one project would probably be something like make a perl script do some bash stuff, and my level 20 is to make a compiler for both Perl and Raku. I also like how 'functional' it is, it is like an older Nim imo. Tell me, are sub-routines/procedural programming the primary imperative discipleine, or are there OOP features that can make my Ruby/Crystal heart flutter? Either way, I love this! Liked and subbed!

[@wrongdogreckons-cp2sy]

I've just started playing with Raku a little and can see where it has a lot of uses - particularly in making your own domain-specific-language. I have never paid attention to Crystal until your comment, but it looks amazing and can't wait to play with it a bit. I like the macro system and the built-in ability to call native libraries. Exciting stuff - thanks for the pointer!

[@andresgz]

The 'split' function typically takes a regex pattern as its first argument. However, if a string containing only a single space character is used (as shown at 10:24), it emulates the behavior of AWK: any leading whitespace in the expression is removed before splitting occurs, and any contiguous whitespace (not just a single space character) is employed as a separator.

[@wrongdogreckons-cp2sy]

This is a great call out - I think I am relatively safe there, since I am normalizing the white space on ingest - but another point that things don't always work the way you expect them to.

[@tuzzogetti]

For the record, what brings Perl to the table that Python don't in terms of features? There is applications in Python that i cant make in Perl and viceversa? or more small differences like pass by reference/value of Perl, that Python doesn't have? Greetings

[@wrongdogreckons-cp2sy]

This is a good question - but you can definitely do anything in Perl that you can in Python. There are differences in syntax and approach - and there will be different supporting libraries available - but Perl is a very capable language.

[@pirate21]

Perl is MUCH better than Python when it comes to regular expressions. Not even close.

[@wrongdogreckons-cp2sy]

@@pirate21 I agree with you that Perl has powerful regex, but I am curious about why you think it is much better than pythons. The only significant difference I can find is that Python doesn't support embedded code in regex - and there is a strong case for why no language should support embedded code in regex. On the other hand, Python does support variable length look-behinds, while Perl does not. Curious about the killer feature that Python is missing for you?

[@wrongdogreckons-cp2sy]

github.com/shane-mason/perl-chain-generator

[@EmilNicolaiePerhinschi]

... perl subs are pass by value; if you pass a reference it will send the value of that reference to the other side, which will be a reference to the same place ;-)

[@wrongdogreckons-cp2sy]

I think understand it best from the top comments on this thread from stack overflow on why I am saying pass by reference - in the end the function just gets a flat list of scalers and needs to unpack/dereference that list. stackoverflow.com/questions/5741042/are-perl-subroutines-call-by-reference-or-call-by-value My understanding is likely very wrong - which it definitely was at the beginning of this video - the point was that the approach is more confusing to me than Python's convention.

[@EmilNicolaiePerhinschi]

@@wrongdogreckons-cp2sy Perl is very different from the rest, and taught as if it was a normal language for example Perl is strong typed ;) except the types are "scalar", "array", "hash", "code" and the "sigil" is a "dereference operator" (think of * in C) ... once you get used to the sigil being a dereference operator Perl becomes a lot easier to read not trying to start a flame war, I really enjoyed your video

[@wrongdogreckons-cp2sy]

@@EmilNicolaiePerhinschi - Perl is indeed a very different language and has its own pragmas and dogmas. The best thing about it has been that the community has always been willing to help by sharing their understanding - just like this conversation. I always love learning and hearing about other's mental model. And I'm super happy that you enjoyed the video and I appreciate the conversation. I'm here to learn.

[@fuzzix]

@@wrongdogreckons-cp2sy You could use the signatures feature, enabled automatically if you `use v5.36` or higher. If you have an older version of perl, `use experimental 'signatures'` should enable some usable version of the feature. You *could* then use the `:prototype` attribute if you want to coerce pass by reference... Or better, just explicitly pass references when you need them ¯\_(ツ)_/¯

[@wrongdogreckons-cp2sy]

@@fuzzix - thanks for this, and it is indeed true: Whatever you want to do, Perl offers many ways to do it :)