Why Are Lisp Macros So Great!?

This tutorial is focused on Common Lisp and Emacs Lisp macros. Lisp macros are one of the things that has kept lisp relevant to this day. Since no matter how languages try to mimic them they just never fit as well as they do in Lisps. I hope to do a future video on hygienic macros like you see in Scheme and Clojure. In addition, cover macros on a deeper level in the future.
- More Example Macros
lisp-journey.gitlab.io/blog/c...
- Common Lisp Pattern Matching
github.com/guicho271828/trivia
- C syntax in Common Lisp
github.com/y2q-actionman/with...
- Clojure's core.async macro internals
• Core Async Go Macro In...
Let me know what you think and if there is anything specific you would like to learn about.
---------------► Wanna Support Me? ◀︎---------------
Github:
www.github.com/sponsors/gavinok
Patreon:
/ gavinfreeborn
---------------► Wanna Checkout My Dots? ◀︎---------------
Vimrc:
github.com/Gavinok/dotvim
Emacs Config:
github.com/Gavinok/emacs.d
---------------► Come Join The Community◀︎---------------
LBRY:
odysee.com/@GavinFreeborn:d
Matrix Space:
matrix.to/#/%23gavinfreeborn:...
Discord:
/ discord

[@mmille10]

My experience with macros in Lisp is that they go through two phases: The first phase is where you use code to construct the code you want evaluated, once the macro "exits." In this phase, the environment is not available to the macro. Only the parameters that are passed into the macro are available. In the second phase, the macro parameters disappear, but the constructed code has access to the environment, and it's evaluated in that context. This is why when you tried '(+ a b) inside the addm macro, you got the "a is not a variable" error, because the constructed expression (+ a b) only became available for evaluation in the second phase, when the macro parameters had gone away, and the global environment did not have a or b defined. With quasiquoting, (and using ,), you can evaluate a and b in the first phase, wrapping the result inside a quoted list, which is then evaluated in the first phrase. Then, the second-phase environment sees (+ 1 2), which results in 3. Side note: The reason the first macro you constructed with (+ a b) worked was that the a and b parameters were available in the first phase, and since the expression was not quoted, the result was 3. In Lisp, numbers are special atoms, which evaluate to themselves, so when the result (3) is evaluated in the second phase, the result is still 3.

[@Pariatech]

All my personnel projects are in Common Lisp nowadays. Being able to so much repetitive code with macro is a godsend. I have yet to do all the real fancy stuff you can do with them (Just look at Let over Lamba for examples). It's so frustrating when doing my day job and have to deal with all the boilerplate of inferior languages.

[@NathanHedglin]

LISP is eternally beautiful

[@jrgomez]

I'm still dipping my toes into lisp world and I'm yet impressed on what the language can do! Thanks!

[@dantheman52420]

Love this explanation. Must be why Lisp isn't used on big teams. Can you imagine the macro hell, a new C++ codebase is often daunting to study just to figure out which of 30 possible paradigms they're following and whether the macros are just a constant or a fully pre-compiled http server. But now I have to do a side project in Lisp to find out what macros can really do.

[@PixelOutlaw]

Nice little demo. :) Somewhere along the line, the Clojure-like lisps (there are a handful) lost the "Programmable Programming Language" motto. It's sad, they've got macros but it's largely a badge of honor to avoid them in that community. Macros take time, documentation, and understanding. But nobody complains when they use one like COND. Some top uses for macros: 1. Creating a complex piece of code from some simple code. (LOOP is a great example) 2. Changing program flow COND (and a ton more) 3. Custom sequences that let you use an iterator inside. (DOTIMES). 4. Setting up an environment around a command WITH-OPEN-FILE 5. New syntax and Domain Specific Languages. (think LOOP which doesn't look like Lisp) The macros mean YOU the programmer could have added these. Even if Common Lisp didn't ship with these forms, macros allow YOU to make them. That's very different than other languages where the syntax is wholly defined by somebody else. THAT is the power of macros, they leave the door open to add your own syntax not just functions.

[@joachimschmidt7662]

I'd like to thank you for your videos, Gavin. They are really excellent.

[@MENTOKz]

gavin u the best man you push me to keep learning programing. i am doing golang right now

[@itsfarseen]

Thank you so much for making this video ❤

[@sdsa007]

wow I finally understand macros, I finally know how to play with them!

[@hajovonta6300]

The material is excellent but the audio is a bit out of sync from the video, which is a bit disturbing.

[@pranavp.a1200]

Watch it in 1.5x speed

[@MajinHico]

Thank you.

[@kuhluhOG]

from my experience, language which let you construct a language "inside" of it (be it with macros or other things) either: - end up barely used in the wider industry - the construct isn't used to actually do this in practice (be it because people forbid each other from doing so or other reasons) the main reason is because these languages turn very quickly into some sort of barely maintainable mess (for other people) if these constructs are used to do that so, I am quite wary of things like macros btw, one small things about Zig: I wouldn't say that Zig's comptime are macros. And if, it's a very limited form of it (which is good imo). In comptime you just say 1. the code gets run at compile time and 2. types are also values. So it's a form of compile time reflection.

[@GavinFreeborn]

From what I've seen zig uses comptime for far more than reflection though that is one use I've seen. I've seen it used similar to constexpr in c++. I may have given zig a bit too much credit. I more wanted to relate the concept of a compile time function. Templates also make a similar attempt but comptime is far closer to what you get in common lisp. Another example of a language that mimics some of lisps macros that has gained a lot of adoption would be rust with its proc macros. You will see tons of libraries making use of them to simplify code. If you ask me features like this often don't sell a language to people for good reason though they will be used as an example to sell its power. That is just my opinion though not a fact.

[@kuhluhOG]

@@GavinFreeborn Imo there is just a big difference between code which gets executed at compile time and macros. The deciding factor imo is that macros work on a level a above the language, be it at AST level or something else, while code which gets executed at compile time does not.

[@aoeu256]

Imagine if on the benchmarks the lispers generated C and Rust or whoever won code instead of using OPTIMIZE method

[@Lantos1618]

Macros always make me feel so smooth brain. It's like regex to me. Only hack it together and keep running it until it works.

[@PMA65537]

Starts at 3:56

[@trannusaran6164]

My go-to channel for Lisp-2 content when I'm not in a Lisp-1 mood ;P

[@garklein8089]

know of any good lisp-1 channels?

[@trannusaran6164]

@@garklein8089 I might be starting one to fill the gap in the market lol

[@modolief]

Could you do a deep dive on SectorLISP sometime? From the webs: SectorLISP consists of 223 lines of assembly. It provides a LISP system that's powerful enough to let you write your own LISP interpreter in just 40 lines of LISP. It's compatible with all PC models dating back to 1981 which have at least 64kb of RAM. This isn't a toy because SectorLISP can run the proof assistant that was included in LISP 1.5.

[@kuhluhOG]

How long are these 40 lines?

[@Freddy78909]

Watch the video "The Curse of Lisp". The strengths are also weaknesses. Like rolling your own syntax using macros might be fun for computer nerds but horrible in a production setting

[@pavelxbushmakin7252]

What about Kernel Lisp where macroses is first-class members like functions

[@Gepapad0]

I'm a total lisp noob but from what I've seen by looking at elisp code I think (defmacro concat-comp (&rest strings) (apply #'concatenate 'string strings)) is a bit more idiomatic. I have no idea how it affects performance (though I suspect - biased as I am coming from a C background - macros are rarely called with enough arguments for that to be a concern).

[@Gepapad0]

I just tested it in Emacs and it turns out that using apply is, in this case, insanely faster. I tested it against /etc/dictionaries-common/words. Probably the same story as StringBuilder in Java, which makes total sense.

[@GavinFreeborn]

Ya that would probably be a better fit. Was just kinda going off the top of my head. I in most languages use (reduce #'concat strings) in elisp and the equivalent. In cl though apply would definitely have been the better choice

[@kuhluhOG]

0:12 The main reason is not the lack of macros, it's the absurd amount (and span) of implicit conversions. At least imo.

[@GavinFreeborn]

I'd say that is part of the problem though with macros a lot of these pain points could have been resolved. Unfortunately even the modern revisions suffer from those implicit type conversions

[@defnlife1683]

Is it me or is lisp insanely powerful? I was just looking at some lisps made for embedded systems. insanely simple.

[@fakeaddress]

Can u give some examples? Curious

[@aoeu256]

I heard that lisp is so powerful so that every coder made their own version of Lisp and no one wanted to work with other peoples lisp code since they made their own eDSLs. Although maybe it’s not true, if you want to hire Lisp coders to do full stack dev, or maybe make macros to optimize your code plz hire me.

[@munzamt]

There is no need for macros in JavaScript. This example with "oh, there is a goddamn var, better wrap it in a macro!" is a spherical cow in a vacuum. No one in a sane uses var in a modern JS. We use const (and const as const in TypeScript, especially). We use let. We don't handle a lot of repetitive code that needs to be somehow simplified. And especially, we don't need to parse something as a raw tokens to construct some monstrosity object. Building tools got us all the optimization we needed, such as for const propagation, string concatenations, and evaluation on literals. In some specific cases, when we want some external parsing, we can use tagged templates. Prisma uses it. ApolloClient uses it.

[@GavinFreeborn]

first of all thank you for the feedback. I think you may have missed my point. I realize that the language has evolved. My point was that the language had to change people had to update ecmascript to make it what it is today. I know and illude to the existence of better options in modern ES (kinda figure it's common knowledge these days). If macros existed JavaScript would have been capable of this from the start as well as features it still lacks that people desire like pattern matching. This isn't an issue with lisps. Common Lisp to this day has been able to evolve without a need for a new standard to be created. Since macros allow for heavy extension of the language lisps will nearly always be capable of supporting a hot new language feature such as the aforementioned pattern matching. In js land we are mostly just stuck and it's a lot of the reason JS has had such a bad stigma surrounding it. There are compilers like bable which kinda fill this hole but it's a far cry from being able to quickly extend the language the same way a simple macro can. Does any language "need" macros? No, but can many languages benefit from them? Yes

[@munzamt]

@@GavinFreeborn I got it, but I still won't agree. ECMAScript evolves in a certain way for some reasons. Most of all, there is a deal: JavaScript has bad design from the start. It has a lot of approval (more than needed). For example, typeof null is an object. It was a mistake, and then it was just fixed in the standard. Most of that JS has a bad stigma, and the new feature won't fix it but will only add complexity to the already confusing language. About all kinds of language extensions, I adhere to an opinion: macros are good at constructing something that would be hard or even impossible to implement in a language. Rust frontend framework Yew got the advantage of using macros for the purpose of constructing HTML elements. Can we deal without it? Through the React perspective, yes (React.createElement is to confirm this). But do we need all this complexity in Rust? Ofcouse no. Other languages have different ways to handle extensibility. Swift and many functional languages allow you to create custom operators. Rust and Common Lisp use macros. C# uses changes in .NET runtime and many "flags" for applying changes (because many C# features are just syntactical sugar). But do we need a tool to extend a language in any way if we use this language in many browsers? If we do that, it will be a mess when ECMAScript implements something in the standard for which many developers have already written a lot of macros. This is more than a guess. I already have troubles when building projects because of many warnings like "Modern JS already guarantees Array#sort() is a stable sort..." and blah-blah. It's very difficult to make changes to a badly designed language. Imagine that everyone could do it. Moreover, I still can't come up with a good way to use JS macros. You know, pattern matching isn't the number one requirement in JS.

[@GavinFreeborn]

@@munzamt totally reasonable. Everyone is entitled to their own opinions. For the record I'm not against tools or languages evolving. I just think there is a lot of power in macros and while it can be too powerful to some. To me it's a nice to have and my example is just to show that they could have saved us years of pain and possibly could have made typescript runable in the browser natively. On the topic of "what if everyone could change the language" it might sound scary but from what I've seen with lisp like react, typescript etc the more widely enjoyed extensions and frameworks are the ones that become adopted. It definitely could come at a cost. It's more a case study than an actual ecmascript proposal of course. As of now JS is good enough for what most people want and there are already proposals for features I'd love to see.

[@aoeu256]

There are lots of things you can do with macros that js needs like a macro to turn a async to normal function and back again, macros to optimize code like custom compilers, macros to add static analysis to your code, a macro that tells you that code is static low level like C so compile to optimized web assembly, run code at compile time instead of runtime to optimize inner loops, etc…. Other features that are nice in lisp other than macros are image based development, optimizing compiler, condition system that has hand,eval without unwinding the stack. I heard Julia is based on Lisp, I wonder if you can use web assembly with that language online hmm.

[@EdouardTavinor]

Mm, goroutines aren't the difficult thing. The difficult thing is the select statement.

[@GavinFreeborn]

Which in Clojure was also implemented as part of core.async (what I was referring to as implementing goroutines). They are implemented in channels. Making a Go like select statement notation would be a trivial macro which I have a feeling already exists. Select statements are just a DSL for non-deterministically selecting between channels without requiring you yo spin the CPU. I know guile's fibers library implements selection as a macro in a similar way.

[@EdouardTavinor]

@@GavinFreeborn mm, I'd like to see that. The trouble is, you don't know which channel you need to select until something turns up on one of them. I'll have a look at this, but my intuition tells me that this is impossible to implement without using the operating system API. I really don't know how I'd go about writing my own select.

[@GavinFreeborn]

​@@EdouardTavinor ``` (go (let [c1 (chan) c2 (chan)] (go (while true (let [[v ch] (alts!! [c1 c2])] (println "Read" v "from" ch)))) (!! c1 "hi") (>!! c2 "there"))) ``` For reference here is the docstring for `alts!!` Like alts!, except takes will be made as if by !!, will block until completed. Not intended for use in direct or transitive calls from (go ...) blocks. Use the clojure.core.async.go-checking flag to detect invalid use (see namespace docs). github.com/clojure/core.async/blob/6ac8ed2a26c67bbc3cc43e4e650f97c984666b0c/src/main/clojure/cljs/core/async.cljs#L175-L201 Gives a little bit of a peek under the hood showing how all the locking is being done behind the scenes.

[@GavinFreeborn]

This also gives an overview of the general implementation as well kzbin.info/www/bejne/nn6oiWmim6eYiK8

[@EdouardTavinor]

@@GavinFreeborn cool, that looks really interesting. almost an exact copy of go :) i'll try to write a small project with it :) it certainly looks much easier to reason about than the threading primitives found in java :/

[@imeatingboiledeggrightnow7644]

lisp is the best language whahahahahah

[@ayoubelmhamdi7920]

are you only understand the lisp ?!