Solving Problems the Clojure Way - Rafal Dittwald

After overcoming a fear of brackets, the next challenge for would-be Clojurians is less superficial: to stop writing Java (or Javascript, or Haskell...) with Clojure's syntax, and actually start "thinking" in Clojure. It is said that Clojure is a "functional" programming language; there's also talk of "data-driven" programming. What are these things? Are they any good? Why are they good? In this talk, Rafal attempts to distill the particular blend of functional and data-driven programming that makes up "idiomatic Clojure", clarify what it looks like in practise (with real-world examples), and reflect on how Clojure's conventions came to be and how they continue to evolve.

[@l4zy_jake]

22:14 “Minimize, concentrate and defer. What’s defer? Well, we’ll get to that in a moment.” Why am I the only one who thinks that was totally hilarious!? XD. (Great talk, BTW.)

[@jonarmani8654]

You stole the comment from my fingertips.

[@inanitas]

You are not the only one, you never are. I hate this phrase. I am not going to say "Am I the only who hates this phrase?", because I am not.

[@colindupee]

@@inanitas I think his point is that there was no reaction from the audience. It seems possible that Rafal didn't even mean it as a Dad Joke, but it was a class-V Dad Joke.

[@VojimirGolem]

Excellent talk, not Clojure specific, recommended for everyone interested in differences between programming paradigms (procedural, oop, functional, data-driven). If you are not interested in Clojure just skip first eight minutes.

[@seastove]

Been said many times already but this is by far the best talk I've seen explaining functional programming. Thank you so much Rafal!

[@TRex-fu7bt]

what’s cool about this functional programming demo here is that it did not stress higher order functions or composition at all. it was just about managing state which is the hard part of a system.

[@robertsmme]

Amazing talk. Well structured, great examples - I think one of the best talks I have heard on this topic

[@softwareminimalist]

Great talk and exactly what I have been saying for a couple of years. People think that OO was invented to help modularize code, which is actually only a side effect of it. The original purpose was to limit the scope in which b- reference passing of data was happening. Without OO you had the whole application as the scope in which one part of code can change data in another part of code, which made it impossible manage. With OO you got these objects which were simply an encapsulation of a smaller part of the design that can modify data by-reference. More manageable, great! It also forced designers to think in a more modular way and started a discussion about design patterns and principles. Through this discussion many understood the trend of minimizing the scope in which by-reference changes happen, and taken to its conclusion, it meant that functions should just work on their arguments and return their output, thus be pure. The scope should have always been is only individual function. THIS IS DATAFLOW. I summarized this as "Software architecture design is understanding of the flow of data, its sources and sinks and type definitions around this flow." This is the way how code should have been written the moment we created Virtual Machines and Runtimes, handling memory for us. Instead the industry got stuck using a little more sophisticated assembler (Java, C++, Objective C, C#, etc.) and kidding ourselves we understand it. We created SOLID principles and patterns, simply to fix this problem that persisted still i.e. passing data by reference. The answer was staring us in the face in the form DATAFLOW in functional languages like LISP with immutable data and first class functions and graphical languages like LabVIEW with actual dataflow as the basis of the language. Now the industry is waking up, thanks to the rising popularity of Clojure, for which I am eternally grateful to Rich Hickey for the genius idea of LISP on JVM. With a functional language the SOLID principles naturally vaporize, they are not needed, because all the polymorphic extensibility is done by simply passing pure functions by-value as first citizens of the language. SOLID patterns also vaporize for the same reason. The only patterns that persist are those related to modularizing code and using external dependencies, but that is an orthogonal issue. Soon the tides will turn and we will collectively all understand that its not us that need to adapt to writing machine language like code, but rather the machine should read and understand what we want to achieve. Personally I believe the future is graphical programming in style similar to LabVIEW, but it would need to become opensource, and there are other big hurdles, like ecosystem and popularity. Until then I love and will use Clojure for everything, the last programming language we will ever need :) Do you agree? Sorry for the long post, I will make it into a presentation soon ;)

[@kwccoin3115]

Hope you can give a link here for the presentation. Thanks.

[@noircc]

@@kwccoin3115 Presentation found here : kzbin.info/www/bejne/pp3Qqmdqapd8bsU

[@ValentinKostadinov]

Great talk. Keeps it simple and gives one of the clearest, most pragmatic and easy to understand explanations of the key aspects of functional programming. I've done lots of React, some Haskell and I'm now learning Clojure for a new job. Made perfect sense to me.

[@theteachr]

Nitpicking, the term "pure" is used incorrectly. The extracted functions don't have side effects, which is only one of the requirements of purity. Some of them are not deterministic, which is also a requirement for functions to be marked as pure.

[@mertnuhoglu]

The refactoring was excellent. Thank you very much for making those abstract concepts so accessible.

[@proxy9321]

22:08 "...and deferring, well, well we'll get to deferring in a moment." LOL well played

[@natassiatavares4568]

This is exactly what I wanted to know in an incredibly easy way. Thank you so much

[@kimblemojimble7967]

The functional refactor was great to see in action. Looking for state, mutation and external side-effect was a really great way to demystify the thought process. Any chance you could supply the code (imperative, oo, and functional)? I wouldn't mind printing it out (old school) and scratching my way through it.

[@noircc]

I would be also interested

[@PixelOutlaw]

Big floppy maps and vectors become a mess over time. Prefer composable piecewise functions which generate what you need clearly and nest those. You'll get MUCH more power that way and you don't enter the brittle realm of copy/paste nonsense inherent in *only* pure data structures. And if your structure changes you'll adjust it in 1 place vs ALL over your code peppered with literal data representations. An unbound map literal is just a more advanced version of "magic number" in your program. ;; Clarity through composition shows intention (make-house ; Generates vector to hold rooms (make-room "foo") ; Generates {:room "name"} (make-room "bar")) ;; Better yet (make-house (make-room "foo" "bar" "baz" "qux")) ; make-room now multiple arity with & rest ;; Floppy garbage [{:room "foo"} {:room "bar"}] ; What does this represent? A house? An office? A prison? Nobody knows!

[@chrispaul8050]

This was a really great talk, thank you for uploading

[@alextjn]

Great talk. Here's my takeaway from it Process of making code more functional (eliminating impure code) 1. move all impure stuff to the top (the top component or a top function that calls everything else) 2. refactor out pure code from the top func 3. reorganize all mutable states into one (global) state object. 4. A dispatch function to return a new state (instead of mutating) 5. [Extra] instead of keeping a mutable state object, use a states array to keep all intermediary states. The top function is just a function to turn out states. 6. [Extra] use recursion to eliminate loop and mutation of state

Thank you so much for this talk! It was amazing!

[@user-fr3qm7rf5q]

Amazing talk!

[@NathanFranckGameDev]

Awesome talk, this really illuminates my thought process and journey of fighting against OOP and brings it home in a way I didn't know I needed. I will have to try Clojure out! I've been trying to force some of the described ideals into javascript and typescript with decent success but I clojure might be the ultimate last leg of my journey

[@kwccoin3115]

It is one of the best video. But without interaction with the user even just simple terminal I/O, no external meant you can function all the way but that is not how reality work.

[@vladislavlisovskii8904]

Really really like style of this talk! Rafal, you should do it more often! 100 likes for that vide and you!

[@georgH]

35:10 "as a haskell programmer I would do the middle one" Maybe you could use the reader monad to pass the state, which would look like the right one in practice, yet being statically type-checked

Great refactoring! Also, pretty cool categorization of methods to avoid side-effects! 😄

[@zed9zed]

I absolutely love this functional refactor tour. I would happily watch a whole series of these kinds of refactors applied to different problems. The final version of the game code with just one green line... fantastic.

[@jmrah]

Wonderful talk. The big take-away for me was the three idiomatic "pillars" of functional code that was talked about: Minimize, Concentrate and Defer. I'll be remembering those. I think it's important to point out that the final functional code you ended up with could still organized in an OO style - that is, organize the functions with the data they operate on (methods), and make the methods pure. In fact, I think if you're forced to work in an OO language (like Java), favoring a 'functional-OO' style is still beneficial for all the reasons outlined in this talk.

[@rafd]

It's a practice in some OO circle nowadays to have 'data objects' (structs with just data) and 'method objects' (no data, but methods that act on data objects; or if immutable, return new data objects; in effect, since these are just 'collections of functions' they are like Clojure's namespaces --- which fun fact, can be compiled down to a singleton object with methods).

[@jmrah]

@@rafd I'm going to have to give that architecture a shot on my next personal Java project.

[@bobbycrosby9765]

Lots of people program like you outline. It's just harder, and less efficient, in languages that aren't written to support it. The libraries you use probably won't work like this either. Furthermore, separating out your operations that interact with the world from the rest of your code has been good OO practice for a long time. For example, it's very difficult to test a class that both reads a file and parses it, so those two things should be separated.

[@DomainObject]

Awesome talk.

[@jujijiju6929]

I like how he deferred defining the defer technique. Lmao, very clever.

[@neilclay5835]

This is a brilliant talk.

[@johnblomberg389]

There is a bug in the refactored code, the first iteration of the code was logging 'state.turn' before it was updated to 'state.turn += 1'

[@androkon6920]

I think I kinda get it. A pure function might be more verbose, but what it grants you is to make the everything else less verbose. And the drive to deriving implicit information in as many places as possible, instead getting it the boring way.

[@brdgamz]

This perfectly demonstrates how to tease the pure out of the impure, the thing I want programmers who haven't seen it to see. Thanks for taking the time to showcase it.

[@elarous]

A good talk, thank you

[@coppurt]

This talk is such a hidden gem, it's one of the very few I've got backed up locally just in case

[@rastislavsvoboda4363]

I'm not clojure programmer, just started to look at it, but this talk was very cool, and refactoring imperative to FP style was great however you made couple of bugs during this process: - newScores() adds just 1, not bountyCard - report() should skip initial state, there are no lastBountyCard and lastPlayedCards values for initial state (and lastState is not defined) - endCondition should be flipped (using not or length == 0) (and lastState is not defined) anyway, it was cool and I wish you have couple more talks where you show how to "transition to FP", thanks!

[@sifbiri4269]

Excelent talk, well chosen examples.

[@kukasmog]

Fantastic, thanks!

[@11STANE11]

great talk! Thanks

[@JihChiLee]

Its really great talk! thanks!

[@user-rz1lw2dh3b]

Absolutely brilliant.

[@nicknesler]

Great talk, thank you for walking us through that. Is there a bug in the recur function? The end condition looks like it would end on the first iteration. Also, the recursive call to recur() doesn't have the last two parameters. It's ok if there's a bug, I just want to make sure I'm understanding everything. Thank you again for all this information!

[@rafd]

Yes, you're right on both counts. The endCondition should either be negated in the recur, or changed to == 0 at its source. The recursive call is missing the extra parameters. (Powerpoint is not a good IDE :P )

[@nicknesler]

lol, understood. Thank you for the response.

[@purposepowerlove]

Fantastic talk. Thank you very much.

[@uday4816]

Such a great talk !

[@hank-uh1zq]

Question: did Smalltalk programmers realize all of this eons ago?

[@davidmark1673]

Pretty much yes - smalltalk is 90% functional and 10% OO by his description

[@jofla]

57:50 is just mind blowing

[@halneufmille]

Wow I think it's the first time I really get what functional programming is all about. I still can't get fully on board with the recursions, the accumulation of function calls and the accumulation of useless state variables though. A question to KZbin's functional programmers out there, would it be possible to specify, instead of state_k = f(state_k-1) something like state_k = f(state_k-1) and by the way store state_k where state_k-1 was because I don't need it anymore?

[@bobbycrosby9765]

This is going to be language specific. Clojure has immutable, persistent data structures. When you modify a vector or list, it shares the memory that didn't change. When you change a map (which is what you use for data structures), the keys that didn't change are shared. It will still be more expensive than just mutating in place, but it isn't as expensive as literally allocating a new chunk of state and copying everything over. If you absolutely must mutate state for performance reasons in Clojure (measure before you do this), Clojure aims to be a pragmatic language and has great interop with the host (JVM for Clojure, JavaScript for ClojureScript), so you can just go with mutable state if you really, really need to. For Elixir/Erlang, it has the persistent, immutable data structures too, but you tend to organize your code by Processes, and all data is deep copied when it crosses the process boundary. This has certain benefits (e.g. each process has its own garbage collection), but also certain negatives - so you need to be careful of your process boundaries. I don't know much about the other FP languages.

[@davidmark1673]

yes - if you only have one reference to the prior state you can mutate in place (there's no-one else around who can see you doing that) - it's all about aliases

[@EvenStarLoveAnanda]

What is it he was saying in the beginning? Peredit? I could not find it.

[@fnumatic5558]

paredit A way of structual editing source code. You don't edit lines of code but things inside braces. calva.io/paredit/ kzbin.info/www/bejne/q5PGdoWqed5_etk github.com/shaunlebron/history-of-lisp-parens

[@diegonayalazo]

Thanks

[@OlimSaidov]

Good talk. Little note: selectRandom is not pure

[@De1n1ol]

It is shown in the right bottom corner at 43:06

[@kahnfatman]

2022 -- enter AWS-CDK -- the snake bytes its own tail: AWS Cloud Development Kit (CDK) accelerates cloud development using common programming languages to model your applications.

[@sangkim6338]

This is superb..

[@qu4ku]

that was, actually, a good idea to don't use clojure code in here.

[@user-sd7hh8ek1c]

I hate to nitpick, but can you really call `selectRandom` a pure function? It can produce different outputs on the same input. Anyway, I really enjoyed the talk, thank you.

[@johngibson4874]

You are not nitpicking. You are correct. He also addressed this in his talk. It is at the bottom right corner at 43:06.

[@rafd]

You are correct. It has hidden state. In languages that support giving rand() a seed, we could have made this state explicit.

[@davidmark1673]

pseudo random number generation itself is pure but many libraries keep the state in a global variable

[@Marc6666]

Link to the talk that he mentions at the end: kzbin.info/www/bejne/sKvRqH55nst2Z6M

[@Verrisin]

58:00 - that's rather different. They just describe their API in data, then generate clients for it. It's similar to swagger. - That is very different to writing the program, the server itself as largely configuration driven. - and I think code is great, and if you try to put a lot in "configuration" at some point it will become much more work than it's worth. - Also, "configuration" is harder to write: usually zero intellisense, discoverability (+correctness) even harder than with just dynamic types ... - That being said, I really DO like configuration in VSCode - it has lots of intellisense etc. - It becomes a lot more usable then, but writing this for YOUR "dsl" becomes way, way more work again...

[@Verrisin]

ok, if you consider considering making intermediate data structures "data driven" as in reagent / reitit , then that's great, yeah, I agree.

[@EvenStarLoveAnanda]

What do the Function data structures look like? You have not shown any.

[@rafd]

en.wikipedia.org/wiki/Purely_functional_data_structure Clojure has functional implementations of Vectors (aka Arrays); Lists (aka Linked Lists); Maps (aka Dictionaries) and others. So, the basic data structures you're used to in other languages are implemented in an immutable but efficient way. Here's a functional B+ tree walk through: kzbin.info/www/bejne/oJXRZ2RtgphghpY

[@cybernessful]

10:00:50 OK, folks reinvented PHP

[@saadowain3511]

I think this is a historical talk. Do you agree !?

[@desertsniper87]

Where's the slides?

[@rafd]

Here you go: docs.google.com/presentation/d/13nRcrbnbvpUZGtABY9UCPCqOZL4Gjc8fI0IkNpePLzc

[@desertsniper87]

@@rafd Thank you. This was an excellent talk.

[@kwccoin3115]

Great great

[@davidmark1673]

Don't quite follow why he suggests that having global state is considered to be having side effects - if the closure of the system is well defined then it's just a matter of organisation.

[@rafd]

A function that makes use of a mutable variable from outside of it is not a pure function, but, often this is a fine/useful trade-off.

[@davidmark1673]

@@rafd that's only true if one restricts the closure of the function's inputs and output to the stack (which is just a piece of memory). If one allows the inputs and outputs to be located else where then you still have a pure function (but are using another piece of memory) - really it is tracking the closure carefully that makes a function pure or not, not the location of which piece of memory in time a value resides. The system and the function cannot tell the difference. Admittedly most functional languages don't track global variables in that manner - which probably also makes them needlessly complex - but there is nothing inherent in type systems that prevent them from doing that. I see it as a language design decision rather than a fundamental of purity and idempotence.

[@francisking708]

"After overcoming a fear of brackets..." Is this a rational or irrational fear? Haskell doesn't need as many brackets, after all. Is the opening parenthesis really necessary? Can we replace the end parentheses by something simpler?

[@jasbrg]

It’s irrational, there’s much more benefit to having parens than the superficial improvement of more complex syntax. That’s the essence of structural editing.

[@bigstones84]

Haskell doesn't need many brackets, yet it is scary. Maybe the fear doesn't come from the brackets but from the unfamiliar. In "Data and Reality" (§10.2.21) there's a beautiful discussion of simplicity that warns exactly about that.

[@FlavioBarrosProfessor]

For those who need a more detailed explanation of the GOPS: kzbin.info/www/bejne/jYa3lGR3e5iUZqs

[@YisraelDovL]

"how do you know things work".... "it does", that isn't a good answer.

[@EvenStarLoveAnanda]

This is very confusing. A mutable state is a variable. Why did you call it Mutable state? Why do you change naming conventions that are very clear and everyone already understand? And in this case Side-effects is an effect you want. It is a result. Why do you call it Side-effects when it is the effect or result? Side-effect is something you didn't intend for but it happened anyways.

[@rafd]

Re: "variables", it's a term typically used to describe specific syntax and behaviour of 'mutable bindings to mutable values', allowing for statements like "x = x + 1". But, variables are just one of the many ways programming languages let you create 'mutable state', others include: the hidden state inside of a random() function; an external database; Clojure's atoms. Re: "side-effects", I agree that they could just be called "effects", but, "side-effects" is what the common term for "statements in a function that affect states external to the function" is in computer science. See: en.wikipedia.org/wiki/Side_effect_(computer_science)

[@whkee]

(❤️)

[@EvenStarLoveAnanda]

How is function NOT a behavior??? You are confusing the hell out of me.

[@rafd]

If all that function does is return a value (ex. f(x) = x + 1 ), then in functional-programming speak, we don't think of that as having any 'behaviour' because the evaluation of the function does not affect the system in any way. A pure function can be used in an "expression" (vs a "statement"). Expressions have "referential transparency" en.wikipedia.org/wiki/Referential_transparency

[@alexgorodecky1661]

Good, but too much microfunctions. Too hard to compile knowledge about what that code actually do. Imperative solution with some _purification_ was "good enough". Microfunction programming paradigm is always kinda asspain.

[@henrish]

This consideration resonates a lot with me, what is your heuristic to decide

[@clickrush]

That is mostly due to him using plain JavaScript in his example. Functional libraries or languages don't have this problem nearly as much. Clojure code is actually very concise, partly due to it having a large standard library, which helps you to write code in a much more abstract way, rather than the very explicit, low level code we saw here.

[@beders]

There's no such thing as microfunctions. There are easy to test functions and not-so-easy to test functions. There's good names for fns and bad names.

[@0netom]

If you care to test your functions, then you will realize that having very small functions simplifies tests immensely. It also makes it a lot easier to find a descriptive name for a function, if it doesn't try to do multiple things. Documentation for the function will be shorter too, or even unnecessary. Such a function is also more likely to become reusable, further decreasing the total code size. My guess is that your annoyance with using a lot of small functions has multiple reasons. In javascript for example, it's too verbose to define short functions and the common code formatting conventions even stretch 1 function across 4 lines at least. With ES6, it's less of a problem, because you can just say `const someFunction = x => x + 1` on a single line or typically 2 lines if the expression is longer. No need for parens around arguments, no need for the `return` keyword, no need for curly braces for a single expression and u might even drop the empty line separating subsequent small functions. This will result in something, which resembles a few entries from a dictionary. You essentially build up a small language, which is tailored to talk about your problem domain concisely and it fits on 1 screen (or even half a screen), where 1 line is 1 sentence, 1 thought. The other source of annoyance might be that you are tempted to make a function do 2 things, because in your specific case you don't see how 2 separate functions would be better, since you don't foresee using them independently. This attitude then leads to struggling to finding a good, short name for the function, introducing bugs, being tempted to use some local state to coordinate/compose the "2 things". Then you can't express yourself within a single expression, so you need the curly braces, multiple lines, return statement and you end up with 1 long clunky function name, which spreads across 5-6 lines, instead of 2 more obvious function names on 2 lines. And by "you", I mean your colleague, because you are pressuring them to write longer functions probably. 3rd source of annoyance could be the hesitation over naming. You want something short, but descriptive. You want to use a term from your problem domain, but if the function happens to be generic, then you are tempted to name it in a more reusable name, just so you don't have to duplicate the definition later. Eg, why would you create a function which adds a trial period to a subscription period, when you can just use your current date library's addition function? Well, because the code reads cleaner, eg `withTrialPeriod(subscriptionPeriod)` vs `subscriptionWithTrialPeriod = moment(subscriptionPeriod).add(trialPeriod)`. It's more flexible, because what if the trial period is not constant anymore? what if you change your date handling library? what if you want to simulate your time-dependent logic with simple integer clocks? Then you have the `withTrialPeriod` function in one place to change.

[@EvenStarLoveAnanda]

Why do you call the thing on top the "root"??? Roots are underground. Your whole structure is upside down. If you turn it around it would be a tree so you can have the root underground. When we build anything physically it is built from the ground up. You can't have the roof built first. Because it will fall down. The foundation is the lowest, but the most important structure to hold everything else up. It is hard to learn programming because you guys flip everything the wrong way and you can't even keep it consistent. And you use screwy naming conventions that differ from real world meanings.

[@aurorazuoris6654]

In programming, trees are upside down. They grow from the top and the leafs are at the bottom.

[@EvenStarLoveAnanda]

@@aurorazuoris6654 Maybe but you are using the wrong nomenclature. You should have used Pyramid nomenclature. And the only reason you do it this way now because we write things top to bottom. But there are other languages that write, right to left and bottom to top. You just need to be consistent with real world vocabulary, otherwise you are confusing people.

[@rafd]

@@EvenStarLoveAnanda Words have different meanings in different contexts. In programming, 'trees' are a data-structure with the 'root' at the top. en.wikipedia.org/wiki/Tree_(data_structure)

[@TB-wvvvw]

All of the things you were asking about here in your many questions like "why is it called side effects" are very well known ideas in Computer Science and Mathematics for many years. They didn't originate with this presentation, and they aren't going to change to suit you because you have a problem understanding them and think they should be different. If you want to work in this field, you need make the effort to take some CS classes or at least read some books to learn the basics like everyone else instead of expecting everyone to waste their time teaching you 2 + 2 = 4 on forums like this. A question or two about things you could not find out yourself after some effort and research on your own is reasonable, but you are being a pain in the ass.