Maybe Not - Rich Hickey

[@TensorProgramming]

Good old Rich Hickey. Even though I don't use Clojure as much as I should, I still love listening to Rich talk about development every year.

[@awvalenti]

15:51 "Oh my goodness... I mean... We've... We've known it all along... Like, our languages embed essential concepts." Love your talks ❤!

[@RequiredAccountsSUX]

Rich Hickey is the clearest thinker and speaker of his generation that I know. He's on the Guy Steele level.

[@ryanleemartin7758]

Ya know, I've been getting into functional programming for a while, using F# and I learned how amazing Option is (Maybe) and here comes Rich Hickey to laugh in my face. Great talk as always!

[@elliottcrifasi3143]

Haha same 😂. It's an improvement on what I had used in other languages, but clojure and this design seem to take that idea and make it even better. I thought Haskell's type system was awesome, but after watching this I'm seeing the benefits of a different way at looking at types/schemas. I'm definitely interested in finding a job using clojure

[@lukeyd13]

Im not a Clojure person anymore but looking at spec in 2022 it seems like it is how rich said it shouldn't be (optionality specified at the aggregate level) rather than in a separate step with selections. Does anyone know what happened to the ideas in this talk I found the answer as of 2022, it’s still in development under the name spec2 there is a GitHub repo

[@Hemigoblin]

I didn’t realize this till rewatching this talk, but when Rich was first describing Clojure in a talk (maybe in “Clojure Made Simple”), he said that Lisp relying on lists instead of tress was a mistake, and that trees are the better paradigm. In this talk, with his discussion about how spec should change to better handle trees, he’s saying he missed the same lesson, and is trying to fix it.

[@ZsoltDonca]

I don't think the issues presented by Rich Hickey with using Maybe or Option are significant, and there are perfectly valid functional solutions. When making parameters optional, one can use Maybe/Option without breaking callers by *introducing a new function*: keep the original's signature as is, introduce a new function with the optional parameter, and make the old function call the new one. Old code still works, and new code can benefit of using the new function without providing a value (passing Empty/None). The same approach works for the return type case as well.

[@DenisG631]

but is it really the same? this way you are kinda lying to your API users by providing same API but with syntactic sugar manual forwarding code. e.g. maybe return by doing extra wrapping, the api caller doesn't know you changed anything inside. He doesn't know that you don't return None anymore. i.e. If you change your signature -> you force the client to make changes in Kotlin, if you change the signature -> your api client is not forced to change anything, but will see that your function doesn't return Optional anymore i.e. in Kotlin syntactic sugar with Optional wrapping is done implicitly in Haskell you have to do forwarding/extra function explicitly AFAIU

[@shalokshalom]

He did say nothing about Option, just Maybe/Either. Option types (in F#) are real union types, if I am informed correctly.

[@mononix5224]

​@@shalokshalom sadly you've been misinformed, since Option in F# is a sum type (_disjoint_ union type), not a union type. Given a type T, T Option doesn't have any members in common with T, since all the original members of T have been 'wrapped' by Some. You can think of T Option as being the union of all members of T 'wrapped' by Some and the symbol None, so that you get Some(T) | None, while a union would be T | None. The fundamental difference is the 'wrapping'/'tagging' that happens when making a disjoint union which is absent with simple unions.

[@AlexRodriguez-gb9ez]

@@DenisG631 This problem isn't limited to Maybe, its also a problem with monads and all wrapped values in Haskell in general。 Also you have to differentiate between >>= and >> in Monads, if decide later on that you want some of your parametrs to go into the container instead of revealed you have to change a lot of code like changing sequence to sequence_, traverse to traverse_, etc...

[@ethanresnick6456]

This talk makes Typescript look pretty good (with its true union types, and `Pick` for select), which I guess isn't surprising, given that that JS and Clojure are both highly map-oriented, and Typescript and Spec are shape-oriented optional overlays.

[@josevargas686]

Using `Pick` and other utility types in TS is absolutely awful though. First, you need to provide a bunch of string values, already a very ugly thing to do. Second, you can't nest it at all. Third, you have a whole new Type and if your function accepts the Pick and the T, you have to discriminate them at runtime with some hacky if statements like `if(!T.prop)`.

[@LoSc00]

@@josevargas686 Syntactically they look like strings, and I remember thinking of them as such when I first learned the language. But semantically because Typescript supports literals as types they're really equivalent to actual identifiers - your code won't compile if you typo them or remove a field that you're 'Pick'-ing elsewhere. I agree with the nesting issue though - the ergonomics of pick fall apart really fast when you're trying to partially accept "object, containing field foo, containing object, containing field bar" rather than just dealing with top level fields. I'm not sure what the utility of accepting both 'Pick' *and* T would be? Generally you use Pick to say 'these are the fields from T I actually use', and T is always going to be assignable to any subset of T derived via Pick.

[@DanielJomphe]

Funny unintended pun: "...there's a spec-trum of what you can communicate..." (56:50)

[@deveugene7]

I whole-heartedly approve. Stripes and scarves are definitely the way to go...

[@derekfrost8991]

Dr Who.. :)

[@janmsavage]

I don't know what would I have done without Clojure. Lisp is the most important discovery in infotech, and Clojure the most important improvement on a Lisp.

[@typon1]

Another excellent talk by Hickey. Not sure if I buy into his attack on Maybe, but it made me think about it more deeply.

[@batlin]

Typescript and Crystal both have intersection types, although I'm not sure how nice they are to work in practice, nor if the kinds of changes Rich showed in the beginning would still be a breaking change. It's an interesting argument though -- whether the benefits of the compiler's completeness check (i.e. did you handle both Maybe a and Nothing?) and the clarity of the new type signature are greater than the costs of a breaking API change.

[@marcellerusu]

> whether the benefits of the compiler's completeness check (i.e. did you handle both Maybe a and Nothing?) and the clarity of the new type signature are greater than the costs of a breaking API change. you get this in kotlin, swift, & strict mode typescript (a common default these days), idk crystal enough tho As in i change my function to `function f(): number | null` in typescript, the consumers will have to check if what's returned is null or not, its just that typescript is powerful enough to not even require runtime wrappers like `Maybe`, it can be statically enforced. Also `number | null` is a different type than `number`

[@unformedvoid2223]

I actually like how your code breaks when you change your function's signature. It helps me to keep my code consistent and find bugs at early stage. Using nullables in C#, undefined and null in JS, undef in Perl wasn't even close to be so pleasant as using Option

[@TankorSmash]

Yeah, I'm with you. I don't like that the code breaks at runtime. The computer is smarter than me, I'd like it to help me and tell me what I'm doing wrong. With nice type systems, it's pretty smooth, and you've got so many fewer questions

[@aoeu256]

If you don’t what you need since you are programming what you don’t know, and you have thousands of callers it could get annoying😂

[@josevargas686]

But it shouldn't break if the requirements are being eased, that's the point... It should break if the requirements are being restricted, but spec does this.

[@franciscoflamenco]

The idea isn't that your code shouldn't break, but that it shouldn't break when you're relaxing the conditions. Of course you should go fix everything in your code if a certainty becomes a possibility. But if a possibility becomes a certainty your checks might become redundant, but they shouldn't break your stuff.

[@TankorSmash]

Did s/schema or s/select ever come out? I don't see it in the docs

[@milosnedeljkovic3737]

To be fair, no knowledgeable Haskell (or even more broadly, ML family) blogger should (and usually don't, actually) call Maybe or Either a union type. Those are instances of disjoint union type, or else called coproduct (sum). And those are just as precise mathematical terms as simple unions, just with a different semantics. The rest of the talk makes a pretty interesting view of a topic, but this little accusation that Haskell or Scala users are living in some sort of fallacy is a bit unfair

[@shalokshalom]

Do you know if F# does this right?

[@lukasjuhrich503]

I'm not sure I'm following you. Doesn't „union“ usually refer to the disjoint union when talking about types?

[@szelpsz]

Yes and no. A mathematical disjoint union / coproduct operator should still be commutative and associative, shouldn't it? A (+) B is ismorphic to B (+) A.

[@hoggmann7217]

Came to say something similar, this was way more informative. A+

[@DrewIsFail]

"yes and no" so "maybe"? Sorry i couldn't help myself

[@freddaoud4432]

Good talk. I understand the point about the spec being separate from the schema. However, this does not solve the problem that Maybe solves. If your function says that x is optional, how do you deal with it if it is present? Are you back to using ifs? This does not provide what Maybe provides, i.e. safe .map, .chain, and so on. Did I misunderstand?

[@RobertPankowecki]

Wonderful and deeply insightful. Thank you!

[@elgireth]

I guess the same argument can be expanded to Lists. If 0..1 can be a *selection* concern and not shape, then 0..* could be too. Maybe specifying the size of the collection as a requirement?

[@ViktorKronvall]

There are a lot of inaccuracies about Haskell in this talk. However, there are some good points being brought up. Treating (Maybe x) as a a supertype of x would indeed be nice but subtyping and polymorphic parametricity don’t go that well together. The main issue is that you are losing the great tools of functional programming such as map and fold if you introduce subtyping without giving up and throwing away the type checker. There has been some work to get back a bit more subtyping to Haskell and now the type system is extended to allow subtyping of data that have the same data representation through type preserving coercions. But being able to reason about subtypes created by intersections of required fields would be a nice way to model some functions. Also returning supertypes generated by unions seem ideal. I’m not willing to throw away parametricity but I hope the two perspectives can move closer in the future.

[@megasuperlexa2]

the first example does not make sense in case you have implicit cast from a value to its maybe container. And in c# it is super easy (and safe). Not so in f# or other systems I suppose

[@robchr]

I haven't used the Haskell Lens library but I think it makes it possible to pull out data from nested structures without the user needing to know anything about the structure. This can solve some of the issues with readers of data being polymorphic over any structure.

[@cryptoAsabiyyah]

@9:00 why would it break existing callers if the argument is now optional. He explains why it would break for "Maybe returns" but why would it break for a required parameter turned optional?

[@gnethercutt]

I believe this was a implication that Maybe is an option type (or a monad, if you're feeling frisky), and it alters the function signature and thus callers would need to change invocations like foo(x) to be foo(Maybe.fromValue(x))

[@MishaSalnikov]

because old code is calling `f(a)`, and now it has to call `f(Maybe a)`, so you need to wrap your variable in Maybe to pass it now

[@jergason]

Hassen Ben-Tanfous you typically have to wrap the value you’re passing in in a type constructor. So to pass in a Maybe String you’d need to wrap the value when calling in Just “my string” or whatever

[@WarrenLeggatt]

It breaks the caller because they have to pack the argument as Maybe, it switches from a type to a union type. You need to supply "Some x" or None. This all comes down to type theory. Many C languages have nullable types so it does not break the caller but the opposite is also true in that you now have no compiler checks around nullable. C# is about to bring in a breaking change for non-nullable reference types by default. Personally I like Maybe, Either etc as they force the consumer of the value to think about what is contained. Implicit nullability means "null reference exceptions" at run time :)

[@andrewkiluk]

Because the type of the input has changed -- if the caller had a line like `foo $ 7`, that must change to `foo $ Just 7` in order to match the new type and compile.

[@EvgenyOrekhov]

54:16 [a] -> [a] says exactly that: reverse will return a subset of the same list it was given, because "a" is not a constructor, and "reverse" doesn't know how to instantiate "a", so the result could contain only those elements that was given to the function.

[@dlwatib]

But reverse does not return a subset of [a], it returns a permutation of [a]. And besides, that's not what [a] -> [a] actually says. All it says is that if you give me a list of 'a' you will get back a list of 'a'. There is no guarantee that the 'a's will be from the list you gave me, and there's no requirement that the list I give you wont have a larger cardinality than you gave me. 'a' is a type variable. It can be instantiated to a constructor.

[@Ven_de_Thiel]

`[a] -> [a]` can mean that, can mean `id`, can mean `take` (or `empty`), can replicate and turn it into an infinite list (if list was at least length 1), etc.

[@DenisG631]

is it really a subset? can not you prepend/append subarray of the original array?

[@isodoubIet]

@@dlwatib This argument is saying essentially "I can't completely specify the behavior of the function in the type system therefore I won't specify anything at all", which is just stupid. From the declaration [a] -> [a] I may not have all conceivable semantic properties of the function (and how could I? that's its _definition!)_ but I certainly know that won't pass a list and get back a zebra.

[@csbnikhil]

The a's have to be from the same list. The function does not know how to create an a.@@dlwatib

[@johanovlinger9020]

Karl Lieberherr's work on "adaptive programming" addressed the use of select-like accessors (traversals). That was in the context of imperative java. His group did not really delve in to how to specify requirements on arguments. I imagine that something like go-ish interfaces might work.

[@pcaisse]

I'm not an expert by any means, but at 7:43 it seems like he's using `Maybe` to make an argument optional which I don't think is what you'd do in Haskell -- you'd just create a new function. In pure languages, the type signature is like a contract which describes the function's behavior, what it _must_ do. You wouldn't ever want to change the types a function takes or returns without creating a new function. As such, his gripe seems to stem from a misconception about what a function signature means in a statically typed world.

[@dlwatib]

No, you misunderstand. If you have to create a whole new function in order to make what should be a trivial change, where's your reuse? You've totally failed to reuse your code.

[@pkop4]

Just wondering, why are the root attribute types (50:18) nillable ? Wasn't the point that the base schema should have concrete types (string vs string?) and optionality is handled in the selection?

[@trippyoctopus]

They are not nillable. But I think I see the source of confusion. For example, `int?` does not mean “integer or nil” in that slide, instead it refers to the *predicate* `int?` and all it does is test whether a value is of the type int. The description of `:user/id` says that it is only a valid attribute if the value it refers to is an int. Therefore `nil` is an invalid value for that attribute.

[@PaulSebastianM]

45:00 what if you get the user from the wire, from a different context, and you have no idea what the schema contains or doesn't contain, then how can you be sure that you're satisfying the select statement?

[@alexgalays910]

Typescript has had unions and intersection types for a long while now :)

[@jaredsmith5826]

Typescript's type system is surprisingly good. But I do have one major beef with it: it can't (at least last time I checked) accurately track the argument types of partially applied functions (FB flow can do this most of the time).

[@Ven_de_Thiel]

@@jaredsmith5826 TS recently merged a PR adding return types to the inferencer, so now it should be muuuch better on such things

[@sfyire]

Union and intersection types are not the same as schema and selection, selection can be nested in the same way you can use Graphql to express a requirement for multiple keys at different levels

[@barrkel]

I didn't hear anything about how specs compose recursively, when a function you call starts adding more implicit requirements on your arguments. To do it automatically, analytically, you end up in basically the same place as functional programming with parametric polymorphism.

[@awvalenti]

So many good ideas on software development... I think devs keep, decade after decade, struggling on the same problems, some of which could be solved already. Less important stuff keep showing up every year and few people discuss the basics. Shouldn't these ideas become written words in books for us and also the next generations?

[@bojanmatic024]

Minor nitpick but I think Rich is wrong on the value proposition of parameterized types. A function from List x to List x is telling at least something (that you will return the same data shape as you accept) and it's not obvious that given a List x you will return a List x. You could return a whole lot of things that may or may not have a connection to x or even a list. For example, given a list of strings I could return to you a list of integers. Or a single string. Or a list of maps. Or a list of lists of strings, etc. Otherwise, great stuff. As always.

[@Ven_de_Thiel]

`[a] -> [a]` in Haskell means the function can be `reverse`, `id`, `take` (or empty), or replicate the values. Rich wants to make it so that when you see `a -> m a`, it might actually expand to `Maybe a -> Maybe (Maybe a)` which would give `Maybe a -> Maybe a`, and now your signature just became `a -> a` magically!

[@cynicist8114]

He is just saying that this information is not valuable because it tells you nothing about what the function is actually going to do with the list. It takes a list and returns a list, but you don't even know if it is the same list, just transformed, or maybe a different list entirely. There is no context, and presumably the reason you are asking this question of the function (what are you taking and producing?), is that you are trying to ascertain whether or not it is behaving according to expectations, and types alone don't provide you with that information.

[@MisterComment25]

So, has this actually been implemented and added to spec? Can we use this type of spec he proposed?

[@BjarkeEbert]

20:02 hehe

[@timpotter6365]

Are the two newlyweds in the intro two couples or two halves of one couple? Inquiring minds want to know.

[@murakas55]

Although I do enjoy Rich Hickey's talks, here I think he knows too little about Haskell to comment. I.e at kzbin.info/www/bejne/j4OYiJd9p9Wiq8k, the "when" could be encoded as `data Car when = Car { make :: String, model :: when String, year :: when Int }`, and then instantiate `type MaybeCar = Car Maybe`, `type DefinitelyCar = Car Identity`, `type CarOrError = Car (Either String)`, where `Left` marks an error for why this field is missing. And functions that don't care about when can use a straght `Car a`. Other `when`s can be invented, including adding constraints to that type parameter, etc. All this can look really nice and tidy in the end. It does take effort to learn all this though.

[@Ven_de_Thiel]

The Scala program doesn't compile. "Maybe/Either" are not evidence of a lack of first-class union types. No one calls them that, they aren't. It's the same debate as Monad Transformers vs Effects: Do you want `Maybe (Maybe a)` to be a thing, or do you want `a | Nil | Nil` which is `a | Nil`. Just pretending Maybe/Either are bad solutions makes absolutely no sense. Having both is good. You may not care about parametricity, but Haskell people certainly do. "|" being commutative also means you can't have the same type on both sides. Once again. Having both is good. A Haskeller would most probably not using `data Person = Person String String Int Float String String`, it's disingenuous. Haskell people value their types and newtypes. You use types to reason about your program, you'd not use raw Strings floating around. Compare that with Clojure where, not to get lost, people prefix their value keys/keywords with "mytype/". I think I prefer the approach that checks types. "I'm not gonna let you write brittle systems" This has a vibe of "Clojure is the only one doing it right". "You're saying more than any type system let any other people say" . Doing the same thing with key selection is already possible with HLists in Scala and with Generic in Haskell. It's not new, it's just that people generally don't want that. Good talk.

[@brandonlewis2599]

I liked this talk, and the speaker's fashion sense. But when changing types leads to type errors ... that's a *feature* not a *bug* ! I *want* to break the caller. I *want* the compiler to find every offending call site. Anything less is unsound. Changing types is breaking an interface. When you break an interface, non-conforming code *should* break at compile time. FlowJS offers true set-theoretic type unions and "type refinements". The good part: you don't have to wrap every nullable in a Maybe at the call site. It fails type-checking if you forget to check for a nullable for null. *That's the point* ! Because when you "just let information flow", you silently allow null (or NaN) to propagate far away from the source of the error. In the limit, You end up with nightmares like log4j.

[@tricky2014]

Its not a feature that when you change your return type from maybe x to x, that the caller breaks. You just increased your guarantee of what you provided. This should be a compatible change and in languages that employ true union types it is. The fact that its necessary that your caller breaks is not dictated by the semantics of the change but the (inadequate) tool you used to express these semantics. Singular types are subtypes of union types that contain them, X is not a subtype of Maybe. Thats a problem. And you can enforce checking for NULL or an error with Union Types too.

[@csbnikhil]

@@tricky2014But the caller would have unnecessary code to handle the possibility of there being nothing even when the function changes the signature to guarantee that there will always be something.

[@apestogetherstrong341]

So what if there is reduntant code? The compiler is a smart guy no? He will optimize. And when you refactor, you’ll clean it up too.

[@kahnfatman]

The programmer/engineer/developer himself is part of the use of a programming language. A developer with 30 years of experience can reason without any compiler or IDE assistance. My point is: The tool is just as good as the person who uses it.

[@Milhouse77BS]

Sheep-oriented programming?