The Dangers Of Promise.all()

[@Cableguyyy]

This video left me fulfilled. Thanks Theo

[@meqativ]

replies[0].content

[@frankkuipers2024]

@@meqativreplies?.at(-1)?.content

[@johnddonnet5151]

@@meqativ replies undefined

[@meqativ]

@@johnddonnet5151 const replies = [ { user: "meqativ", content: "replies[0].content", }, { deleted: true, }, { user: "johnddonnet5151", content: "replies undefined", mentions: { user: "meqativ", short: null, }, }, { user: "meqativ", content: [Circular *1], mentions: { user: "johnddonnet5151", short: null, }, }, ]; replies.at(-1).content

[@BikoGames]

@@johnddonnet5151 if (Array.isArray(replies))

[@tuomaskoivistoinen6476]

Promise.all is good for doing multiple queries with no side effects where you need all to succeed before the next step. You could do all settled and add retry logic but I’d rather have it in the work function or have the whole thing retried instead

[@awmy3109]

Exactly. Devs just worry about BS these days 😂

[@kartashuvit4971]

wouldn't it be better to start a transaction instead

[@tuomaskoivistoinen6476]

@@kartashuvit4971 that question is a bit orthogonal to how you handle promises. If a transaction would prevent significant bugs then yes, otherwise you get better performance committing a single statement at a time.

[@nikensss]

sounds to me this video is needed because people don't read the documentation thoroughly enough

[@developersteve1658]

​@awmy3109 I think it's that they less worry about bullshit and more and more that the majority use case doesn't require a surgical solution like this. If you have some really slow process that runs many times (let's say some api call that costs for time used), it would make sense to try and cache the successful results and retry the failures, because it could save a lot of time and money in the long-term. For loading 1kb of json from your own api for a component to render "hello [user]", yeah that can be retried wholesale.

[@samuelgunter]

if you use all instead of allSettled, this video would be over at 0:10 because that's when the first Exception is thrown

[@ropoxdev]

Haha

[@Notoriousjunior374]

You should have a function that catches that error, do whatever you want with it, null, undefined or whatever. Then it wouldn’t be a problem?

[@avidrucker]

Also interested to know

[@Alec.Vision]

4:09 Bruh, that's what Exclude is for. The best thing about Exclude is that it will exclude anything that EXTENDS the second type param... so you can exclude a whole object type from a discriminated union by specifying only the discriminating property. You don't have to know the types, just the difference.

[@Glyra12]

didnt knew about this nice one

[@kevinmitchell6141]

Could you give an example of how you use this?

[@LLF1234]

@@kevinmitchell6141 If he doesn't, ChatGPT probably does

[@pellyryu2344]

nice tip thanks

[@robgioeli]

I’m glad you got this all settled, thanks Theo 😉

[@EvanBoldt]

Seems like another example of how returning errors results in better error handling than throwing errors.

[@alexandrucaraus233]

Throwing is more OOP, returning is more functional approach. TS supports both, so it all depends how and when you use it.

[@bryanleebmy]

@@alexandrucaraus233 Nothing about throwing or returning is inherently OOP or functional. Throwing is just the model most commonly used by Java, which happens to be the most widely taught OOP language. In an alternate universe, returning errors or result types could have been the standard way of doing things in Java. OOP != throwing.

[@kartashuvit4971]

That's why I'm slowly falling in love with Go

[@alexandrucaraus233]

@@bryanleebmy yes, java, c#, c++ and all other other OOP languages use more the throwing approach, and all other pure functional prefer returning values. So blue!=green, like I said :D.

[@felipeandrade6629]

Said the Go developer, lol

[@Xe054]

I love this type of content where you share new discoveries about a language. Please make more videos like this.

[@alanscodelog]

I usually use .all and catch in the entry promises and handle any errors and cleanup there and return or push the errors to an array (I find this nicer and more flexible than allSettled). The final .all then only serves to send off /format the errors if needed, or to take the final decision on what to do, not handle the errors themselves.

[@gosnooky]

To be fair the "^" operator being XOR and not exponent is common to most languages? As for the topic at hand, I'm guilty. I use Promise.all all the time.

[@CottidaeSEA]

I believe it's the standard. Most languages have something like Math.pow() or pow() instead of **.

[@theoneandonlymeshe9174]

“I hate JavaScript” ~ Theo, Aug 2023 1:34

[@elevyg]

It is easier to check if the item of the result has value as key, then you know that the status is fulfilled

[@Bluesourboy]

I have this issue in some production apps that were not written by me that this will fix. Theo you are a godsend!

[@AlexSpieslechner]

summoning ts wizard matt to make a video on this type

[@williamdrum9899]

"Don't make promises you can't keep" seems to be the moral here

[@schlopping]

honestly this is the best argument I've seen for a Result type, and it wasn't even intended.

[@MKorostoff]

Great video, but for the most part, I want Promise.all to reject if a single underlying promise rejects. For instance, I was recently writing some code that split PDFs into chunks for search indexing. If one single page cannot be chunked, I don't want to store incomplete data, I want to show a message like "example.pdf failed on page X." Promise.all is essentially like a database transaction in that way, and very convenient for that purpose (though I agree, allSettled has many great use cases).

[@xinaesthetic]

Do you have a strategy for cancelling other tasks when one fails?

[@MKorostoff]

@@xinaesthetic like if you had promises foo and bar, and you want to init them at the same time, then cancel bar if foo fails? Personally, I wouldn't, I'd just let bar finish.

[@xinaesthetic]

@@MKorostoff fair enough. No point coding it when you don't need it. In a situation where it did matter I suppose I'd probably have some shared reference to a bit of state that flags if the operation should be cancelled, but not sure what particular patterns there are that might be good to know.

Basically if your requirement is to process a bunch of steps sequentially, which means one after the other, please forget Promise.all because it process everything CONCURRENTLY.

[@dtinth]

4:54 - When seeing `.filter` followed by `.map`, using `.flatMap` will usually result in a cleaner code: `results.flatMap(r => r.status === 'fulfilled' ? [r.value] : [])` When you use flatMap, filtering and mapping is done in a single step, so there is no loss of type information between these 2 steps.

[@NickServ]

True, but transduce also fixes this problem without introducing intermediate arrays, which will use more memory and power to merge.

[@magne6049]

@@NickServ which transducers library do you recommend for JS?

[@karamuto1565]

I am more into observables by now. There I have multiple choices to handle multi asyncs and it feels easier.

[@TJKlimoski]

I actually just used promise.all in a project that to me was appropriate. Essentially I'm running two async functions in parallel that return an array of values. Once I get back both arrays I compare the two arrays and keep the values that are shared between the two. If one promise rejects (which only happens because they found no values to return in their array) I don't need to know what the returned values in the other async function will be because I know there will be no matching values for me to pull. So the fact that it jumps to the catch statement, and I don't get the values for the other promises, all work out.

[@guilhermegoncalves6743]

Promise.all is good when youre using puppeteer. Great video!

[@vncntjms]

I prefer to just retry everything. That's the reason why I put them all together in one promise in the first place.

[@Tszyu01]

I usually use a library like p-all or p-limit to also ensure promise all runs with a specified amount of concurrency.

[@Voidstroyer]

Nothing dangerous about promise.all if you know what it does (You might argue with me because I said "if you know what it does" but I say that because the function itself is not dangerous, it's the misuse of the function that is dangerous). Promise.all requires all functions to resolve otherwise nothing is returned (which does resolve in discarded work being done). Promise.allSettled is a good alternative though but if the requirement is that all promises MUST resolve (and you don't have a retry mechanism which could potentially also cause other complexities because endless retries would cause your program to hang) then promise.all is the preffered choice.

[@daniellchukwu]

I can't reject this one. God bless

[@pankuka23]

If I need all results to succeed, and I don't want to keep it running on failed as .all does, I just use an async for loop - works wonders ;)

[@sobanya_228]

I use it with something that's not an array. When you request multiple things, but definitely need all of it. An array api in that situation is kinda bad, I even defined a dictionary version of Promise.all, where you give names to each entry.

[@Holdino_]

I've been awaiting this video

[@luc122c]

I find using a promise pool is a good balance. They usually return results and errors as separate objects, and have other features such as setting concurrency

[@RealDrDoom]

Maybe this is an anti-pattern but what if you catch inside the promises? If there's an error return null, otherwise return the result. That way the promises always resolve and we can filter out results after

[@festusyuma1901]

If you use a for loop instead of a filter, you'll get the correct typing

[@umuden]

Yes, the error behavior isn't too inferable. Once, instead of reading the docs, I wrote my own makeConsecutively function. Which was fine.

[@trappedcat3615]

I don't get it. I'm still awaiting the catch. Let me know when this is all settled.

[@alexandrucaraus233]

All is settled, nothing to catch, go home :D.

[@ColinRichardson]

New Title = I used the wrong method for what I really want.. but what I want is not what 95% of everyone else wants.

[@ziadx3]

thank god typescript fixed this filter issue

[@Aetherius218]

You could also implement error catching in your asynchronous function (e.g., WORK) and have it return the potentially caught error. Then you can filter the returned array by instanceof Error

[@Notoriousjunior374]

This, I don’t know what’s wrong with the limited amount of creativity in devs these days.

[@Alec.Vision]

Lol, Promise.all(promises.map(p=>p.catch(e=>e))) Can I haz utoob job plz?

[@ellisgl]

I just keep thinking of the song "Promises" by Fugazi...

[@xaviersavinon5851]

Thanks, I didn't know that allSettled existed. Before this video, I only used Promise.All like this: The function WORK would be an async func with a try/catch so, if the promise got resolved would returned the value if not returned a null. After that filter the array of nulls.

[@bozhidarmanev6772]

Theo is close to embracing a properly typed language!

[@johnmcparland3277]

"JS Dev discovers Result and/or Option" is my favorite genre of programming video

[@valenciawalker6498]

Thank you very helpful.

[@pelle4971]

How might our understanding and application of asynchronous functions change if we were to prioritize clarity and predictability of outcomes over simplicity or convention in our coding practices?

[@SkyaTura]

The type issue couldn't be solved by using Exclude?

[@3ventic]

Isn't the .filter and .value issue one of those things that ts-reset fixes?

[@sn0wb0ardfreak72]

promise.All() seems more reasonable in a serverless backend env during boot / initialization (eg. AWS SSM)

[@jewbarrymore_]

It was super hard not to notice and resist commenting, but why is waitFor declared as async? Seems kinda extra to wrap the return value when you're already returning a promise? Obviously, js handles it behind the scenes gracefully, but still, there's no await used inside.

[@d3vilm4ster]

Well not all is Settled in JS async yet

[@dev1333]

oh no... dont do this to me theo

[@Nackenschelle]

splendid moustache, theo 🤝🏼

[@joshrogan3577]

Is it fine to do promise.all with tanstack query mutateAsync? Say I need to make 5 separate independent mutations on button click.

[@RTWeaver]

I use it when building my test environment on the backend. I don't really care if the server or database failed to standup, if either of them fails then its pointless to run the test suite. Same with performing a bunch of bulkCreates with sequelize. If one of the bulkCreates fails then the database is corrupted anyway and I'm going to have to find out whats wrong and fix it regardless so it doesn't matter if the promises still run in the background or send back results.

[@Fernando-ry5qt]

The bulk case if you work with a bunch of operations of that nature you might use a Sagas pattern and revert/delete the corrupted data and retry it in some cases, but yeah I get what you mean

[@benkogan1579]

Another thing is that the Promise concurrency functions are only useful for IO / network related tasks, since the requests are in flight in parallel. However, if you just have a very expensive task computationally, splitting it up and Promise.alling it wont speed it up at all, since JS is single-threaded and it will only ever work on one task at a time. If, however, you used multiple threads operating on multiple cores, then it would make sense.

[@aredrih6723]

You kind always turn computation heavy task into io with webWorker. It just requires a lot more setup (create worker, send relevant data, await response from worker)

[@NickServ]

This is a bit of an oversimplification, since while it can result in performance issues compared to threads, single-thread concurrency with delaying Promises until next tick can still improve performance in many situations.

[@benkogan1579]

@@NickServ for example?

[@aredrih6723]

@@NickServ do you mean performance (how fast your code run end to end) or reactivity (how long your code take to react to an event)? If reactivity, I agree. If performance, if you have say 1000 lines of code to run to complete a function, splitting it into 10 group of 100 lines won't make things end faster.

[@NickServ]

@@aredrih6723 Both. Promise concurrency still helps with e2e wall time, especially when getting results early with all/race/any.

[@kylegilbert5089]

Wow, this is actually an issue in one of my apps and I wasn't sure what the problem was. Ty Theo.

[@paulomattos6753]

Promise ALL is the best for select queries

[@lengors7327]

The filter issue was solved with TS5.5 🎉

[@errormaker1]

That is unsettling

[@snatvb]

promises is sucks because they are not lazy and not cancellable and this might leads to unexpected results, data-race in async useEffect in react for example

[@DryBones111]

The typing on the settled promise is so gross 😭

[@klausburgersten]

wdym hard to work with async stuff in js? have you tried to do this in python?

[@alexaka1]

This behavior makes me like RxJS.

[@appuser]

I haven't finished the video yet, but I'm going to guess most of the time taking a less FP approach and using "for...in" within an already async function is the recommendation coming up, rather than managing the nesty confusion of Promise.all

[@appuser]

I was so wrong and learned something useful.

[@owenwexler7214]

Don’t worry, the Promise puns will never not be funny… although I’m sure some people will… await the day when this isn’t true.

[@mattmmilli8287]

Use await in a for-in loop(you can indeed!) to do them sequentially and promise.all for all at once

[@munnakumarbarnwal1694]

Great video 😍, Thanks

[@kartashuvit4971]

The type issue is actually easy to solve npm uninstall -g typescript

[@willbdev]

Thank you

[@john.dough.]

1:06 -> the carrot operator `^` is bitwise OR in JavaScript. You probably wanted Math.pow(). nvm, you caught it.

[@alexanderpedenko6669]

Who is this video for, who immediately started writing on React without learning JS basics?

[@Kay8B]

I only used promise.all when theres a dependancy chain, a relies on b relies on c. If A failed we dont want to continue. Otherwise I agree settled for everything else.

[@t3dotgg]

Wouldn’t you need to await in sequence if they are dependent?

[@Kay8B]

@@t3dotgg Right my example wasnt clear, they would rely on eachother or another dependant requires all 3. Not sequentially. not in fact a proper "chain" more like a web

[@MrJester831]

Futures > Promises because polling can be halted

[@code.smiles]

Sorry for the *chain* of puns...

[@spicybaguette7706]

JS error handling in general sucks IMHO, especially with promises. Even with typescript you have no way of knowing if a function could throw and what it would throw, I hate it

[@_worksonmymachine]

I'm just here for the puns.

[@stevenhe3462]

Because JavaScript Promises are fucking eager.

[@danypell2517]

another gem

[@possumkeys]

Use case: Start up app requirements being fetched in calls. If any fails, we need to start all over again.

[@t3dotgg]

Wouldn't it be nice to know WHICH failed?

[@garretmh]

@@t3dotgg You can add catch handlers to the individual promises and benefit from the short-circuiting behavior.

[@possumkeys]

@@t3dotgg not really, I can't see that on network debugging. It's all strictly required so we start all over again.

[@mananbhatia8575]

Name that vs code theme please

[@calledtigermttv7184]

poimandres dark theme

[@mananbhatia8575]

thanks fr@@calledtigermttv7184

[@rudro314]

thanks for letting me know how bad js is

[@hanes2]

if u think async is hard in javascript. well.. it's freaking pain in other languages, even just Python it's quite a nightmare compared to Javascript.

[@magisteryura]

python is a pain as a whole

[@truepicksyt3323]

But I heard never settle

[@bhardwajthummar9313]

Hail allsettled() !!!!!

[@StephenRayner]

There’s an opportunity to make all these suggestions accessible. Something a kin to ESLint warnings / recommendations. I’m imagining a plugin that shows brief descriptions of “avoid this” with some if “logical/situation”. Then we the community can add their own, vote, attach KZbin videos. So if you write Promise.all you see this video / a code example.

[@avidrucker]

Love it

[@StephenRayner]

Damn 😮

[@DevinBidwell]

Sorry did you say Exception? I think you mean Error ;)

[@tobafett2873]

solid vid, good stuff.

[@miguelemmara5046]

its 'all or nothing'

[@aredrih6723]

I agree on the correctness side but disagree on the efficiency side. If you need _all_ request to complete to perform a task, exiting early is more efficient. If you want some retry logic, placing it in the catch handler of each individual promise would allow you to retry immediately instead of having to wait on the slowest to resolve first. Also, isn't that be the use case for AbortController ? You tie every request to a signal and as soon as the controller abort, you interrupt as early as possible because the result is no longer required. That said, I remember node logging a warning for memory leak for trying to have more than 10 listener to a signal...

[@CottidaeSEA]

There are cases where all you care about is that the promises are done, not that they succeeded.

[@wichtel1805]

I have exactly such a case in a current project: It does a few async operations in parallel, all of which must resolve. In case of an error, the shared signal cancels the rest. Works very well since we do not need the individual results in case of an error - and an early result is more valuable than having all error messages should more than one fail. But that's clearly very situation dependent.

[@aredrih6723]

@@CottidaeSEA catch(() => {}) is always an option if you don't care what kind of error each promise encounters. If you're looking for stat on which action succeeded and which one failed, then yeah, allSettled might fit better. Also, at the end of the day, allSettled is just a promise.all with a well chosen then applied to each promise (e.g. .then((e) => ({e, ok:true}), (e) => ({e, ok:false})))

[@CottidaeSEA]

@@aredrih6723 That is not compatible with await syntax, but yes, that works. Or you could just use allSettled and go on with life.

[@IvanRandomDude]

Thanks F#

[@BenRangel]

4:30 stuff like this contributes to me still using JS instead of TS. Scenarios where your working with some JS code where all the quirks are well known and accepted, but to use it in TS without an error you must do a quirky workaround. It doesn't feel like it adds anything to my code but confusion. I mean, I'd love to use types for the parts of my codebase where it feels significant, but when it affects everything it seems to be more of an annoyance than a feature.

[@lapulapucityrider3227]

because of small issue you ignore all the benefits sounds an excuse or just being lazy.

[@magisteryura]

lol, i was writing in js since es3 in ie, now i use TS (for 5+ years or so) and i really hate when i need to read plain js code

[@d.ilnicki]

You can still await for any of underlying promise in the array. They aren't going anywhere, references are still here.

[@mehulsharmamat]

Just realized one of my production bugs might be due to EXACTLY this. brb checking

[@flatmapper]

Great

[@st-tube]

Fire and forget is a common pattern so Promise.all is fine but Promise.allSettled is more appropriate in most cases

[@ciach0_]

I Promise to you Theo that this video is going to be resolved for the all-settled users.

[@ymi_yugy3133]

For interacting with library code is is a good idea. Thanks for pointing it out. In my own code I don’t use exceptions. So promise.all works fine

[@CrzyMan_Personal]

I think it's more about your mental model. `Promise.all` isn't "wait for all of these to settle", it's more of a "treat these all like one big promise". Once you treat it like that, it makes more sense. after using Supabase a lot, I've become a big fan of my async functions being something akin to `function(...args): Promise`. Where they catch themselves and always resolve nicely. That kind of coalesces the behavior of `Promise.all` and `Promise.allSettled`. Either way, it still forces you to consider the error case a lot better

[@yofish101]

Yavascript folks…

[@96shahab]

@Theo telemetry is a common place for me. I usually push up data in a fire and forget manner and for some metrics (granted, these are not actually all that many) it's better to use all instead of allSettled just because it allows the publishing function to return faster, reducing the time the function has to run. Those promises in the background can fail and it'll just be treated as a missing data point, which for a lot of metric types is actually not critical.

[@NuncNuncNuncNunc]

How do you know whether or not you have systemic errors? If you have common errors, your data risks being biased.

[@96shahab]

@@NuncNuncNuncNunc it depends on the metric being sent. Something like network tx and rx for example it's okay if publishing fails once every so often. A user may also select a metric to be non essential and that would be treated the same way. Of course, if every publish fails then that's a different matter that we do look out for, though all we do in that case is log to console, as we can't be sure whether the problem is ours or if the user has something in their config or something like a proxy

[@mkwpaul]

Isn't the ^ (caret symbol) bitwise XOR and not exponentiation? From what I can remember ** would be exponentiation.

[@neoney]

he says that in the video

[@jkbrodotdev]

Great video, Promise me you leave the puns out next time

[@JAt0m]

I use this helper here to group data by the 'type' field. Could obviously also be a used with a 'status' field. export type Visitor = { [P in T["type"]]?: Extract[]; }; const groupByType = (data: T[]): Visitor => A.groupBy(data, (item) => item.type) as unknown as Visitor;

[@bryanleebmy]

That `as unknown as Visitor` is a terrible pattern and ugly as hell.