Building a simple concurrent task queue on PostgreSQL only with Clojure code sample

[@andrey.fadeev]

☕ If you liked this video and want to support my channel, please consider buying me a coffee. Your contribution helps me create more content like this: 👉 Buy Me a Coffee: www.buymeacoffee.com/andrey.fadeev 👉 Ko-fi: ko-fi.com/andreyfadeev Please also subscribe to my other resources: 👉 Substack: blog.andreyfadeev.com 👉 Telegram: t.me/andreyfadeevchannel I'm truly grateful for your support, and thank you for watching! 🙏

[@JukkaLehtimäki]

Good explanation on how to build an unlimited concurrency task queue. The real challenge is when requirement comes in that some tasks can't be run in parallel. You would need to add a new column "group" into the table and ensure that only one task in a group can be executed at any time. Tasks without group can be run as shown here.

[@caiocesar4827]

Thanks a lot for you video! I was hopping to see test cases that check the concurrency aspect of this service.

[@andrey.fadeev]

Hi, thanks! Sorry I was a bit lazy when I was preparing the video :) Could be a good exercise and I can merge it to the repo if send a PR. It could be something like create N of tasks and run processing on a threads pool (eg with claypole library) and validate that each item was processed exacly once

[@caiocesar4827]

@@andrey.fadeevsounds nice! I will try myself too. Thanks a lot, and also I didn’t know this library, I’ll have a look! 😊

[@andrey.fadeev]

@@caiocesar4827 I have a video about clojure concurrency on the channel, maybe that will be useful!

[@FrancoGasperino]

There is a PG extension named pgmq, which supports this within postgresql.

[@andrey.fadeev]

Cool, good to know, still useful to know underlying mechanisms

[@AnandKumar-dc2bf]

I am planning to Learn Clojure. (I already know GO, Python).. Can u let me know how are the job prospects for Clojure ..

[@andrey.fadeev]

Clojure job market is not too bad but it also depends on location, Europe/UK and USA - there is plenty of Clojure companies. Of course it's a niche language so the absolute number of jobs is much less. Good news is that pay is usually quite good and overall atmosphere in the companies.