Memory safety, type safety allowed fearless concurrency. But we are used to async pattern. I feel go did a great work. Balancing between performance and how simple it is do it.

Yes! Register here: share.hsforms.com/1VeS7vcAdRyqme1TLriu-YA444dk Watch the live here: kzbin.info/www/bejne/iIPIpWZuetyGpdE Join us on Discord to ask any questions (and download the guide): discord.com/invite/c3esdEfywj We'll expect you at 1pm EDT tomorrow 👀

Thanks for the question! I might have been wrong in the live demo about not being able to use worker update in that particular case. It was hard to quickly think it through while not getting out of the planned schedule. Golem supports two ways to update a worker to a new component version. Automatic update basically replays the whole history of the worker using the new codebase. This potentially can fail, if there were "too big" changes in the codebase. If it fails, the worker is restored to its original version and it works as if nothing happened (and the failed update attempt is available in the worker metadata). So this means that you can change anything (including the external interface) that has not run yet, but if something ran, then the possibilities are limited to small fixes that do not change the (observable) outcome. For example if you invoked function A, and it returned a value, that value is persisted, and if the modified version would return a different value, that is a divergence and fails the update. But if the change only affects things that are not observable (persisted) then it's fine. Of course there are cases when you have to do such a change that cannot be applied to a worker automatically, but you still need to preserve your worker AND have the new version running. For this we support something we call manual update; in this case you have to implement a `save` function in the old version (if it was not there yet, it can be added and auto-updated as it's something never ran yet!) and a `load` function in the new version, and you have to serialize/restore your state manually from an arbitrary byte array. Once the functions are implemented, the process of running them as part of the update process is orchestrated by Golem.

@@DanielVigovszky Thanks for the great response! Having two cases makes sense, but doesn't it mean that Golem has to store the history of all arguments of each interface function invocation in order to attempt automatic update? That seems to be quite demanding on storage: does Golem support remote object stores for such metadata? And even then, if the first version of the worker was launched say a month ago and actively used, wouldn't automatic update be prohibitively time-consuming for any change to that worker? Meaning that save/load implementations would rather be the default approach.

@@yfjolne You are right that for automatic update we always need to use the full history, even if snapshots would be used to optimise recovery (which is by the way on the roadmap but not enabled right now). So this is a tradeoff the user can make between having to implement custom code, or use the built-in machinery. Both makes sense depending on your workers and their lifetime. So if we need to store the journal forever, how do we do it? It is stored in multiple layers, the primary layer is the one being written in live - that's currently a redis stream. There are (configurable number of) secondary layers, either further Redis streams, or S3 buckets, where old entries got moved in compressed chunks. The default configuration right now is 1 archive layer in Redis and 1 in S3. Eventually if you are not accessing a worker, it gets completely moved down to the bottom layer. We only have redis+s3 implementation now, but it's designed in a way that it should be easy to implement connectors for other data storages (just need to implement a keyvaluestore, an indexedstore and a blobstore interface) See more at learn.golem.cloud/docs/operate/persistence#oplog

@@DanielVigovszky I see, thank you for the detailed reply.

Oh yeah. It definitely balloons when trying to be precise, and we’ve probably surpassed the point of “is it worth it?”