Cache Management with GitHub actions

[@MaximilianNeu-m6n]

Thanks, setting up caching with this guide was a breeze. Easy to follow and very clear. I was able to cut down my npm install time from 2 minutes to 3 seconds when the cache hits!

[@CristianChiovari]

If i use my own on prem runners , the cache is something shared between multiple runners or will be local to each runner ? Thanks !

[@MickeyGousset]

The cache is not stored on the runners it is stored by the repo and pulled down to the runners when needed. So there is a centralized cache where a copy is shared with each runner as needed.

[@shockstin]

Why is your camera output set to 10FPS? I like your content, but you should look into the FPS settings.

[@MickeyGousset]

Hey! Thanks for the feedback. I will look into that as that would be an error on my part. I just upgraded all the equipment I use so I’m hoping the new equipment will solve that problem.

[@MohdUmar-e1r]

Hi, in my case new cache is added on every PR github action with the same key, But I want to add new cache only when node_version or package-lock.json changes

[@MickeyGousset]

So in that case you would want to make the name of the cache dependent on hash of the package-lock.json. So if the file changes, the hash changes, and it knows to rebuild the cache

[@MohdUmar-e1r]

@@MickeyGousset I am doing this "with: path: | **/node_modules key: ${{ runner.os }}-node_modules-${{ hashFiles('**/package-lock.json') }}-${{ steps.node.outputs.version }}" still on every new PR its creating cache, can you give me the correct key | code if I a doing something wrong

[@MohdUmar-e1r]

@@MickeyGousset Hi still waiting for the solution

[@sid_work]

What is the cost involved to caching? I understand the 10 Gb limit per repo, i just want to know whether it is billed or is it a free offering? We have GitHub enterprise license and most of our repos are private.

[@MickeyGousset]

No cost

[@sid_work]

Thanks for the reply!@@MickeyGousset

[@protecta22]

Is this cache global to all PRs (as long as the yarn.lock files don't change) or is it per PR? Also, wouldn't we gain more time by still using the caching properties of the actions/setup-node in the advanced version? How do you restore the cache between multiple jobs of the same workflow?

[@MickeyGousset]

Cache is “global” to all PRs in a repository, as the cache exists at the repository level. Cache is restored based off the cache name so if multiple jobs use the same cache name, they will access the same cache.

[@protecta22]

@@MickeyGousset thanks for you quick answer. And thanks for the video :)!

[@WalterFeddern]

Is there a way to get this functionality to work with self hosted runners? I have output of a workflow in the range of 20-30GB in a folder that needs to be referenced by the next run of the workflow. Running self hosted runners in EKS cluster, and have been trying to mount EFS volume to hold the data, but wondering if this might offer a better solution.

[@MickeyGousset]

That’s going to be too big for repo caching, as the max cache size is 10 gb. You’ll have to figure out something else, such as storing it on a shared volume, which is the direction you are headed I believe

[@akhilr94]

Caching node_modules is not recommended as per the docs. What do you think about this?

[@MickeyGousset]

@akhilr94 drop me a link to the docs, and I'll talk to some people and get an opinion.

[@MickeyGousset]

Caching node_modules can speed up your workflow runs by reusing the installed dependencies instead of downloading and installing them every time. However, caching node_modules also has some drawbacks, such as: It can increase the size of your cache and the time it takes to upload and download it It can cause issues if you update your dependencies or switch between different package managers It can expose sensitive information if you store any in your node_modules folder So I guess it depends....

[@akhilr94]

@@MickeyGousset Also, I tested out both versions. node_modules has less cache size than yarn global cache. Also since I'm on self hosted runner, I have to pay the penalty of downloading and uploading the cache from github runner. This network cost is leaning me towards caching node_modules altho the recommended approach is caching the global cache. The only reason against using node_modules is the node version inconsistency (according to the docs), but like you mentioned in the video, we can make the node version a part of the key to make it consistent.

[@airpoint9715]

Super clear! Thanks for this

[@MickeyGousset]

Thank you!

[@mignun15]

Hey great video! I was wondering, imagine you're using megalinter for instance, could you cache the pulled image so it is "reusable" everytime you run the workflow?

[@MickeyGousset]

You "could", but normally the cache is not meant for storing container images. If you keep it under 10 GB, then its doable.

[@mignun15]

@@MickeyGousset How would you go about it if you don't mind me asking? Ideally as the image is pulled I would like to cache it however I can't seem to quite understand which path to specify to do so.

[@yugandhar2007]

Thanks a lot!!

[@IntermattOfThings]

Great video! Thanks!

[@MickeyGousset]

Thank you!

[@sketchwaretagalogtutorials]

What about caching the apt

[@MickeyGousset]

Can you explain what you mean a little more?

[@sketchwaretagalogtutorials]

@@MickeyGousset I have apt packages in Ubuntu/Linux such as python3 and pip3 dependencies, I want to cache them for faster build times.

[@MickeyGousset]

@@sketchwaretagalogtutorials gotcha. Then it should just be a matter of finding the location on the build server where you have those files and adding them to the cache

[@sketchwaretagalogtutorials]

@@MickeyGousset nice. Thanks.