Autoscaling GitHub Runners: How to Create Elastic Runners in GCP

[@classicthirdeye]

Hey Mate, As the name of this channel speaks "Out Of DevOps" you are really thinking excellent, well explained enough. Its a bit hard to replicate for me but you have a very well documentation but easy to grasp the core idea. Hats off to you mate. Keep doing videos. All the best.

[@OutOfDevOps]

Thank you very much for your comment ❤

[@MrRamakrishnav]

Very well explained and clearly documented. Thanks a lot, it greatly helps.

[@OutOfDevOps]

Thank you very much. Feedback like yours pushes me to keep making vids.

[@DevopsCuriosity]

Excellent video! The combination of the well-explained written tutorial and the video made it simple to follow along with the steps. However, I've encountered a problem when using the GCP "Open editor" feature. It appears that the .yaml file is automatically modified (presumably by GCP), resulting in the following error: "unexpected EOF while looking for matching `)'". Do you have any insights into what might be causing this problem? Would you still recommend building an elastic runner using this method?

[@DevopsCuriosity]

I was able to resolve the issue. I am not quite sure what changed, but it seemed to work all of a sudden. One thing to note: I had another issue following this where the trigger would actually not trigger. There would be no error message, nothing. I eventually figured out, that the runner service account causes the issue as it seems to not have enough permission rights. I was able to resolve it by using the standard cloud build service account (not specifying a specific runner for the trigger). However, I am not sure which permission is actually missing. On another note: For me the runners take about 5min to spin up and actually work. Is this the same in your case? Because it seems in the video it only takes a few seconds.

[@OutOfDevOps]

Hey glad you solved… and yes the runner takes a bit too much. It is mainly because the VM gets rebooted to complete the configuration and there are few dependencies to be downloaded if I remember correctly. If you want to optimise the speed you could create a GCE image with everything installed, that should bootstrap in seconds.

[@DevopsCuriosity]

@@OutOfDevOps Great, thanks for the tip, I will try it.

[@tugorin]

@@DevopsCuriosity I am facing the same issue(no trigger and no error messages) and can't resolve it even running with cloud build service account.

[@humanfitcode1721]

Hello, nice video but one queation, i have 404 error when i add the cloudbuild webhook url to github.any idea thx

[@mpnewmanz]

Changing the 'region' to 'global' on the webhook fixed this for me.

[@classicthirdeye]

I have a bit of confusion here. In build-config.yaml which is run by CloudBuild service is creating a file called /workspace/ci.yml which is then mentioned in metadata -metadata-from-file user-data=/workspace/ci.yml . So who runs this ci.yml and where. In metadata startup-script and shutdown script which are bash script which I understands but in metadata user-data when it is run and why this user-data file is in yaml and who and where it runs. ? And also could you explain what is cloud-int, I am not across it ?

[@classicthirdeye]

If I am not wrong, Cloud-init is to setup the VM with the required files and packages everytime using ci.yaml which uses the yaml config like docker. Well We could use BASH in user-data script to run to install what is required ? BTW why we wanted to install all the software everytime as its timecounsuming instead we use a pre-installed template with all the packages installed ? Or can't we use start-up script to install the required packages ? Is there any disadvantage is following the above thoughts I mentioned or are we getting any enough advantages in using Cloud-init with user-data ?

[@OutOfDevOps]

Hey thanks, yes you are right it’s a bit of a waste of time to install all the dependencies every time a job runs. I think I wanted to prove that you can scale to zero the cost by even removing the need of storing a VM image. But in a real world scenario I would just create a GCE image with all the dependencies, to optimise for startup time instead of costs.