Running Jenkins In Kubernetes - Tutorial And Review

Should we be using Jenkins for our Kubernetes CI/CD pipeline needs in 2021?
How to deploy Jenkins in k8s? How to run Jenkins builds in Kubernetes? How would a full CI/CD pipeline look like?
#jenkins #kubernetes #k8s #ci #cd
Consider joining the channel: kzbin.info/door/fz8x0lVzJpb_dgWm9kPVrwjoin
▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬
➡ Gist with the commands: gist.github.com/b0a32051da5ae3dea09e596689d7cf24
🔗 Jenkins: www.jenkins.io/
🎬 GitHub CLI: kzbin.info/www/bejne/eHqsZ42PZ7ehosU
🎬 Kaniko: kzbin.info/www/bejne/e5jah4SEa6yBgMk
🎬 Kustomize: kzbin.info/www/bejne/iqjXk5psgauhd8k
▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬
00:00 Intro
00:42 Installing Jenkins
06:57 Creating Jenkins jobs
07:44 Exploring Jenkinsfile
13:01 Exploring PodTemplate
15:16 Defining roles and permissions
16:05 Creating GitHub webhooks
17:55 Creating a pull request
20:05 Building pull requests
21:04 Building mainline
22:49 Should you use Jenkins in 2021?
▬▬▬▬▬▬ 🚀 Courses, books, and podcasts 🚀 ▬▬▬▬▬▬
📚 DevOps Catalog, Patterns, And Blueprints: www.devopstoolkitseries.com/posts/catalog/
📚 Books and courses: www.devopstoolkitseries.com
🎤 Podcast: www.devopsparadox.com/
💬 Live streams: kzbin.info
▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬
➡ Twitter: vfarcic
➡ LinkedIn: www.linkedin.com/in/viktorfarcic/

[@XxNocturnxX]

I'm interested in seeing how you deploy gitops through jenkins! Thank you for the vid.

[@DevOpsToolkit]

Adding it to my TODO list... :)

[@squalazzo]

@@DevOpsToolkit me too!

[@mikhail.mikhailov255]

+1

[@martinborgman9150]

Yes, please do show us how to do GitOps with Jnkins.

[@bobbyray8457]

As always a pointed analysis of an extremely relevant topic. Jenkins was basically ubiquitous and is still probably useful for several use cases so a deep dive on running it on k8s was very much appreciated.

[@roybargil]

This is an amazing tutorial ! , Well done very explained simply and in depth , super relevant and usfal Thank you !!!

[@RyanKienstra]

Great video, interesting that pipelines in Jenkins are good! A really balanced look at it.

[@FernandoAlmeida1973]

I have to live this comment here. Why the hell are you always publishing a new video about the problems I'm dealing with when I need it? Very very thank you...

[@DevOpsToolkit]

That's my superpower. I can read people's thoughts :)

[@vimalneha]

But always great tips that you share, thanks for that.

[@Peter1215]

Amazing video as always! I'm more on the "don't like it" side and can totally relate to the maintenance pain. For me tools like tekton, which are lower level but kubernetes native, are much better fit and weirdly enough less operational/maintenance pain once setup correctly.

[@DevOpsToolkit]

Exactly. Tekton (or Argo Workflows) might present a bigger initial investment but once they are doing whatever they should be doing, maintenance is relatively low.

[@bjorndemicoli8254]

great stuff!

[@windfreak]

Thanks for the video. I know Jenkins quite well, but you also learn while checking out how other use it. I currently still use Jenkins in Azure instead of Azure Devops as I don't want to depend too much on Azure and want to keep the Jenkins knowledge when I need to create pipelines for other environments (and I will have to do that )

[@GuillaumeMaka]

Yesterday I was jsut thinking about "I need to check if Viktor have a video about Jenkins on k8s" The next day right in my recommendation: "Running Jenkins In Kubernetes - Tutorial And Review" and its a new video. What a great day!

[@DevOpsToolkit]

You see? I can read your mind :)

[@damienlust]

yes let's go for GITOPS with Jenkins , tks

[@DevOpsToolkit]

Great! That seems to be a popular request so I'm pushing it closer to the top of my TODO list.

[@sanketjaiswal1137]

The world is waiting for Jenkins replacement.

[@DevOpsToolkit]

There are many replacements for Jenkins. Pipelines might be one of the most saturated segments of the market. My main beef with pipelines is not that much that there are no replacements for Jenkins, but, rather, that there isn't much innovation in that area. Most of the tools look and behave more or less the same.

[@vimalneha]

Sure i would like to see gitops part you mentioned

[@DevOpsToolkit]

Great. Now that I know that there is interest, I'll add it to my TODO list.

[@sanketjaiswal1137]

Yes GitOps with Jenkins

[@morbidflames]

I´ve been looking for an alternative, but at the end of the day Jenkins does the job, I´m afraid of been stuck in the golden hammer anti-pattern. I have some a: what can be a good modern alternative to Jenkins? I usually do automation stuff with it, but I can see how Jenkins is becoming an old tool. Your content is awesome, thanks for sharing your experience and knowledge.

[@DevOpsToolkit]

What is the alternative to Jenkins really depends on what you want and what you do not want to do. For example, Argo Workflows and Tekton are great, but they are relatively low level and require a lot of work. GitHub Actions and GitLab CI are also good. They require less work but might be limited in what they do. Codefresh is good as well, but SaaS (or hybrid) might not be a good choice for everyone.

[@venkataramanaakkineni3583]

Thank you for sharing the knowledge, could you please do video on spinnaker. When to use spinnaker and Argo ?

[@DevOpsToolkit]

Adding Spinnaker to my TODO list... :) As for Spinnaker + Argo, it depends which Argo project you're referring to. If it's Argo CD, it only does deployments or, to use better words, synchronization between manifests in Git and what's in a cluster. So, Argo CD can take over the deployment part of Spinnaker. On the other hand, if you were referring to Argo Workflows, the two are designed to accomplish the same goals. Both are pipelines and, normally, you would use one of the other. I'll do my best to make a video and go through those and other points in more detail.

[@gcezaralmeida]

Congratulations Viktor, awesome video! Could you compare Argo and Jenkins? What is better in your opinion? Pros and cons?

[@DevOpsToolkit]

I'm planning to create a video that compares the major CI/CD pipeline tools, including Jenkins and Argo Workflows. However, such types of video require a lot of work and it will likely not be done in the next few months. Until then, the major differences are: 1. Argo Workflows was designed to be k8s-native (CRDs, scaling, etc.) while Jenkins was modified to work inside k8s. 2. Jenkins is a jack of all trades while Argo Workflows is focused on k8s workflows. 3. Jenkins is a higher-level type of tool and provides more out-of-the-box. 4. Jenkins maintenance is much more expensive. From my perspective, within the "self-managed" category of pipeline tools, Argo Workflows and Tekton are the winners without any doubt. Most of us will be using it (excluding those who prefer SaaS) either directly or through other tools built on top of it.

[@zamboz01]

Please show how to do GitOps with Jenkins :)

[@DevOpsToolkit]

It's coming... :)

[@Feryero]

I had to maintain Jenkins and it is a huge pain. It's the main reason why I hate it and try to stay as far away from it as possible. Problem is, if the customer has a lot of legacy applications in old infrastructure, I'm afraid it's the most viable solution.

[@marwan.alrihawi]

Deploying Jenkins master out of K8S and jenkins agents in K8S is better and more stable solution

[@kingdonbarrett3753]

Deploying Jenkins in Kubernetes declaratively, with GitOps, to build and push images for GitOps deployment, is next level "third frame of that one meme" energy 🤯very stable and very cool

[@danielacosta9662]

Please show how to do GitOps with Jenkins

[@EduardSchildner]

Really nice video 👍 I'd put down another disadvantage to Jenkins running in k8s: pods run only in sequence / or at least I haven't been able to to run multiple pods in parallel. Imagine multiple containers running inside one pod - what's the pod's resource consumption? The answer: add up memory & CPU requirements for each container in such pod... From the start of the pod till the end. What if I have a resource hungry container running one stage, and then some other stage running in a small container? Add the time and add the requirements. An alternative is to create multiple single container pods... But these can only run in sequence :-( I'd be happy to find out I'm wrong and even if I am - the verbosity. Argo Workflows + DAG = my happy thoughts. Can't wait for dumping Jenkins finally :-)

[@DevOpsToolkit]

You're right. Jenkins k8s plugin is focused on containers inside a Pod, and not on running each step as a separate Pod. That's much better solved in Argo Workflows or Tekton. That being said, sharing the state suddenly becomes more complicated, but that's a small price to pay when performance and resource consumption are concerned.

[@SurjitBains]

you can run each stage in parallel here's a snippet of this being carried out... github.com/polarpoint-io/groovy-jenkins-shared-library/blob/master/vars/java_v_0_2_0_workflow.groovy#L108-L128

[@DevOpsToolkit]

@@SurjitBains Jenkins can definitely run containers inside a single Pod in parallel. However, the initial question was whether it can run each step as a separate Pod. Is that what the snippet does?

[@SurjitBains]

@@DevOpsToolkit that's correct (it's using a plugin ;) ) and as part of a shared library

[@DevOpsToolkit]

@@SurjitBains Where do you tell it to use a Pod for each step instead of a container inside a single Pod?

[@DevOpsToolkit]

Is Jenkins relevant in 2021? IMPORTANT: I used the old terminology in the video. Please read cd.foundation/blog/2020/08/25/jenkins-terminology-changes/.

[@StvNklaR]

We are actually doing GitOps in Jenkins already (with fluxcd v1 still). How would you share deployment logic (for temporary deployment to run tests on) and having flux/argo at the other side. Love DevOps Paradox :-) Thanks!

[@DevOpsToolkit]

Typically, I would add new definitions whenever a PR is created, update those definitions when PR is updated, and remove them when PR is closed. Those definitions would be using a unique namespace (e.g., repo + PR number). That way, every PR has an app running only until the PR is closed or merged. If you combine that with, let's say, KNative, than those apps would be running only when someone is using them.

[@nagachaitanyab3363]

Hi Thanks for the video. I have jenkins setup running on k8. What if I want to add users or credentials in values.yaml file and helm upgrade..instead of doing it in the jenkins console. How do I do that?

[@DevOpsToolkit]

You can do that through Groovy, but it's not a pleasant experience. The major problem is that Jenkins was not designed to be an API-first type of tool (most from that era were not), so many of the things we take for granted today are complicated with Jenkins.

[@baydakovss]

In addition with gitops I wonder if you create a research with circleci instead of Jenkins please

[@DevOpsToolkit]

Adding it to my TODO list... :)

[@yasirkaram]

How do you rate gitlab-ci, gitlab-runner with Jenkins?

[@DevOpsToolkit]

GitLab is good. It might not allow all the permutations you can have with Jenkins (no tool does that), but it's simple and it just works.

[@GuillaumeMaka]

What's your thought about JenkinsX?

[@DevOpsToolkit]

I stopped following it around 6 months ago. Give me a bit of time to see what's new and get back to you.

[@AlexSchiessl]

Mostly agree what you're saying... And I stopped using Jenkins because of mentioned "con" reasons... But there is JenkinsX... Have you looked into that, too?

[@DevOpsToolkit]

I did much more than that. I was working full-time on the project in different capacities. Eventually, I left the project due to conflicts about the direction the project should take. Since then, I'm not following it closely so I cannot comment on its current state nor give any recommendation on whether to use it or not. I do not want my past experience with JX to be projected onto it since it might be in a very different state right now.

[@m19mesoto]

To much hate... Not keeping the focus, was it useful, not that much. The main reason, Companies refuse to change their CI/Jenkins. In general all you said is true and valid, but just try to keep focus better, I wold suggest you to use IDE, is just more descriptive. Learning from hacking console/vim is not that easy, except someone like to lost his time. I think in 2021, jenkins is still relevant, main reason is because companies required to and is up to use to make as you said more portable etc. I also don't agree that writing Argo/Yaml is easy then writing Jenkinsfile or gitlab-ci. I don't see a reason why kubernetes should become now configuration manager.

[@DevOpsToolkit]

I would not say that there was "too much hate". Instead, I am trying to give an honest opinion that includes both the good and the bad. One of the main reasons I started this channel is because I'm disappointed by people who always present everything as perfect and are afraid to call out the bad sides of tools and processes. I believe that is subjective and not very useful. As a side note... I used Jenkins since it was first released ~15 years ago. You can say that I was the early adopter. Years later, I joined CloudBees, and Jenkins (as a product) was my full-time job (in different capacities). I was heavily involved in the decisions and roadmap of Jenkins (as well as Jenkins X) and many of my friends are Jenkins maintainers and/or people working in CloudBees. Jenkins has a "special spot" in my heart. Nevertheless, that does not mean that I would only talk about its good sides, and not the bad ones. That would defy the purpose of this channel. I did my best to be as objective as I can, but that does not mean that I succeeded. If you felt the video was subjective and full of hate, all I can say is that wasn't the intention and that I obviously failed.

[@m19mesoto]

@@DevOpsToolkit Thank you for your replay.. I think what you said is good and valid. I am not the biggest fan of jenkins at all. For the past year or so I am trying to make our jenkins work like Git-Lab, however due to many reasons this is simply not possible. I think the in the most of time Jenkins has bad repostation because people are just using in in the wrong way. Anyway the bottom line is the your videos and other videos like your's are helping all of us to see the outer perspective / other way / other point of view / I have so many comments on how jenkins is working / plugins etc. Also how people are using it.. I would like to hear your opinion about Git-Lab.

[@DevOpsToolkit]

Unfortunately, I haven't used GitLab enough to be able to judge it. I'll do my best to dive deeper into it soon.