Shipwright - Building Container Images In Kubernetes

[@DevOpsToolkit]

What did you think about ShipWright? Could it become your go-to container image builder in k8s type of tool? IMPORTANT: For reasons I do not comprehend (and Google support could not figure out), KZbin tends to delete comments that contain links. Please exclude from the links to ensure that your comments are not removed.

[@bobbyray8457]

This is in my opinion one of the biggest pain points in the entire application lifecycle and I am so pleased to see progress made in terms of simplifying image builds. Thank you!

[@mayukh_]

Feels happy to say that I am an active contributor in Shipwright!!

[@DevOpsToolkit]

It's a great product and you should be proud.

[@a5a5a9]

Awesome ! waiting for the shipright + ArgoEvents + ArgoCD integration !!!

[@thesunilvarma]

Yes please, it will very helpful if you can make video on cicd pipeline with shipwright + other tools. It will be perfect timing at least for me I currently in need for this... 😅

[@CloudNativeJanitor]

this reminds me of the buildconfig resource on openshift in combination with image streams. loved your episode, simple to follow, agree with your conclusions as always ;-), would love where this is going with eventing/workflows

[@nitinkansal]

Thanks for this one ! I would like to see another video combining argo events with shipwrite for complete pipeline.

[@DevOpsToolkit]

Great! I'll start working on it after the next video.

[@DudeSkinnyTall]

06:50 Source to Image or S2I is from RedHat's Openshift. I used it in OKD 3.11 about 2 years ago so all I can remember now is it was rather advanced tech back then. It allowed me to automagically build container image from an application's source code. List of supported languages was somewhat limited but this might have evolved by now...

[@DevOpsToolkit]

Thanks for the info. I knew it was something familiar, but could not remember what it was

[@CloudNativeJanitor]

@@DevOpsToolkit its figures out the building tool and schema from the source code artifacts or by guiding it via option or builder image, customizable at three levels, pre-build, post build and how to build IIRC

[@christianibiri]

Awesome tool Viktor! I can¿t believe I did not hear the word "silly" in this video jaja

[@DevOpsToolkit]

That was a huge oversight from my side. I'll do my best to return "silly" to the upcoming videos :)

[@harshad_dev]

Wonderful video. Thank you for all the efforts. Keep it up. :) Just a small suggestion, it would be nice to reduce the pop up sounds for your lower thirds or titles.

[@DevOpsToolkit]

I think that they are now lower or completely removed. Can you check one of the newer videos and let me know if it's ok or it should be lowered more?

[@harshad_dev]

@@DevOpsToolkit I just watched your latest monitoring and logging video. I guess the title pop-ups sound is still there. It is not as such bothering but since I watch videos with my noise cancelling headphones and hence I feel the sound of pop-ups more. But this not something like showstopper. Your videos are really worthy and high quality content. :) PS. Would be nice to connect with you on social platform, following you on LinkedIn though, could't find you on any other media like FB/Insta

[@DevOpsToolkit]

That's great feedback. I'll tell the editor to lower those sounds in the upcoming videos. I'm not on Instagram and FB. You can find me only on twitter and linkedin and in a bunch of slack workspaces (devops20, Kubernetes, CNCF, crossplane, etc.). My user is always vfarcic.

[@harshad_dev]

@@DevOpsToolkit Perfect. Thanks!

[@squalazzo]

absolutely interested in integrating this with other tools to complete the process and bypass the missing features! thanks!

[@DevOpsToolkit]

Great! Adding it to my TODO list... :)

[@freeseacher]

Tnx! New visual style is awesome, but sound for them is bit loud and attracts more attention that it really needs

[@DevOpsToolkit]

Do you mean the sound from the animations or me speaking?

[@jonassteinberg3779]

This is *precisely* why I'm not convinced about the declarative paradigm. It's starts out like "yeah see all you do is line up these kv pairs in yaml and badda-bing-badda-boom" and then two minutes later you're like "I need a variable" and then someone's like "okay cool just use helm" lmfao. And at that point...why tf aren't we just doing procedural again?

[@ehudyonasi6064]

Great idea! Would be cool to see if it can be integrated with earthly and as you said with some configuration to know if there is a new tag or commit for the repo.

[@adarshjaiswal8256]

We need one more updated and detailed video on shipwright

[@DevOpsToolkit]

Adding it to my todo list...

[@mrocholl]

@@DevOpsToolkit Second this!

[@countbrapcula-espana]

Victor. I watch all your videos. Also we both look like "The Count" on Sesame Street. We're like brothers! Actually please please do the Argo Event/Argo Workflow as invocation video with ArgoCD and this Shipright. I would pay to see that video. Thank you for all the spectacularly educational videos. Cheers from Spain

[@DevOpsToolkit]

Adding it to my TODO list...

[@haimari871]

Thanks for the Video, The idea is great, but i'm not sure about implementation. in most cases we use a CI/CD tool or some kind of pipeline or any other solution to build and create a release tag (semrel) which will be later used for the image tag. Also If I'll use cli with any of the mentioned container build tools I can dynamically add metadata to the image, custom args easily by simply passing them and make them part of the CI if needed. I think at the time of writing this, Shipwright obviously going in the right direction but at the same time lacks simple and basic features which are required in most cases of container image building. The build pattern you mentioned combined will be great, however don't forget you will lose visibility in the pipeline tool you are using, since you will have no indication to what happened during build, unless you intend to use kubectl command from CI and not use GitOps approach, and that is what we want to avoid in the first place (cli tools in CI)

[@DevOpsToolkit]

You can run ShipWright through CLI in your pipelines (e.g., `kubectl apply ...`). Now, whether that approach brings enough benefits to use it is doubtful. Nevertheles, you are right. "Event based" approach to pipelines is in its infancy (just as most of k8s tooling and processes). But we are getting there. Shipwright is going towards the right direction, but it is indeed a young project and lacks a few things.

[@thescourgeofathousan]

@@DevOpsToolkit Hey Viktor, similar to many microservice-based solutions I see using event-based I wonder if the increased complexity, impact to traceability/visibility and emergent behaviour risks are really worth the benefits. It also seems that this solution is coming from the “we love Kubernetes and believe it is the future of containers for ever and ever” pov and building a solution only and always for kubernetes. Ie lock-in. Whereas when I’m building CD capability for an enterprise customer I’m working hard to keep my head in the CD headspace and not make the solution locked in to a specific application execution environment because they often have multiple types across servers, vms, container-based and we don’t want to build a whole stack of CD for each one…. I note that this video is from 6months ago, have you done any in the intervening time that cover progress in both the event-based pipelines concept and the do everything inside K8s paradigm? Cheers mate

[@DevOpsToolkit]

There is always a lock-in, the question is only what you lock yourself in, not whether you're locked. Kubernetes is the most widely used platform with contributions coming from almost all the vendors. That, from my perspective, is less dangerous type of locking than if I'm tied to a single company (e.g. Docker, AWS, RedHat, etc.).

[@thescourgeofathousan]

@@DevOpsToolkit but I’m not locked into a single company either. By maintaining a strong adherence to separation of concerns and modularity I’ve had a lot of success building a common set of CD orchestration modules & tech stack specific build/test/deploy modules all of who’s chosen tools can easily and independently be swapped out without impacting the up or downstream components. It is not as flexible as an event driven system I absolutely have to admit. Ie being able to simply deploy a new k8s resource type that responds to an event to either replace an older type or add to the virtual pipeline. At the moment though, the lack of determinism in an event-driven system as well as the added difficulty of creating a visualisation layer that displays the pipeline state in a coherent manner really puts me off. But that’s why I was interested in any videos you might have made in the time between now and then: maybe there are more fleshed out examples now might completely address my concerns.

[@DevOpsToolkit]

Unfortunately, I haven't done any newer video on the topic of event-based pipelines. Let me add it to my TODO list...

[@OlegKorsak]

does kaniko support multiarch builds like docker buildx?

[@DevOpsToolkit]

I do not think it does. You would need a separate kaniko execution for each arch.

[@HaraldKubota]

Great content (as always), but please remove the audio reflections of the room.

[@DevOpsToolkit]

Unfortunately, the next video was shot at the same temporary location and might have the same issues with the audio. Starting from the one after that, I'll be back home where the sound is much better.

[@nas1k]

Hm... I think it's not a good implementation yet but looks really cool as an idea. I mean it's really awesome to use Tekton as the low-level framework for the final solution.

[@DevOpsToolkit]

I would characterize it as a young project with a lot of potential, but not yet to be used by those not happy to experiment

[@thecomputercat8095]

Hello Viktor, amazing videos. I learn so much from you. This looks interesting but at one point do you run test against these containers during building and even before committing them to a repository. Currently I am building containers using Jenkins Jenkins builds containers and runs tests against them and if only when they pass they get pushed to ECR. in this case what is the benefit of Shipwright ?

[@DevOpsToolkit]

Tests that do not require the application at runtime (those before deployments) can often be included in Dockerfile itself (multi-stage builds). That simplifies both local development (devs can tests continuously by running builds continuously on their laptops) and the CI process no matter which tool you're using to build images.

[@impactcoders5996]

Hi Victor, thanks for your videos, they are really inspiring. I have a question based on this video and Argo Workflows - I wonder if you can help as it'll be useful for others I'm sure. I am creating the buildrun from an argo workflow and I'd like to sync the state of the workflow with the state of the buildrun. It was suggested on Shipwright slack channel that: "You should look at .status.conditions[type=Succeeded].status. This is a tri-state value, Unknown while running, True when successfully finished, False when failed." and I know that the relevant yaml in the workflow is successCondition & failureCondition for the step in the workflow, but I've really struggled coming up with the correct expression for this. Any ideas?

[@DevOpsToolkit]

You can make a simple Shell script for that. It could export the state into a variable (e.g., `kubectl get ... --output jsonpath="..."`) and then a simple `if` statement that would return non-zero if the value in the variable is `False`. Now, I'm not sure whether I fully understood the question so please let me know if I didn't or if you need more detailed instructions.

[@impactcoders5996]

Also, for everyone else, I needed to parameterise the shipwright buildrun but as per your video that wasn't possible. What worked for me was a pipeline that creates the build and then uses the newly created build to create the buildrun.

[@DevOpsToolkit]

@@impactcoders5996 The team behind Shipwright is working a way to parametrize builds. I can't say when it's coming, but I can confirm that its coming.

[@SebastianWillkommen]

Danke!

[@DevOpsToolkit]

Thanks a ton Sebastian!

[@jirityr]

Interesting project but I don't like the hard dependency on Tekton ;o( Otherwise the audio of the video wasn't great. You need to bring the mic closer to your mouth or use a clip-on mic.

[@DevOpsToolkit]

I also didn't like initially the dependency on Tekton. Nevertheless, I started looking at it as an implementation detail which just happens to be visible, which is a better option than if the ShipWright team made something "special" that would be only harder to debug and find info. Thanks for the tip about the audio. Can you please let me know if the audio in kzbin.info/www/bejne/q4aQdnSHbbuVrM0 is better. If it is, the problem will be solved by itself once I move back to my apartment (I am currently on vacations with hastily assembled temporary studio that has a lot of echo). If it's not good in that one either, I'll have to change the mic. Right now I'm using a shotgun mic (Rode).

[@jirityr]

@@DevOpsToolkit The other video sounds better but I can still hear quite a lot of echo there.

[@DevOpsToolkit]

@@jirityr Thanks for letting me know. I'll add more acoustic panels and see whether that removes the echo.

[@ec-eui5161]

No semantic versioning, so you have to explicitly define the tags. Having to create a PR with a BuildRun every time you want an image. Using Gitlab it takes about 3 lines of code to build a docker image... here it takes half a dozen manifest files? and a new one every time you want to buildAgain... I am scratching my head, but what is the point of this? I really don't see any advantage to using this.

[@DevOpsToolkit]

It is not a replacement for GitLab or any other pipeline tool but a potential replacement for one of the steps in pipelines (container image build). If you build in Kubernetes (which you don't have to), Docker is not an option and this is a kind of a wrapper around builders like Kaniko.

[@DavidBerglund]

I started to look at this project today. How to I get shipwright to authenticate and pull source code from a private Gitlab repo?

[@DevOpsToolkit]

Is github.com/shipwright-io/build/blob/main/docs/development/authentication.md#authentication-for-git what you're looking for?

[@DavidBerglund]

@@DevOpsToolkit looks like what I need, thanks!