Argo CD GitOps Promotions with Kargo (by Akuity): A Brilliant Idea with Flawed Execution?

[@DevOpsToolkit]

What do you think of Kargo? Do we need such a tool (promotions) and, if we do, do you think Kargo is the one?

[@Luther_Luffeigh]

I believe GitOps friendly promotion tool like Kargo is really needed. I think in general CI/CD tools have a hard time enabling all use cases and workflows because every company has it’s own process or conventions. So I don’t blame Kargo for being opinionated because it’s probably unattainable to fit all workflows. They also said in V1 announcement that they intend to support Terraform as well…which I don’t know if that’s a marketing hype or a real intention 😅

[@koushyk]

Im using kargo on 2 my projects, and generaIly I like it

[@Luther_Luffeigh]

@@koushyk did you had adjust your existing setup/workflow/release process to fit in Kargo?

[@koushyk]

@ nope, I’ve just replaced deploy steps with Kargo, now with promotion directives Kargo is now more flexible, I think they will more flexibility with time. Sharding is working fine as well. But I haven’t used analytics templates for now. I had to separate monocharts and split it to bunch of separate projects it Kargo

[@philipclark5216]

I like Kargo a lot! I’ve been watching it for awhile, waiting for it to “mature”. One thing to note, their docs show using branches for stages, but you definitely don’t need. I use it with a standard path based approach just fine

[@dirien]

Thanks for finally making the video! I completely agree with you about the documentation-it’s absolutely terrible. Period. I’ve given two talks about Kargo: one before version 1.0 and one after. In both cases, it took a lot of effort to get my use case working (Pure GitOps, no branches, and updating the dependency of a Helm umbrella chart). That said, as you pointed out, Kargo has the potential to become the tool for all things promotion. Whether it’s with a polished UI (hello, Argo CD!) and manual gates (enterprise-ready!) or fully progressive workflows, it’s on the right path. I also think it won’t be long before there’s an option to create custom workflow steps, which should address some of its current shortcomings in time.

[@deado7282]

"I assume you know Argo, if not: 1. Be ashamed... 2. Once the shame wears off: learn Argo" 😂😂😂 Im rolling

[@Pianodestroyer1]

You can use analysisrun to run arbitrary k8s jobs after promotion, incl. tests...

[@ZoisPag]

I came here to say this. Looks like Viktor made some assumptions as well! 😅

[@GeraldNunn]

Isn't an issue with analysisrun is that everything runs in parallel? Sometimes you want to execute post-promotion things in a specific sequence

[@phillipsma]

Thanks for making the video. You've basically expressed where my headspace was at with Kargo, and how it might or might not replace my current workflow. I'll stick with an Azure pipeline for stage manifest generation + PR for merge/deployment gate. It works for me but only as I use a custom operator to couple argo events back to the pipeline so it can continue with other testing tasks, knowing the deployment is up and healthy. It's that integration that is problematic as far as I can tell.

[@powersurge5576]

Amazing! tx for this video Viktor!

[@joebowbeer]

Thanks for making this! Simply put, Kargo is a controller for some new resources incl. "Freight" plus a cool UI for observing and approving promotions? I wonder if the k8s events from these resources are sufficient for integrating other tools. Philosophically, I'd like to see the rendered manifests treated as "artifacts" that are published to OCI, but I have heard some counter arguments.

[@vagrant4154]

It's a nice tool for devs, and could be great for shift-left. Unfortunately it's unusable for a lot of people due to the fact that it doesn't support things like Datadog version tags in annotations, so basically updating anything outside of the basics like image tag when using kustomize wasn't possible in our testing. Still watching it though.

[@JamesStrachan]

if you render the YAML for all of your stages from your kustomize/helm via CI (adding whatever Datadog annotations you need - or the labels Viktor was thinking of in the video) like this video: kzbin.info/www/bejne/fIioY2VqbNSWqqs you can then use kargo to promote/rollback by just telling ArgoCD to point at the git SHA of the properly generated YAML with all the annotations/labels you require - this also lets you avoid having lots of branches; just 1 branch for your GitOps releases with a folder per stage/cluster

[@ЮрийЮрьев-х2т]

true

[@zoop2174]

Thank you, very informative.

[@IvanRizzante]

Thanks for another great video! I see the idea behind Kargo and I agree that it's still not clear if it's going to be "the" approach. It gived the same impression to me that it's something trying to replace pipelines. Now I don't know Kargo but I do promotions using a simple git repo where environments (or stages) are in different folders and then I use argocd-diff-preview to show users what they're going to change. The main concern I have is: would developer love having to cope with Warehouses, Stages, Projects and learn all this when they can do pretty much the same by using a simple git repo and moving manifests among folders? It seems to me Kargo is a platform team (or Ops or whatever) rather than a developer friendly tool. That being said I'm still curious to see how all this evolves

[@len4ezz]

Pretty sure that devs will love the UI, where they can promote versions of their artifact by themselves. Also nicely can see which version deployed where

[@koodauskanava9096]

Have you open GitHub issue about the Event or integrations to workflows?

[@DevOpsToolkit]

I did not open an issue but I did discuss it with people from the project.

[@starstreamlab]

Could Argo Rollouts be used for something similar?

[@DevOpsToolkit]

Rollouts come later. Once you promote a release Rollouts can do progressive delivery (e.g. canary deployments) of that release.

[@sergeyp2932]

For some time (even before adopting GitOps), I used docker tags as promotion flag in pipeline scripts (e.g. "docker pull app:XXXX && docker tag app:XXXX app:prod-XXXX && docker push app:prod-XXXX") which can be executed automatically or manually after testing. After switching to GitOps, I considered using Flux Image Automation to automatically deploy these tags in corresponding environments. If it were done that way, it would have become my own analogue of kargo. But it wasn't, because of the same synchronization problem: pipeline has no way to know the moment when promoted version is deployed in specific environment. So it was finally become classic and obvious "yq -i && git commit && git push && flux reconcile --with-source". P.S. Interesting. Just checked docs on Flux Notification Controller and found that it supports emitting events to NATS. Maybe, asynchronous pipelines using image repo as outbound event bus and NATS as an inbound one is already possible today, but it requires some testing.

[@DevOpsToolkit]

I wasn't aware that Flux is emitting events to NATS. That is the general solution I thing everyone should choose as a start. After that, if someone wants to add a more specific way to stitch things together... Great. Otherwise, events should work.

[@fpvclub7256]

Can you not use Argo Events to trigger test automation based on the deployment events from Kargo?

[@DevOpsToolkit]

That's certainly an option but argo events themselves are problematic and probably the Argo project that received the least love. On top of that, while that is possible it becomes unnecessarily complex. I am doubting the usage of kargo over simply updating yaml in workflows. If events need to be added my doubts only increase.

[@m19mesoto]

kustomize 🤮