Exploring KCL: Configuration and Data Structure Language; CUE and Pkl Replacement?

[@DevOpsToolkit]

Is KCL the data structure language to rule them all or yet another out of many?

[@gowthamg6760]

completely agree with you

[@geofftsjy]

What do you think of using Pulumi to author complex yamls like k8s manifests? I believe it gives you the option to create the file only and not kubectl apply them. If you're already using Pulumi, the k8s resources already exist and work with intellisense. They can be combined in all sorts of logical ways. Policy as code can be applied to them. I've even toyed with the idea of making a custom git commit provider to perform the commit to a gitops repo/pr after formation. It would save you from injecting another tool (if you're already on Pulumi). I'm coming from using Kustomize and haven't played with the other tools you've been speaking about in this area. KCL looks promising, btw =)

[@DevOpsToolkit]

@geofftsjy I think that those should be behind an API. End users should send requests to APIs (no matter which tool they use for that) and let controllers convert them into whatever should be the end result.

[@dreamcat4]

Thanks so much for such lovely video... i agree with you a lot here and it's great to see you ahead of us trying out and working with these new tools, both the CUE and KCL and sharing how it really is. And I don't really trust anybody else as much as your good self here, in terms of these kubernetes and devops space to paint the picture how it is. So grateful & just hoping that the longevity aspect that they will last longer timespans to remain relevant going forwards. To reduce those constant learning churning tooling burdens. Which is such challenges within devops communities. All my best my friend

[@dirien]

I love KCL! Thanks for making this video! 🎉

[@DryBones111]

Great video. I will definitely have to give KCL a closer look. I think the perfectionist in me will continue to like CUE, but the pragmatic side of me says KCL will be easier to get a team on board with it. I'm just not sure I'm willing to give up the CUE scripting engine.

[@klassica]

I'm trying to use KCL to generate reusable IAM configurations to deploy with Crossplane. Definitely a masochistic but enlightening journey so far!

[@IndigoVFX]

Just as I’m getting to grips with CUE 😂 Thanks for the heads-up - and I agree with the self inflicted pain being worth it.

[@vtomilov]

Main issue with CUE is performance, that is impossible to overcome at some point. The fact that writing and reading CUE code is so hard can be argued like 'maybe I'm not smart enough and I need to be a better person to do that'. But performance was a killer for my project, so we migrated to KCL in three weeks with over 20k lines of configuration code.

[@cowgod77]

That's pretty exciting. Cue sounds like it ticks all the boxes I want, but I've tried THREE times to work through the "getting started" documentation, and for some reason just never manage to wrap my head around it. Meanwhile, your example KCL project in this video seemed immediately intuitive. I'm diving right in -- this might finally be the Helm replacement I've been looking for for my company's internally-developed applications.

[@DevOpsToolkit]

A simple explanation of KCL could be "as powerful as CUE, but easy".

[@DryBones111]

Thanks! I was also thinking of looking into KCL but I knew you would too, so I waited for this video 😊

[@AndreaGhensi]

Now that I have your blessing, I'm going all in with KCL! And I'm also glad that they already fulfilled your request of having a nix package!

[@Jarek.]

Thanks for this video, very useful - as always! A few weeks ago I've started exploring KCL and indeed, it seems more compelling than the others in the bunch.... My usecase is: as dealing with Vendor-provided Helm chart I can't really generate manifests, just override value yamls. Admittedly, they are quite complex and having a lot of environments does make managing them quite complex. So looking for templating these yamls using KCL. CUE failed at importing chart schema, KCL at least was able to import it....

[@juanbreinlinger]

Not sure if I understood your use case, but sounds like you can use helm dependencies and inject your own templates in your own custom chart to add functionality.

[@DevOpsToolkit]

@juanbreinlinger It is not that much whether I can or cannot do that with Helm but, rather, that Helm was not designed to work with data. It is a wrapper around Go templating which is free text templating completely unaware of data scructures and schemas. Go templating is just as good (or bad) at working with, let's say, html as with data. CUE, KCL, and a few others were designed to work with data and yaml, the output, is nothing more than structured data.

[@Jarek.]

@@juanbreinlinger Sorry, I should have been more clear. The chart I got from the vendor is fairly complete and does not need any modifications or extensions. It's "just" to generate value yamls used during the deployments (helm install). The difficulty is that there is a lot (and I mean - a lot) of environments differing not only basic things like IP addresses but very often - architecture. So now I need a tool to template these yamls and generate them based on the env-specific config file. Right now solution I've inherited has some basic templating but this needs to be taken to the next level - and KCL seems like a good fit.

[@KasunRathnayaka]

💌 KCL is awesome. need to learn more about it . thanks for sharing

[@EphraimMower]

Argo CD applicationsets seem like they were made just for us! We have many tenants (Argo CD applications) who use nearly the same manifests. But this presents an issue if we'd like to move toward an approach where we are generating pure yaml to be applied by Argo CD. Do you have any thoughts on reconciling these two concerns?

[@DevOpsToolkit]

Assuming the the number of actual differences is relatively small, I'd go with kustomize.

[@joebowbeer]

KCL is currently the 2nd-most active CNCF project in terms of end-user contributions

[@ErickCarty]

How does KCL compare to CDK8s, would they be complimentary or overlap in defining K8s apps? Would one be used more for certain use-cases or to help address platform/app complexity? Thank you for sharing your insights!

[@DevOpsToolkit]

Both serve the same purpose; transform code into yaml. Hence, the main difference is the suntax, the language itself.

[@adrien-barret]

one advantage of cue would be timoni, I know that you can find some pck manager for kcl too but may be not complete as timoni atm, may be you can talk about that in another video :)

[@DevOpsToolkit]

Upcoming changes to CUE might remove the need for tools like Timoni. More on that soon (when those features get released). On the other hand, Timoni has additional features that neither CUE not KCLand Pkl have. It's more like Helm that contains kubernetes specific features while others are limited to equivalent of Helm temolating that produces yaml.

[@east4ming]

As an Ops using yaml/kustomize/helm to configure k8s, I would like to use this kind of language, which one is more suitable for me, CUE/Pkl/KCL?

[@DevOpsToolkit]

I'm in between CUE and KCL. Both are great with CUE being more robust but with steeper learning curve. If I would have to choose only one of those, I'd probably go with KCL since it's easier to write and read and is easier to propagate to others.

[@IvanRizzante]

Thank you for another great video 🎉 again I am biased I use jsonnet with tanka and I still haven't figured out anything about KCL that is so missing in jsonnet to make me think about replacing it, except maybe for type checking which is good. Still I am trying to be fair with my judgement and I must say that I like the approach KCL took, I'll definitely have a deeper look at it

[@geofftsjy]

I wonder if KCL is a callback to JCL for mainframes... (Job Control Language)

[@DevOpsToolkit]

It's more of a combination of Json and Go.

[@iamunknownperiod3355]

So, I checked out CUE because of this and the recent Pkl video. Weirdly, I haven't seen your CUE video yet lol I haven't worked with CUE, KCL or Pkl extensively, just tried some getting started examples and browsed the docs. But CUE seems to me to be the most flexible of them all. Those scripting features and the standard library seems to enable functionality that just ties it all up nicely in one tool. Pkl seems to be the easiest to pick up in all honesty. KCL seems more like an easier CUE without those scripting features (or maybe it does and I'm just not aware?), and exclusively for JSON/YAML (they say "etc" but the CLI only accepts json/yaml and I can't find if I can extend it) Overall, CUE seems to be the tool for me and I'm honestly thinking of integrating it into my workflow. I'm not entirely working on YAML manifests so KCL is a no go, I need the flexibility to template other things and the added scripting features is an added bonus.

[@martingreber1700]

You could use the builtin template package and then just print out the rendered template. Or create a lambda that transforms your objects to something else and print it out.

[@hkupty]

As a cue enthusiast I'm happy you kicked out pkl. 😂

[@theaugmenter]

I don’t know where you work but no job and no team can follow all the new technologies you are talking about in each new video😮

[@klassica]

He works for Upbound, the makers of Crossplane, which is another great DevOps tool (with a learning curve... Viktor has a series!)

[@countbrappcula]

Then a lot of the orgs not at least considering GitOps with Terraform with either FluxCD or ArgoCD Controller or Crossplane and ArgoCD are already outdated

[@mvam75]

One of the big features these tools miss are `diffing` functions. While i would love, love, love to replace our homegrown and incredibly slow and difficult to maintain yaml generation tooling - its one saving grace is the ability to live diff you're changes against a cluster from your workstation. without that, no tool will get traction in my org.

[@DevOpsToolkit]

Are you referring to diffs in yaml produced before and after the changes? If you are, diff tools do not have to (and probably should not) be a part of kcl or similar tools.

[@mvam75]

@@DevOpsToolkit Diffs against the cluster for changes you are making. For example: deploying a new version of a CNI, once the version is updated locally in a branch, the diff can be generated between the local branch and the cluster prior to merge (requires access to the cluster ofc). Without any such feature, any off the shelf tool is instantly considered a non-starter - regardless of how much easier said too would be to use and maintain.

[@DevOpsToolkit]

There is `kubectl diff`. It's baked into `kubectl` so there is no need to even install anything. Then there is diff provided by Argo CD if you're using it. Most of Kubernetes dashboards have diffs. And so on and so forth. So, there are plenty of tools that do the `diff` between the desired state (YAML) and what's in the cluster. Even `helm diff` works in the same way. Behind the scene it executes `helm template` to generate YAML and then it compares it to the actual state in the cluster. That leads me to think that I misunderstood your question and that you're referring to something else. Is that the case?

[@ev4043-x6e]

just for thinking of studying Cue or Pkl

[@remi2215]

I've just learned cue for a project to generate docker-compose manifests and now I must consider kcl 😢 cue is not very easy when playing with loops and conditionals on structures and mostly lists (but I succeeded to do so). I'm not very happy to execute go get ... k8s to download a lot of stuff before being able to apply k8s constraints, and the equivalent for docker-compose is not effective because of an issue with docker-compose go structure implementation. Also cue modules are a little strange to use with implicit/not obvious behaviours concerning files taken into account/packages, OCI is experimental. I would like to use cue to help non technical users to create configuration files. I wonder if kcl offers better features for all these concerns. Another techno another journey 😅

[@DevOpsToolkit]

CUE is more powerful and feature rich than KCL. On the other hand, it has a steeper learning curve and you might not need all those features. KCL is a great alternative to CUE especially for those who do not want to dedicate considerable time learning it.

[@juanbreinlinger]

My only advice about all these tools , based on my own experience, is whenever you are faced to choose one... be mindful . Helm has become a standard... , it is in every single project I want to install. Is it perfect? Far from it... but in 5 years time It would be interesting to see how many of these other projects survive. The infrastructure tools ecosystem has always behaved in waves... a new tech comes out... multiple alternatives show in the market... eventually just a few survive. As professionals our responsability is to provide the best solution, but also considering the project long term mantainability.

[@DevOpsToolkit]

You're right about survival. Tools and languages are born with the goal to solve a problem that was not already solved or to improve an existing solution. Most of them die. Still, not all of them die. Some survive and end up in our tool belts until a new one replaces them. CUE seem to be here to stay and KCL is one of the fastest growing projects in CNCF. Both have a chance only if we give them the chance. Others in that area are not that promising. ytt is an example of a tool that serves a similar purpose yet is likely not going anywhere. Ultimately, it is us, users and contributors who make those choices. Finally, I do agree that helm is the standard for third party apps. Every project has to have a helm chart. Still, that does not mean that it is always the right choice when defining our own stuff. When i talk about kcl and other languages (helm is not a language though) I am focused on using them to define our own stuff rather than for third party apps.

[@scottscoble2500]

nix-instantiate to output json ... it's quite slick and is what terranix uses to create terraform json

[@michals.2604]

I feel like these intermediary languages are all crap mainly because Go or Rust suck at making good DSLs. KCL,CUE are extremely complicated for what little value they provide at the end of the day.

[@DevOpsToolkit]

i can understand how cue can be labeled as complicated but not KCL.