Crossplane - GitOps-based Infrastructure as Code through Kubernetes API

[@MarkusEicher70]

Hi Viktor, hello Community. For me as newcomer to the IaC, GitOps, DevOps space currently planning and building my own development and runtime environment, it is very interesting and helpful to have your videos and expertise. Also very helpful comments in here. I like this place. ✅

[@soubinan]

Crossplane + ArgoCD (or FluxCD) is so powerfull!! Clearly, I add this combination in my top toolset

[@DevOpsToolkit]

Yep. It's an amazing combination. The only thing that prevents me from using it 100% is that Crossplane still has a relatively small number of CRDs so it's useful mostly in situations when what you need is what it's currently supported (unless there is time to add the missing things and contribute to the community).

[@jl17876]

When you run a Terraform plan, you can see drift detection against the desired state. It isn't automatic, but tools like Terraform Cloud do automate this & provide regular drift detection scans. Regardless though, Crossplane looks promising 😀

[@kevinruder9652]

You have the best explainer videos on Cloud Native Technologies. Great Stuff

[@zarbis]

I knew it! As soon as I've seen Google's Kubernetes Config Connector, I knew it was a matter of time that something cross-platform would appear at some point.

[@DevOpsToolkit]

Crossplane has been around for a while. I got in contact with the a year ago when it was just starting. However, at that time, it had only a handful of modules. I felt they were on the right path, but not yet ready for prime time. Now I think that it is a good candidate for IaC needs. It is nowhere as "big" as Terraform though. Still, if we all wait until something new is as big as something older, new stuff will never get a chance.

[@mikedonot]

Lively style, thanks for the video. First, I asked myself why Salt Project wasn't mentioned in the beginning, but after a while I figured out that maybe it was for the better.

[@DevOpsToolkit]

I feel that Salt is slowly fading into oblivion. I gained some traction a while ago, never never enough to warrant its sustainability. Still, I can do a review of or compare it with others it if that would help.

[@sep69]

Great stuff ! Thank you for this video. For my customer I already have k8s and argocd running. They want to move to Azure and this looks like a great way to manage the infrastructure. Thanks again and all the best :)

[@juancarloschristensen]

Thanks for the video Viktor. I would love to see a video on Pulumi vs Terraform vs Crossplane, and another between ArgoCD vs Jenkins X.

[@DevOpsToolkit]

I added both suggestions to the TODO list, with the latter being in a slightly different form. Argo CD and Jenkins X have very different objectives and try to solve different problems. Argo Workflows + Events could be, somehow, compared with (a part of) Jenkins X. At the end, I'll probably compare Jenkins X against all Argo projects combined.

[@DevOpsToolkit]

I decided to do a video on Pulumi first, and then another one as a comparison between Pulumi, Terraform, and Crossplane. The Pulumi review video is coming tomorrow, and the comparison will probably be published next week.

[@milenkomarkovic]

Great explanation! Hvala Viktore!

[@aarontavio]

Hey Viktor! thanks for your 20 minutes videos. I'm learning a lot about interesting DevOps Tools. After watching your crossplane videos I think I definitely have to give it a try.

[@srsh77]

Thank you so much for sharing this information. A tool like Crossplane is what we are missing truely. I'll try it today.

[@DevOpsToolkit]

Please come back and let us know what you think about it after using it for a while.

[@karimnaufal9792]

You're a genius Victor! Amazing use of crossplane, thanks for sharing!

[@JohnWu]

Thanks for the video. Now I am going to implement crossplane actually. Wish to get more videos on specific use cases and examples of how to use crossplane, argocd and other tools in combination to deploy a web app with database for example.

[@DevOpsToolkit]

I will publish more videos about crossplane and other tools. However, those will probably go to the Upbound channel instead of here. Since I joined upbound, I am avoiding crossplane on this channel to avoid being labeled as subjective.

[@slavafomin]

Hello, Viktor! Thanks again for a great video. Please correct me if I'm wrong, so in order to deploy a complete IAAC solution, I will need to first manually create a small GKE cluster to which Crossplane and Argo CD are getting installed, and then use these tools to actually provision separate clusters for my applications? In your examples you are using local minikube, but in production this should be a real GKE cluster, right? What would be your recommendations for how to automate the deployment of this initial "control" cluster then? Also, what are the requirements for this cluster in production? Is there a way to calculate how much resources should it take? Thank you!

[@DevOpsToolkit]

You do need a k8s cluster to run Crossplane. Once you do, you can use Crossplane to manage that cluster (and anything else). The alternative is to use Crossplane Cloud which gives you a control plane with Crossplane. > What would be your recommendations for how to automate the deployment of this initial "control" cluster then? Create a cluster any way you would normally do, install Crossplane, and use it to manage that same cluster from there on. To do that, you'll need to set the `external-name` in your definitions so that Crossplane knows which existing cluster it should manage from there on. > Also, what are the requirements for this cluster in production? Crossplane is very lightweight so a minimum cluster should do. That being said, I normally do not have a cluster dedicated to Crossplane but use the same cluster for other tooling (e.g., Argo CD, Workflows, etc.). > Is there a way to calculate how much resources should it take? I don't have the exact figure. It has a very low footprint and so (minimum) size should do. If uncertain, install Crossplane locally and run `kubectl top pods --all-namespaces`.

[@SebastianScherer-oi4pn]

@@DevOpsToolkit what would the IaC solution for this crossplane cluster look like? should we create another cluster, crossplane-2, to manage this crossplane-1 cluster? 😉

[@DevOpsToolkit]

@SebastianScherer-oi4pn you need to create a management cluster with Crossplane one way or another. It can be, for example, a local cluster (e.g. minikube) where you install crossplane and instruct it to create a "real" cluster with Crossplane inside. From there on, that "real" cluster can manage itself as well as any other resources you might want to manage with Crossplane. Another alternative is to grab a crossplane cluster from upbound. Now, to be frank, if you need to manage only a few resources, crossplane is not the right solution. It shines on larger scale and, in those cases, the "trouble" of creating the first cluster is often negligible. P.S. that is a similar story with any other kubernetes solution. For example, if you would use Cluster API to manage clusters, you would still need that initial cluster.

[@prasadnaidu6580]

Hi Viktor, Thank you for this amazing video with a clear explanation. I have a use case for provisioning cloud resources like Cloudfront, and RDS for application along with an EKS cluster. Would like to know in such cases would cross-plane be a better choice compared to terraform where we can define all resources in the tf file?

[@DevOpsToolkit]

The way i see it, crossplane is one of the next gen tools for managing resources and it is certainly better than terrsform. On the other hand, i am biased since i choose to be actively involved with crossplane (after i made that video) so I strongly suggest trying it out and making your own conclusions.

[@thelmansanchez5733]

Great Video, thanks i am starting with crossplane, lets go!!!

[@christianibiri]

This video blow my mind…… thank you for doing this 😁😁😁

[@kirinofy]

this channel is godsend! thank you!

[@jlsan92]

Amazing tool. Going to try it once I have the chance. The first thing that comes to my mind is the "genesis" cluster, probably using crossplane to define itself is madness, but this cluster would need a special treatment, mandatory velero and protection hehe. Thanks for the video!

[@jlsan92]

Haha, you mentioned the same at the end

[@DevOpsToolkit]

That is a "chicken and egg" problem that exists in many tools. What comes first? For example, to manage infra with terraform, you need persistent storage to store its state. I would like Terraform to manage my storage, but I also need storage for Terraform to work. Another example would be Argo CD. Argo CD should, and can, manage all Kubernetes resources, including Argo CD. However, the first installation of Argo CD needs to be done without Argo CD. After that, it can manage itself, but not initially.

[@TAICHI1SCO]

Thanks for sharing Viktor.

[@DevOpsToolkit]

My pleasure!

[@davehoffman515]

Outside of using kubeapi, Saltstack has been doing this for years. Expandable by writing your own modules in python. Crossplane does sound cool though

[@VisibleToAnyone42]

definitely, I am dreaming of a time when I can easily do everything I need for my app in terms of infrastructure wily I deploy the app itself with helm or whatever I use. now I have too complicated CD pipeline where I must run terraform to ensure that the app will be functioning properly after deployment via helm, this terraform step requires extra attention and care

[@maslak0208]

Hi, do you had opportunity to check tf-controller from flux? It's the missing link for me that constantly reconciles terraform same way as yaml definitions and can be very easily integrate with vault. For my small POC it worked great.

[@DevOpsToolkit]

Adding it to my todo list... :)

[@anishnagaraj]

First of all, thank you for the amazing video on crossplane. Could you provide your inputs on using ArgoCD+Crossplane+Terraform to combat the issue of lack of providers. Also your idea on Canary Release for IaC will be verymuch welcomed.

[@DevOpsToolkit]

There is a Terraform provider that allows you to use Terraform projects inside Crossplane. It's not ideal, but it does allow relatively fast switch (but without some of the benefits). As for the lack of providers... Jet providers have close to 100% coverage for the "big three" (GCP, Azure, AWS).

[@CloudNativeJanitor]

Ansible is growing strong, and they have a new product Ansible automation platform, CFEngine is quite big in embedded device management e.g. set-top boxes, unfortunately, due to COVID the cfgmgmtcamp did not hold in-person meetings for the last two years, as well as FOSDEM which makes a bit outdated with the current status of cfgmgmt tooling and ecosystem

[@j0Nt4Mbi]

Good stuff Viktor thanks for sharing. I think this tool can simplify the deployment and managing multiple clusters in multiple cloud providers. For now I think Crossplane fits in my requirement let's see what happens. Thanks for your explanation

[@DevOpsToolkit]

In that case, you MUST watch the follow-up video about compositions (kzbin.info/www/bejne/d6XFhGSrZ89qptE). They make the real difference.

[@j0Nt4Mbi]

@@DevOpsToolkit thanks Viktor. I am looking something for Azure and AWS I see for aws has more modules kind of, for Azure at this point see few modules.

[@DevOpsToolkit]

@@j0Nt4Mbi The community will soon release Terrajet modules. Those are providers for AWS and Azure (GCP coming soon) generated from Terraform code. Those should have the same coverage as Terraform. They are not the end-game and the development on "crossplane-native" providers continues. So, Terrajet is mostly about reaching close to 100% coverage quickly. From the user perspective, you will not see any difference between crossplane-native and Terrajet usage.

[@j0Nt4Mbi]

@@DevOpsToolkit Yeah sounds amazing, the good approach would be getting cloud native without dependency of other tools but Terrajet sounds great deal to move fast. Thanks again I really appreciate your job posting this kind of tools.

[@RideLikeAChamp]

Mind boggling - truly ubernetes

[@midediamond6314]

This is a great video Victor, after watching this video. I have being doing more of Crossplane. But how do I create a DigitalOcean VPC and Bastion with Crossplane?

[@DevOpsToolkit]

Unfortunately, DigitalOcean provider did not get as much focus as some other providers (e.g., AWS, Azure, GCP, Kubernetes, etc.). As a result, some of the DO resources are not yet available in the provider. The good news is that the work on the DO provider is ongoing and I hope to see those you mentioned available soon. In the meantime, you can join the people working on the DO provider and help them out. If that's not an option, I recommend opening an issue requesting those resources to the added. The more people show interest, the more likely it is for those maintaining the provider to prioritize them.

[@midediamond6314]

@@DevOpsToolkit Thanks for the reply Victor. I really appreciate

[@CloudNativeJanitor]

Just discovered Terrajet while checking what providers are out there, will be super awesome to cover it in the future, I am thinking on-prem e.g. vSphere, VCF, bare metal, etc that might have a terraform provider already

[@DevOpsToolkit]

I did cover terrajet but in the upbound KZbin channel. I need to check whether someone already did providers you mentioned. If no one did, it should be relatively easy to add them through terrajet.

[@CloudNativeJanitor]

@@DevOpsToolkit will double-check the unbound channel

[@amjads8971]

What happens when crossplane cluster goes down ?

[@DevOpsToolkit]

Assuming that you have your manifests in Git, the answer is "nothing". The resources managed by Crossplane continue running and once you create a new cluster with Crossplane and the same resources from Git applied (e.g., Argo CD), Crossplane will continue as if nothing happens. There is a small potential issue with resources that cannot be named or without IDs (e.g, a few in AWS), but those can be solved with the `external-name` label. Ultimately, you might want to have a backup of etcd if you want a piece of mind. As a side note, a similar issue exists with Terraform. What happens if the state store (e.g., S3 bucket) goes down? As a matter of fact, Terraform state is much more fragile than Crossplane state (stored in etcd).

[@chandup]

Good explanation. Kudos! Could you please do a video on distributed tracing using Jaeger and Grafana Tempo , please.

[@DevOpsToolkit]

Added it to my TODO list :)

[@DevOpsToolkit]

Jaeger is done! kzbin.info/www/bejne/fHyTpptjbNN3ick

[@RakeshKumar-eb9re]

Every year new tool that what devops is :)

[@DevOpsToolkit]

DevOps is such a vague term that all and none of the tools are part of it.

[@ravikumarhr4524]

Thanks for the video Victor.

[@DevOpsToolkit]

You're welcome.

[@caesarion6]

Thanks for this one Viktor! Interesting as always. I think Crossplane is super interesting but I have my doubts if it will ever read the popularity of Terraform. First it will be hard to replicate a terraform plan equivalent, second they need to nail how to reference state/resources from previous manifests (like get subnets or vpcid); I guess the approach is to use secrets and configmaps? And lastly it will be hard to compose modules like you can with terraform using the k8 api. Ofc I could be wrong on all accounts, time will tell.

[@meyogi]

Maybe you can have a look to 'Composition' crossplane.io/docs/v1.0/getting-started/compose-infrastructure.html That seems to be the way Crossplane handles a reusable 'module-lile' pattern

[@DevOpsToolkit]

Replicating a plan should not be hard. It certainly knows what is the difference between the actual and the current state, and what is the diff, so it should be able to replicate `terraform plan`. Also, it can already reference existing resources and manage things not initially created by Crossplane. Lastly, composing modules and extending Crossplane is already working. At the end of the day, both Terraform and Crossplane manifests are reflection of APIs they use to interact with Azure, Google, AWS, and others. My main concern is the first note you made, but I would rephrase it a bit. "Will it ever reach **sufficient** popularity?" It does not have to be right away as big as Terraform, but it does need to have decently big community that will guarantee that it continues growing. When Terraform started, it faced the same problem when compared with Chef, Puppet, and Ansible. It outgrew them over time. Now, the fact that Terraform did it, does not mean that everyone will do it. For now, I think that Crossplane is a good attempt to "shake" IaC market. Probably the best one since the inception of Terraform. Time will tell whether they will succeed. Ultimately, it will mostly depend on whether it will attract early adopters. I am one of those, but I am not using it exclusively.

[@DevOpsToolkit]

Oh yeah. As a matter of fact, I had it in the demo, but ultimately removed it so that I maintain

[@alex24x7]

Thanks for review Viktor! My thought was about Terraformer import, which can be an easier alternative of automation drift detection

[@DevOpsToolkit]

It's not that much about the ability to import but rather about having a process that continuously compares the states and makes sure that there is no drift. Terraform does not do that, as far as I know.

[@alex24x7]

​@@DevOpsToolkit sure it does not work out of the box, any solution need some steps to set it up. In case of terraformer it might be a cron job to import and compare with previous terraform state. as well as additional steps to sent a notification and roll back unexpected changes ... but when you already have a lot of TF code, it is an easier solution then migration to crossplane

[@DevOpsToolkit]

@@alex24x7 Oh yeah. If you already have TF manifests, it is likely easier to do those additional steps than moving everything to Crossplane. Terraform is in many ways better (and some ways worse) than Crossplane, so it's not clear that Crossplane should be the solution when starting from scratch. When there is already an investment in TF, switching to Crossplane is likely not a good idea.

[@richardmarchesi]

loved video crossplane drift management is amazing looks promising long term as possible alternative to terraform specifically combined with gitops, any chance video on k8s operators/ crds?

[@DevOpsToolkit]

Adding it to my TODO list... :)

[@jirityr]

Nice video. Is there a way how to review what exactly is going to be changed/added/destroyed before it actually happens? It would be very unfortunate to see my precious data being deleted when a non-careful change to a DB cluster triggers its rebuild. Could you perhaps also show how Crossplane can use Terraform runtime or show other GitOps tools that can use Terraform for building the infrastructure?

[@DevOpsToolkit]

As far as I know, crossplane does not have an equivalent of the terraform plan command. You would need to review changes in a PR and run tests before merging it to master and applying to production. I believe we should do this with terraform as well since executing a plan does not give a full picture. Still, having a plan (dry run) in crossplane would be nice and should be considered a missing feature.

[@DevOpsToolkit]

As far as I know, there is no translator from crossplane manifests to terraform hcl so one cannot be used with the other. As for GitOps tools that would use terraform... If you are not interested in having a daemon that continuously watches for changes between the desired (hcl) and the actual (infra) state, all you have to do is create a pipeline in your favorite tool (e.g. Jenkins, Codefresh, Circle I, etc.) that will apply terraform every time you make changes to a git repo where you keep hcl. That would be GitOps without drift detection and continuos sync.

[@junaid18183]

Hey Victor nice video as always, Just small doubt after I create a real cluster from my k3s based small cluster, is there any way I can import ( just like terraform import ) the resources?

[@DevOpsToolkit]

That is the subject of the upcoming video that will be released this Wednesday. It will not go live here but on the Upbound channel (kzbin.info/door/m_v2HL0pdqtShHD-ZDDTaA).

[@milenkomitrovic8611]

Just one small question, how we deploy initial k8s and crossplane config?

[@DevOpsToolkit]

You can, for example, do it from a local k8s cluster and, later on when crossplane creates a "real" cluster, move it there and let it manage itself.

[@DevOpsToolkit]

Take a look at kzbin.info/www/bejne/f53EinqdrsxjbNE

[@bhaveshghelani6403]

CrossPlane needs K8S cluster to run, but ideally K8S cluster should be provisioned through crossplane. How do we address this?

[@DevOpsToolkit]

You can either use managed crossplane (by upbound) or use a local cluster to provision a control plane cluster and then move the same manifests to that cluster so that it can be managed (together with other clusters). In other words, you need an initial cluster (it can be temporary local cluster) to create a permanent cluster which can be managed by the same resources used to create it.

[@technoe02]

This is god damn amazing

[@bogdanM29]

I tried to deploy resources cross account in aws with crossplane and assume role via ProviderConfig but it doesnt seem to work, the aws role for example gets update in cli, but is not being created. I am not sure which is the correct ProviderConfig for cross account deployments. I'd like to use the service account approach instead of access keys

[@DevOpsToolkit]

Is github.com/crossplane-contrib/provider-aws/blob/master/AUTHENTICATION.md#using-assumerole what you're looking for?

[@rotemshoshani424]

What about advanced scenarios? Like multiple IAM policies attached to the EKS role, VPC definition, SGs and so on? Referencing terraform resources, you can really have a fine-grained control on how the env looks like

[@DevOpsToolkit]

You can do all those things with Crossplane as well. On top of that, you can create compositions (not explained in that video) that combine all the resources you need and expose them as a new CRD.

[@rotemshoshani424]

Well... I guess I have to try it out 😜 So hard to keep up with all these solutions, just when I got used to Terraform.. I guess that's both a good and a bad thing

[@DevOpsToolkit]

It is impossible to keep up with everything. We should be informed what's out there and then choose what to change next and what to leave as-is.

[@aadyan2rashid751]

Thank you!

[@anilrana8823]

Great Video.Viktor. I have deployed gke using crossplane. Now I need to also deploy kubernetes on On-Prem baremetal machines. How can I do that? Thanks

[@DevOpsToolkit]

How are those machines managed? Is it Equinix, VMWare, etc.?

[@anilrana8823]

@@DevOpsToolkit They are just ubuntu based systems. What is the simplest way to enable them for crossplane? If I can just install a software and then crossplane can do the trick then it would be great. Thanks.

[@DevOpsToolkit]

Crossplane needs an API to talk to. If it would be, let's say, VMWare, crossplane could use it to create machines (VMs). If it's just Ubuntu, you're probably better off with something like Ansible.

[@anilrana8823]

@@DevOpsToolkit What are the choices I have? VMware, and what else? Thanks

[@DevOpsToolkit]

Tools like crossplane, cluster API, terraform, Pulumi, etc. are mainly focused on managing services by interacting with their APIs. Ansible sounds like a better choice if it's baremetal without some API on top. If, for example, it would be equinix metal, crossplane or others would be a better choice.

[@vusalalekberov5574]

Great content Viktor, thanks! I would also like to get your opinion on the Tekton.

[@DevOpsToolkit]

I already have that one on my TODO. It's coming, I just do not yet know when.

[@DevOpsToolkit]

Done: kzbin.info/www/bejne/bZ7Zo6Our8R1nKc :)

[@PelenTan]

Me likey. A lot. One of the things I _really_ don't like about Terraform is its use of its own language for the file. It _could_ be argued that HCL is a "better" language for setting things up. I disagree for many reasons, but that could be argued. However, _everyone_ knows yaml. I think we'll see something along the same lines as Betamax(Terraform) and VHS(Crossplane). Additionally, I definitely don't see having to run up a K8's cluster to manage a bunch of actual "work" clusters as a "bad" thing at all. Opposite entirely. I think you are correct that K8's is going to become the de-facto api for all things cloud. Especially since K8's can be mostly self-healing while storage really isn't. So question for you. Where do you feel Rancher would fit into all of this? Or does Argo pretty much make that an obsolete option already?

[@DevOpsToolkit]

Rancher is a k8s distribution, similar to Ubuntu being a Linux distribution. People should be able to manage their Rancher clusters using any IaC tool and it should be in Rancher's interest to contribute modules to all the popular projects. Now, Rancher, apart from being a k8s distribution, does management of k8s clusters, provides dashboards, and so on and so forth. I'm not very fond of that part. I think that it should stick to being a distribution, and facilitate usage of other tools to do the rest (e.g., Terraform, Crossplane, Argo CD, Flux, etc.). There is a disclaimer though. I haven't used Rancher for 6 months (if not more) so it might look different today than how I remember it. I'll add it to my TODO list to go through it again and probably make a video.

[@xuejunli6817]

Thank you for making these great videos. Could you talk about Cluster API?

[@DevOpsToolkit]

Adding it to my TODO list... :)

[@DevOpsToolkit]

It took me a while, but now it's finished and available at kzbin.info/www/bejne/bqq4dYiej5uUodE

[@xuejunli6817]

@@DevOpsToolkit Great Viktor! And thank you for following up!

[@vn7057]

Hey Viktor or Folks Do you know what if an upgrade API form remote side and my side still using the old api Will the gitops sync crash the infra? Lets say I have something K8s API/betav1 create many years ago And then k8s no longer support it But I forgot to update the api And now only api/v1 is allow However if the crossplane yaml still mention in betav1 I am worry it will keep destroyed and create …dead loop

[@DevOpsToolkit]

I'm not sure what happens in that case. I always check whether specific versions are removed before upgrading a cluster. Or, even better, upgrade manifests with new API specs while the old ones are deprecated (before they get removed). In other words, I don't know :(

[@vn7057]

@@DevOpsToolkit haha it’s fine I will test it out if I have chance, I am sick with Terrform HCL already.

[@debkr]

Agreed. The documentation needs to be improved.

[@DevOpsToolkit]

It's much better than when I created that video, but there's still room for improvement.

[@luciendasilva3862]

"Chef and Puppet had it but they dead" hahahaha!

[@DavidBerglund]

It's still very very limited for use with Azure though? Can I create an Azure function, a Linux VM an App Service etc?

[@DevOpsToolkit]

You're right. The number of Azure CRDs is currently limited and smaller than for other "big" providers (AWS and GCP). That is the major issue with Crossplane. It is a relatively new project so the number of libraries is much smaller than for Terraform. Ultimately, the speed with which CRDs are added depends on the focus the community has and contributions from Cloud vendors. If you need more than what is currently available, and you cannot contribute the "missing" CRDs, the best option is to stick with Terraform (for now).

[@DavidBerglund]

@@DevOpsToolkit I thought so, wish I had the time to learn Go and to contribute.. I'll just hope that Azure support is a lot better in a few months

[@DavidBerglund]

They have machinery to generate providers from Terraform modules, that sounds promising. Smart to leverage the huge Terraform ecosystem

[@DavidBerglund]

Perhaps this is something you could find time to investigate and share with us? I just discovered your channel today and subscribed pretty much immediately. Great content!

[@DevOpsToolkit]

Adding it to my TODO list... :)

[@Marxyon]

Here is what I don't quite understand. If I need a Kubernetes cluster to run Crossplane, and I can use Crossplane to create clusters, then who and what should manage my first cluster that I run Crossplane on, it can't be Crossplane, after all

[@DevOpsToolkit]

That is an unfortunate problem with using control planes. You need it up and running before you van start using it. So, you need crossplane installed and for that you need a k8s cluster, be it local or remote. Once you have it, if it's a remote (real) cluster, you can manage it with Crossplane from there on. One potential solution, besides obvious like AWS/GCP/ Azure/etc console, can be Upbound cloud. You can use it to bootstrap a temporary control plane with Crossplane and it it to create the first cluster that will be, from there on, managed by crossplane running inside it. Many of the crossplane users tend to use it for large scale. If all you need is a single cluster, prerequisite work might be too much.

[@TankaNafaka]

What would happen if your origin cluster (minikube) is gone? Also is there any way to stash crossplane state and reuse on next minikube?

[@TankaNafaka]

Also Terraform can import existing resources and include in tf state. But yes, crossplane is true gitops tool as event is auto triggering state from origin k8s (minikube) as you have shown in your demo.

[@DevOpsToolkit]

I a "real world" situation, I would not use minikube, but a "real" cluster (which could be later on managed by Crossplane as well). Normally, you should be able to recreate Crossplane if a cluster goes down based on the manifests you used. Nevertheless, that alone might not be the best thing to do since such a solution would assume that you are using `external-name` labels to identify infra resources you're managing (otherwise, Crossplane assigns names to those resources). So, I normally create backups to be on the safe side. Specifically, the state of Crossplane resources, like any other Kubernetes resources, is stored in etcd so that's the one that needs backups (with or without Crossplane).

[@TankaNafaka]

@@DevOpsToolkit Im aware of minicube was just a placeholder for demo purposes. Though not sure if destination k8s would continue to trigger events back to origin k8s (minikube in this case) once you have origin + backup in place? Perhaps demo with k8s backup would be good to have here? Also with tests to verify state is sane. Personally, i dont mind idea of having a multiple k8s in the setup, but I favor simplicity above all. As for etcd, sure you can use HA rds setup but this increases cost and requires additional infra, where with terraform, your next apply run can be reused from anywhere, even from your local dir as long you have latest tfstate stashed (tf state pull > terraform.fstate)

[@DevOpsToolkit]

@@TankaNafaka I'll create material that explains how to do backup/restore, move from one cluster to another, etc. I might not publish it here though, to keep this channel independent and objective (I joined Upbound recently). I will likely publish it in kzbin.info/door/m_v2HL0pdqtShHD-ZDDTaA

[@TankaNafaka]

@@DevOpsToolkit To rephrase my prev post: destination k8s does not trigger events, but origin k8s scans for a state change and then triggers a job to re-apply state sync from crossplane cdr manifests. So any state backup apply should doit. But without a state backup, is there a risc ending up with orphaned k8s? Anyways, Im looking fwd to see any follow up on this once you have the time. Thx for your efforts and great work Viktor. 😃

[@barefeg]

Can we get rid of eksctl now?

[@DevOpsToolkit]

It's hard to say... Crossplane has a potential to be the next-gen infra, but it is also very young. I'm slowly increasing my usage of Crossplane and decreasing others, so I'm still not there 100% (and probably will not be any time soon).

[@nootajay]

can I call it the puppet killer?

[@DevOpsToolkit]

I would call terraform the puppet killer and that would make crossplane the terraform killer :)

[@VisibleToAnyone42]

the idea of "crossplane" is cool, I'd say. But the number of resources you can manage is super limited. also you cannot always 'get' property of resource and reuse in other respurce. for instance, you have a subscription and a SA, you want this SA to be assigned to subscription, but you cannot do this. you must hard code value. and many more, but most important one is that you simply do not have everything you need in terms of resources, at this point even Terajet did not do a mirracly, still looks like Crossplane needs more time. last time I wanted to use it was half a year ago and I decided to postpone until it becomes mature enough, looks like I have to give it more time again.

[@DevOpsToolkit]

I'm not sure I understood what you meant to say.

[@VisibleToAnyone42]

@@DevOpsToolkit sure, you could not understand, I updated my comment, I hope it is better now. thank you for your work!

[@DevOpsToolkit]

You're right. Crossplane is much newer project than other IaC tools (even though I would not call it IaC since it's not only about infra). As a result, the number of providers and resources that can be managed through those providers is smaller than what you get from others. That improved with Terrajet but it's still no enough. The good thing about Terrajet is that is simplifies contributions. It's much easier than before to contribute a provider and, as a result, I find a new provider at least once a week. The problem, however, is that those providers are scattered across people's repos so we need to figure out a way to put them all into one place (and validate them).

[@kreebo_]

If crossplane needs kubernetes to run, that means you can only use crossplane after you provision your cluster. weird

[@kreebo_]

Any suggestions?

[@DevOpsToolkit]

That's true. That initial cluster can be anything, including local k8s like Docker Desktop, Rancher Desktop, minikube, etc.

[@danielhd6719]

Issue is you want this first cluster to be accessible for everyone in the devops team in case you are sick. So it has to be a real cloud based cluster that has to be created manually or using terraform to then be imported to crossplane which makes it vulnerable to mistakes which in return can render your whole infrastructure state void/removed. TL;DR - cool idea but it wont fly high.

[@DevOpsToolkit]

You can create that first "real" cluster using crossplane running in a local cluster and, after that, install crossplane in that new cluster and apply the same manifest so that it manages itself. A similar problem exists with terraform as well. For a serious usage, you need storage for tf state. How do you create that storage? Manually?

[@jonassteinberg3779]

terraform your minikube cluster lol

[@Teacification]

Давай по-русски братан :)

[@DevOpsToolkit]

Я бы так и поступил, если бы мог говорить по-русски без Google Translate.

[@SV-tc8cu]

Учи английский, братан