Terragrunt Tutorial: Create VPC, EKS from Scratch!

[@AntonPutra]

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - [e-mail]

[@gmeister3022]

People who aren't DevOps engineers might not fully understand the tremendous value you share in your videos. In this video, you intuitively and comprehensively teach Terragrunt, an amazing tool used by some of the best DevOps teams in the industry, and not only that, you use it as an instrument to easily teach advanced AWS topics. If that wasn't enough, you cover the entire scope of infrastructure environments; development, staging, and production. What more can I say other than it's absolutely perfect?

[@AntonPutra]

Wow thanks! I appreciate it!

[@eschult2001]

I've been on the software development (CI) side for 30+ years and only recently had the opportunity to contribute to the CD side of a new startup. Thanks for sharing these great tutorials! Helped me a lot while getting started on my DevOps journey.

[@AntonPutra]

Thank you so much!!

[@cantbefooled8044]

i dont think you understand how much your explanation and combining so many different elements of devops into one video has helped me so much. Like, exactly what i was looking for. thank you so much!

[@AntonPutra]

Thank you! Glad to hear that!

[@roberto_camp]

Anton you are such a "go-to" guy for this stuff. Keep up the amazing work you are doing for the community. We appreciate you so much. 👊

[@AntonPutra]

Thanks Robert!

[@SuperAleksandar96]

@@AntonPutra 10/10 videos Thanks

[@Wzooff]

Terragrunt rocks! We use it for managing complex multi-client infra. Bunch of includes, dependencies, etc :) terraform without terragrunt usable only for very simple projects.

[@AntonPutra]

agree

[@agun21st]

Awesome video for using Terraform Mudule based with Terragrunt and store state in aws S3. I followed all the way and now I am maintaining this infrastructure for my production. Waiting for next video related with this video sir.

[@AntonPutra]

Thanks you!

[@jlamoree]

This is a fantastic tutorial. Extremely helpful to me.

[@AntonPutra]

Thank you so much for the support!

[@MohamedTst-ri3wf]

Hi Anton Putra, Thank you for the excellent tutorial on setting up an EKS cluster! It was incredibly helpful and easy to follow. I wanted to ask for your guidance on adding Ubuntu 20.04 nodes to my EKS cluster. Could you provide some instructions or point me towards a resource that explains how to do this? Thanks in advance for your help and for the great content you create!

[@AntonPutra]

Sure, you need to build your image using this official packer script - github.com/awslabs/amazon-eks-ami They are called self-managed nodes here is a reference - docs.aws.amazon.com/eks/latest/userguide/worker.html

[@aksel58]

hi Antonio, could you please elaborate why you preferred to use folders named by the env name (dev/staging) rather than the terraform workspaces?

[@AntonPutra]

There's a significant limitation in how to isolate different environments when using Terraform workspaces. I have a dedicated section somewhere in this video where I explain exactly why you would choose folders rather than workspaces - kzbin.info/www/bejne/fZi0dmtrdtZlsJY

[@renatosouza1k]

This channel is very util. Hello from Brazil.

[@AntonPutra]

❤️

[@conrad707]

As always, great content by Anton on TG. Super helpful. I'll be applying many of these concepts. Thank you!

[@AntonPutra]

thank you!!

[@333HaKan333]

Why you wrote your own module instead of using official VPC module ? is there a reason or just for teaching purposes

[@AntonPutra]

In the long run it’s much easier to maintain your own resources. Less abstractions less dependencies. I’m talking about 1+ year. Less technical debt you acquire is better for you and your team.

[@MrRin_]

Great Tutorial, make sure to use the same instance size as in the video and NOT t.2 micro. My autoscaler wouldn't launch and I could not figure out why, now I know that it's the instance size. My guess would be that in order for autoscaler to work - instance size should not be less than t3a.xlarge. Anyway, thanks Anton, I learned a lot

[@AntonPutra]

Cool, thanks for the tip

[@prashlovessamosa]

Thank you 🙏 for sharing this i was searching about this yesterday and you uploaded this Anton you are great.

[@AntonPutra]

My pleasure!

[@umka7536]

Thanks a lot for a tutorial that covers Terragrunt from the beginning. Спасибо!

[@AntonPutra]

pojaluysta =)

[@abdallakenawy2855]

Anton, you are such a great DevOps Engineer ...keep it up, my friend!

[@AntonPutra]

Thank you!

[@ahmedsaif4541]

"Great demo! , first I noticed you configured the Terraform backend (S3 bucket) in each terragrunt.hcl file within the Dev environment subfolders (eks, kubernetes-addons, vpc). Wouldn't it be more efficient to define the backend configuration once at the root level and use Terragrunt's dependencies block to inherit it across modules? Just curious about the approach. second thing: you exclude the eks addons when you applied it , because it depends on eks creation first , do i need to apply it seperately , last thing, why you didn't push the infrastructure-live-v4 " where the Terragrunt files are located" and prefered to keep it local ?....... Thanks!"

[@GabrielPozo]

Thank you, Anton! Amazing tutorial.

[@AntonPutra]

thank you!

[@pier_x0]

thumb up and subscription strictly required!!!

[@AntonPutra]

Thanks!

[@anestd]

Amazing explanation, especially the last part. I could relate it with the code we have in our live environment. Got my doubts cleared. Thanks.

[@AntonPutra]

thank you!

[@devops22]

Amazing Video🌟 got a question, how could Github infra accessed private Github module without a token ?

[@AntonPutra]

go to settings and add "deploy key"

[@es3t]

11:42 that dev/vpc/dev/vpc/terraform.tfstate file doesn't make completely sense

[@festus-obi]

i'm curious, is there a reason why the tf files were prefixed with numbers ? like the below 0-provider, 1-vpc and so on

[@AntonPutra]

It's just for the tutorial to sort them in the IDE and show the logical order of how you would create infrastructure components.

[@VuqarIE]

is there possibility to replace cluster autoscaler with karpenter?

[@AntonPutra]

well of course, it does the same function, scale your k8s cluster. it just instead of adjusting desired size of auto scaling group it creates standalone ec2 instances. i have few videos covering karpenter

[@VuqarIE]

@@AntonPutra this channel should be on top. The content you are providing is marvelous. thanks a lot

[@MisterUsman]

Hey Anton, amazing video. The build-up to the final .hcl file put it all together. Can you explain what the differences were between the live-v1/v2/v3/v4 folders and when doing this for a completely different environments, do we need to go through all those steps or can we go straight to v4

[@AntonPutra]

Thanks! v1 is just raw terraform code v2 uses terraform modules v2 & v4 both terragrunt and different modules You can go straight to v4 if you use open source modules or build your own and go for v4

[@sergey_tech]

Fantastic video! I truly appreciated the concise and fast-paced explanation. I've liked and subscribed. Thank you!

[@AntonPutra]

Thank Sergey!

[@fauzuwannazri5267]

Hi Anton great tutorial as usual, thanks a lot, but i wanted to ask, how do we handle eks upgrade with terraform , is it enough just by changing the cluster_version and eks will handle the rest, or do we need to do something to scale the cluster first, and do the drain and cordon etc. fact: Putra in my language means prince.

[@AntonPutra]

Thank you! Based on my multi-year experience managing Kubernetes clusters in AWS and GCP, sometimes it can be as easy as updating the version and applying Terraform. In that case, Terraform will upgrade the control plane and then node pools one by one (drain, terminate, etc). In some cases, for example, when upgrading to 1.21 or 1.22, there were breaking changes in Kubernetes which forced us to create a brand-new clusters and migrate apps there due to the new and deprecated APIs.

[@ethaniel86]

great videos! is there tutorial how to add more kubernetes-addons like aws load balancer controllers, efs csi drivers etc. using this deployment method?

[@AntonPutra]

Thanks, sure you just need to add all of your addons in this module - github.com/antonputra/tutorials/tree/main/lessons/160/infrastructure-modules/kubernetes-addons and create a specific variable such as enable_alb_controller.

[@kayoutube690]

I think git as a source is difficult to use specially if the organizations are using MFA auth for their git. But over all your video is awesome!

[@AntonPutra]

I disagree. GitOps is a standard; even in Google SRE books, they specifically mention following the same git approach.

[@kayoutube690]

You're right! But I'm referring only to jenkins. I never use GitOps but i would love to explore this tool.

[@AntonPutra]

@@kayoutube690 Due to the recent fork of Terraform to OpenTF, I suggest waiting to see how all those "providers" will be adopted by OpenTF. Terragrunt will no longer be able to use Terraform and will switch to OpenTF because of licensing issues.

[@dannotes9396]

Thanks for the video. very insightful.❤ quick query: How can we share values between staging and production using this architecture?

[@AntonPutra]

you can use read_terragrunt_config function example: locals { common_vars = read_terragrunt_config(find_in_parent_folders("common.hcl")) } reference - terragrunt.gruntwork.io/docs/reference/built-in-functions/#read_terragrunt_config

[@joshualegg3750]

Really useful having all this information for a beginner. I was tearing my hair out going from blog post to blog post.

[@AntonPutra]

thanks Joshua

[@kundupradip]

I created my channel just to comment on this video. This is video is very good. You are helping a lot of people like me. You are going a great job. Please keep it up. Thanks.

[@AntonPutra]

thank you! ❤️

[@mdmoniruzzaman703]

Hi, in 58:40 when i run terragrunt run-all plan command, it's not working. showing can't read resource with the name of eks

[@AntonPutra]

Check if you used the same name for the cluster

[@tiagobarreto7853]

Thank you so much Anton! Amazing knowledge!

[@AntonPutra]

Thank you, Tiago!

[@YordisPrieto]

55:29 It would be amazing to update the project to do all the terraform IAM setup (showcased at the end using the AWS console) using terragrunt as well.

[@AntonPutra]

Thanks for the feedback

[@muhammadzeagham4092]

I keep getting cyclic dependency error when I run "terragrunt run-all plan/apply " on dev folder. I downloaded "Anton Putra's" code and and tried running on it and it still gives same error. Can anyone help me. Also I am using local backend and modules.

[@abhishekmurthy5318]

How do you reference data blocks that we use to fetch from AWS like authentication credentials in terragrunt?

[@AntonPutra]

I didn't get the question. In Terragrunt, we usually create all infrastructure using Terraform itself and use dependency references between different parts of Terragrunt. Data blocks are usually the exception, not the common practice.

[@fedorvinogradov2587]

Great tutorial, love it! What do you think of Terraspace?

[@AntonPutra]

Thanks! I haven't used Terraspace yet.

[@saibd]

Do you have any git link to have the code please

[@kundupradip]

You are the best. The tutorial is really good. Thanks for it. Please keep it up.

[@AntonPutra]

thank you!

[@HamletHakobyan-p7k]

Thanks Anton, great content. Just a question: isn't better to use for_eash instead of count in subents?

[@AntonPutra]

Thanks! Yes, 100% use for_each to iterate over objects in Terraform. count is mostly used as a flag to indicate whether to create an object or not. Do you have an example for me using count?

[@renatosouza1k]

Valeu!

[@AntonPutra]

thank you!!

[@mwanthidaniel1254]

Hello, why is my pod not triggering for scale up of the worker nodes, `pod didn't trigger scale-up`.

[@AntonPutra]

Best way to find out is to check autoscaler controller logs. Kubectl logs …

[@AntonPutra]

Best way to find out is to check autoscaler controller logs. Kubectl logs …

[@ChrisChristensen143]

Fantastic video. Thank you so much!

[@AntonPutra]

Thank you Chris!

[@andrestorres7343]

Top quality, thanks for sharing 🎉

[@AntonPutra]

Thank you Andres!

[@raypi2297]

Why you have no ".terraform" dir when init ?

[@AntonPutra]

i believe it's just hidden

[@AndroMuteli]

спасибо, кучу времени сэкономил)

[@AntonPutra]

pojaluysta :)

[@idansoffer5675]

@AntonPutra in production environment, when you need to upgrade the eks version… will you do that via terraform ..? or thats where terraform ends its job. because i guess upgrading a live eks cluster with deployments is risky . Thanks

[@AntonPutra]

Not really, EKS would first drain nodes and respect the constraints that you place. You need to make sure you have pod disruption budgets for all your apps to avoid any downtime. kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/#configure-poddisruptionbudget

[@tctfone1]

Nice video and perfect speed. Spasibo

[@AntonPutra]

Spasibo vam =)

[@Nitinbakaya]

Sorry to say, but you are going very fast and it becomes very hard to understand and have to watch the video repeatedly which is time wasting.But overall your videos are very helpful.

[@AntonPutra]

Thanks for the feedback, trying to make them short and concise

[@pazfelipe]

Could you provide this same example but without using terragrunt? Just terraform?

[@AntonPutra]

Sure, here for exampe - github.com/antonputra/tutorials/tree/main/lessons/167/terraform Also, I have terraform module version - github.com/antonputra/tutorials/tree/main/lessons/156

[@sujithsurendran7686]

Excellent

[@AntonPutra]

Thanks

[@sujithsurendran7686]

@@AntonPutra can you make some videos on Kubernetes CRD and some automation on that.

[@AntonPutra]

@@sujithsurendran7686 Operator is coming soon, but what do you mean by automation?

[@haroldsalmazan7261]

Very Helpful for beginners like me. Appreciate the effort of posting these. I just have question, when I'm adding the Labels and Taints inside the node_groups, terragrunt doesn't seem to detect it. What should I been missing? I just wanted to add multiple taints and labels for each nodegroup. I haven't added sec group as well to the node group, that will might be same instance.

[@AntonPutra]

Thanks! Well you would need to add taints to the module itself here - github.com/antonputra/tutorials/blob/main/lessons/160/infrastructure-modules/eks/3-nodes.tf#L26 Example - github.com/antonputra/tutorials/blob/2d1421a4c6b91e8ee22017f8252aee170586b711/lessons/152/terraform/7-nodes.tf#L98-L102 But instead of hardcoding use github.com/antonputra/tutorials/blob/main/lessons/160/infrastructure-modules/eks/3-nodes.tf#L2 variable

[@haroldosalmazan]

​@@AntonPutra Thank you, I was able to add in on the module. However, I'm still trying to figure out how to make an object(taint/labels) inside the node groups to not require an input/s. I have set labels and taints inside the module by using "each.value" but when I skip it on a node group, it fails saying "Unsuitable value for var.node_groups set using the TF_VAR_node_groups │environment variable: all map elements must have the same type." Still looking on what should I have to add to make this possible. Here what I add on module tho, "labels = each.value.labels" taint { key = each.value.taint.key value = each.value.taint.value effect = each.value.taint.effect }

[@100faouri]

hi thanks a lot for the video!! can we migrate existing terraform code running on AWS to Terragrunt is that easy to do?

[@AntonPutra]

Yes, you can, but it's not easy since you need to import state for all components you already have.

[@lukas-pastva]

AMAZING!!!

[@AntonPutra]

Thanks, Lukas!

[@StephanDuToit]

Thanks!

[@AntonPutra]

Thank you, Stephan! I appreciate your support!

[@sssanjaya]

Hi, do you recommend Terragunt for large scale projects? sometime it becomes hard to maintain.

[@AntonPutra]

I do, especially for big companies with lots of environments, but keep it simple and beginner friendly.

[@rohanshahi1608]

we can use terragrunt to wrap up for a bigger infrastructure on a same cloud provider It can be a library for all resources

[@AntonPutra]

agree it's very helpful when you have lots of environments and components

[@tautvydasbujauskas8104]

Great video, I am managing a few clients but but with only 2 env, so don't see that much gain of Terragrunt, when initial setup and refactoring would be very complex. But information is very valueable. The ideo of using terraform modules like these addons is super, gonna start this from Monday :) Another thing that is still stopping me from using Terrafrunt is using Terraform cloud as backend, since i couldn't find good explaind example. Maybe that could be idea for next video? :)

[@AntonPutra]

Thanks, well you can always create your own or try to use www.runatlantis.io

[@raypi2297]

how to know when should use Terragrunt instead Terraform?

[@AntonPutra]

when you have more then 2 envs

[@vunguyen2246]

thank you bro!

[@rohmersicat3429]

Hello master can we do manage own security group instead of using the default one ? and if that so how can i add the security group to use with K8s ?

[@AntonPutra]

It creates one for you. You can get id of the group using "cluster_security_group_id" output variable and add additional rules using the following resource "aws_security_group_rule".

[@rohmersicat3429]

@@AntonPutra yes what i mean if i want to allow port 8080 how can i add it on vpc ?

[@AntonPutra]

@@rohmersicat3429 I need to know your use case. For example if you your application (pod) needs access to RDS, you add EKS security groups as source in your configuration. Optionally you can setup Security groups for pods

[@xavierordonez2484]

Can you share the code

[@AntonPutra]

Sure, you'll find link the video description

[@xavierordonez2484]

@@AntonPutra please provide email to keep in touch. Some projects I have

[@coda2k]

Thank you very much for this AMAZING video! If you're creating your cluster in a region other than 'us-east-1', it's necessary to modify the kubernetes-addons/1-cluster-autoscaler.yaml file to add the 'awsRegion' value to the Helm chart, as it defaults to searching for the node group in "us-east-1". Otherwise, you'll get the error "Node ip-xxx.xxxxx.compute.internal should not be processed by the cluster autoscaler (no node group config)"

[@ianmubangizi]

Amazing!!!

[@AntonPutra]

Thanks Ian! =)

[@timorling5197]

If you change the region to something other than `us-east-1`, the autoscaler needs `awsRegion` to be set in `infastructure-modules/kubernetes-addons/1-cluster-autoscaler.tf`

[@AntonPutra]

You can set it using this variable - github.com/antonputra/tutorials/blob/main/lessons/160/autoscaller.yaml#L66

[@timorling5197]

@@AntonPutra How is that yaml file being consumed? Perhaps I needed to re-init terragrunt?

[@timorling5197]

I figured it out, put the autoscaler.yaml into the `dev/kubernetes-addons/` directory and add the values like normal with `helm_release` `values = [ file("${path.module}/autoscaler.yaml") ]`