8 Terraform Best Practices that will improve your TF workflow immediately

[@TechWorldwithNana]

Follow me on Instagram for behind-the-scene content 😊 bit.ly/2F3LXYJ If you want to support this channel, please leave a like :) ▬▬▬▬▬▬ Learn more about Terraform? 🚀 ▬▬▬▬▬▬ Terraform explained in 15mins ► kzbin.info/www/bejne/ombOYpSflKx1eqc Complete Terraform Course for Beginners ► bit.ly/3OCoCPu Terraform in complete DevOps process ► bit.ly/3WtBARg ▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬ 0:00 - Intro 0:25 - Terraform State & State File - Best Practices around State 1:18 - BP 1: Manipulate state only through TF commands 1:46 - BP 2: Remote State 2:44 - BP 3: State Locking 3:43 - BP 4: Back up State File 4:23 - BP 5: Use 1 State per Environment 5:36 - BP 6: Host TF code in Git repository 6:56 - BP 7: CI for TF Code 7:39 - BP 8: Execute TF only in an automated build 8:28 - Wrap Up & More TF Resources ▬▬▬▬▬▬ Useful Links 🔗 ▬▬▬▬▬▬ ► Remote State: www.terraform.io/docs/language/state/remote.html ► State Locking: www.terraform.io/docs/language/state/locking.html

[@RiteshAgarwalGwalior]

Hey Nana, I searched for your terraform course on Udemy but find it but costly considering in Indian currency. Is there anyway I could get some discount on it?

[@yashagarwal1515]

Hi Nana, A very nice video on Terraform. Can you also create one a basic Terraform Tutorial video and share your views on it. Thanks

[@andrewmiko3445]

- Use remote state with versioning and locking; - Use workspace for multiple environments; - Use for_each instead of count if it's possible; - Never save TF state files in git, they can contain sensitive information in plain text format; - Use modules for code reuse (DIY); Thanks for a video :)

[@lmind2]

For bullet 4 - why? If we're speaking in best practice terms, you shouldn't have sensitive information in plain text anyway. This should mitigate any issues with maintaining state files in git, no?

[@juangomez4133]

@@lmind2 It could store auto-generated secrets like service account keys. Also if you already have remote state why do you want it on git? It would be a hassle to maintain the current state across your branches.

[@envZero]

We are honored to sponsor this great video!

[@Djsanddy]

one of the best videos that actually shows how big orgs do it. Thanks

[@MariuszZalewski]

Thank you Nana. This is a handful for Terraform beginner developers. What I would add from my experience - creating only one state file for one environment can be an issue if the environment has a lot of resources. A bigger state means a much longer plan and apply (and very often more $$$ if someone uses CI/CD as SaaS), so my suggestion is to divide code into workspaces and use state files for each of them. Keep up to 50 resources per state.

[@OutOfDevOps]

Mariusz, I agree with you, for me the most important thing in Terraform is to avoid the Terralith. Terraform code should be split in multiple dimensions: environments, frequency of updates (of infra components), type of resources, permissions (avoid to run with TF with SuperAdmin SAs), providers. The downside when splitting terraform in multiple dimensions is to keep track of the dependencies. I think IaC is still very early stages and needs more automation.

[@sagarahire6531]

She have already mentioned this point

[@robertocastano4014]

Danke!

[@TechWorldwithNana]

Danke für die Unterstützung Roberto! 💙

[@robertocastano4014]

Gern geschehen, Nana, deine Kurse sind wundervoll. Schöne Festtage! @@TechWorldwithNana

[@muelladresse10]

Hi Nana, I just wanted to give kudos to you for your Terraform course on Udemy. I finished it some weeks ago and I can use my new knowledge in my daily work.

[@TechWorldwithNana]

Thank you Tilo for your feedback! It's really great to hear that my course is helpful for your job 😊👍

[@niravchotai9721]

- Use workspaces to better organize state files - Use pre-commit hooks to do basic Terraform fmt, linting before commiting changes

[@mariocortes2670]

Hi Nirav, I never have used workspaces in real life, I think that if you can use different state files for different environment it wouldn't be necessary. Could you give any reason to use it? I could be wrong. Regards

[@TechWorldwithNana]

Thanks for sharing Nirav! 👍

[@lmart7402]

actually using workspaces is a pretty bad practice I feel as differents workspaces are juste subset files of a main state so all your states are hosted within a single remote backend, so it becomes a single point of failure, you wouldnt want you prod state deleted by a mistake made on a temp workspace. There is a book called terraform up and running published by Oreilly and if I recall well, it is stated toavoid using workspace if you arent doing it on terraform enterprise.

[@mariocortes2670]

@@lmart7402 Thanks for your reply.

[@johannesallen5593]

@@lmart7402 what about terragrunt to manage envs?

[@Nagenderkotnala1]

Nice video, you can also include -: Using of TF modules to follow DRY code practices. Use of terraform workflows to deploy similar type is multiple environments. Securing sensitive variable to output on console. Securing state files wherever kept.

[@benjishohet]

I just love the way you present the information in your videos! Best videos out there

[@yogeshdharya3857]

This env0 for both Terraform and DevOps as a whole looks and sounds so cool! Great information there Ms.Nana. Thank you so much!

[@professorlamp]

One thing worth mentioning regarding tfstate and putting your terraform into version control is that you do not want to version control your .tfstate files. These can potentially contain secrets in plain text. The best thing to do in this case is switch to a remote storage method like s3 even if you're not part of a team, just to be on the safe side.

[@richardwaldron1684]

As a newbie to Terraform this is an excellent video, I wasn't aware as yet of most of these items so I've learnt a lot. I still don't know anything about CI/CD so that's my next visit in your library. Thanks

[@maxweber6556]

Hi Nana, you are awesome.the way you explain things is super easy. You know what, I became Devops expert only watching your videos.

[@MoranGuyVideos]

Perfect timing!

[@KahanDataSolutions]

Great job! Love your presentation style and have enjoyed many of your videos over the past year.

[@AGsaga13]

Your videos are few in thousands that I enjoy watching till the end.

[@VijayKumar-xt7es]

Using terrform modules instead of repetative resources and pass the required input arguments to this module, this module can also be versioned and kept in git and use tags (with incremental versions eg: v0.1) to refer in the main code. This enhances our code to become better in terms of readability. Also, we have to hide the sensitive content in tf output vars ( if any) and donot hardcode sensitive protected info in tf code, instead handle them through CI ( store them as jenkins credentials ), write jpac to read it and pass it as CLI arg to tf commands as needed.

[@theuniverse8948]

Yes I Got Best KZbin Channel For My DevOps Journey

[@Stvcloud]

Amazing best practices, one to add is to create names or identifiers dynamically for the resources that cannot be deployed multiple times, this way you can deploy IaC for features branches to test your changes before merging to the main branch

[@be_erod]

Love the content, and I’m happy to report that I’ve been following these best practices for several years now. Only thing I’d recommend is to use modules as a best practice. It’s just easier to manage components vs one file with all the resources.

[@jonykiran9901]

Love your videos as always, concise, precise and crisp, thank you

[@jitender83601]

Hi Nana, Thanks for the video. Terra grunt can be use for DRY your terraform configuration across multiple environments.

[@OMNS777]

Thank you 🙏. Gonna give env0 a try!

[@therealpa3ng]

Thx for sharing! How about a video on managing terraform modules? Or maybe how to separate out IaC for staging, production, and dev environments :)

[@BernardoGarcia19]

+1 about terraform modules, and perhaps outputs

[@joesome_]

+1 that would be a great one, looking forward to it

[@sathyanarayanareddy5192]

Very good explanation now i have good idea about TF State. Thanks

[@myvlogs1414]

Terraform modules also one of the best practice to share other terraform users for quick start/update without reinventing or duplicating the terraform code

[@hiteshsuthar1097]

Always like, and Already Subscribed with opening bell icon 😅

[@christinachen9669]

great video Nana, Love all of these

[@metalejo]

Hello Nana. Another best practice it's trying to avoid throwing resources to the main files in all the environments that's has the same deployment. This should live in a separate versioned/tagged module allowing to have a standard way to deploy new environments.

[@yyao3895]

I’m actually interested on how to test IaC code.

[@stuffedcode]

Great Best Practices Explanation! Really awesome explanation on how to utilize 1 state per environment and how to organize states!

[@TechWorldwithNana]

Thank you :)

[@mariocortes2670]

Great video!

[@ynaumenko]

Just a quick tip regarding remote state. In my practice I'm using state separation per service as well, not just per env. For example, gke cluster or cloud SQL tf modules should store their state files within different folders. It could be easily configured in Terragrunt - backend.tf could be generated automatically with required configuration. Finally it will look like: /terraform/state/environment/service. Usage of terraform workspaces is not a good idea especially with distributed teams. IMHO.

[@Rolanvarez]

THANK YOU!

[@thomash.8297]

Perfect Nana!

[@jeremysmith6459]

Stellar content! Can you discuss environment management strategies for Kubernetes workloads leveraging the GitOps model? 🙏🏽

[@punj]

Thanks for this video Nana. One thing I was expecting in this was to organise terraform code in reusable modules.

[@rasmeysareth6409]

It is a very good practice to set default_tags on aws infrastructure. And also to create a resource group to group easily all cloud resources by filtering on some tags.

[@jyotsnagupta1248]

No, when using Amazon S3 as a backend for state file storage, the default state locking mechanism does not automatically utilize DynamoDB, so your statement at 3:27 is not correct.

[@andyhuynh2450]

Hi Nana, great tutorial. Is there a way to get notify when someone in the team execute a terraform plan?

[@jagtarsingh2058]

Thank you

[@tobylifers3390]

Nice video. I'm surprised you didn't mention Terragrunt though.

[@random_user_dev]

hey, one question regarding the state file for each environment, since I'm creating a state file for each env, is it best practice for all to be in the same bucket with different directories or different buckets? and regarding the dynamodb, should be one db for the state files?

[@minakhalil7043]

Thanks for this great video. I would also add the use of Terraform Workspace to separate different environments as best practice. Thoughts?

[@Matt-ct5ks]

Nice video, if you remake please include more about secrets management and what files you don't want to commit to any public repositories.

[@Praveenkumar-vg3pn]

Your explanations are concise and to the point. Keep creating these videos it helps a lot.

[@josetapia4211]

i love this video, thanks for sharing.

[@ankitbishnoi4067]

Very useful information 👌

[@BeckerBoarder]

Great content, I really appreciate your time. I am just curious, what tool do you use to animate your slides?

[@matzmr]

Great video as always. Can you tell me what application did you use for working on the animations in your video?

[@_mansoor]

Awesome!!! Thank you Nana

[@sabyspeaksonline]

Would request a tutorial series or bootcamp on MLOPS or AIOPS?

[@HannesWithoutJo]

Hi Nana, could you share your source for the popularity statistic of IaC tools at 0:03? I've been looking for something similar for my Bachelors thesis. Thanks in advance

[@sanooosai]

thank you Nanaaaaaaaaaaaa Glory to you

[@jonasag4584]

Hello Nana Great job as usual 👏 I think there is a another important point to discuss regarding Secrets in plain text Thank you 😊

[@code1530]

Hi Nana! why are these courses not available in Udemy Business? :(

[@TechAtScale]

What do you use to make these nice presentations?

[@Harshal977]

Can you please upload the video on Terraform vs Terragrunt

[@wz6887]

Hi Nana, may I ask which software are you using to edit such beautiful and interactive videos? Thanks a lot

[@izuchukwuokoye734]

Hi Nana, how do I upgrade argoCD version from 1.5.5 to 1.6? Looking forward to hearing from you

[@mossasameer3338]

your youtube tutorials are very high-quality content, I wanted to purchase a course and i did for the terraform course, but for the DevOps course it's very expensive, could you consider please some discounts :)

[@studyastudyb2886]

Can You please cover What all topics to study for Hashicorp Terraform : Associate Exam

[@StephenRayner]

4:40 you didn’t say how to have multiple state files. I recommend you do it with work spaces.

[@OutOfDevOps]

Hi Nana, thank you for the great content you produce, what's your preferred choice for testing IaC (terraform in particular)? Thanks

[@flesz_]

DBAs at my work decided to remove 2 servers and keep number 3,4,5 only This was a problem to do it in TF as we use count and count was 5. If you changed count to 3 it would remove servers 4 and 5 So I had to amend the module we had and also change indexing inside the tfstate to make it work

[@iham1313]

i don't get the best practise having a terraform state file remotely. why not use git for that too in order to have "locking" or versioning and a pipeline to push that against a server? i don't see why use git for terraform code, but not for state files. maybe i am missing something

[@amirkhemissi2421]

Does using a repository to host your terraform code eliminates the need to have a dedicated storage for the state file ? Or , we use repository to host only the code and after completing the pipelines the new changes will get the latest state and update it ? Thanks in advance ^^

[@zamankhan4003]

Can anyone tell me how to get a software developer job as I am a fresher. And also learning the Devops bootcamp for further knowledge

[@poulmh]

It is perhaps obvious, but I think it's worth mentioning that auth tokens etc. should not be stored in variables file, but in terraform.tfvars. And this file should not be included in the source project.

[@mothman1988]

I'm currently using a tool called atlantis, that should be an alternative to env0

[@wassap124]

env0 supports atlantis workflow and has more features

[@prasannavenkatesh4881]

Hi Nana, I am struggling to download the latest builded artifact from artifactory which is a war file basically, I have many artifacts on my artifactory repo and I need to download the latest builded one..can you is there any way to do it?unfortunately I’m not using the artifactory pro version which makes it more complicated to do.please help

[@itsmebhanuteja]

Hello, Your explanation was simply superb and easy to understand as always. As mentioned in this video 4th best practice -> I am using GCS bucket and storing my terraform state file there after every terraform apply but unfortunately I have deleted GCS Bucket and now i have lost state file...Could you please let me know how can I create or get back the terraform state file. Any solution.

[@petertolgyesi6125]

According to best practice 6, it is in Git, so according to best practice 8, let the CD put it back, once you have a new bucket. (I know this is theoretical now...)

[@achillemongatcheucheu2866]

Great video thanks. Do you have some recommendation on how do we test terraform code in CI/CD pipeline?

[@leoyang887]

terratest ? Which runs your tf code against a empty cloud environment, then terratest interacts with the cloud, to validate whether the freshly provisioned environment meets expectation. There are different alternatives to terratest, my understanding is terratest works best with Azure, as it provides better integration with AAD to access the provisioned resources for validation.

[@ashutoshbhatte11]

use terraform validate to test terraform code

[@ivantang5795]

How about secrets management?

[@Midhunchowdary]

@TechWorld with Nana you missed security best practises like storing secrets and also terragrunt.

[@user-sz5slm]

Thank you Nana 😎 Since you are using Git, the development process must follow Gitflow, you must have branches for your dev, test and prod environments, and the state files must match those branches. This can be considered a best practice. But everything is learned in practice in a specific project.

[@dAvid3_f]

Hello Nana, yet another great tutorial 👍 But I was just wondering why we couldn’t use git or another scm tool to store stats ?!? 🤔

[@sagarmamidala5242]

We should not store state file in SCM as it contains credentials. You can still store but it is not recommended

[@dAvid3_f]

@@sagarmamidala5242 thanks for the feedback

[@JenilCalcuttawala]

Woahhhh .! My org follows all of these…

[@alexiscalderons]

I'd like to say the same, I watched the video 2 years ago and my org still doesn't. I've applied tf as automation on my own but I'll prepare a presentation with the information @TechWorldwithNana gave us 3 years ago to help my org to impact in a good manner the way we are doing things specifically in the cloud.

[@yehudamakarov]

We didn’t speak about security of secrets!

[@fahadbawazir1771]

Good but if you could make complete TERRAFORM video in which case studies to implement in different platforms of technical perspective

[@TechWorldwithNana]

Hi Fahad, I have a complete course, in case you want to check it out. Check the video description :)

[@moeenkamali1288]

at the end why just dont use git with those state files?

[@Dakshithnaresh]

Please make full course

[@philippeconway8636]

Nana has one here: www.udemy.com/course/complete-terraform-course-beginner-to-advanced/

[@shaileshsingh1445]

Nana, are all these tips integrated with the boot camp?

[@TechWorldwithNana]

Hi Shailesh, yes they're integrated. You can check out the curriculum!

[@ehsanhasin]

great

[@hlubradio2318]

Thanks but most are best practices everywhere else too. If anyone has been with the computer in a shared environment they would have used a few already especially the locking mechanism

[@danielbc82]

Nice video but I think you missed the point with GitOps and Terraform, in order to implement GitOps, a tool like Crossplane instead of Terraform must be used (or at least create your own terraform tool)

[@gimboland]

I don't understand point 4 (backing up/versioning terraform state). Isn't the terraform state just supposed to be tracking the actual state of the deployed resources? If that's the case, and it gets nuked, am I not in the same position as I was in before I ran "terraform apply" or "terraform plan" for the first time? Can't I just run "terraform apply" again? Seems to me it's an easily-recomputable resource - so why back it up? (But maybe I'm missing something?) And as for versioning state... again, why? I don't see any use case for rewinding to older state (as you seem to suggest) - again (as I understand it) it's just supposed to be a reflection of the current actual deployed state, so rewinding would just make it inconsistent with reality. If you want to rewind your system state, that's what versioning your actual terraform code is for (and yes of course do that!): check out an old version of _that_ and plan/apply. The only purpose I can think of for versioning state is as a historical record for audit purposes (i.e. attesting that "this changed at this time"). But maybe I'm missing something...? Genuinely asking, as I'm relatively new to terraform. Everything else you're saying makes sense to me, so if there's something I'm missing here I'd love to know about it. Thanks!

[@ValeriySolovyov]

1. You created VPC -> subnet -> VMs. How on the earth terraform will know if you want recreate subnet with terraform taint without 'state'? 2. You 120+ servers have names server-random suffix (module random string) how terraform will know if 99th server should be replaced?

[@Jorres-qx1lu]

Hey, could somebody please explain to me one thing? Why is it necessary to store `tfstate` file at all? An alternative: just query the provider about what infrastructure it has and use this response result instead of tfstate on the moment of code execution. It would require to implement a small number of additional API for providers, but it reduces so much complexity for the end users... I can see only one possible reason - it is difficult to perform locking, when N people simultaneously execute `terraform apply` without state file. But here comes the punchline - since Terraform does not provide this feature out of the box, you still have to enforce mutual exclusion yourself, this reason must not be the actual reason why was it not done! As I see it, querying state explicitly would solve all the complexity of maintaining this state shared between team members, which is a SIGNIFICANT complexity if you did not use static file storage in your project before. Literally, WHY???

[@mo4721]

What is the difference between a software engineer and programmer, even though they both write code

[@nick7902]

Why didn't you mention in 1st best practice nothing about terraform import, state list, state rm ?

[@rajesh-ee3bj]

hi..nana ..you can make one video for learners...write terraform script and deploy through ci/cd pipeline...please my request...madam .

[@darshmostafa6138]

I am a beginner and I learn HTML CSS and I want to be a Back End Developer ( Node js or Spring BooT ) Please reply me and Thanks 🥰

[@emaayan]

i'm not sure i understand about restoring previous versions of corrupted state files. if the state files represent the cloud resources, then using a previous version would mean it will not be in sync with what actually exists in the cloud? in regards to using your own TF repository, i have to say i object to that ,(even though it what we do) because in my opinion, tf code should reside in the SAME repo as the application code that uses it, this is because the commit should contain both the resource creation and the code that uses that resource. assume a developer needs to establish an event bridge, SQS, SNS and s3 bucket resources, his code would need to address and use them, which would require creating them in advance, which would require devops to do this process, making the developer wait till they are finished, (same goes for changing said resources ) thus making devops a bottle neck (yes i'm aware env0 is meant to solve this, but to me it seems kind of an over kill) additionally let's say i'm using a new syntax in terraform (like using dynamic blocks) which may not always work, the constant iteration of a build server deploy would THAT bottle neck as it would start queues on these changes. and lastly , i'm wondering, if i can combine localStack into this, buy somehow managing the code to be selectively executed on localstack container, (where i want only certain amount of resources created and override all endpoints to my own.

[@ValeriySolovyov]

Probably you don't use ring deployment. Probably you don't have multiple DCs and rules. Probably you never faced the infrastructure changes not related to the code (mostly it has another lifecycle).. Probably you never faced the migration - for example your code is not enough to handle 2x rpm and need to create second EventHub or emergency to recreate a SQS. Probably you never faced the shut with a buil of the code when you can't release when it needed for infrastructure changes.

[@emaayan]

@@ValeriySolovyov shut with a buil?

[@ValeriySolovyov]

@@emaayan autocorrection:) The problems with Artifactory or dependencies(servers or repo shutdown)- when you cannot release a new version but you have an urgent infra changes.

[@emaayan]

@@ValeriySolovyov yes, but those could happen any way, i"m talking about a system the has it's tf source code and state committed in each repo that uses the code for it, but the process that launches and coordinates it is in one place using Tf remote to access the actual templates, otherwise, what will happen is that your own code will always fall out of sync with cloud resources. I have no idea how it's being managed these days

[@ksaittis8516]

Another best practise is to use terraform modules, which will allow you to reuse your terraform across all your environments.

[@charlestangwa7188]

Best

[@garancha]

Probably it's because of Terraform's approach, or because of some restrictions of targeted infrastructure, but solving parallel editing using locking is ridiculous, especially considering reinvention of the wheel; there is "code" in infrastructure as code, and nowadaysr the best solution to maintain the code is using source control.

[@marslogics]

How about terragrunt?

[@NitinMathewGeorge]

Wish to have an email connection ☺️ Also would like to know more about integrating CIs or env0 into PRs.

[@grymmjack]

DRY with terragrunt, modules for reuse.