Deploying a Glue Job to AWS with Terraform: A Step-by-Step Tutorial

[@calvinfernandes1054]

Great video! It would be great if you cover Job bookmarks with Glue

[@DataEngUncomplicated]

Thanks, What are you looking for the video to cover, just the concept at a high level?

[@gauravmathur56]

Job bookmarks are a bad ideas , bookmarks can be screwed very easily

[@alessandroariu6090]

Thank you! Great video, is it possible to also stop a job run and start again a job? I mean for a CI/CD process or it is necessary to use the cli

[@DataEngUncomplicated]

You're welcome! What do you define as job run? A glue job run?

[@alessandroariu6090]

Thanks for the reply, e.g. if I have a streaming job running and I want to deploy a new version of the script

[@DataEngUncomplicated]

I think you will need to add a step in the ci/cd process to stop the current job if you can't wait for it to finish on its own. Calling the cli in the ci/cd pipeline is one way or having a Python script using boto3 is another

[@alessandroariu6090]

I see, so it is not possible only with terraform. Thank you again

[@maheshkharawatkar4356]

Thank you for this wonderful video, The content is so useful, I have a question; Is it possible to create an aws glue job with json file instead of python file using the terraform?

[@DataEngUncomplicated]

So glue jobs are written in python. If it's a spark job, you have an additional option of scala. I'm not sure what you mean by using a JSON file?