[Python] Build a CRUD Serverless API with AWS Lambda, API Gateway and a DynamoDB from Scratch

[@billybob2a]

I've followed about 6 videos before trying to get this working and was unsuccessful. A couple of things I took away from yours that made it successful for me: - Your attention to detail and explanation of the code was very helpful. - Setting the timeout to longer than 3 seconds I think is the golden thing that puzzled me, other videos didn't advise it, yet I think it helped! - Case sensitivity is important. I named my key productid instead of productId and had to use cloud watch to figure out why mine didn't work. - Not only does it work, I now have a code base as well as a base understanding to keep going. Massive thank you!

[@dhochee]

I had to add lambda/dynamodb functionality to a simple website in just a few hours and was worried I wouldn't have time, but this vid totally saved the day. Everything worked perfectly. Thanks!

[@FelixYu]

Glad that it helped 👍

[@shaneatvt]

This is super helpful material. Your level of content and pace are great. Thanks a lot Felix!

[@FelixYu]

Tyty glad that u found it helpful!!

[@daqa290885]

Hi bro, excellent video, in the first time, was desperate jejeje, because I changed some variables and put the wrong variables necessary for all the code, also, I could add all tests for every method that you mentioned in the video, was very interesting, because if you don't know how to check the logs in. cloud watch, or you don't the correct syntax to write the dynamodb resources, you always get and internal server error 502. Thanks for this video you won a follower for your channel. Note: I broke my brain, trying to fix all my errors, but this is our world, we try to understand other codes and to practice every day until all are excellent. thanks again and regards.🤓

[@trevspires]

Felix - any chance you can share the code repo?? I'm a python noob, and getting intenral server errors when hitting API GW with a 502. Source could be helpful as I troubleshoot what I've done wrong.

[@asfandiyar5829]

If you are getting internal server error then make sure that the code is correct and that dynamo table name is spelt as productId (capital I not i). If you are getting a 404 not found error then make sure you have spelt your variables correctly. I had PATCH as PACH. I've provided the code written in this tutorial below: # lambda_function: import boto3 import json import logging from custom_encoder import CustomEncoder logger = logging.getLogger() logger.setLevel(logging.INFO) dynamodbTableName = "product-inventory" dynamodb = boto3.resource("dynamodb") table = dynamodb.Table(dynamodbTableName) getMethod = "GET" postMethod = "POST" patchMethod = "PATCH" deleteMethod = "DELETE" healthPath = "/health" productPath = "/product" productsPath = "/products" def lambda_handler(event, context): logger.info(event) httpMethod = event["httpMethod"] path = event["path"] if httpMethod == getMethod and path == healthPath: response = buildResponse(200) elif httpMethod == getMethod and path == productPath: response = getProduct(event["queryStringParameters"]["productId"]) elif httpMethod == getMethod and path == productsPath: response = getProducts() elif httpMethod == postMethod and path == productPath: response = saveProduct(json.loads(event["body"])) elif httpMethod == patchMethod and path == productPath: requestBody = json.loads(event["body"]) response = modifyProduct(requestBody["productId"], requestBody["updateKey"], requestBody["updateValue"]) elif httpMethod == deleteMethod and path == productPath: requestBody = json.loads(event["body"]) response = deleteProduct(requestBody["productId"]) else: response = buildResponse(404, "Not Found") return response def getProduct(productId): try: response = table.get_item( Key={ "productId": productId } ) if "Item" in response: return buildResponse(200, response["Item"]) else: return buildResponse(404, {"Message": "ProductId: {0}s not found".format(productId)}) except: logger.exception("Do your custom error handling here. I am just gonna log it our here!!") def getProducts(): try: response = table.scan() result = response["Items"] while "LastEvaluateKey" in response: response = table.scan(ExclusiveStartKey=response["LastEvaluatedKey"]) result.extend(response["Items"]) body = { "products": response } return buildResponse(200, body) except: logger.exception("Do your custom error handling here. I am just gonna log it our here!!") def saveProduct(requestBody): try: table.put_item(Item=requestBody) body = { "Operation": "SAVE", "Message": "SUCCESS", "Item": requestBody } return buildResponse(200, body) except: logger.exception("Do your custom error handling here. I am just gonna log it our here!!") def modifyProduct(productId, updateKey, updateValue): try: response = table.update_item( Key={ "productId": productId }, UpdateExpression="set {0}s = :value".format(updateKey), ExpressionAttributeValues={ ":value": updateValue }, ReturnValues="UPDATED_NEW" ) body = { "Operation": "UPDATE", "Message": "SUCCESS", "UpdatedAttributes": response } return buildResponse(200, body) except: logger.exception("Do your custom error handling here. I am just gonna log it our here!!") def deleteProduct(productId): try: response = table.delete_item( Key={ "productId": productId }, ReturnValues="ALL_OLD" ) body = { "Operation": "DELETE", "Message": "SUCCESS", "deltedItem": response } return buildResponse(200, body) except: logger.exception("Do your custom error handling here. I am just gonna log it our here!!") def buildResponse(statusCode, body=None): response = { "statusCode": statusCode, "headers": { "Content-Type": "application/json", "Access-Control-Allow-Origin": "*" } } if body is not None: response["body"] = json.dumps(body, cls=CustomEncoder) return response ######################################################################## # custom_encoder: import json class CustomEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, float): return float(obj) return json.JSONEncoder.default(self, obj)

[@tomtricoire4774]

I had some problem with the CustomEncoder and had to change this : import json from decimal import Decimal class CustomEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, Decimal): return str(obj) # Convert Decimal to a string return super().default(obj)

[@rafaeldeghi587]

Anyone had this error? Cant find a solution, its occurs when i try to use patch method, the value is updated, but the response from the api is 500 [ERROR] UnboundLocalError: cannot access local variable 'response' where it is not associated with a value Traceback (most recent call last): File "/var/task/lambda_function.py", line 46, in lambda_handler return response

[@biswanathsah9732]

Thank you @asfandiyar5829. I wrote the whole code by watch but got Internal Server error . I couldn't recognised error . Thank you for the correct code , it worked for me 😊

[@AhmedKhatib-c1w]

This was very straightforward and helpful. Thank you! Experimented with stuff and followed along and learned much. Just one thing, your python variable / object naming convention feels like Javascript (camelCase), while in Python the convention is variable_name. So was confused for a short time xD.

[@ehsanarefifar196]

Very nice start-point walkthrough video. Thanks Felix! Way to go!

[@FelixYu]

Thank you!!

[@tarcisiosteinmetz3472]

Great work, Felix Yu! You successfully explained API Gateway and Lambda in a very detailed way. Thank you.

[@FelixYu]

Thank you 😄

[@LS-qg2zn]

Very very helpful tutorial for a beginner.. Thank you so much!

[@JoseRodrigues-vd3si]

Thee best ever explanation I have saw about this subject.

[@FelixYu]

Glad that u found it helpful :)

[@romihans]

I have copied all of this code into VSC and got this working. No hassles there, barring a few glitches. I am thinking of creating a Git repository that could help out people who are on a learning curve wit Lambda & API (it's always a learning curve actually, but can be real steep in the beginning) In a real life scenario, would it be a better idea to have different Lambda functions for delete, modify and add functions of a CRUD API? Just discussing the pros and cons vis-vis maintainability and testability

[@clivebird5729]

Very helpful and insightful Felix. Thank you for sharing this, very much appreciated.

[@FelixYu]

Glad that it’s helpful :)

[@CMishra-kl4rb]

Hi Felix, Can you please share the code because I'm getting error on my system as "Internal Server Error". Please share code anyone.

[@victoradejumo566]

I tried updating using patch and I am getting Internal Server Error message. Wonderful video you put up. It was very helpful

[@FelixYu]

that means there is an error in the lambda function. take a look at cloudwatch and see what the problem is

[@anojamadusanka8914]

got error. { "message": "Internal server error" } 502Bad Gateway. How to resolve this. No errors shown in the log events. Thank you.

[@FelixYu]

did u print out the request event? how did it look like?

[@fitnesswithvaibhav]

Got the same 502 bad gateway { "Message": "Interval server error" }

[@fitnesswithvaibhav]

@@FelixYu please help me

[@asifhossain1874]

@@fitnesswithvaibhav got the same error

[@fitnesswithvaibhav]

I have checked and found it was my mistake

[@ajaysinhavadithya]

Awesome video... could you please create a video for RDS instead of Dynamodb

[@maheshbabuuda3059]

Hi Felix Where can i get the lambda function code ??? python script ??

[@stephensuico5741]

Thank you! Lot of value here

[@FelixYu]

Glad that u found it helpful!!

[@kothon1]

Bro you're a Lambda Beast!!! Alteast to a mere mortal novice!!!!

[@FelixYu]

Thank you and glad that it’s helpful :)

[@SatyaNand592]

Awesome Video , very helpful and the standard of code is also adorable. one request Felix please create a separate playlist for python aws functionalities with the same standard of coding please that would be very helpful to the mass.

[@FelixYu]

glad that its helpful 👍

[@camichaves]

Outstanding video! Thank you.

[@FelixYu]

Glad that u found it helpful 👍

[@andynelson2340]

Awesome, thanks for making this video😊

[@FelixYu]

Glad that it’s helpful 👍

[@YogaTriLoka]

This is a great video, super helpful! If I wanted to get all products with color "green", what would the scan body look like? Or would you not use scan for this functionality?

[@christianechica4270]

can you share the code via github?

[@ttaylor9916]

Thanks!

[@GauravRoy1972]

Thanks for this Felix, could you please create a tutorial to explain the CRUD operations in dybamoDB via Lambda. Querying in dynamoDB seems to a whole subject in itself.

[@DestroidAdicted]

I have a problem that when I make the query the event does not bring the httpMethod and path information, the event comes empty.

[@lennyc2568]

Hi getting [ERROR] KeyError: ‘httpMethod’ …. On post getting a 200 with the healthPath but the above error when trying to retrieve from an existing dynamodb

[@FelixYu]

when u print out the request event, do u see httpMethod as one of the attributes??

[@mukuljain8383]

Thanks for making videos for nodejs and lambda function super happy for that, also could you please make videos for js, your videos are great and i want to learn aws with nodejs and not python

[@FelixYu]

The js video link is in the description!!

[@clintonebai1351]

Hey Felix, you are doing a great job and thank you for this wonderful tutorial. I coded along with you in this tutorial but after trying to invoke my function via postman, I got a 500, internal server error, however, that is not my most significant concern. How are you able to put all these pieces of code together, how are you able to know where and when to use a particular function, module or class? I understand the basics of python but putting them together to form one big program like what you just did is a nightmare for me. How are you able to write almost 250+ lines of code forming a single program without mistakes? Is there a manual you guys use when coding? what is that cheat code you use bro?

[@andrzejwsol]

Were you able to resolve the internal server error? The health check works but then I get 502 error when I try the POST request.

[@andrzejwsol]

I fixed my internal server error! Turns out my DynamoDB table's partition key was misspelled. I had it as "productid" instead of "productId" (capital i was lowercase i). So for anyone getting a 502 I'd say go back and make sure all the small details are correct...

[@clintonebai1351]

@@andrzejwsol Thanks for the information I will look at my code again

[@andrzejwsol]

@@clintonebai1351 let me know how that goes. I’ve also read that Lambda is very particular and will throw a 502 if you have single quotation marks versus double (‘’ vs “”) so that’s worth checking out too

[@clintonebai1351]

@@andrzejwsol Alright, no worries but Can you share your code with me?

[@yashmodi5761]

Please create tutorials using AWS cdk and boto3.

[@luizarnoldchavezburgos3638]

Is it better to have CRUD in one api or C R U D in 4 diferents lambdas?

[@romihans]

could we have created a zip file that includes the custom encoder and uploaded that to Lambda? I think this should work. Your inputs would be very helpful

[@navidshaikh9146]

Im getting 502 bad gateway in postman with 'internal server error' message, how should I solve this

[@FelixYu]

That means there’s an error in the lambda function. U can check the lambda log and see what the error is

[@navidshaikh9146]

@@FelixYu can you please provide lambda code in description or somewhere

[@Omanshuaman]

downgrade node 18 to node 16

[@bodyshapeandmotivation]

Can you share the code for the same

[@surajthallapalli4227]

Loved it, thanks

[@FelixYu]

Glad that it’s helpful :)

[@shaikshoaibakthar-f5d]

can u plz start classes of python from beginer to advanced level??

[@amrithanshu3478]

{ "message": "Internal server error" } how to resolve this

[@LavanyaVeluswamy]

Can you make the video for SpringBoot? It would be great help

[@happylearning6543]

@Felix Yu, this video was really helpful thanks a lot!! I am stuck in integrating my lambda function, dynamodb and api gateway. Are you open to giving feedback on individual questions? Will be easier if I show you my approach and your coffee is on me for sure!

[@amineghadi1524]

Thank you for this video it's very useful , can you do same one with Redshift Database

[@willianmaesato4618]

Good night, first good work, I have a doubt I wanted to make a pagination in this model would it be possible for you to show how to do it? I tried to find something and try but I didn't succeed.

[@onlymullapudi]

Thank you for the tutorial. Can you provide lambda code with gitlab link?

[@riazahmad5975]

Hello sir , please make vidoe that how to insert csv data to dynamodb in serverless framework using lambad in nodejs

[@aryabasu814]

thanks for this great video... where can I find the code? thanks

[@tejashreepotdar9318]

My update is failing I have written the same function as your's can you please share me the right code for Update

[@FelixYu]

What error message are u getting?

[@ParthPatel-rh1ct]

Having an issue when running the health check on Postman. { "errorMessage": "'httpMethod'", "errorType": "KeyError", "requestId": "f2c90a37-afe5-44f7-99a2-696dd8811efc", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 24, in lambda_handler httpMethod = event[\"httpMethod\"] " ] } Can you please post your code? Because, I don't know what happened with the code.

[@gshan994]

your event parameter doesnt have a key as "httpMethod". you can pass event[] as a parameter in json.dumps(response) instead of response

[@FelixYu]

When u log out the whole event object in line 21, how does it look like? Does it show httpMethod as one of its attributes?

[@rafaeldeghi587]

Anyone had this error? Cant find a solution, its occurs when i try to use patch method, the value is updated, but the response from the api is 500 [ERROR] UnboundLocalError: cannot access local variable 'response' where it is not associated with a value Traceback (most recent call last): File "/var/task/lambda_function.py", line 46, in lambda_handler return response

[@sahirbhat9297]

whre i can get code of this project

[@aaryanravi5265]

​ @Felix Yu Thanks a lot for this video, its really helpful. I am getting an error message as follows; "message": "Missing Authentication Token" any solution on this?

[@FelixYu]

Did u accidentally enable authentication required in api gateway?

[@shrutikamandhare5046]

Hi was your issue resolved?

[@ablevoice2428]

Great video. Thanks....pls I will appreciate if you can share this code. Something like the repo link

[@khoatd7726]

Hi Felix, I see 502 bad gateway when call GET /product API. What could I check to solve this error? Thanks.

[@FelixYu]

That means there is an error in ur lambda function. Check the cloudwatch log in the lambda and see what the error is and then resolve it accordingly

[@khoatd7726]

@@FelixYu Thanks a lot!

[@Omanshuaman]

downgrade node 18 to node 16

[@Omanshuaman]

downgrade node 18 to node 16

[@MyGui1000]

It's work very well to me in POSTMAN and Requests in Python, but when I try make a request in my simple web page using Javascrip I'm having a issue call CORS "CORS policy: Request header field acess-control-allow-origin is not allowed by Access-Control-Allow-Headers in preflight response", someone know what is?

[@virtualvessel0]

Hi, thank you for this. Could you send or post the text source-code please. Thank.

[@Kukshalshrey]

hey!! great video this really helped me clear few doubts!! but I am getting this error >>> "errorMessage": "'httpMethod'", "errorType": "KeyError", could you please help?

[@FelixYu]

When u log out the whole even object in line 21, how does it look like? Does it show httpMethod as one of its attributes?

[@Kukshalshrey]

@@FelixYu if i print(event) iam getting { "key1":"value1", "key2":"value2", "key3":"value3" }

[@FelixYu]

@@Kukshalshrey u needa configure ur lambda with api gateway and then use postman to hit the endpoint. U can’t just hit test in the lambda console

[@asifhossain1874]

Please upload the code also so that we can test it

[@shaikshoaibakthar-f5d]

can i get the code??

[@tasmiyamuneer8886]

{ "message": "Internal server error" } getting this error .. could u please solve it

[@fitnesswithvaibhav]

Bro i am also getting same error i u get any solution please let me know

[@mallikarjunsangannavar907]

@@fitnesswithvaibhav any luck?? i'm getting error at response = getProcessDomain(event['queryStringParameters']['process_domain']) line.

[@zaheerbeg4810]

This tutorial is okay for AWS free tier?

[@FelixYu]

Yes, it’s within the free tier limit if don’t run it a lot

[@zaheerbeg4810]

@@FelixYu Thanks

[@kosalendra1387]

will anybody please let me know how much will it cost for this above project in the vedio @Felix YU

[@chinmayakumarbiswal]

Sir can you share your code link

[@FelixYu]

Idk if I saved the code when I did it. Let me check when I get a chance 👍

[@Nikhilsharma-tc5wr]

Hi felix , your video in such a helpful can you share these code with me👍👍

[@dahavlogs]

Love your video. But i am getting an error. Even I followed everything. Can you please help me out. Error: { "errorMessage": "'httpMethod'", "errorType": "KeyError", "requestId": "7a3b8589-935a-41fe-bd22-5cfade4a3fa3", "stackTrace": [ " File \"/var/task/lambda_function.py\", line 22, in lambda_handler httpMethod = event['httpMethod'] " ] }

[@dahavlogs]

PLEASE RESPONSE

[@dahavlogs]

ARE YOU THERE ?

[@AniraKanu]

Could you please put your code on git and share a link.

[@trainsam22]

can you please give the code?

[@hemantchaudhary3465]

Hey! First of all thank you for a such a great video it really helped me alot:) So when i completed everything i was able to work with the post function after changing json.load to json.loads and the items did reflect i the table but the 'GET' funnctionality is not working. got this error on the postman with error code 502 { "message": "Internal server error" } and then check the cloudwatch log and got the log as below. [ERROR] TypeError: 'NoneType' object is not subscriptable Traceback (most recent call last): File "/var/task/lambda_function.py", line 32, in lambda_handler response = getProduct(event['queryStringParameters']['productid']) Anyone encountered similar problem and was able to navigate through it?

[@lucasmagnagodeoliveira9842]

Same here

[@Drappyyy]

I think you need to replace productid with productId

[@JFischbeck]

Can you share your code?

[@gamerz5135]

did any one got the code? can u share it with me

[@camelpilot]

Code repo please?

[@hrishinani]

Hi I tried running the code but it shows this error : botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Missing the key product_id in the item, and thanks for the tutorial this one helped a lot

[@FelixYu]

My guess is that ur table has a primary key set to be product_id but u passed in productId in the request?

[@hrishinani]

@@FelixYu yes ! Thank you the problem is solved 🙏🏻

[@FelixYu]

Glad that it’s working now 👍

[@maheshbabuuda3059]

where is the script

[@tasmiyamuneer8886]

could you please post your code

[@saptanilchowdhury1851]

plz share the code

[@asifhossain1874]

"message": "Internal server error" }

[@jimmypeng5552]

I got a error as below when doing the product get method. { "message": "Internal server error" } and then check the cloudwatch log and got the log as below. [ERROR] TypeError: 'NoneType' object is not subscriptable Traceback (most recent call last): File "/var/task/lambda_function.py", line 32, in lambda_handler response = getProduct(event['queryStringParameters']['productid']) thanks for your great class.

[@jimmypeng5552]

I resolved the issue. that's for the input value in Json format. such as the PATCH method to /product can solve the error "Internal server error". wish it can be helpful { "productid": "10001", "updateKey": "price", "updateValue": "1000" }

[@FelixYu]

Glad that u got it to work 👍

[@hemantchaudhary3465]

Hey! I got the same error and i am not able to resolve it. How did you do it? Thanks:)

[@cpacash3964]

Dislike, because you don't share the example code, what I supposed to do, copy it from the video?

[@liviucristianionescu]

Such laziness....

[@mayanksinha6591]

Used the same code and the same configuration but getting this error [ERROR] KeyError: 'httpMethod' Traceback (most recent call last): File "/var/task/code.py", line 23, in lambda_handler httpMethod = event['httpMethod'] [ERROR] KeyError: 'httpMethod' Traceback (most recent call last): File "/var/task/code.py", line 23, in lambda_handler httpMethod = event['httpMethod']. Logging the event in cloudwatch I am getting this [INFO] 2024-01-12T11:22:01.150Z 12836e18-d826-4cad-8593-20e29a5f7b70 {'productid': '1101', 'color': 'red', 'price': '100'}

[@saptanilchowdhury1851]

share the code please it is not working for me get, patch methods. POST and DELETE is working

[@iantaylor5871]

Where can we find the code?