AWS SQS + Lambda Setup Tutorial - Step by Step

[@TM-cb2te]

The reason why you fired one lambda for each of them is because your trigger window is set to 0. it needs to be at least the smallest allowable number presumably for it to wait enough to scan beyond the first element in the queue.

[@yashgangrade5460]

Thanks for clarifying my doubt!..

[@haoguoxuan411]

good call

[@iroshanaravishan1388]

Recommend for beginners✌️ Clear . Simple . Superb

[@eliasperez5168]

I love watching your videos you are amazing at what you do can’t wait to get there

[@BeABetterDev]

Thank you so much!

[@andysobrandy]

That was what I needed. Thanks

[@daniloqueiroz1143]

Thank you my friend! very good

[@eduardocardoso6286]

Great video man

[@AlvinC-sz3li]

SQS to Lambda has scaling issue. This trigger is a push based trigger, Lambda get pushed regardless of Lambda throttle. The result will be, all messages are dropped or be sent to DLQ. I remember internally SQS to Lambda won't pass PE review.

[@BeABetterDev]

Would love a link to the design if you have it !

[@jairomartinezantonio6759]

Thanks a lot for the knowledge.

[@senthilsds]

Great tutorial. Thanks a don 👍

[@BeABetterDev]

Glad you enjoyed it!

[@SirW0nka]

That was very helpful. I have a question though: are the messages that weren't successfully processed going to the DLQ instead of going back in the main queue if this feature is set up?

[@liangyu3771]

awesome video

[@camiloportilla-z7f]

works like a charm,

[@firstname-lastname]

Yo mate, as always, great tutorial 👍

[@BeABetterDev]

Thank you! Cheers!

[@sparshjain6077]

Since we have kept the Batch window as 0s, wouldn't it nullify batching? (We will wait 0s for a batch size of 10 messages and invoke the lambda). Is my understanding correct?

[@bhomiktakhar8226]

Yes

[@g0raxh812]

Exactly...I tried to make the window > 0 and the batching works fine ! so yeah putting batching Window as 0 means we are nullifying the batching...

[@BeABetterDev]

Good eye! It seems this is the case.

[@anandupadhyay1675]

@@g0raxh812 even if the batch window is set to zero, SQS will still batch messages received from different lambdas/events at the same time, depending on the batch size and the number of messages received. This is because SQS batches messages before polling for new ones, and it does not take into account the source of the messages.

[@priyankavaidya110]

hey , you made really great video on this topic , could you please make more videos in details for the same , it would be more more helpful

[@rudrakhuntia5583]

thank you very much

[@thiagolopessilva]

Is there any approach to only run one lambda at time to avoid parallel processing after it received the sqs message?

[@gabrielscotafernandes1461]

Hello, could you please make a tutorial using SQS + Lambda to trigger a standalone Task on a Cluster?

[@gabrielscotafernandes1461]

Actually to reach this behavior, I'm using boto3 to run a Standalone task on my lambda code. Is there another way to do this?

[@ruwanvimukthimettananda7847]

I want to setup same setup, but Lambda in on one AWS account and SQS on another AWS account. How we can setup cross-account connectivity. If you could create a video, it will be great

[@flo_bady]

Hi Dan great content thks a lot Quick one, how are streams created ? Should all the batch items be treated by one lambda invocation? You seem to say that it is not the case but if you have more explanations on when a new log stream is or is not created that would be interesting. Thks a lot Cheers

[@BeABetterDev]

Hey Florian, New log streams are created when a new Lambda container starts up. They aren't necessarily tied to any stream. Hope this helps clarify.

[@chakhmanmohamed9436]

Hello, could you please do a tutorial on how to set up file storage in aws so that the website users can upload their stuff. Just like a database. I don't know how the process is made. Thanks

[@georgesmith9178]

Your question is a bit general, but I will give you a quick answer. First off, you need to allow your users to register and ensure the registration process cannot be abused by bots (assuming you don't want to create an account for every user manually and distribute the credentials via email). Once you have that in place, you can, you can auto create either a separate bucket per user, with the corresponding permissions to upload (IAM roles control that, you will need to create some, or auto create them with tools like Terraform), or create a key in a bucket and modify its permissions to allow only a specific user to upload under that key. Finally, you need to set up quotas, so that users cannot upload massive amounts of data. Hope that helps. Best of luck

[@renejacques8288]

Say the batch size is 10 and the Batch Window is 0, does that mean msg's will be processed as soon as they come in even before the batch size gets to 10?

[@dimalisovyk5277]

How does it actually work with Lambda? Lambda is supposed to be invoked, but SQS assumes polling. So it is pull, not push (unlike SNS). Does it instantiate a lambda with 100% uptime?

[@BeABetterDev]

Hi Dima, Lambda instantiates pollers that can scale up and down depending on how many messages are in the source queue. So yes, it is a pull model as you described.

[@dimalisovyk5277]

@@BeABetterDev thanks! Good to know

[@khanakia]

I followed your tutorial it worked but my queue was stuck in Message in Flight mode those are not auto deleted but you are auto-deleted i can see the video. Note i am using a Standard queue.

[@BeABetterDev]

Hi Aman, This probably means there was an error in the invocation of your Lambda function. I would suggest checking the logs in Cloudwatch. The message only gets deleted upoin successful invocation.

[@dhanavardhanreddy1297]

I need to code json

[@csoutsource]

Why would we use SQS when we can implement Step functions?

[@BeABetterDev]

Cost mainly and use case dependent. Sometimes step functions can be overkill for when a SQS message will do.

[@notaladeen4156]

😂