Build and Deploy LLM Application in AWS Lambda - BedRock - LangChain

[@appikumar-d8l]

Please do more on AWS Bedrock to develop on RAG applications......your explanation is simple and effective.......stay motivated and upload more videos about LLM

[@AboniaSojasingarayar]

Thanks for your kind words! Sure I will do it.

[@akshaykotawar5816]

Yes same thing i want ​@@AboniaSojasingarayar

[@AboniaSojasingarayar]

Here the tutorial link to Deploying a Retrieval-Augmented Generation (RAG) in AWS : kzbin.info/www/bejne/nZrGpJVvpZyooJY

[@raulpradodantas9386]

Save my life to create lambda layers... I have been trying for days. TKS!

[@AboniaSojasingarayar]

@@raulpradodantas9386 Glad to hear that! You most welcome.

[@zerofive3699]

Very informative

[@AboniaSojasingarayar]

Glad it was helpful!

[@humayounkhan7946]

Hi Abonia, thanks for the thorough guide, but i'm abit confused with the lambda_layer.zip file, why did you have to create it through docker? is there an easier way to provide the dependencies in a zip file without going through docker? Thanks in advance!

[@AboniaSojasingarayar]

Hi Humayoun Khan, Yes we can but Docker facilitates the inclusion of the runtime interface client for Python, making the image compatible with AWS Lambda. Also it ensures a consistent and reproducible environment for Lambda function's dependencies. This is crucial for avoiding discrepancies between development, testing, and production environments. Hope this helps.

[@NJ-hn8yu]

Hi Abonia, thanks for sharing. I am facing this error . can you please tell how to resolve it "errorMessage": "Unable to import module 'lambda_function': No module named 'langchain_community'",

[@AboniaSojasingarayar]

Hello, You are most welcome. You must prepare your ZIP file with all the necessary packages. You can refer to the instructions starting at the 09:04

[@tapiaomars]

Hi, its possible integrate DynamoDB for store and retrive context of last user prompts in lambda function?

[@AboniaSojasingarayar]

Hello, Yes , DynamoDB, S3, or in-memory storage depending on requirements. Each piece of context is associated with a user ID, ensuring that contexts are isolated per user with conversation ID. Hope this helps.

[@tapiaomars]

@@AboniaSojasingarayar Thanks, I'll try it and let you know how it goes.

[@Bumbblyfestyle]

👍👍

[@AboniaSojasingarayar]

😊😊

[@akshaykotawar5816]

Very Informative thanks for uploading

[@AboniaSojasingarayar]

Glad it helped!

[@RajuSubramaniam-ho6kd]

Thanks for the video. Very useful for me as I am new to AWS lambda and bedrock. Can you please upload the lambda function source code? Thanks again!

[@AboniaSojasingarayar]

Glad it helped. Sure you can find the code and complete the article on this topic in the description. In any way here is the link to the code : medium.com/@abonia/build-and-deploy-llm-application-in-aws-cca46c662749

[@fkeb37e9w0]

Can we use openai and chromadb on aws??

[@AboniaSojasingarayar]

Yes we can! In the below tutorial I have demonstrated how we can create and deploy lambda layer via container for larger dependencies : kzbin.info/www/bejne/nZrGpJVvpZyooJYsi=F_X7-6YCAb0Kz3Jc

[@fkeb37e9w0]

@@AboniaSojasingarayar yes but can this be done without eks or containers?

[@AboniaSojasingarayar]

Yes! You can try it by creating a custom lambda layer.If you face issue try to use only the required libraries and remove any unnecessary dependencies from your zip file.Hope this helps.

[@user-wr4yl7tx3w]

Can we simply rely on open source only without using Amazon? What if it is just prototyping?

[@AboniaSojasingarayar]

Yes, we can use open source completely.

[@Nabeel27]

I get error: Runtime.ImportModuleError: Unable to import module 'lambda_function': Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there. followed all steps as in your video.

[@Nabeel27]

Looks like I had to setup the lambda as arm64 and the layer (created on mac Docker) also as arm64. Next, it also requires Bedrock setup and access request to llama model to use. llama 2 is no longer available, have to request llama 3 8B or something else.

[@AboniaSojasingarayar]

Hello Nabeel, Are you still facing the above issue?

[@Nabeel27]

@@AboniaSojasingarayar Thank you so much for following up! the error I am getting now is this: "errorMessage": "Error raised by bedrock service: An error occurred (AccessDeniedException) when calling the InvokeModel operation: User: arn:aws:sts::701934491353:assumed-role/test_demo-role-sfu6wu6d/test_demo is not authorized to perform: bedrock:InvokeModel on resource: arn:aws:bedrock:us-east-1::foundation-model/meta.llama3-8b-instruct-v1:0 because no identity-based policy allows the bedrock:InvokeModel action",

[@Nabeel27]

@@AboniaSojasingarayar I was able to solve it. I got the permission to use llama3 and also had to update role permissions to use Bedrock.

[@AboniaSojasingarayar]

@@Nabeel27 Great 🎉