We used python to create this LLM app.

Glad it was helpful :)

Thank you so much! 😊 I’m glad you found the tutorial helpful!

@@AboniaSojasingarayar Do you know if there is a tool that can convert the Annoted json from the Anylabeling tool to the yolo format?

@johannes7856 Hi Johannes, You may try following library github.com/rooneysh/Labelme2YOLO If not you can convert any labelling json to coco json and again convert it to yolo using the above library. Hope this helps.

Sure, here it is: gist.github.com/Abonia1/fc442374e1c20c86db8effbf95d93eb6

You are most welcome! Glad it helped.

Glad it helped 🙂

Happy to help

Sure! Thanks for your suggestion.

Looks like I had to setup the lambda as arm64 and the layer (created on mac Docker) also as arm64. Next, it also requires Bedrock setup and access request to llama model to use. llama 2 is no longer available, have to request llama 3 8B or something else.

Hello Nabeel, Are you still facing the above issue?

@@AboniaSojasingarayar Thank you so much for following up! the error I am getting now is this: "errorMessage": "Error raised by bedrock service: An error occurred (AccessDeniedException) when calling the InvokeModel operation: User: arn:aws:sts::701934491353:assumed-role/test_demo-role-sfu6wu6d/test_demo is not authorized to perform: bedrock:InvokeModel on resource: arn:aws:bedrock:us-east-1::foundation-model/meta.llama3-8b-instruct-v1:0 because no identity-based policy allows the bedrock:InvokeModel action",

@@AboniaSojasingarayar I was able to solve it. I got the permission to use llama3 and also had to update role permissions to use Bedrock.

@@Nabeel27 Great 🎉

Thanks!

Most welcome. Great and glad to hear that finally it worked. 1. Of course we can use the flask API and ColabCode package to serve your mode via endpoint in ngrok temporary URL. github.com/abhishekkrthakur/colabcode 2. And another way is using flask and flask-ngrok. pypi.org/project/flask-ngrok/ pypi.org/project/Flask-API/ Sample code for reference: from flask import Flask from flask_ngrok import run_with_ngrok app = Flask(__name__) run_with_ngrok(app) @app.route("/") def home(): return "Hello World" app.run() If needed I'll try to do a tuto on this topic in future. Hope this helps:)

@@AboniaSojasingarayar thank you Have a nice day~

Hello, Yes , DynamoDB, S3, or in-memory storage depending on requirements. Each piece of context is associated with a user ID, ensuring that contexts are isolated per user with conversation ID. Hope this helps.

@@AboniaSojasingarayar Thanks, I'll try it and let you know how it goes.

Absolutely, I’ll work on getting it ready shortly. If there are specific areas you want me to concentrate on, just let me know! Also, do you have any custom dataset you'd like to use for this tutorial? Thanks

Glad it helped

Thank you Venkatesan. I'm glad you enjoyed the discussion.

Hello, You can use local VScode or any IDE to create .env New file -> name it as .env And add your API key as follows: ROBOFLOW_API_KEY=your_api_key Once done drag and drop it in colab. Hope this helps.

Glad it helped. Not really! Just using the built-in recording and iMovie to edit it.

@@alvaroaraujo7945 Hello , thanks for your kind words. It may be related to your ollama serve.Are you sure Ollama is running ?

Glad it helped. As mentioned in the description, you can find the code and explanation in this article walkthrough. medium.com/@abonia/deploying-a-rag-application-in-aws-lambda-using-docker-and-ecr-08e246a7c515

Yes, we can use open source completely.

Your most welcome. Here I have introduced basic usage of SAM 2 models. If you want to evaluate your finetuned model you may try mean IoU score, for a set of predictions and targets or DICE, precision, recall, and mAP.

@@Basant5911 can you share your code base and error or issue that you are facing currently please?

@@DenisRothman Thank you for your kind words. I'm grateful for the opportunity to review the book and share my thoughts. Your work is well-deserved and truly one of the most insightful books I've read.

To build RAG with pdf in AWS ecosystem, you need to follow steps that involve uploading the PDF to an S3 bucket, extracting text from the PDF, and then integrating this data with your RAG application.

@@AboniaSojasingarayar Can I locally extract text from pdf and build vector DB locally using vscode and then build the docker image and push it to ECR AWS like what you did in the video?

@@MohamedMohamed-xf7wh Yes, you can locally extract text from PDF files, build a vector database and then prepare your application for deployment on AWS Lambda by building a Docker image and pushing it to ECR. But which vector db are you using? It can be accessible with API?

@@AboniaSojasingarayar FAISS .. what is the problem with vector db?

@@MohamedMohamed-xf7wh Great!

@@htayaung3812 Thank you so much for your support! I'm working to bring more tutorials.

@@raulpradodantas9386 Glad to hear that! You most welcome.

@@SidSid-kp4ij Hello Sid, Sure can you post your error message here please?

Glad it helped. Sure you can find the code and complete the article on this topic in the description. In any way here is the link to the code : medium.com/@abonia/build-and-deploy-llm-application-in-aws-cca46c662749

As we are using lang= "en" so it uses roberta-large. We can also customize it using the model_type param of BERScorer class For default model for other languages,find it here: github.com/Tiiiger/bert_score/blob/master/bert_score/utils.py

Glad it helped. Sure I am already on it.

Hello, You are most welcome. You must prepare your ZIP file with all the necessary packages. You can refer to the instructions starting at the 09:04

Hi Humayoun Khan, Yes we can but Docker facilitates the inclusion of the runtime interface client for Python, making the image compatible with AWS Lambda. Also it ensures a consistent and reproducible environment for Lambda function's dependencies. This is crucial for avoiding discrepancies between development, testing, and production environments. Hope this helps.

Thanks for you kind words and glad this helped. Implement a strategy that combines semantic chunking with K-means clustering to address the model’s contextual limitations. By employing efficient clustering techniques, we can extract key passages effectively, thereby reducing the overhead associated with processing large volumes of text. This approach not only significantly lowers costs by minimizing the number of tokens processed but also mitigates the recency and primacy effects inherent in LLMs, ensuring a balanced consideration of all text segments.

​@@AboniaSojasingarayar Video was great and very useful.. can you make the small video on this clustering method using embedding ?

@@VirtualMachine-d8x Sure will do, happy to hear from you again. Thanks for the feedback.

Here we use open-source Mixtral from ollama.But, yes we can use OpenAI models as well.

Glad it helped!

Yes we can! In the below tutorial I have demonstrated how we can create and deploy lambda layer via container for larger dependencies : kzbin.info/www/bejne/nZrGpJVvpZyooJYsi=F_X7-6YCAb0Kz3Jc

@@AboniaSojasingarayar yes but can this be done without eks or containers?

Yes! You can try it by creating a custom lambda layer.If you face issue try to use only the required libraries and remove any unnecessary dependencies from your zip file.Hope this helps.

Thank you Vijay Gandhi, for your insightful comment! You've raised an excellent point about the importance of considering the limitations of LLMs in the context of abstractive summarization, especially regarding their context length and scalability issues when dealing with large documents. Indeed, one of the significant challenges in using LLMs for abstractive summarization is their inherent limitation in processing long texts due to the maximum token limit imposed by these models. This constraint can be particularly problematic when summarizing lengthy documents or articles, where the full context might not fit within the model's capacity.

It's my pleasure.

Glad it helped!

Thanks Akshay. Glad it helped!

Glad it helped 😊

Thank you! Glad it helped🤓

Thanks for your kind words! Sure I will do it.

Yes same thing i want ​@@AboniaSojasingarayar

Here the tutorial link to Deploying a Retrieval-Augmented Generation (RAG) in AWS : kzbin.info/www/bejne/nZrGpJVvpZyooJY