Chat with an image | LangChain custom tools tutorial | Python Streamlit | Computer vision

Code: github.com/computervisioneng/...
0:00 Intro
0:54 Start
2:29 Project overview
8:13 Main process
12:38 Auxiliary functions
27:06 LangChain custom tools
35:58 Create agent
51:32 Demo
52:40 Let's have some fun!
54:22 Outro
#computervision #python #webapp #computervisiontutorial #langchain #streamlit

[@ComputerVisionEngineer]

Did you enjoy this video? Try my premium courses! 😃🙌😊 ● End-To-End Computer Vision: Build and Deploy a Video Summarization API bit.ly/3tyQX0M ● Hands-On Computer Vision in the Cloud: Building an AWS-based Real Time Number Plate Recognition System bit.ly/3RXrE1Y ● Machine Learning Entrepreneur: How to start your entrepreneurial journey as a freelancer and content creator bit.ly/4bFLeaC All my premium courses are available to the Computer Vision Experts in my Patreon. 😉 www.patreon.com/ComputerVisionEngineer

[@python360]

Very cool project, video deserves many many views! Subscribed.👍

[@ComputerVisionEngineer]

Thank you! Glad you enjoyed it! 😃💪

[@WelcomeToMyLife888]

awesome, your hard work is much appreciated!

[@ComputerVisionEngineer]

😃 Thank you! Glad you enjoyed the video! 🙌

[@ingluissantana]

Looks like a great project!! Gracias!!

[@ComputerVisionEngineer]

Yeah it is a great project to get more familiar with LangChain! 😃🦾 De nada!

[@user-wt3fh6kq5e]

it's great,make the knowledge easier and more interesting,thank you very much.

[@ComputerVisionEngineer]

You are welcome! Glad you enjoyed it. 🙌

[@ishujain9343]

Awesome work Sir 💯

[@ComputerVisionEngineer]

Thank you! 😃

[@sreekartammana]

Waooo, this looks awesome💥

[@ComputerVisionEngineer]

😃 It is a very cool project!! Glad you enjoyed it, Sreekar! 🙌

[@RudraGoswami-wz8hv]

How to fine tune the model on custom data? As it is specific to the training dataset associated with Open AI

[@adnanahmad4168]

Bro please please create a video on how to deploy Yolov8 trained custom model on Nano Jetson. I have submission of my Final Year Project within a week and I can't find help regarding that. Please do it.

[@hiteshram7421]

Great Project😃😃

[@ComputerVisionEngineer]

Yeah I enjoyed it a lot! 😃💪

[@tancheeken]

Hi, thank you for the awesome video. I do have a question. I understand that the concept of building an agent with Langchain. In your example, the function returns the caption of an image. The caption is a man on a horse with a dog. If I were to query the color of the dog, it would not be able to re-process the image and focus solely on the dog, right? If so, what's the purpose of using Langchain agent and LLM? Wouldn't it be better to image-to-caption and store the caption to normal database? Or am I missing something here?

[@ComputerVisionEngineer]

Hey, this tutorial is an example regarding how to use LangChain in a computer vision project. I agree we could continue working on this project and add more features to it. 💪🙌

[@AlessandroTogni]

It would be awesome to add a functionality that allows users to edit the uploaded image via prompt. "Cut the [detected object]", "Change the lightning from day to night", etc

[@ComputerVisionEngineer]

Yeah, it would be awesome to add additional image processing functionalities! I will continue improving this project in future tutorials. 💪💪

[@mrsmile9399]

Love your videos Is there an alternative to chatgpt that you recommend from huggingface?

[@ComputerVisionEngineer]

Take a look at HuggingChat. 😃🙌

[@rishabhinc2936]

I have a question ? What if we dont create object detection and image caption tool?? Will it still answer our query related to object detection and captioning ???

[@ComputerVisionEngineer]

Nop, without those tools it won't answer any query related to object detection and captioning.

[@santhosh1387]

sir , you have put that lane crossing detection video in private can you please put it back sir , my work depends on it and that video has helped me a lot , thank you

[@ComputerVisionEngineer]

I am preparing a more recent version of the lane crossing detector 😃. It will be available soon! 🙌💪

[@VAIBHAVCHAUDHARY-fl1hn]

Can we do this by using yolov8 and using our custom dataset and asking about that product ?

[@ComputerVisionEngineer]

Do you mean creating a custom tool to perform object detection with yolov8? Yes, it is possible. 🙌

[@connectrRomania]

include installation of transformers in your requirements from the main hugging repo

[@ComputerVisionEngineer]

Oh, I missed it! Thank you for the heads up! I will update the requirements file shortly. 🙌

[@ishasingh1039]

Hello , i followed this tutorial and made the app. But for whatever question i ask , its response is caption ( except for object detection) . Like when I asked it to generate story , it returned the caption of the image . I have done everything as same as shown here

[@ComputerVisionEngineer]

Thank you for your feedback, I will try to test it soon and I will update the code if needed.

[@rohitpillai2307]

I have faced the same issue@@ComputerVisionEngineer

[@tnztanim1623]

Can we do this with free hugging Face model instead of Open Ai? Open Ai requires subscription.

[@ComputerVisionEngineer]

Yes, we could use a free alternative, instead of openai. 🙌

[@tnztanim1623]

​@@ComputerVisionEngineer will you plz implement something instead of openai? I really liked the project and tried my best but couldn't make the project run with a free model.

[@allies4183]

Which model have you used?

[@ComputerVisionEngineer]

I am using the model facebook/detr-resnet-50 for object detection and Salesforce/blip-image-captioning-large for image description. 💪

[@rohitchan007]

I'm getting this error. PermissionError: [Errno 13] Permission denied: 'D:\\Courses\\Computer_vision_engineer\\Ask_image_question\\tmp3chlmrdj' I have granted read write permission to the folder. Need help

[@ComputerVisionEngineer]

What is your OS?

[@rohitchan007]

@@ComputerVisionEngineer Windows

[@rohitchan007]

@@ComputerVisionEngineer I fixed it. it was due to NameTemporaryFile() issue. Replaced with with open("temp.jpg", "w+b") as f:

[@ComputerVisionEngineer]

cool, thanks for the update!

[@rohitchan007]

@@ComputerVisionEngineer no problem. I really love the projects you're working on and sharing the knowledge