Building a Chatbot with ChatGPT API and Reddit Data

[@bababear1745]

I'm a professional trainer at Uber and I want you to know that you're one of the best instructors I've ever seen. Keep doing this.

[@Pdruc]

This lady is legit, not like many other “pro data analysts on KZbin “

[@IVNVKNG]

I think you have found your calling. Your professional approach to teaching / tutoring videos are elite.

[@Thuvu5]

Wow, thank you Ivan!! ❤️ I think so too haha

[@aliffnabil5542]

damn this is such a very good project and not to mention the production of this video also!

[@Thuvu5]

Aww happy you liked it Aliff 🙌

[@obc9794]

Very interesting ! It seems so easy watching the video but there are a lot of work behind the scene. Thanks for all your effort !

[@tomaszkaminski3762]

This notebook part was very interesting. It's difficult to show that in this way. Congratulations and thank you!

[@andreyseas]

Great video! Love the peaceful background music

[@jamesgachago8306]

I followed you while learning to use Alteryx. Fantastic work you do everywhere. Thank you so much!🙏🏾

[@vinaymama]

Damn... This is Awesome Mam. With one single video, i have learned lot of concepts and techniques. Thank You

[@Thuvu5]

So glad to hear! Thank you for watching 💛🤩

[@Osteomorphis]

I was literally just talking with someone about this an hour ago at work! Great timing!!

[@Thuvu5]

Ooh that’s awesome Matt! I guess this kind of chatbot can be useful for a lot of things at work too

[@drunkenboat_dark_domain]

I’m a data science intern and I really like this video, thanks you 🥳🥳I’ll definitely try datalore

[@nazneenakram2629]

Omg! I love your idea of chatbot. Now, I am inspired to create my own version. Thank you so much

[@FaruqAtilola]

When you like a video just before you consume its content... 😊

[@Thuvu5]

thank you Faruq! I’m so glad to hear 🤩💛

[@Marc42]

ikr :)

[@cristianmoraaguado596]

7m kkikkkk

[@Davidkiania]

This is superb love the content, the cadence and the explanations. It is designed for someone who's certainly not a newbie and does enough to trigger curiosity and further quest for knowledge. Thank you so much and keep doing amazing work.

[@thaoquach1377]

your informative and helpful video always helps me to figure out a bunch of project ideas. Thank you very much chi Thu

[@aelabassi]

This is so far very good project, well done.

[@bradleycardenas9012]

Amaizing proyect. I love it ❤ Thanks Thu vu

[@Thuvu5]

So glad you like it Bradley!💛

[@timcesar1]

thanks so much really a lovely notebook and the way you put in words and video to guide us I have create mine and is working really thanks for share I encourage you to keep going regards from NYC

[@Thuvu5]

That's awesome! Thank you 💖

[@obwislacipa]

Very interesting. I really enjoy the way you explain !

[@fraineri]

Awesome video! A lot of information but really easy to understand with the format of this video🥰 It helps me a lot for a little project I have in mind but I can't finish putting it together. If context information is, for example, a book. Will GPT be "clever enough" to figure out the answer of some questions about some topic? Should I do some kind of fine-tunning to this type of tasks?

[@Thuvu5]

I think you don’t need to do fine tuning unless the book is about a very rare topic and has a lot of difficult language. I’ve seen start ups who do fine tuning for Chatbots on legal documents for example

[@victorl.mercado5838]

This was an excellent video. Very comprehensive. I learned a lot, and had to subscribe to your channel.

[@Thuvu5]

Awesome, thank you! 🙌🙌

[@khiemfle]

Thanks for the helpful sharing! Besides, be careful with the API Key and secrets shown in your video.

[@alexyoung8185]

I love the project and the video. Great production! Do you know if there are any free alternatives to the chatgpt api? I wanted to do a project for school but the amount of data and money part scares me lol

[@artborovik]

it is the best tech so the answer is no, but maybe some scholarship program could be a deal :)

[@cartoonchan182]

Tbh i was planning to build something like this for the industry I'm working in but was little lost and not sure how good it will be... I think your video will be a good way to start.. haven't finished the video yet let me go back and finish it 🐒

[@DavidSoles]

I love this project. Thanks for sharing.

[@Thuvu5]

You are so welcome!

[@paulchin2593]

Thank You Thu Vu. Amazing presentation

[@_.vassa33]

Literally Liked the video even before watching the video because Thu Vu never disappoints (lol) ...

[@Thuvu5]

This made my day Peggy! Thanks for watching ❤️🙌

[@RatafakRatafak]

Nice commercial for Deepnote :D

[@Adam-uu7iz]

Thanks for the video! Just wondering is I can do all this under the free plan provided by Data Lore?

[@Thuvu5]

Thanks Adam! Yes I believe you can. Only bigger machines options and GPU are not available for free plan, but I don’t think you need it :)

[@markring40]

Thank you! Great video and project 👍

[@mohammed_shabaz]

Wow really great...., could you please make a video on how to create GPT4 plugins.

[@alfredocentarini6241]

muy buena investigacion!

[@NamasenITN]

Video starts at 19:45

[@deenyokabi]

I love your storytelling

[@sawanpalasiya734]

Amazing video and project 👏 👌

[@Thuvu5]

Thank you! So glad you liked it 🙌

[@1Esteband]

Great presentation. Thank you! Please correct me if I am wrong ,it seems to me that when you create the llamaindex in reality is a local repository that will use to search locally so I guess chatGPT is not using it at all to learn or bias its answers.

[@gviacava]

awesome!!! thank you!

[@kevinus2710]

Great video ! I'm curious how the langchain works with OpenAI API, they did get the api_key from OpenAI but in OpenAI API ChatCompletion have only role as system, assistant and user. Not sure how we can add large data like thousand rows of reddit data into the system role.

[@firasnacef001]

Thank you for the detailed process and the video quality. Do you think the model can be finetuned to a language that chatgpt doesn't fully understand yet? (Let's say dialects of chinese or Arabic for instance, but written in latin characters). Does this method create a new vector space representation that will enable the model to understand this new language based on the input data? Or does it have to understand the input language first in order to generate the proper responses on which it will be trained?

[@Jingizz]

Quite hard to wrap my head around this. One question, would it be possible to download the chat history of a discord server and create a bot of a person

[@awesomeexcel]

Awesome Video. I like it so much.

[@hieunguyen-dd1nm]

Thank you. Greate video!

[@JacquesGauthier-t3w]

Great video!

[@gabrielrodriguez3194]

I love these videos..

[@ObservingBeauty]

Very interesting. Thanks

[@tonnoztech]

GPTSimpleVectorIndex has changed parameters in the last version, perhaps an update is needed?

[@RetropunkAI]

Just found this today. Does the Reddit portion still work considering latest change with Reddit? Thank you.

[@neversm6207]

Hello, I’m really interested in going through with a similar project for a textbook that I converted into a txt file but I’m always worried about how much ChatGPT will charge. It looked affordable but I won’t know till I try. My question is, did you only have to pay the .04 -.01 cent as a one time payment or is this ongoing as you continue to use the chatbot, based on how many tokens it takes to give an answer each time? In other words am I only paying once or is this kind of like a subscription based on my usage over time?

[@冷石-r9z]

just amazing!

[@alphonseinbaraj7602]

Really I love it. I am going to try now. Please help me if any problem..thanks

[@juanpasalagua2402]

awesome! Thanks :)

[@sparshsumani2338]

Very Interesting!

[@markrosenberg4369]

All very nice, but how reliable is the Reddit Data?

[@Thuvu5]

It’s more a source of community knowledge, not everything is verified of course 🙂

[@shuvojyotirakshit5808]

Will this q and a system have context memory ? For example if I ask something like what's NLP ? And then in next question I ask give me some examples of it. Will it understand that I am referring to NLP ? I did this same project using embedding models first and then shifting to gpt turbo but I wasn't able to get this continues conversation system (which is required for my specific project).

[@gmnayeem2291]

Just WOW ...!!!

[@diaz072]

Toll Danke!!!

[@eittorres]

great video. thank you.

[@goldenknowledge5914]

Interesting. I kinda trust reddit more than reviews on google

[@yiukins]

Thank you very much for the tutorial. I tried to run your Reddit EDA and Chatbot, but it shows an error, can you please check?

[@Thuvu5]

Hey, thanks for following through the tutorial! I’m not sure what’s the error you encountered. If you need help pls join my Discord community (see link in the channel’s About section)

[@behrouzbeheshti]

If GPT3/GPT4 doesn’t answer correctly (e.g., about the code), simply tell it in the next chat :”the code in your previous response doesn’t work”), or (“the code you provided in your previous response gave me this error: error description”. ). GPT will apologize and tries to give you a better answer 😊

[@gunabalang9543]

Love all your videos ❤❤

[@HappyHogan-jf3ue]

You look pretty good

[@JustAn0therSoul]

wouldnt it make more sense to use gpt3 or lower, as those models can be trained, so you dont have to use 700 tokens for each requests

[@johanbonaparta]

Could you explain this please?

[@JustAn0therSoul]

@@johanbonaparta now every request sends the reddit data with it which uses 600+ token for each call, but some models from openai can be fine-tuned, i assume that you could insert the data there once so it wouldnt use as much tokens for each request

[@johanbonaparta]

@@JustAn0therSoul Now I understand why with my data and just a few request, OpenAi charge me 350k Tokens.

[@syedabushoaib]

lots of love to you

[@annpik392]

hello, I am pretty new here. I very much like the video but struggle with get_emotion function. Any chances the source code is available somewhere?

[@nastaran1010]

I need to have data from a specific range year , unfortunetly in time_filer I cannot provide this. With pushift we can identify range year, it did not work :(

[@phartemah]

Hi Thuvu, I’m also in the Netherland and I’ve been learning data analysis recently. If you don’t mind, can I connect with you?

[@nastaran1010]

my another problem is, I can just 1987 , even limit = 3000?, this is a limitation of API or is my mistake?, how cn i handle to get more data?

[@guimaraesalysson]

Great video

[@DarrenTarmey]

What ide are you using

[@licalgado1]

❤

[@pepperpeterpiperpickled9805]

wisdom of reddit? uh-oh

[@veerutrivedi9791]

Hi, I am trying to replicate your code to practice. But getting an error that module ‘reddit’ has no attribute. I also writing this code on datalore. Tried web search but no success. Would you be able to assist me here please?

[@kritsaphongphuthibpaphaisi1509]

Already subscribed ❤

[@DangVietHa]

I encountered this error when using free plan chatgpt. Can we bypass it if we use pro plan? ``` ValueError: A single term is larger than the allowed chunk size. Term size: 694 Chunk size: 600Effective chunk size: 600 ```

[@DangVietHa]

I can bypass the error by increase chunk_size_limit to 1600, but encountered another error ``` AssertionError: The batch size should not be larger than 2048. ``` Because my input data is too large? And can we have any way to fix it?

[@HardikPatel-ou1bh]

Awesome!

[@md.alnahian4613]

chatgpt is not there as you set the threshold 2 is it should be 1.

[@jaylacsam]

She didn't really ask chatgpt a question that can only be found in the reddit data.

[@kenchu7303]

Will chat gpt 4 replace data scientist?

[@Thuvu5]

We are yet to see 🤔😅

[@DivineDutz]

Where can I find the secret key pls

[@tantomanontroppo8582]

someone knows how to do do this but with falcon 7b from hugging face instead of openAI?

[@applemontea]

Reddit Mad about their free API use for train AI, which now has a value of billions of dollars

[@TuanNguyen-gr4cd]

chị là ng việt à?

[@diaz072]

humm could i do this with google colab? ....

[@Thuvu5]

Yes definitely! Only thing is Colab doesn’t have the built in interactive widgets

[@AnujKumar-dn1lo]

You are very beautiful

[@letsbefunny]

Information overloaded, you should consider explaining thing little slower.

[@manidinesh89]

ValueError: Encountered text corresponding to disallowed special token ''. If you want this text to be encoded as a special token, pass it to `allowed_special`, e.g. `allowed_special={'', ...}`. If you want this text to be encoded as normal text, disable the check for this token by passing `disallowed_special=(enc.special_tokens_set - {''})`. To disable this check for all special tokens, pass `disallowed_special=()`. I encounter this issue. Any resolution?

[@kennylaikl299]

Encountered error at: comments_posts_df_sub['emotion'] = comments_posts_df_sub['comment'].astype(str).apply(lambda x : get_emotion(x)) comments_posts_df_sub

[@kennylaikl299]

RuntimeError: The size of tensor a (532) must match the size of tensor b (512) at non-singleton dimension 1

[@mianelson9489]

@@kennylaikl299 If you're still having this problem, I had to update that part of the code to say : comments_posts_df_sub = comments_posts_df_sub.assign(sentiment=comments_posts_df_sub['comment'].astype(str).apply(lambda x: get_sentiment(x))) comments_posts_df_sub It worked the same but it stopped giving me that error.

[@rocaivan]

Very good! Thank you