Create ChatBot Based On The Data Feed By You - GPT-Index | OpenAI | Python

[@njorogekamau3820]

This tutorial truly stands out from the rest! After struggling with coding along other KZbin tutorials on GPT-index and langchain, I finally stumbled upon this gem, and behold, my code worked like a charm. Thanks for sharing your expertise and making it easy to follow. You're a lifesaver!

[@glebp7020]

Thank you, Shweta. I was able to get something working based on your code. Please note that lots of libraries are now outdated (gpt_index is now llama_index). My complete code (works 100%!): from langchain import OpenAI import sys import os from llama_index import SimpleDirectoryReader,GPTListIndex,GPTVectorStoreIndex,LLMPredictor,PromptHelper,ServiceContext from llama_index import StorageContext, load_index_from_storage def create_index(path): max_input = 4096 tokens = 200 chunk_size = 600 #for LLM, we need to define chunk size max_chunk_overlap = 20 prompt_helper = PromptHelper(max_input,tokens,max_chunk_overlap,chunk_size_limit=chunk_size) #define prompt llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-ada-001",max_tokens=tokens)) #define LLM docs = SimpleDirectoryReader(path).load_data() #load data service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper) vectorIndex = GPTVectorStoreIndex.from_documents( docs, service_context=service_context ) vectorIndex.storage_context.persist(persist_dir="storage") return vectorIndex def answerMe(): storage_context = StorageContext.from_defaults(persist_dir="storage") index = load_index_from_storage(storage_context) query_engine = index.as_query_engine() q = "What is the best plan?" print(q) print('------------') response = query_engine.query(q) print(response) #create_index('data') answerMe()

[@shweta-lodha]

Thanks Gleb. I have created few videos on how to fix these broken ones and few more are on the way. Please feel free to let me know, if anything else is not working. I will try to provide solutions at my earliest convenience. Thanks once again.

[@stalinamirtharaj1353]

Great bro... It was helpful to test it out.

[@sumankalyanghosh645]

@@shweta-lodha Currently I am getting RateLimitError with whatever account I try. Is there a way I can bypass this and test out

[@sumankalyanghosh645]

With the above code I have tried as well mentioned by Gleb.

[@deepaksartape9502]

Please cover UI part as well in one of the coming video. Thank you so much in advance.

[@rayfellers]

As someone just learned how to use GPT-4o Mini to write python code, your explanations are well done.

[@Spaz42]

Thank you so much for this! You made something very intimidating to me super easy to understand. I am very grateful for the time and effort your put into this video. Thanks a lot!

[@MoturuPraveenBhargav]

Ms. Shweta, your tutorials are superb!! Stands out first from the rest.

[@shweta-lodha]

Thank you! Cheers!

[@DandaFundas]

Thank you Swetha. It can’t get any better than your explanation with show and tell 👍

[@usmanahmed1073]

Very good explanation. You speak very calm and make audience to follow all step in detail. Keep good work

[@mangesh29506]

Your instructions are really good, thank you so much Shweta Ji

[@jeffersonhope2668]

Outstanding preparation and presentation. Thanks so much!

[@shweta-lodha]

Glad you enjoyed it!

[@congtycophanxaydungo.m.d1735]

thanks for your contribution, it's easy to understand for a beginner like me. You can continue to make the next video to guide the question and answer continuously like chatgpt.

[@ddsharma2327]

Question about execution flow: Hi Sweta - Thanks for a very helpful tutorial. Trying to wrap my head around the execution flow. For example, createvectorIndex probably sends the document to openAI servers, the servers return embeddings that are then stored locally. During answerMe - is the prompt sent back to openAI for embeddings? Does answerMe again send document embeddings back to openAI servers? How much is done in the local process (on PC) vs how much is done by openAI servers? Thanks.

[@shweta-lodha]

One way to understand this is by disabling the internet connection once you have embedding stored on your local device and then try to call answerMe. Doing this will clear all your doubts 😊

[@faisalsaddique3323]

This is really useful, Shweta! Thank you so much for making such an awesome content.

[@shweta-lodha]

Glad you think so!

[@LearnWithSibin]

I noticed that in the implementation shown in the video, the GPT model is being called every time a user prompt is received, and this can be expensive in a real-world scenario where the application is serving multiple users concurrently. Each request to the GPT model requires a certain number of OpenAI tokens, and this can quickly add up and become expensive.

[@shweta-lodha]

Indeed! For production scenarios, you have to extensively used vector database

[@fascinatingstories101]

Thank you. Is there a way to capture user inputs like contact details (email, name etc)? That would be really useful.

[@SmritiChaudhary-k5b]

Hii, I am facing this error RetryError: RetryError[] while calling answerMe function, can you please help me out .

[@M-ABDULLAH-AZIZ]

having data in a file and real time embeddings vs embeddings in a db for chatbot for an application (provides information about an application)?

[@FnKnowBot]

Hi Shweta, Awesome video, great learning. Thanks. Just one concern - when we grant access to ChatGpt to our custom data stored in our machine, then is there a risk that the data can be copied/ used by users of ChatGPT or Open AI or otherwise?

[@mikhailtrufanov9356]

Hi, thank you for the video! I have tried this code to answer questions based on data about a very specific product. It answers well, however, it answers general questions as well (such as "What is Earth?") having no information at all about this in the files provided. How can I make this code answer based on the information that I provided only?

[@shweta-lodha]

Thanks for watching Mikhail. Give a try to my this video, it gives better results - Use Your Locally Stored Files To Get Response From GPT like ChatGPT | Python

[@harshilpatel323]

Hi Shweta, that was a great tutorial! However, I have a question. Just like you used the custom data from the local disk here, How can we use the data from aws/elasticsearch?. I have a huge database (About 20 million records) which our employees access via querying on elasticsearch, and If I wish to create a custom chatbot trained on that data, how would one achieve that?

[@maggynewnham]

Hi Shweta - I am planning to follow this tutorial, it looks amazing, I was wondering if you know if the local data we used is kept and is local only or if any of the data gets make public or goes back to OpenAI?

[@shweta-lodha]

Thanks Maggy, glad you find it useful. Data will go to OpenAI servers

[@atony229]

Great Video, you are inspiring me to start learning how to code. I am doing this as a project to see if I can follow along withyou. Thank you.

[@shweta-lodha]

Best of luck!

[@tchalaful]

awesome !! very inspiring, thanks a lot for your work !

[@bhagatsingh..]

Very nicely presented its a marvel

[@shweta-lodha]

Thanks!

[@shweta-lodha]

GPTIndex Medium link : medium.com/@shweta-lodha/create-chatbot-based-on-the-data-feed-by-you-gpt-index-openai-python-772291576bee LlamaIndex Medium link: shweta-lodha.medium.com/create-chatbot-based-on-the-data-feed-by-you-gpt-index-llamaindex-openai-3efd7abe3ed9 For PromptHelper and documents error: kzbin.info/www/bejne/e2epmmWQi9CCZ7c

[@sevarakenjaeva7138]

your video and medium source article do not sync up. you mention createvectorindex in one place and createindex in another. maybe you should take a look as this does not help anyone who is trying to follow ... watching video and copying code from medium.

[@rajparmar6420]

hi shweta, does this take structured data like csv or excel for manipulate any data?

[@sqlsql4304]

Very good tutorial, one question if have some logs which contains sensitive data like ip etc, if I feed the logs to the llm will my data move out of my system to the llm model provider's server. If so is there a security threat. I have a presentation to make on using llm to my company and for sure this question will rise, any feedback will be appreciated

[@shweta-lodha]

Thanks. Yes, it will go to model provider’s server and it could be a security threat. If privacy is your concern, then my suggestion would be to use Azure OpenAI

[@giadavolpin3564]

Thank you so much Shweta! You got me so passionate on the topic. After completing your tutorial how do I move this into a chatbot that I can bring to my app? I’m stuck

[@shweta-lodha]

You need web UI

[@fakeshirin4642]

@@shweta-lodha can you drop a tutorial on that because that would be very helpful

[@lifedesignguru]

Amazing tutorial, thank you!!

[@pauldad1]

This looks like a brilliant tutorial thank you. Please excuse the silly question, but where are you editing your code, I opened IDLE but it must be some other editor/console? Thank you again.

[@shweta-lodha]

I am using Visual Studio Code (VS Code). You can install it as it very easy to use. Please make sure to install Python and Jupyter extension from Extensions panel.

[@pauldad1]

Amazingly helpful, thank you!

[@naveennoelj]

@shweta-lodha, wonderful video and article. Thank You. Please keep up the good work. One qs: is there a way to implement the same solution using a web based solution - angular, react etc.

[@shweta-lodha]

Thank you so much Naveen, glad you liked it. I was thinking to try this out but couldn't find API endpoints. Please let me know, if you come across any such documentation.

[@tuyenhuan-i3y]

Thank you very much. I really hope you will have more useful videos like that.

[@shweta-lodha]

Thank you, I will

[@danish1794]

Hello Shweta Lodha, I have a couple of questions. Firstly, do you have any ideas on how we can reduce the cost of tokens? It would be really helpful if you could make a video explaining any potential solutions. Additionally, as a learner, I'm facing issues with expired APIs. Do you have any suggestions on how I can resolve this problem? Thank you in advance for your help.

[@shweta-lodha]

I have already created a video on cost factor, please check my playlist. I didn’t understand- what do you mean by expired API

[@darkmode4467]

What is the token for? Is it limited to ask the bot? Is there a way it can be unlimited?

[@SGSaisreenath]

@Shweta this is brilliant! as usual top notch like any of your other videos, thank you so much for taking time doing this. One question here, can you please provide the differences in the parameter if the same had to be achieved through Azure Open AI with openai.api_base, openai.api_type, openai.api_version, deployment_name - I have been trying to meddle with this but no luck so far..

[@shweta-lodha]

Sure, I'll cover this in my Azure OpenAI series, which I started recently :)

[@prem280182]

wonderful. you made it look so simple.

[@shweta-lodha]

Thank you! Cheers! If you can't make things simple, it means you yourself didn't understand :)

[@Priyanka-js8zl]

Thankyou for this detailed tutorial, so you asked 2 question from it, can you please share how much openai credits does it used in this complete operation? that would be really helpful

[@shweta-lodha]

It should not be that much. I'm uncertain about the exact number as I sent multiple requests around same time frame :(

[@dealersagent]

Thank you for this great video!

[@oreos_overdose]

Hello and thanks for the video. Very useful. Is it possible to use a sphinx generated website (essentially a handful of html pages) as the resource documents instead of a plan txt file?

[@satishkumar-ir9wy]

I have not found such a detailed explanation of Open AI based Chat bot, Thankyou for Sharing. Can you please make a video about how i can feed my NLP based ML model to create a Chat bot based on ChatGPT.

[@shweta-lodha]

Yes, soon

[@hammidfire]

Does it have to be a txt file? What if I have a csv data table? Also does it have to be in a directory?

[@hammidfire]

Do you think I should have the script convert the csv to a text file first?

[@shweta-lodha]

It is not mandatory to take text file. You can use other file types too provided you are able to read them and convert the text to vector. No need to put in directory, if it is single file. But in that case, you have to look for different function.

[@AliRaza-u3u1m]

Hi Shweta, You did a great job. Can you please help me how to set that environment on which your are doing code.

[@shweta-lodha]

Sure. Drop me an email and we will connect

[@sriramyapriya4110]

Hi, thanks for the great video. I'm a beginner. How do I proceed now that gpt_index has been replaced by llama_index? I'm getting the error "No module named 'gpt_index'

[@shweta-lodha]

Yes, there are changes. Please refer my How-To-Fix video, it has the fix.

[@michaelxue5335]

Exactly what I need! thank you

[@prudhvi70]

Any plans for coming up with part 2 for this with custom data that needs to be continuously indexed in the background?

[@shweta-lodha]

Will plan soon. Thanks Prudhvi for the pointer :)

[@spicytuna08]

there is no longer GPTKeywordVectorIndex. There is GPTSImpleKeywordTableIndex() are they the same?

[@EllenSimpson-w9n]

Hi Shweta, this tutorial is amazing! I have one question after running the bot, on my OpenAI usage I am getting text-davinci requests as well as text-embedding-ada-002-v2. Any thoughts on why I am getting the davinci requests?

[@shweta-lodha]

I didn’t understand your question completely. Your embeddings would be using text-embedding-Ada-002-v2 model. Don’t you want to use this?

[@EllenSimpson-w9n]

@@shweta-lodha Thank you for your reply. My usage looks like this does this make more sense? 10:55 PM Local time: Apr 26, 2023, 8:55 AM text-davinci, 2 requests 3,805 prompt + 75 completion = 3,880 tokens 10:55 PM Local time: Apr 26, 2023, 8:55 AM text-embedding-ada-002-v2, 1 request 8 prompt + 0 completion = 8 tokens

[@EllenSimpson-w9n]

The bot is also able to answer questions about topics that I have not fed it. I think that it is accessing other sources than what I have fed it.

[@Gapxpag]

Hi Shweta, amazing job, hope you can help me: when running vectorIndex = create_index('Knowledge'), I get an error message: Output exceeds the size limit. Open the full output data in a text editor. maybe you know why?

[@shweta-lodha]

Hope 'Knowledge' directory exists in the same path, from where you are running your script.

[@joserodolfobeluzo3100]

HI Shweta Lodha! First... Thanks a lot for the tutorial! :) I have a problem with an error. Can You help me? I wrote the same code and use the same data. But when I try to run the code, this error appear: TypeError: __init__() got an unexpected keyword argument 'documents'. I Printed the variable and it contains the book text... The error is on the line: vectorIndex = GPTSimpleVectorIndex(documents=docs,llm_predictor=llmPredictor,prompt_helper=prompt_helper) I looked for it on internet but I didn't understand why there's a problem with the arg "documents"... :/

[@joserodolfobeluzo3100]

I solved! :D For anyone that had the some problem I had, here is my function: def createVectorIndex(path): max_input = 1024 tokens = 256 chunk_size = 600 max_chunk_overlap = 20 #define LLM llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-ada-001")) #load data docs = SimpleDirectoryReader(path).load_data() service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, chunk_size_limit=512) vectorIndex = GPTSimpleVectorIndex.from_documents(docs, service_context=service_context) vectorIndex.save_to_disk('vectorIndex.json') return vectorIndex The problem is: I removed this line: prompt_helper = PromptHelper(max_input,tokens,max_chunk_overlap,chunk_size_limit=chunk_size) Should it be a problem? It worked here without it and with other texts... If it's a problem, could anyone explain the part of the code?

[@shweta-lodha]

Please have a look at documentation, in case something has changed. At the time of this video, it was all good.

[@shweta-lodha]

You rock! I just created a video on how to fix this :)

[@RitvikGautam-p9z]

Hi shweta please tell which extensions you have used in VS code for this video

[@shweta-lodha]

Jupyter

[@SanjanaMagar-d5q]

hi Shweta, i tried this code on my end but in { vectorIndex.save_to_disk('vectorIndex.json') } is giving me an error so i tried { index.storage_context.persist('vectorIndex') } this is not creating a vectorIndex.json file on my system its creating a folder name vectorIndex.json inside which i has 4 json file docsstore, graph_store, index_store and vector_store.. can you pls tell me where i am going wrong

[@IgorBožić-o8n]

Thank you Swetha for this easy and well-formed tutorial! Is there a way to connect this bot to a WordPress site? I have created a custom bot explained in this tutorial which uses a dataset from a knowledgebase site. Now I'd like to connect that chatbot to that site so that users can directly communicate there, in addition, to seeing articles in a standard way.

[@shweta-lodha]

Thanks and glad you liked it. Did you try plug-in?

[@gauravrathi5028]

Hi Shweta, Very useful tutorial and I followed the same but m always getting the response None for any question, However I kept the document at same place where you have mentioned. Could you please help me out, what could be the reason for the same.

[@shweta-lodha]

Please give complete/absolute path and try it. If still it doesn’t work, then issue is not with the input file

[@youwang9156]

appreciate for your work, just wonder if this method works for extracting specific number ? like i want to extract some price for mutilple products

[@shweta-lodha]

Yes, I tried this and in most of the cases, I got accurate output.

[@youwang9156]

@@shweta-lodha thank u so much for reply

[@noronhasanjay]

Thank you. Very useful.

[@thinanadl4939]

Hi @shweta, can this bot integrate with any database..?

[@rezahamzeh3736]

A tutor that is looking for the result of a func without calling the func :))))))!!!!!

[@shweta-lodha]

Haha 😃

[@leoheise9967]

hey, any tips on how to fine tune a model based on a very large pdf document without the " " to split prompt/resolution? I thought maybe have a script break down in every question mark? Or is there some other way?

[@shweta-lodha]

I'm going to publish a video on this today. Stay tuned!

[@mneama4383]

Thanks a lot great video , just want to know if all my data is stored in sharepoint how can i take them all into the same mechanizm u did thanks again for this great tutorial

[@mcsquared361]

This was awesome, thank you so much!

[@shweta-lodha]

Glad you enjoyed it!

[@alexdomla]

HI! This is a very helpful tutorial. I have a question: How would you bring this to a website? I mean, creating a chat website where you can ask questions and the bot answers based on your custom data. Is it possible? Thanks in advance :)

[@shweta-lodha]

Glad you find it useful. Either you can create a website or you can create a widget and plug it onto your website

[@shivampatel4179]

ImportError Traceback (most recent call last) Cell In[29], line 1 ----> 1 from gpt_index import SimpleDirectoryReader, GPTListIndex, GPTSimpleVectorIndex, LLMPredictor, PromptHelper 2 from langchain import OpenAI 3 import sys What should I do for this error???

[@shweta-lodha]

Please refer my how to fix video entitled as breaking changes. Things are broken because of API enhancements

[@adityakadam2256]

That's a great video thanks for sharing this. I have a question if you don't mind. How different is this method rather than using Open AI APIs for chat completion, embedding and completion? Thanks.

[@shweta-lodha]

Here you can save the vectored index locally and re-read it.

[@adityakadam2256]

@@shweta-lodha Thanks a lot for a prompt reply. One last question. The method you explained under the hood it uses text-ada-002 model for embedding and text-davinci-003 for completion, right?

[@shweta-lodha]

Yes, you're right :)

[@adityakadam2256]

@@shweta-lodha thanks for the response and clarification 😁

[@ralphanthony5198]

Hello, Shweta Lodha! Great job. I really love your videos. And I have a quick question: Do I have to have a paid GPT plan in order for the code to work?

[@shweta-lodha]

No necessary. It would work with your free account too, assuming you have required credits.

[@ralphanthony5198]

@@shweta-lodha Thank you so much., Shweta. But I will get me a subscription today anyhow. I don't want to ran out of credits. With Love from Tijuana. You are great!

[@mayurbhor2231]

Thanks a lot . Can we use llama_index instead of gpt_index ?

[@shweta-lodha]

Yes, you have to use as gpt_index is deprecated. You can refer my GPT_Index breaking changes video for that.

[@mehdirizvi3305]

For me, this code was throwing an error. If you update the code by first importing ServiceContext from gpt_index (along with the other stuff you were already importing) and then update the code by adding the following lines, '-----#load data docs = SimpleDirectoryReader(path).load_data() service_context=ServiceContext.from_defaults(llm_predictor=llmPredictor,prompt_helper=PromptHelper) #create vector index vectorIndex=GPTSimpleVectorIndex.from_documents(documents=docs,service_context=service_context) --- it should solve your issue. P.S: I used this code from another one of your videos :))

[@shweta-lodha]

Please refer my How-To-Fix video. All this happened because of breaking changes in API 😊

[@syedrizwanali5212]

Hi Mam, Does this work today also? Gpt_index does not have any function?

[@shweta-lodha]

Perhaps not as lot many things have changed recently, in terms of SDK and API. Lot many functions are renamed and moved here and there.

[@syedrizwanali5212]

@@shweta-lodha Thank you for the reply. Would you have any latest video on same topic? Or would you know if someone have created on this topic in simpler way like you had done here.

[@smudgepost]

Very good video thank you. I must have blinked and it was over! Which specific function uses langchain?

[@shweta-lodha]

LLMPredictor

[@rickp.5778]

I appreciate your video very much!

[@shweta-lodha]

Glad it was helpful! Stay tuned for more...

[@SPonharshitaP]

could this learn from new unlabeled questions provided by the user ? Is this a trainable chatbot which could learn new things every time user asks a new question

[@shweta-lodha]

No

[@smart-sg5cs]

hi shweta your tutorial is really helpful and your knowledge on open Ai. i have also emailed you please reply

[@shweta-lodha]

I'll try to respond at the earliest.

[@nikunjpatel6522]

also getting this as well Output exceeds the size limit. Open the full output data in a text editor

[@shweta-lodha]

This is because you have restricted the output length for Jupyter. You can open it in notepad to see full error. Not a big deal 👍🏻

[@shahbazzaman4143]

I have implemented code, works perfectly, Thanks for that, I have question, it is just providing answer from context, Is gpt_index is all about context ? can not get general information like, who is trump or any other information? if there is way please help me with in this code

[@shweta-lodha]

This video is about how to get answers based on context. If you want general info, then you can simply achieve that using OpenAI

[@karthikeyand6678]

@Shweta, Awesome video and Thanks for sharing.. I have data in CSV which has many text columns and many rows. I want to build chat application based on the data. Can you please let me know how can i implement this .

[@shweta-lodha]

You can read CSV in memory, chunk it and you’re good to go

[@liyuanzhu6111]

Thank you for this wonderful video🎉. I have a question when I was trying it. I run the codes to create vector index,but I cannot find it in the OSDisk. There is no error😂

[@shweta-lodha]

If you didn't provide complete path, then it must be in your current directory. Current directory is the one, from where you are running your script.

[@iulihardt9368]

Hi, I done it with sucess! really thank you! However its possible to start one chat as the last part of the video and save where it stop and come back in other moment?

[@shweta-lodha]

Glad you find it useful. You need to save your chat history and refer it whenever you're starting your conversation next time.

[@narjessabbassi124]

Hello, this tutorial is really amazing but i was trying to replicate the same work but got an error when executing the function createVectorIndex saying that the function __init__() in GPTSimpleVectorIndex got an unexpected keyword argument 'documents' did anyone get the same issue ?

[@shweta-lodha]

Please check my How-To-Fix video

[@srujanashankar9517]

This helped! Thank you

[@partheshtiwari1808]

How to restrict the bot to only search for answers based on data provided or how to make it generate an error if the question asked is outside the available data

[@shweta-lodha]

For this, you need to tweak your prompt.

[@algarve-video]

is possible to creat a simple page with a search page that request for the prompts?

[@shweta-lodha]

Yes, it can be done

[@kareemmatouk]

Hey so im wondering How I can create a interface once I have made my language model.

[@shweta-lodha]

You can go for flask

[@Mohammed-lo7xr]

Thank you for the tutorial, it was great! How would I deploy this app to the internet onto my own custom domain?

[@shweta-lodha]

You need to create web app for that 😊

[@radbirden6111]

Hi thank you for a great video. Is there a way we can combine the code and add gradio ui ?

[@shweta-lodha]

Yes, you can.

[@cmaly6167]

Are there any other vector index options, I'm not getting the expected results.

[@shweta-lodha]

There are few vector databases you can use like pinecone, redis, etc

[@jmisc]

How do we load multiple data, and will the script and OpenAI remember the prior conversation?

[@shweta-lodha]

For memory, you need to change this implementation a bit

[@Bludgeoned]

Hi, it seems I can’t find VectorSimpleIndex, has it been replaced by VectorStoreIndex?

[@shweta-lodha]

Please check my latest video: How to Fix[GPT-Index]: Fixing GPT-Index Related Broken Pieces

[@Bludgeoned]

@@shweta-lodha because it's llama_index right?

[@shweta-lodha]

Absolutely!

[@farhanjunaid9180]

Can you explain how you setup your IDE here?

[@shweta-lodha]

I am using VS Code and installed extensions for python and Jupyter

[@zhiyingwang1234]

Is my own data imported to openai system in this case? I don't want to breach the company's data confidentiality code.

[@shweta-lodha]

Yes

[@SergeyBazyliuk]

ValueError: One of documents or index_struct must be provided. what i need to do with this error?

[@shweta-lodha]

Looking like it is not able to generate JSON properly. Please validate your docs

[@ibrahimjaballa2706]

I am a beginner When using gpt chat, do I have to get the API from it, or can I get it from anywhere?

[@shweta-lodha]

Please get it from Openai.com

[@MunchildAbaga]

Thank you 🙏🏽 ❤

[@shweta-lodha]

You are so welcome

[@educationdelightenglish3819]

madam, I am from India, after doing tons of research on ChatGPT API ,this is the best! I am working on Linux platform and have Jupyter notebook, will it work in that env?

[@shweta-lodha]

Yes, it should.

[@_Darth_Panda_]

for this line vectorIndex = GPTSimpleVectorIndex(documents=docs,llm_predictor=llmPredictor,prompt_helper=promptHelper) I am getting the following error __init__() got an unexpected keyword argument 'documents'. Any tips?

[@shweta-lodha]

Please have a look at documentation, in case something has changed. At the time of this video, it was all good.

[@_Darth_Panda_]

@@shweta-lodha You are right. There appears to be some change in new version of gpt_index. This modified function code worked for me. Thanks for this amazing tutorial. was looking for something like this for a long time. def create_index(path): max_input = 4096 tokens = 200 chunk_size = 600 #for LLM, we need to define chunk size max_chunk_overlap = 20 #define prompt promptHelper = PromptHelper(max_input,tokens,max_chunk_overlap,chunk_size_limit=chunk_size) #define LLM - there could be many models we can use, but in this example, let’s go with OpenAI model llmPredictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-ada-001",max_tokens=tokens)) service_context = ServiceContext.from_defaults(llm_predictor=llmPredictor, prompt_helper=promptHelper) #load data - it will take all the .txtx files, if there are more than 1 docs = SimpleDirectoryReader(path).load_data() #create vector index vectorIndex = GPTSimpleVectorIndex.from_documents(documents=docs,service_context=service_context) vectorIndex.save_to_disk('vectorIndex.json') return vectorIndex

[@shweta-lodha]

Thanks Darth. Lot many people are getting this error. Hence I created a new video explaining this change. Cheers!

[@Cantoneseaudiobook]

thanks for the amazin video, can it read podf file instead of txt?

[@shweta-lodha]

Yes you can! But there would be different class/API/function to do so.

[@vy1330]

Mam it is use this without inter ah mam

[@shweta-lodha]

No

[@reymardp]

Hi, just want to ask if the data that I will feed will be exposed to public or to openAI? Is this safe for business? Thank you.

[@shweta-lodha]

IT would be exposed to OpenAI. If you are concerned about security part, I would recommend you to check Azure OpenAI.

[@Shailukool]

Hii need your help. I have followed exact same steps but facing below issue. I have given pip install gpt_index and got response same as shown in your video then when I am giving from gpt_index import SinpleDirectoryReader code then I am getting error Module not found “gpt_index” I tried uninstalling, re-installing and checking on chatgpt but failed for everything. Plz plz plz plz help

[@shweta-lodha]

Please check my most recent How-To-Fix video. It contains the solution

[@aaron6822]

i got this error "TypeError: BaseGPTIndex.__init__() got an unexpected keyword argument 'documents'". dont know if i will get any assistance what the heck.

[@shweta-lodha]

Please refer my How-To-Fix video

[@spicytuna08]

getting this error on line PromptHelper(): chunk_overlap_ratio must be a float between - and 1

[@shweta-lodha]

Yes, you need to change this between 0 and 1 due to recent API changes

[@snowboy1688]

Hi, where I can download the IDE tools?

[@shweta-lodha]

You can just search it over Google - VS code