PandasAI - Generative AI Capabilities in Pandas via OpenAI ChatGPT / LLM models

In this video, we'll take a quick look at a new library called PandasAI - this library enhances Pandas with capabilities from OpenAI's large-language-models, allowing you to ask questions and pass prompts to your Pandas DataFrames, and get back results - without writing any additional code!
For example, we can ask a question to a DataFrame, and get back text, numerical outputs, charts/graphs, and even Pandas Series vectors.
☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲:
To support the channel and encourage new videos, please consider buying me a coffee here:
ko-fi.com/bugb...
𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮:
📖 Blog: www.bugbytes.i...
👾 Github: github.com/bug...
🐦 Twitter: / bugbytesio
📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
pandas-ai: github.com/gve...
Titanic Dataset: github.com/dat...
OpenAI API Keys: platform.opena...
#python #llm #pandas #artificialintelligence #datascience

[@Davidkiania]

This is awesome content valuable and timely I love it ... and well done.

[@bugbytes3923]

Thank you !

[@emmanuelkolawole6720]

How does it handle large datasets? Two million rows and above?

[@silkogelman]

Super interesting! Prompt based visualisations! 🤩 Combining your own datasets with LLMs has so many cool applications. I love that you check the accuracy of the LLM answers. Thanks BugBytes! 🙏

[@bugbytes3923]

Thanks Sil! 🙏 Always important to verify the output of these LLMs… for now anyway, until they truly take over 😂

[@fernandtape9363]

Good one! I'll try it later. Are other languages supported in the prompt?

[@bugbytes3923]

Thanks! Good question - I haven't tried this out tbh. But the prompt sends requests to Open AI models, so the answer will depend on whether these models support your language I think.

[@fernandtape9363]

@@bugbytes3923 I've tried chat Gpt in French and Chinese before. So I guess It could answer. Get to try to confirm.

[@thirukumaran6780]

getting this error tell me how to fix it UnicodeEncodeError: 'latin-1' codec can't encode character '\u200b' in position 7: ordinal not in range(256)

[@lavanyavijayan9227]

I have multiple CSV files of different dimensions. Is there a way to load all of them and get the result? Excellent video btw!!

[@uroosaimtiaz]

Very cool! Wasn't sure how AI will continue to change how we program, but I didn't expect this. I wonder what is next :0

[@ArsenalvsEverybody]

why data visualization doesnt work in my jupyternotebook? when i write prompt = chart the survivor count for each sex

[@joseluisbeltramone599]

Excellent video, comprehensive and clear explanation. Thank you!

[@bugbytes3923]

Thanks a lot José!

[@Gowtham25]

Is there any way to load our own LLM model? If so, it would be great.

[@noamsiegel]

I am looking for the same thing, please don't hesitate to let me know if you find anything. For now, there is a way to do so using the gpt4all package and models.

[@greggy1872]

can your DF be multiple excels or CSV? I would like Panda to run across multiple files and tell me correlations and potential links between files

[@nitinsingh430]

Why am I getting different outputs for the same dataset and same prompt?

[@bugbytes3923]

ChatGPT is not deterministic, so the outputs will be different each time - the language model behind that PandasAI calls will potentially generate different output at each call. You can pass a "temperature" argument to the OpenAI object to help control how random vs. how deterministic the output is. See the following link for more: gptforwork.com/guides/openai-gpt3-temperature

[@LifeTravelerAmmu]

Hello @BugBytes .. Great Thanks !!!! How can i use MYSQL database instead of using dataframe , csv files.... If it possible ... please let me know ... Thank you !!!

[@bugbytes3923]

Thanks! You could try pulling the data from your MySQL table into a DataFrame - see this function: pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_sql.html

[@hariomgupta2010]

HOw to show chart in django html templates.

[@shekarshekar4988]

hey Although I am using Paid API key, the API is taking a long time to respond. is there any chance we can improve the Performance?

[@bugbytes3923]

This is in the hands of OpenAI - hopefully their platform can improve its performance over time.

[@mehsaniphysicsmathmatics2147]

You are amazing! Thanks for all the beauties you represent to the world.

[@bugbytes3923]

Thanks a lot!

[@emmanuelkolawole6720]

This is the best project in the AI industry till date

[@bugbytes3923]

Bold claim! 😆 But yeah, definitely a cool project!

[@mtmanalyst]

Excellent!!! Thank you

[@bugbytes3923]

Thanks a lot!

[@khanna-vijay]

Well explained.

[@bugbytes3923]

Thank you!

[@reneschwabe8437]

Very interesting. Unfortunately not for business data cause OpenAI gets the data. Would be better with a local LLM.

[@willosullivan1986]

In your experience, how long does it take to build an in-house LLM to serve as a autonomous data analyst?

[@noamsiegel]

Let me know if you find how any other ways besides hosting a gpt4all model and using its package.

[@mtmanalyst]

Very nice demo Thank you

[@bugbytes3923]

Thanks a lot!