Tweet Visualization and Sentiment Analysis in Python - Full Tutorial

[@ramyadav9102]

You just help people in making their minor project free and fully complete in a day. Hat's off. 👏👏👏👏💯

[@LucidProgramming]

Thank you Shikar. If you found the content helpful, you might find the other content on my channel useful as well. If so, please consider subscribing, and thanks for your support!

[@akshayrajiyer]

Thanks for the informative video! Minor change to Part 1, the fetched_tweets_filename should be passed as a parameter when creating the StdOutListener object inside TwitterStreamer.stream_tweets(). This is corrected in Part 2 but wanted to point it out in case folks found it confusing!

[@Claire-ci4vr]

I'm currently struggling with this, I keep getting "TypeError: __init__() missing 1 required positional argument: 'fetched_tweets_filename'". Any advice? Confused about how to pass it as a param. Thanks so much in advance!

[@Sqedabantwana]

​@@Claire-ci4vrdid you manage to fix it?

[@itissrinivasan]

That was quite deep, covered the basics very well and has helped me learn from scratch ! Thank you so much for the depth and for your patience!

[@DaDavid15]

Subtly hilarious "shame on you" omage to the Great Hitchens around 34:00 . Thank you

[@liamfmackle]

Brilliant tutorial. Learning a lot about OOP as well as twitter sentiment analysis, thank you!

[@paulkerrigan9857]

This ain't the way to do OOP, chief. Don't waste time making pointless classes to wrap other classes. Just use a function like a sane person. I've seen too many beginner programmers get into the industry and start trying to force every single thing into multiple classes, bloating code that once required a dozen lines into over a hundred, all to worship OOP. Don't say "it saves time because I'll reuse this class later!" You will never reuse it. That is a lie.

[@nowyouknow2249]

@@paulkerrigan9857 I totally get. It's so annoying

[@zainabelmadhoni6856]

You are a lifesaver. I have a “Social network analysis “ project . And your code matches perfectly what I am supposed to do. Thank you very much 🙏🙏🙏

[@ranaatif943]

plz cn u shr me code o3204022823 whts up no

[@zainabelmadhoni6856]

​@@kellyokios8659​go to twitter developer and you write a request. it takes about a day or two then you get access

[@zainabelmadhoni6856]

@@ranaatif943 there is a link in the description for a github project. all the code is there

[@ranaatif943]

zainab elmadhoni where is link plz share rafrance 03204022823

[@ranaatif943]

15 is last Date university for submitd project 😭😭

[@jayantakumarroy543]

oh angel sent from up above,you know you make my world light up

[@LucidProgramming]

Thanks! If you like the content, consider subscribing to LucidProgramming! :)

[@anne-marieroy8812]

Excellent tutorial, very clear and thorough. Thank you.

[@LucidProgramming]

No problem, Anne-Marie. If you like this content, I hope the content on my channel would also serve you well. Thanks for your comment, and happy coding!

[@_hindu_warriors]

can someone tell me dataset used in this vedio is dynamic or not ??

[@sefatergbashi]

Thank you so very much! This was very comprehensive and satisfying

[@LucidProgramming]

Cheers! If you like what you see, consider subscribing to LucidProgramming! :)

[@paulkerrigan9857]

Amazing. No doubt your next video will be entitled "How to Replace Your Car Engine". Eighty eight minutes will be devoted to picking the correct pair of fluffy dice and two minutes will feature you typing a quick email to your mechanic. I watched this thinking "when are they going to mention the sentiment analysis?"

[@synthefire]

lol. I guess you were never a beginner just evolved straight to advanced huh. Or you don't know how to fast-forward, they even set up cue points in the description. Make your own video :)

[@karthikbankapur3520]

Thank you for this video, everything was explained in detail as it should be , learned something new .

[@vikramchauhan5744]

Explained really well.

[@keshavsharma9380]

Thanks for the code Lucid!

[@novitasariarlim1294]

thanks. You describe in easy to understand words

[@kaushalkumar3017]

Thanks a lot Vincent ... really very helpful

[@Adam-wi4wd]

Hi ! Amazing tutorial. I have a question though : How would you do if you wanted to gather sentiments related to a hashtag rather than a user like in the video ?

[@GustavAgar]

I want to know the same

[@muj6889]

thanks for the video and you get a new subscriber here

[@ReasonToKeepGoing]

You are an angel sent from above! absolutely amazing... Thank you for this tutorial, it was much needed. So in depth, great explanations and really structured. definitely subscribing.

[@LucidProgramming]

I know this is a late response, but thank you! :)

[@ReasonToKeepGoing]

@@LucidProgramming better late than never 😄

[@LucidProgramming]

@@ReasonToKeepGoing Haha, thanks! :)

[@TheDouglasDale]

Excellent video. Thank you sir!

[@simplet3306]

Love you buddy i support you

[@LucidProgramming]

Thank you very much, I appreciate the support!

[@atulkumar781]

Hey awesome work, really appreciate your hard work

[@vivekrajak7474]

Amazing learning, it's like getting Farari for free. Thank you so much❤️......but could you please make a brief video on only sentiment analysis because sometime we get a negative count for a positive tweet as well ☹️

[@esratiras4090]

Thank you very much you are perfect person

[@ronmiakanmode8335]

hi, i am trying to stream live tweets by following the steps axactly as you did but im getting the following error: TypeError: __init__() missing 4 required positional arguments: 'consumer_key', 'consumer_secret', 'access_token', and 'access_token_secret' please help.

[@keirwhitlock1940]

What vim plugins are you using? I'm loving how yours looks.

[@LucidProgramming]

Hi Keir. Thank you for the comment. If you go over to my channel and check out my Vim playlist, specifically the following video, that will give you what you need to setup Vim like mine is. While you're over there, if you find content you enjoy, please consider subscribing! Cheers, and happy coding: kzbin.info/aero/PL5tcWHG-UPH2HrF5M7-IIXK6JSRG0obYo

[@VeeWebCode]

Awesome Video.

[@LucidProgramming]

Thank you! I hope you found the content useful. If you like videos like this, please consider checking out my channel. Cheers, and thanks for your kind comment!

[@KEERTTIK]

Hello world... It's siraj..

[@LucidProgramming]

Haha, he's one of my favorite KZbinrs. Thanks for your comment, and if you like this type of content, please do consider checking out the other videos on my channel. Indeed, I hope that a fellow enthusiast of Siraj may find some entertaining videos. Cheers!

[@KEERTTIK]

@@LucidProgramming i just subscribed

[@LucidProgramming]

Thank you Keerttik, I really appreciate your support! Cheers, and I hope you find the content of my channel to be useful. Have a great day!

[@sumanshu.nankana]

Hello Sir, We have not called the on_data() function. How is it printing the tweets? (around @ 15:00)

[@juhu5504]

I don't have ACCESS_TOKEN and ACCESS_TOKEN_SECRET, only have BEARER_TOKEN. How can I complete this tutorial and stream real time data from Twitter?

[@kitcharizard5281]

Did you find an answer to it?

[@adarshsingh936]

Could you post a video to create a sentiment analyser if we don't want to use Textblob

[@berradi5521]

Thanks for your effort

[@ravibansal8012]

Thanks a lot sir fir this excellent tutorial. I have a doubt that what to do with that twitter streamer class and hash tag list and how these can be used to create time series plots of live tweets data

[@SURAJKUMAR-fp6zb]

how to get tweets using twitter id please help ...

[@Softwareengineeringdev]

Great sir thanks a lot!

[@LucidProgramming]

No problem, I hope you found the video useful. If so, please do consider subscribing and checking out the other content on my channel. Cheers!

[@RahulGupta-wd3lf]

Good video became a subscriber

[@LucidProgramming]

Very much appreciated Rahul, thank you!

[@harshpratapsingh7640]

what compiler are you using?

[@facundolubo]

I'm stuck with the callback and website issue. where i can get an idea to solve this? None of youtube videos shows it... i'm begginer with API's

[@bi26]

Any alternative for someone like me whose developer account application was not approved by Twitter? How can i get my Twitter credentials?

[@iloveyoueggegg]

I might be being stupid but what programme is this? @5:22

[@akilhosang9655]

I've had a problem creating the application. When I filled out the details to create the app and hit create, it shows me a modal window saying "Review our Developer Terms" with nothing else to do. There's no way for me to create my app. Can anyone help?

[@goldenbroccoli2267]

thank you for your tutorial

[@rajpatel2093]

what should i do if i want to print that all data on html file?

[@markchatting820]

Great video man. I've got a question: how do you add the tweet coordinates or place to the df. I'm referring to the stuff you're describing at the 1:00:00 mark. When you look at what everything in the tweets is called using print(dir(tweets[0])) it says they're called 'coordinates' and 'place'. But if I write this: df['coordinates'] = np.array([tweet.coordinates for tweet in tweets]) and the same for place, they come up empty. Any suggestions? Thanks a lot in advance. i'm gonna keep watching your videos

[@ΑθανάσιοςΣουλιώτης-θ2γ]

Can we download tweets of multiple users in the same time ?? OR each time we need to change the screen_name ??

[@GeekXpert]

can you please help modify the code if we further include tweets that have emojis? please help!

[@uditgoel6925]

Check out the medium article they remove the emojis using their unicode

[@GeekXpert]

please provide the link to it!!

[@mostafagafer8621]

Thanks a million for this video . but I am at the minute 33 and I am getting an error 'charmap' codec can't encode characters in position 197-201: character maps to . any help would be great please

[@dsid4172]

are you on windows?

[@Justobolsa]

tweets.append(tweet.text.encode("utf-8"))

[@gleep5002]

how would I do this for a large amount of tweets, maybe around a trend? and then export the sentiment data to perform analysis

[@nisalbandara]

You must first train a data set. visit kaggle.com it has a Twitter dataset. Use that

[@ashufarswan]

In my code item fuction is not working properly it is giving random number of tweets..... what i can do to prevent it.

[@jakaseptiadi9845]

hy bro, what is the classifier algortihm do you used for sentiment clasiffication? is it Naive Bayes? or Support Vector Machine?

[@aniruddh1507]

it really depends on the data set. Usually sentiment analysis and Naive Bayes goes hand-in-hand. But on the real world, it is better to create a classifier that votes the best algorithm based on its accuracy and confidence level since the result is usually not reliable as the accuracy levels are volatile for every training set. Therefore... go with different classifiers and compare their accuracy, reliability. Once you come to a conclusion on which one is the best, rock and roll!! cheers!

[@tsovinar86]

hello i have a question if the tweepy installed but not work on VS can the same we write on anaconda? i had same problem for not intsalling for numpy

[@abdulrahmanalmalood8844]

I'm sorry, but what kind of coding tutorial that starts without mentioning the text editor will be used. I'm confused where to start the project.

[@davidadu1113]

Hi, can the following features be added to the stream.filter (created at, reply)? Thank you

[@AIdevel]

Its really though-provoking I really enjoyed this extraordinary lecture and I am really enthusiastic to apply this and add my own creative ideas ❤🎉🎉 Last question though How can I turn 1. 0,-1 into positive neutral and negative so that I can visualize them ?

[@AngelicaAsahan-ez1mc]

What IDE did you use here?

[@lionpersia]

Vim

[@AngelicaAsahan-ez1mc]

@@lionpersia thanks for the info

[@AngelicaAsahan-ez1mc]

it's confusing to use on windows

[@lionpersia]

@@AngelicaAsahan-ez1mc ur welcome, mate

[@lionpersia]

@@AngelicaAsahan-ez1mc If it's confusing, you can always use sublime or vscode.

[@GiantShepherd]

Hi does it work with Jupyter notebook too ? Coz I’m trying and it’s not letting me excite import twitter_credential

[@DanielWeikert]

Thanks a lot. Could you do that for facebook? Is there a particular reason why you first use a list and then a numpy array to create your column for the pandas DataFrame?

[@LucidProgramming]

Hi Daniel. The Twitter API is specifically designed by and for Twitter. You could do something similar for Facebook, but the code would be different. The reason for going from a list to an array to a dataframe is dependent on what goals I happen to have for the data. For instance when I'm using a dataframe, this is due to the fact that I want to process and analyze the data, and Pandas dataframes are good at such things. I hope you found the video useful. If so, please do consider subscribing and checking out the other content on my channel. Cheers!

[@lewiskori9426]

@@LucidProgramming the video is really amazing!! just checked out your channel .great work.

[@tarishigeetey8039]

Thank You !!

[@shubhambajaj2611]

Hey, I need help. Actually he uploaded this code on github and there are lots of files and i dont know which one to use so that i can show a project for social media sentiment analysis. If anyone can help, please help urgently

[@saarthaksggautam9817]

look for the last file.

[@Korhal55]

Hello, first of all, thank you for your video. It is very instructive and well structured. But I have an issue, I have follow your development but I cannot extract more than 200 tweets in my DataFrame, even if I write 1800 in the count in the "user_timeline" function. Do you know if there are a limitation of extract? I use Anaconda and JupyterLab to code. Thanks by advance.

[@srilathamaddineni3895]

same problem

[@vaishnavideshmukh1972]

Hi, where the text data get the store of tweets and hash tag list also on my computer??? Please reply. I run you code and it run but I want text file of tweets and hash tag list

[@sandy1996kenya]

hi, I would like to do sentiment analysis bt not use the traditional positive, neutral and negative categories, bt rather a much broader one, how do I go about that

[@hardikkumar598]

a question, does TextBlob use a classification algorithm like SVM, Naive Bayes etc. to calculate the sentiment polarity ?

[@jeelbhalodia4501]

Can you please suggest how can I write some unit tests for this project?

[@LucidProgramming]

Look into the "unittest" framework in Python.

[@webdeveloper3116]

bro,your work is nice. but i want to ask that how to use any trained model in place of textblob? i am cleaning text using textblob but i dont want textblob polarity feature. i want to check polarity by my own trained mode like naive bayes,random forest etc,but all trained model are giving negative value for all sentences during streaming.but when i separately predicting same sentences alone ,it is giving positive,negative ,all correct for sentences. please help me how to use trained model in place of text blob for getting correct prediction.

[@ifestos1977]

Great. thanks alot

[@kashifzaidi152]

In this video, which platform you are using to write code

[@sauravkokane3490]

How to restrict no. of tweets?

[@aishwaryar2379]

Hi, could u just let me know how to download 3200 statuses. I am able to download only 200.

[@talesamaral3744]

My tweets are getting cut. I can't get the complete text from it. Always end with '...' earlier. Any clues why?:

[@vishalsirka6397]

Sir, please make a video on Tweeter Visualization and Sentiment Analysis in laravel(php)

[@jsGuzi]

Can the sentiment analysis portion of this video be applied to other data sets instead of twitter?

[@LucidProgramming]

Yes it can!

[@aissoumonder3518]

İs this code works for python 3.10?

[@siddhanthnb]

Hey i wanted to make a sentiment analysis and publish it to github. But is it really safe to publish twitter api credentials into version control? Please tell me. P.s. I have used the same method as you did which is making a seperate python file and import it in our main code. But is it still safe?

[@kindaworks3293]

Is this tutorial using v1.1 or v2 of Twitter API

[@albertorubialesborrego3249]

Hi i saw the tweepy documentation. But i can't solve my problem. I want to obtain tweets with 280 characters, not truncated to 140. How can I do? Thanks a lot! :)

[@LucidProgramming]

This was a year ago, but hopefully, you found a solution. Thanks for watching in any case!

[@santeau]

48:49 realDonaldTrump doesn't work anymore as an example

[@sahanahv4071]

Sir ,can I use pycharm for execution

[@nisalbandara]

Yes, You can use pycharm or vs code both works well

[@sakthim7538]

Even After entering the keys i got 401 wts d reason sir?

[@dhartipatel2298]

I have the same problem. Did you find any solution?

[@sakthim7538]

@@dhartipatel2298 When I execute with cursor fn it goes well.

[@sahanahv4071]

Can I do it in pycharm

[@akeelfarooq1585]

when we analysing tweets how we can save these tweets on back end on json,txt,csv

[@galinasabinovakarova5251]

Can I do this in Jupiter Notebook?

[@LucidProgramming]

Hi Galya. You certainly can, if you're more comfortable with that. I tend to use vim as it's an editor that's light-weight and gets out of your way. If you like the type of content in this video, I hope you can also find similar content on my channel. Thanks for your question, and have a nice day!

[@ankitaarya7966]

sir,does this code can handle sarcasm ??

[@kamaldamian]

Thank you for this code,i got an error ModuleNotFoundError: No module named 'twitter_credentials' when trying to run it on anaconda please help me on that? thank again

[@praveen1991996]

Twitter_credentials python file should be saved in the same folder of your project folder.

[@MrEdd215]

Thanks!

[@Karimschillspot]

When I try running tweepy_streamer.py I keep getting error | ModuleNotFoundError: No module named 'tweepy' Yes I have tweepy installed. It's located in my \AppData\Local\Programs\Python\Python37-32\Lib directory. Any help would be very appreciated.

[@jahanvin1001]

I was facing the same problem. Is this you first Pycharm project?

[@StudyFun]

@@jahanvin1001 in my pc its running perfectly

[@juanpguzman666]

error 401 my credentials are OK i've checked. its have something to be with the clock please help!! Greetings from Colombia

[@harshadbhide5206]

Make sure the clock on your machine is showing the correct time. Then regenerate all the credentials and try again. That worked for me

[@calvinw2884]

I am on the visualizing tweet data time stamp 108:00 and while you are able to get the number of likes from the most liked tweet I was wondering how to get the tweet itself with the most likes. I tried sorting the array and printing the first element in the list, but it keeps giving me a "object is not subscriptable " error, do you have any idea how to do this?

[@itissrinivasan]

print("most fav tweet is: ", str(df1[df1.retw_count==np.max(df1['retw_count'])].text.values)) Basically, find the max value of the "retweets' field. match that in the data frame. then take the tweet in the data frame where it matched. get the value (which here, is the text of the tweet)

[@rajpatel2093]

where you code it? software name ?

[@synthefire]

vim

[@akeelfarooq1544]

help me sir it showa me error every time

[@MrBuri78]

Hi Vincent, First of all, thank you very much for this great content! It is very well explained and very useful! Thank you! I hope you'll find the time to give me an answer: I have two questions: 1.) in the following section: if __name__ == '__main__': twitter_client = TwitterClient() tweet_analyzer = TweetAnalyzer() api = twitter_client.get_twitter_client_api() tweets = api.user_timeline(screen_name='@realDonaldTrump', count=200) you have only one twitter user (realDonaldTrump). Is it possible to "follow" more than one person/institution to get their tweets downloaded? 2.) Is there any possibility to get more than 200 tweets? When I overwrite the "200" lets say with '2000', i still get only 200 tweets. An answer would be highly appreciated. Thank you very much and Regards Buri

[@LucidProgramming]

Thanks for the comment. Unfortunately I didn't see this until just now. If you want a more immediate response, I would suggest checking in on my channel. W.r.t your question, yes it is, and it's stated how to do so in the docs!

[@paulkerrigan9857]

When in doubt, add another layer of pointless wrapper classes. :D

[@hajeerakhanam178]

please share the code

[@ShravanKumar-xc6ij]

By the way, what do you do?

[@LucidProgramming]

It's on my KZbin channel, LucidProgramming!

[@danishmuzafar5572]

Plz help me i have research on Depression Detection using Emotion Arificial Intelligence with the help of CNN RNN LSTM

[@faheemrajwadkar7100]

How to make twitter apps now? apps.twitter is closed and is replace with developer.twitter and to create an app there you need to apply for a developer account which will only be used when approved. Any workaround?

[@alisaghi051]

No,it is not closed!

[@sadhnarohilla1841]

@Faheem Rajwadkar is your problem resolved? becoz m also facing the same probem. I dont know how much time will it take to approve my developer account so that i can have access to twitter api

[@hectorromero685]

Someone try something with instagram?

[@varunjha7731]

I m getting 401 continuously

[@LucidProgramming]

Hi Varun. Did you make sure to place in the proper API access tokens into the credentials file?

[@juanpguzman666]

@@LucidProgramming i have the same problem my credentials are good

[@vincentrusso6563]

@@juanpguzman666 You might be making too many requests to the server. You can either hold off on running the code and making requests for a little while or reset your system clock.

[@mddaud6029]

It's nice,help me

[@LucidProgramming]

Very happy to hear that Md Daud. If you like content like this, I hope you check out my channel where I have other videos with a similar flavor. Cheers, and very happy to hear that you've benefited from the video!

[@rosybanerjee1884]

This entire program coding is not running at all in Python and giving errors. It is like as below : from tweepy import API ModuleNotFoundError : No module name 'tweepy' def get_twitter_client_api(self): ^ IndentationError: unindent does not match any outer indentation level. ------------------------------------------------------- But in reality I already have tweepy in my system.

[@aishwaryakhot9353]

using realDoanaldTrump to analyze tweets, well that account does not exist anymore :D

[@malamhari_]

What kind of sorcery is this..