Twitter Sentiment Analysis (NLP) | Machine Learning | Python

[@pranayzalke3461]

Big thank you Ashwin, it helped me do a project overnight.

[@HackersRealm]

Glad it was helpful!!!

[@kkamit0106]

Same here !!

[@saviovnn]

Thanks for the caption it helps a lot !!! Send hi it to Brazil.

[@rohanpatel8106]

Sir please make some more advanced project like this for cse students

[@HackersRealm]

yeah sure, I will be making more projects!!!

[@pavansai5239]

did u know how to download the dataset ? help me out of this

[@BEESaiShivam]

Are these too basic to put on cv?

[@dataanalyticsfun9883]

Thanks you very much sir for making amazing video .sir I have request to you to make a video regarding future engineering and feature selection . thanks

[@HackersRealm]

Noted, I will create seperate video for feature engineering

[@parthshingari6082]

+1

[@nasreenbanu2245]

Thank you so much.You made me understand complex problem to simple.Hats off..keep going.Can u make video on choosing statistical test.

[@HackersRealm]

sure, I will make separate videos for the concepts!!!

[@purvikabajaj8383]

thank you for a clear explanation

[@HackersRealm]

Happy to Help!!!

[@mwanza826]

Hello sir, i have an issue downloading wordcloud even after installing c++ build tools , what could the issue be

[@HackersRealm]

what error you're facing while installing?

[@mouskey]

Tq sir its more helpuful for my project

[@HackersRealm]

glad it was helpful!!!

[@mouskey]

Yes sir its reallyhelpful, one week i stuck in this program but i got a solution now do more like this sir if you don't mine post tutorial video of adv python its more helpful for beginners lyk me once again tq sir.

[@HackersRealm]

@@mouskey advanced in python or data science?

[@mouskey]

@@HackersRealm both

[@HackersRealm]

@@mouskey sure, planning to do that!!!

[@nugrahaputra3632]

thanks very much brother, good luck

[@HackersRealm]

Always happy to help!!!

[@kajalsoni7721]

great tutorial!! can you tell me how would we go about making a gui for this model? using gradio maybe?

[@HackersRealm]

You can use any gui module like tkinter or others to create it. Backend functions will be same. You can check how to deploy ml project for more information

[@kajalsoni7721]

@@HackersRealm thanks for the reply! i'm trying to deploy the model,running into a lot of errors.do i need to set up twitter api for real time tweet collection?

[@razibdebnath5386]

can i use the same pre-processing procedure for SVM,naive bayes and random forest model?

[@HackersRealm]

yes you can

[@palakparmar9674]

i have to do research paper , where can i find whole code. because it was not there on github

[@HackersRealm]

it's available in github... go to machine learning project repository and find the folder with title

[@djadeja4709]

First of all thanks for such amazing explaination, But I want to ask you that from feature extraction step you didn't use datavisualization or list of hashtags you created. I have done the same whole code as well as second time after preprocessing I jumped on feature extraction and both gave the same output in f1_score and accuracy_score!!! So can you explain how bag of word has helped in classification??? In which step they classified positive and negative tweets?

[@HackersRealm]

Glad you liked the video!!! hashtag won't give us meaning results as it doesn't contain the actual tweet, so i didn't use that. bag of words are used to identify the set of words that considers positive or negative. Initially it's available in dataset, after that it will try to predict it.

[@djadeja4709]

@@HackersRealm Got it!!! Thanks you so much!!!

[@خلیلشاھ-ش9ع]

how can we get confusion matrix eg precision, recall, f1 scores?

[@HackersRealm]

you can use classification report module to get that

[@viraj9954]

hey!! great work....just wanted to know which algorithm did you actually use in above video

[@HackersRealm]

It's logistic regression

[@ephrembekele5241]

Sir you said the objective was to detect hate speech in tweets. so what can we conclude from the results? I am a little confused.

[@HackersRealm]

the negative reviews we are classified are hate speech, using this model, we can able to identify that

[@ephrembekele5241]

@@HackersRealm what does the accuracy scored show us in this case?

[@HackersRealm]

@@ephrembekele5241 how many tweets we predicted correctly using the model

[@flouwer6669]

How can I classify it as neutral as well? and How can we integrate the model into a web app?

[@HackersRealm]

you have to use the dataset that has neutral class and use flask to integrate in web app

[@RashmiTigadi]

Which algorithm is used in this process which u have explained

[@AIResearchTechniques]

Sir on df['clean_tweet'] = np.vectorize(remove_pattern)(df['tweet'],"@[\w]*") when I run this code the output for clean tweet shows none

[@AIResearchTechniques]

How to rectify this error

[@HackersRealm]

please check whether the data is loaded

[@AIResearchTechniques]

Thank you sir ..When I remove the '\w' the output run without error and showed all cleaned tweets

[@HackersRealm]

@@AIResearchTechniques Glad to hear that

[@payalkasera4596]

@@AIResearchTechniques i am facing same issue can u help me

[@harshi_vlogs07]

Thank you 😊

[@HackersRealm]

Happy to help!!!

[@aritrochakraborty8122]

Facing issue regarding installing the wordcloud.... " ERROR: wordcloud-1.5.0-cp34-cp34m-win_amd64.whl is not a supported wheel on this platform. " how to fixed it?

[@HackersRealm]

try install visual c++ build tools and retry the command

[@himanshurathod1178]

could you please tell me how did you load the dataset i am getting no such file error

[@HackersRealm]

please check whether the file is in the same directory or not

[@rapidknowledge3483]

Please make video on live tweets analysis using machine learning

[@HackersRealm]

Yeah, will do that soon

[@mouskey]

Hlo sir, sorry for disturbing you i have one doubt if the letter is not happy which kind of senmential the code will take positive or negative?

[@HackersRealm]

i didn't get you, can you rephrase it!!!

[@mouskey]

Sir, in sentimental analysis positive words are happy like that and negative words are sad like that what we consider if the word is not happy .it's a positive side or negative side sir

[@HackersRealm]

@@mouskey it may consider positive, to overcome these limitations, we may go for lstm models

[@mouskey]

@@HackersRealm okay sir tq for ur valuable reply, Sir in this video you use lstm?

[@HackersRealm]

@@mouskey No, I have used a machine learning algorithm, check fake news detection project, I have used lstm in that

[@himanshunagapure9883]

Your data has label for 1 as racist(negative) and 0 denotes tweets as not racist(positive) .So what did you consider for neutral tweets??

[@HackersRealm]

This dataset doesn't have neutral, you can use other tweets dataset for 3 classes!!!

[@user-gl2nv3fy7w]

Hello sir..i got an error while visualisation the frequent words..it says that only supported for Truetype fonts..plz help me

[@HackersRealm]

are you using different dataset?

[@user-gl2nv3fy7w]

@@HackersRealm yess

[@HackersRealm]

@@user-gl2nv3fy7w the please check the type of the column and may be change to strings

[@user-gl2nv3fy7w]

My dataset has all objects.. so i need to convert them in string? Then this problem will get resolve?

[@HackersRealm]

@@user-gl2nv3fy7w yes most likely!!!

[@viane123456]

Sir I am a btech first year student. can you tell me how can we get current tweets from twitter in csv file using python?

[@HackersRealm]

You can use tweepy to get current tweets; You can also check realtime twitter sentiment analysis in the playlist

[@Moviesslider]

How do you get that dataset? How do I get data directly from the Twitter tweets? Please suggest way for that.

[@HackersRealm]

the dataset can be downloaded from the links in the description. For real time, twitter analysis, I will make a video in the upcoming weeks

[@artfromtheheart9545]

Can You Please Make video for this? "Automated Classification of Societal Sentiments on Twitter With Machine Learning"

[@HackersRealm]

I have done a video on real time twitter sentiment analysis. you can check if that's the case

[@ترافهعشق-غ2ظ]

كيف يمكن الحصول على جدوله البيانات من النص الكلمات ام من اين ومن لا يدخلنا csv

[@landaalhanshaly7989]

Can you recommend a book for python and machine learning?

[@ARkhan-xw8ud]

Since the data was imbalanced, you should use oversampling technique

[@floweraddictxx6238]

Bro why i can't use PorterStemmer? Is there any that i need to install?? Please help

[@HackersRealm]

you can use that if you installed nltk package

[@floweraddictxx6238]

@@HackersRealm but already installed bro

[@HackersRealm]

@@floweraddictxx6238 then it will work, it has all the modules

[@mukeshrajpurohit5593]

It will create data leakage problem if we get BOW before splitting the data into train & test.

[@hrittickdebnath35]

Thanks buddy

[@HackersRealm]

Glad it was helpful!!!

[@yasamvarnika7998]

Hello Sir, Please help me to download the dataset its not downloading

[@HackersRealm]

You can get it from the github repo as well!!!

[@mouskey]

Sir , could you please tell me the code of movie review in twitter sentimental analysis and the project uses also.

[@HackersRealm]

the process remains the same for any kind of sentiment analysis

[@mouskey]

@@HackersRealm tq sir

[@parth1211]

hello sir , why cant we do logistic regression directly ? why we using nlp ?

[@HackersRealm]

we cannot give text data as it is, as the model can't understand. It can only get information through numbers.

[@himanshupandey1167]

sir not able to get the dataset from the provided link

[@HackersRealm]

you can get it from github repo also

[@skeler4430]

Sir how to use SVM instead of logistic regression do you have code or can you please explain

[@HackersRealm]

replace the model name with svm model, that's it

[@AnuragKumar-ij7rl]

There is error while creating graph unable to create graph by this process

[@HackersRealm]

which part you're getting error?

[@ghosthanded]

Sir I am getting an empty array for the function of extracting hashtags how to resolve this

[@HackersRealm]

are you using different data or same?

[@ghosthanded]

@@HackersRealm a different one I have scarped the data myself

[@HackersRealm]

@@ghosthanded then you may need to change the processing steps accordingly

[@ghosthanded]

@@HackersRealm can you give me a reference for it?

[@pratikshapatil7413]

Can you please tell me from where I can Download the Dataset

[@HackersRealm]

you can get it from link provided in description or github

[@kavya.r8385]

hey, i am getting error in the 6th step (remove @user)

[@HackersRealm]

what error it is showing?

[@user-ls4mx2tw1l]

Can you help me , i have assignment I have to convert before August 20 😭 about nlp sentiment analysis Tweet

[@HackersRealm]

Which part you need help?

[@user-ls4mx2tw1l]

@@HackersRealm about sentiment analysis with naive Bayes

[@HackersRealm]

@@user-ls4mx2tw1l you can use the same flow and change the model for this

[@priyachimanchode648]

Can't find the CSV file in any links, it would be convenient if just put the link in the description directly.

[@HackersRealm]

The dataset link is in the description, you can get it from analytics vidhya or you can get it from github repo too, Please check and let me know

[@priyachimanchode648]

Found them, thank you.

[@ashutoshbansal8254]

i don't find CSV file

[@HackersRealm]

please check the github link or dataset link

[@pavansai5239]

please help me out from this i cant download the dataset , please help me .........

[@HackersRealm]

you can download the data from github too, try it

[@pavansai5239]

@@HackersRealm got it thanks

[@VijayKumar-ck7ww]

Can we apply naive Bayes ?

[@HackersRealm]

yes we can apply

[@masternik6126]

I want to make a project on detection of self harming or Sucidial tweets

[@umamahaswarikarre8062]

Sir is this a project for CSE or ECE

[@HackersRealm]

we can't specify any department for the projects, anyone can use based on the applications

[@roshnimustafa8051]

how to download dataset?

[@HackersRealm]

link is in the description

[@roshnimustafa8051]

but i didnt found dataset there

[@HackersRealm]

@@roshnimustafa8051 you have to register in that website to get it or you can get it from github repo link too

[@aquillinantony2770]

Why we use "np.vectorize" in twitter handle removing

[@HackersRealm]

i just showed different ways to pass values to the functions. It's not compulsory to use this

[@RevatiVardekar]

where can i get the dataset

[@HackersRealm]

you can get it from the link in the description!!!

[@خلیلشاھ-ش9ع]

sir can we bert model ?

[@HackersRealm]

Yes but you have to use corresponding preprocessing methods for it

[@خلیلشاھ-ش9ع]

@@HackersRealm thank you

[@srishtimishra5101]

can't find dataset sir ..It's redirecting to some site.

[@HackersRealm]

you can find it in the github repo

[@srishtimishra5101]

@@HackersRealm It's redirecting to analytics vidya and when I go to github and try to access dataset its telling can't open....It says can't show files that are this big right now ..and open raw data like that

[@HackersRealm]

@@srishtimishra5101 In github, you can download the file and use it. or reach me over insta or linkedin, I will share the data

[@theghost_16]

@Srishti Mishra if made the project can you plss share the link of datasets .

[@samreenshamim8161]

Hello Sir! I hope you're doing well, Sir I got an error when I typed this command: df1['clean_tweet'] = df1['clean_tweet'].apply(lambda x: " ".join([w for w in x.split() if len(w)>3])) AttributeError: 'float' object has no attribute 'split' I would be grateful if you would help me out a little :)!

[@HackersRealm]

can you rerun the code from the start? or you're using different dataset for this project?

[@samreenshamim8161]

@@HackersRealm I am working with a different dataset indeed but I've checked the datatype of the column using the info() command and it shows object datatype. So I think this shouldn't come up with an attribute error but it still shows that error.

[@HackersRealm]

@@samreenshamim8161 I think the column may have null values, try to remove null values and proceed

[@samreenshamim8161]

@@HackersRealm okay Sir I'll try 😊

[@samreenshamim8161]

@@HackersRealm Sir my problem got solved! I did a small conversion of x.split() into str(x).split(). It gave me tweets without short words successfully :).

[@meghaagarwal9907]

Please provide the dataset file, it can't be found!

[@HackersRealm]

Please check the description or download it from github repo

[@machinelearning3518]

did u made the regex tutorial?

[@HackersRealm]

No Not anything specifically yet!!!

[@Saboor-Hamedi]

How to get a label on our individual dataset from Twitter?

[@HackersRealm]

You can use pretrained model or you have to manually label them by yourself, if you want to create a complete dataset

[@Saboor-Hamedi]

@@HackersRealm Thank you, but what that label mean? Is it important for research, because I am using my data for thesis

[@HackersRealm]

@@Saboor-Hamedi label is the output column which will be used as result

[@Saboor-Hamedi]

@@HackersRealm How to get that results? I was asking about that

[@shivanitufchi968]

Can you pls share this code here

[@HackersRealm]

Please check the github repo for the code, link in the description

[@oreoluwaadeyegbe4670]

Hey I would love to get in contact with you as I need just a bit of help in my dissertation for my Masters, if you won’t mind of you of course. Thank you in advance

[@HackersRealm]

you can book a consultation using the link in the description or reach me via insta or linkedin, Thanks!!

[@AmirSohel-iz9kz]

Dataset?

[@HackersRealm]

you can check the github repo!!!

[@afridmohammed6711]

@@HackersRealm It is showing that " we can't show the files that are this big right now"

[@HackersRealm]

@@afridmohammed6711 You can download the file and view it or you can use other link to download the file

[@kumarithalathoti9784]

Hey did you got the dataset