Stock Market Sentiment Analysis Using Python & Machine Learning

Stock Market Sentiment Analysis Using Python & Machine Learning
#SentimentAnalysis #StockPrediction #MachineLearning #Python
⭐Please Subscribe !⭐
▶️ Get 2 Free Stocks on WeBull when you deposit $100 (Valued up to $1600):
act.webull.com...
⭐Get the code and data sets by becoming a supporter on Patreon:
/ computerscience
⭐Website:
everythingcompu...
⭐Helpful Programming Books
► Python (Hands-Machine-Learning-Scikit-Learn-TensorFlow):
amzn.to/2AD1axD
► Learning Python:
amzn.to/3dQGrEB
►Head First Python:
amzn.to/3fUxDiO

[@toni2mito]

IF you substitute this: keep_columns = [ 'Open', 'High', 'Low', 'Volume', 'Subjectivity', 'Polarity', 'Compound', 'Negative', 'Neutral' ,'Positive', 'Label' ] with this: keep_columns = [ 'Open', 'High', 'Low', 'Volume','Label' ] you will get even better scores: precision recall f1-score support 0 0.88 0.80 0.83 193 1 0.82 0.89 0.86 205 accuracy 0.85 398 macro avg 0.85 0.85 0.85 398 weighted avg 0.85 0.85 0.85 398 That means that adding the news did not improve model accuracy/precision.

[@yixuan9213]

Wao, so the news titles are meaningless

[@Elywely]

God, this is some golden stuff right here. Simple, clean, to the point. I just started this journey towards machine learning, and seeing this not only taught me some interesting stuff, but also filled me with a lot of hope, since I actually understood every single step that you made. Thanks a bunch for this video, and much success to you!

[@ComputerSciencecompsci112358]

Thanks Elysium!

[@angelortiz9447]

This is glorious, I have been researching "fundamental stock analysis pdf" for a while now, and I think this has helped. Ever heard of - Ganichael Yonharlotte Trick - (search on google ) ? Ive heard some amazing things about it and my buddy got cool results with it.

[@JoshKonoff1]

Computer Science, I become a Patreon and still can't find the data sets. This is incredibly frustrating. How can I find these? Otherwise, I can't follow these videos

[@mahmoudkharoof6699]

Nice video good job! , Can you show how did you get the news csv file?

[@Dante-po7ng]

Great video! The data sets can be found on www.kaggle.com/aaron7sun/stocknews

[@AlexanderShs]

Real hero!

[@Jason-ru7xt]

Thanks alot

[@karlamariafelix6566]

thank you!

[@brittj8630]

My goodness. You absolutely saved my dissertation! Thanks ❤️❤️

[@DanhWasHere]

TLDW Summary: We try predict if a stock price will decrease or increase by using the sentiment analysis of top news articles at that time. 84% accuracy by end of training. Timestamps: 6:50 - EDA (exploratory data analysis) 10:30 - Merge dataset 12:03 - Combine news headlines as one block to feed 15:20 - Clean text data : remove breaks and slashes 19:21 - View clean data 22:32 - Get subjectivity + polarity with TextBlob 28:20 - THE MEAT : Function to get sentiment scores using SentimentIntensityAnalyzer 33 : Data Preprocessing done with Sentiment and Polarity Scores 39:47 - LinearDiscriminantAnalysis train model 41:22 - Classification report My opinion: This is a demo of TextBlob and SentimentIntensityAnalyzer and with some good data preprocessing. If someone wants to learn more they should look into NLP and the SentimentIntensityAnalyzer from vaderSentiment library. Good demo for beginners and big plus for using Colab instead of calling for us to download the libs with pip

[@ComputerSciencecompsci112358]

Amazing summary of the video. Thanks for watching!

[@chanchiwing530]

Yes, agree with you that this is a good tutorial for beginners to learn abt sentiment analysis of stock mkt by demonstrating textblob and vader for news headlines. I think the most challenging point for this domain is that the model should be able to predict next day/few days movement in practice because it is relatively easy to use today headlines to predict same day closing price and get high accuracy if the market is almost like efficient market and be able to absorb the news fast

[@ikikika]

Thanks @Computer Science for the video and @Danh Nguyen for the summary. I'm a beginner in data science. What would you recommend I learn next to: 1. Improve accuracy of predictions 2. Deploy this project onto a server and access the results using an API Any feedback is appreciated and thanks once again!

[@ikikika]

@@chanchiwing530 Hi, any recommendation regarding which libraries could be used to achieve prediction of next day/ next few days movement? Thanks!

[@DanhWasHere]

@@ikikika To improve the accuracy I say study the problem more and tweak the code used as the easiest start, as for this learning as a service I recommend putting this logic in a Flask server or lambda and access the result with a GET request -all the answers can be found with some Googling -thanks

[@mayurgupta4004]

why have u calculated polarity,subjectivity, and then u have used setimentintensityanalyzer to calculate positive negatve and neutal values .Why do we required it and this will not lead to multicollinearity ?

[@SciHeartJourney]

Thank you for this video, but what I wanted to see is how you got that csv file with the new data.

[@7deepakpandit]

Kaggle maybe. You can also try to write a scraper to fetch relevant news headings

[@alighten_]

I notice you didn't shift your labels. Doesn't this invalidate your training and test?

[@BekBrace]

Thank you so much for the brilliant content!

[@tanercoder1915]

Do you have a video on preparing the data sets? Can't follow this tutorial as it depends on having those two data sets.

[@tanercoder1915]

@@michaelmichaelmichael4988 Great! Thx!

[@nidaalyas1301]

@@michaelmichaelmichael4988 I need your help please get me back at [e-mail]

[@aayushsethi3078]

CAN someone please help: ----> 1 merge['Polarity'] = merge['combined_news'].apply(getpolarity) NameError: name 'getpolarity' is not defined ^^ keep getting this error, even though I followed everything to the letter

[@AranDhillon]

Not sure if you have solved it by now but incase not or anyone else has issue. That error basically means that the getpolarity you are using in that line you pasted has not be defined before. This can be an issue of how you spelt it (including capitals). Following the video, he has defined the that function as get_Polarity so it is possible you have done the same there. Without seeing your whole code, my guess would be that you have defined it as get_Polarity and are trying to use it as getpolarity but as you didn't define getpolarity, the complier doesn't know what to do. Try using merge['Polarity'] = merge['combined_news'].apply(get_Polarity)

[@WonderEkpe-w1f]

Hi, please, can I get the data?

[@ComputerSciencecompsci112358]

Hi Wonder, You can get the data set on patreon.com/computerscience

[@alexandertraveler510]

why on earth do you have to pay to get the dataset???

[@razvan9795]

I like your videos but I don't know why you aren't posting links for users to download the datasets. I found the DJIA datasets on Google in less than 2 minutes and I think that you're just losing viewers due to this fact.

[@ComputerSciencecompsci112358]

Thanks for watching and thanks Razvan for your helpful statement, I will take that into consideration!

[@billsava815]

@@ComputerSciencecompsci112358 Please post where we can get the dataset for the top news. otherwise this tutorial is worthless

[@JoshKonoff1]

Computer Science, I become a Patreon and still can't find the data sets. This is incredibly frustrating. How can I find these? Otherwise, I can't follow these videos

[@hera-12]

U can find it on kaggle

[@napent]

I think that this model will perform as well as without your sentiment data -_-

[@samkim3021]

hi! would you let me know how to get stock news (dow jones industrial average news) data ? I got the stock price data but ...

[@avivhoitash9582]

can you please give the dataset.

[@govindhgj9856]

How can I predict future price movement by data given by me?

[@slowhanduchiha]

u can try out some time series forecasting using lstm

[@mimification7679]

Can i use codes from github or kaggle ? I want to start collecting data from twitter to know people sentiment on such issue but i don't know how to start . I have no idea about python ..my background is mass communication

[@ADAS774]

Where can I find the dataset great video thanks

[@ashishgoswami6303]

i also need

[@shreyas_sarda]

Really good stuff for sentiment analysis. I wonder if we can really use OHLC Data for the day to predict up or down move for the same day. This predicts today's move and we don't know High or Low till close of market. Then, how would we take a position or close position. Maybe 'Label' or 'y' could have been shifted (-1) for predicting tomorrow's move up or down.

[@ZEANUWOE]

How to scrape data twitter by user location? Is there any one who can suggest any way or site to find scrapping twitter data by user location?

[@bhavyashah3392]

Where can I get Stock News as csv?

[@jorgehermosovalle3275]

www.kaggle.com/aaron7sun/stocknews

[@ozsuakin]

@@jorgehermosovalle3275 updated data?

[@kartikpandey5335]

Very nice ! could you also make a strategy like moving average based buying selling based on sentiment analysis please.

[@deepanshugupta641]

sir, how can I access news related to different stocks? I'm eager to practice with various stock updates.

[@nguyenduyta7136]

thank for cool things. I think it should be better if it can auto update news line with new price, right?

[@azizullah7881]

thanks for the valuable information, that I got through this video, I have a question how can I bring the news data to the excel file which you have used in this video...

[@maxbezrukov7711]

Thank you! Very usefull information.

[@ComputerSciencecompsci112358]

Thanks for watching!

[@loryo80]

Thank you so much for the video tutorial. I would like that you show us how to improve the model and also put some live scrapping data for a given list of Url. Thank you

[@MehakFatima-mx1ix]

Hey great video! one question though. What if we don't want to merge the headlines on the date field and want sentiment score of each individually. how would you suggest we aggregate the scores to run the prediction?

[@rajesha5798]

Are there Stock News CSV files available for Indian Market - Sensex/Nifty If not is there an efficient way to web scrape Relevant data to create my own News File?

[@tagifts]

How exactly did you get those headline? Manually picked them up or is there a automated system?

[@nicolaspilot]

How can you donwload news/articles into csv format? 😕

[@ComputerSciencecompsci112358]

You will have to scrape the data from the website containing the news/articles. To do this you may have to understand the structure of the website. Then after you've scraped the data, format it into a dataframe and convert that to a CSV file.

[@nicolaspilot]

@@ComputerSciencecompsci112358 is there a video tutorial on something similar?

[@wisalahmad8373]

Please someone tell me where this label column comes from? and how can we add this with our own dataset?

[@SuperReddevil23]

Amazing man. Im just gonna build upon this video of yours in my fintech final

[@ShakTMT]

does anyone know how you would implement this into a notebook containing an LSTM prediction model like hes shown in a previous video 'stock price prediction using machine learning'. For example, if i want the prediction value to change based on sentiment analysis being positive or negative. Is this possible? Does it make sense to do?

[@ydherdn]

Thank You

[@ComputerSciencecompsci112358]

Thanks for watching!

[@pkstock372]

df1 merge df2 , you need ( on =date ) // But merge [ Combined News ] = headLines , ( No need on = date ) ?

[@bjerkenpeter]

This is great! Anyone who would know how to plot the linear discriminant analysis in a graph?

[@jpainmaker]

Just found your channel great work. Thanks for taking your time to educate us.

[@ComputerSciencecompsci112358]

Thanks for watching!

[@Maximus18.6]

From what website you got the head news ?

[@karlamariafelix6566]

This video is amazing! I'm learning a lot! but I have a problem when trying to get the sentiment score for each day. I get an AttributeError that says: 'sentimentAnalyzer' object has no attribute 'polarity_Scores'

[@giuliofacciolo246]

If I want to try the model how can I do? Please help me

[@loganathansiva7063]

Thank you sir. Your videos are very much interesting and helpful for beginners like me. No word to praise you. May god shower his blessings on you.

[@chitrakchakraborty8563]

How do I get the datasets? please help me

[@mimification7679]

how we can get the data that in excl file already u upload it

[@ComputerSciencecompsci112358]

You can get it at patreon.com/computerscience

[@chukov08]

Beautiful! Love your content

[@ComputerSciencecompsci112358]

Thanks Lucas!

[@randb9378]

Hi there, for training/testing the mode, could we 'financial news on that day' + another parameter like tweet sentiments ? So we will have 2 parameters and the target value passed to the model?

[@HopDubstep]

Hey thanks for uploading , good video, my first time messing with both ML and sentiment analysis.

[@figuredoutfitness8154]

How to get This done for Indian stocks?

[@ironstark_007]

how to label the data if we get the headlines through web scrapping?

[@yaraali2311]

thx for the explanation could you send me the csv please ?

[@wilsongomes3360]

very good job.

[@dharmenrakumar8947]

Really great work

[@ComputerSciencecompsci112358]

Thanks a lot!

[@salvadorgutierrez7373]

This video makes me want to.... “Go ahead and create a new cell” Lol thanks for the vid!!!

[@ComputerSciencecompsci112358]

Hahaha

[@ihebbibani7122]

Hello , From where can I get historical sentiment datasets ? Thanks

[@John-dw6jb]

Great stuff, thank you

[@pkstock372]

where is the [ 'Label' ] column come from ?

[@pkstock372]

After the SIA score , the [ Label ] suddenly showed up

[@antiquerepro3239]

combined news.csv already have label column

[@pkstock372]

I think its came from 1 hot Encoder , is it right ?

[@daitavan297]

My question is: why didn't you use normalize the dataset?

[@shuvraneelroy5]

Search on kaggle. Theres only one result you'll find and thats these datasets itself. Download em.

[@daitavan297]

@@shuvraneelroy5 That is not my looking for an answer. My question is that the scale of data is very large. So should we need normalize the data in preprocessing step

[@shuvraneelroy5]

Acc to the video. The polarity and sentiments were already normalised. And the practice of normalising the input feature vastly affects the optimazation process. Different scales can lead to a cost function contour streched towards the feature with the largest scale. So to avoid stretching, we normalise all the input vectors or features

[@grantgre]

Hey man I need an image classifier for some medical imaging’s and I only have maybe 100 of each image so transfer learning neural network image classifer Is needed I need some extra help in terms of manipulation of my data pre-processing and labeling it.

[@afp-li8yj]

The state of the art when you have to increase your datset is to apply a matrix rotation on the existing dataset, you can use also a translation matrix

[@grantgre]

Andrea Previtali So yeah you know I guess you’re saying mathematically increase the data set by doing a math transformation. I know that some of the images can be viewed at different angles as it were or to take multiple shots at different angles to increase the data set like a burst of photographs that can increase the data set as well. But, you’re saying that transfer learning is passé now? I thought the purpose of that was to avoid having millions of pieces of data?

[@quantmean1729]

Like before watching the video

[@robertprobe]

video is blurry! not able see the content..font is blurry

[@ComputerSciencecompsci112358]

I'm not sure why it is blurry for you.

[@robertprobe]

@@ComputerSciencecompsci112358 Sorry! it was blurry @ 10:00 .till 16:00 ..now its fine

[@robertprobe]

@@ComputerSciencecompsci112358 Great content..Appreciate it