Build a Comment Toxicity Model with Deep Learning and Python

[@stevew2418]

You’re my favorite person I’ve found on KZbin this year! Ultra high quality and very cool project. Thanks for all you do!

[@og_pluta]

Today is my birthday so it's perfectly timed video! Thanks for a gift Nicholas!

[@NicholasRenotte]

HAPPY BIRTHDAY!!!

[@Moonev_Fantasy]

If you have issues with enormous dataset size (as me), you can try to resize it this way: df = df.head(60000) or another number which would be appropriate for you

[@shayantankabiraj677]

That was helpful

[@surajramamurthysuresh7446]

That was really helpful

[@trusttheprocess4775]

This was incredibly helpful. My kernel kept dying and i was searching solutions but yeah, my ram couldnt keep up and had to lower the size of the training sample as you suggested. Thanks a lot!

[@panmusic786]

Hey nick!! I had increase the epoch to 10 and the result was : Precision: 0.93 RECALL : 0.92 ACCURACY : 0.45

[@flurrshify5419]

Love the video bro keep it up! You have an insane level of quality in your tutorials. Please Please consider doing distance detection, I would love to see a tutorial on that!!!

[@sheikhshafayat6984]

I never comment anywhere usually. But just to tell you that you're the best thing that happended to me after Khan Academy. Can't express how grateful I am for you!

[@fusionhubb.]

Best guy on internet so far , i wanna give a huge thank you to teach us i successfully make my Final year project as i am a student of software engineering and i have choose my career to Quality Assurance but i love to learn machine learning . This is all by this guy Thank you once again

[@ishanpurohit9762]

Thank you so much for this. Please keep uploading more projects. This is better than learning from tutorial and it's rare to find.

[@thepresistence5935]

31:57 where I can find the pipeline videos, I have searched in your videos, But I can't get where it is, Can you please help with that?

[@NicholasRenotte]

Part 1 in this shows how to do it with the Keras image_dataset_from_directory method. Might actually do a detailed vid or a short on it this Sunday.

[@thepresistence5935]

@@NicholasRenotte Thankyou so much, bro. Love you

[@dipankarnandi7708]

I did two things, I trained the model for 15 epochs, it went to overfitting, so used dropout of 0.3 in the hidden layers, this helped to bump it up. Second, using callbacks, I saved the model, ran for 15 epochs again but the model was already saved at epoch no.4. In the second approach, I believe I did 1 mistake which was using shuffle = true in model.compile, Maybe that shooted the accuracy to 0.96 at the start and then went down and down as the epochs went up. Let me know Nick, then I see if any changes can be updated ^^ In simpler terms we need a higher no. Of samples to train this model. Then it will be better

[@piat795]

Could you provide some insights as to how you used Dropout of 0.3? I see that the tutorial imports it but we don't actually use it. Could you take about how you used it to increase the accuracy of the model? I'm very new to ML, sorry if it's a basic question.

[@ArunYadav-qq1cj]

lol crazy stuff... how much hours it took you to train for 15 epochs. mine took 36 min. for 3 epochs..lol

[@abhilashapanda7406]

how did you save the model ? mine is showing errors

[@dipankarnandi7708]

@@abhilashapanda7406 check if ur model creation or layering has some errors. Saving model is simply model.save unless you wanna save it in onnx or any other format

[@dipankarnandi7708]

@@ArunYadav-qq1cj I have 2 gpus of 24 gb each.so it's quick

[@ishandandekar1808]

Hey Nicholas, in the embedding layer why isn't the value 1800? Shouldn't it be the output length, why did we set it as MAX_FEATURES? Also, what is the significance of that `+1` ?

[@ishangrotra7265]

i tried it with just max_features it works just fine the adding of +1 didnt create a significant difference

[@victorluz8521]

I'm going backwards on your videos after discovering the channel. This is yet a stunning one! Thank You!

[@FootballFlicksTV03]

import pandas as pd import tensorflow as tf import numpy as np from tensorflow.keras.layers import TextVectorization model = tf.keras.models.load_model('toxicity.h5') vectorizer = TextVectorization(max_tokens=200000, output_sequence_length=1800, output_mode='int') input_str = vectorizer('hey i freaken hate you!') res = model.predict(np.expand_dims(input_str,0)) res it give error

[@prxninpmi]

oh my god, my course project at the university of the same topic. I'm glad nick what do you think about BERT? Are there any advanced word representation methods other than word2vec, etc))

[@NicholasRenotte]

This model is pretty basic, AFAIK most of the SOTA models seem to be pretrained transformers these days particularly when it comes to NLP tasks!

[@moondevonyt]

after doing this video, what do you think the best model is for comment toxicity?

[@NicholasRenotte]

This is good (if trained longer) but most pretrained transformers will probably smash it out of the park!

[@SushiTheLeo]

This video deserves one million likes at once form just me...that is how useful I found it

[@owenhochwald506]

could you use a library like sklearn with train_test_split to also split up the data?

[@aavideora6269]

Is this a project/mini project I could add to my GitHub if I followed the steps and did it in a new notebook? If so, what is the protocol for citing the original creator?

[@baddogbrandon1767]

i have tried to tweak the model by adding dropout layer, changing dense activations and changing the batch size, I also run multiple epochs but the accuracy hardly climbed up and there are some false positives and negatives on the result. how do we address the low accuracy? (highest i've got is 52%)

[@jawwadahmed5342]

Try a different tokenisation technique

[@TransalpDave]

Hey Nick thanks for this great video. Had a question about Embedding, is the keras layer enough for good performances because i saw some people using a word2vec before the embedding, is it really necessary ? I'm actually working on something similar to google smart compose so all the tokenizing and vectorizing part is interresting !

[@NicholasRenotte]

The Embedding layer is great because it's fine tuned to that use case, word2vec embeddings values can be loaded into the Keras embedding layer. I haven't tested out my theory but I believe they won't immediately outperform a fine tuned embedding layer because it's a general word representation! It should in theory though, speed up training time if you set training=False for that layer.

[@TransalpDave]

@@NicholasRenotte Okay thanks for your feedback !

[@TransalpDave]

@@NicholasRenotte For word prediction do you think transformers will be much better than standart lstms ?

[@tanmayy07]

how can we train epoch in very less time? any solutions

[@den3917]

youre amazing. i love how much you explain and not just write code

[@KinG-ql5ly]

When i compile my model then i want to look summary, I cant see any result. It gives me 0 parameters and no output shape. How can I solve this problem?

[@KinG-ql5ly]

I found myself :)) Thanks for the video!

[@who-kv6fe]

@@KinG-ql5ly what did you do?

[@emanuelcuracosta3753]

Many thanks! Very clear as always! I have a doubt. Is it possible to have a regression model out of text? I mean, not a classification model, but a continuous variable as output. For instance, predicting Airbnb rent price out of a description? I searched a lot, but can't find any examples out there. Any suggestions from anybody?

[@gouthamansaravanan7692]

yes, we can do it. ( Reviews to rating/price)

[@03_rahuljordanchandra4]

Hey Nick, this was an amazing video. I am a beginner but i learnt so much from here! I made the batch size as 1000 as 1800 gave some trouble and made the epochs 15. Precision was 0.93, Recall was 0.95 and Accuracy was 0.47. Also it is not really predicting the threats label well. I do not know how to fix that.

[@malikahmed5281]

Literally, you are amazing loving your content. I am going to make a project on this.

[@Manju0247]

Can't fix value error:failed to convert a NumPy array to a tensor

[@jessewen9774]

I learned something new,so happy,continue to update,bro,please!

[@YG-re5yl]

Hello Nicholas, glad to see some NLP in your page. Just wondering, why not using some pretrained language model that perform well on that task? Would love to discuss about it with you

[@NicholasRenotte]

Good suggestion! I did this first up as slightly more simple example. I wrote a transformer based model as well, just haven't done a vid on it yet!

[@thepresistence5935]

YES, I am looking for XLNET

[@YG-re5yl]

Yep, I would love to see some more NLP stuff on your channel ( even though I feel it's much more centred toward CV and RL) You do a great job, keep it up 💪

[@hafedkhalil2779]

how would such a video have only 692 likes !!!!!! you are amazing at what you do my friend :)

[@ahmedm.soliman374]

amazing , how can i fune tine number of layer , nodes...etc in tensorflow keras . in sklearn there is gridsearch, there is a tool or technique to get best architecture for neural network..?

[@NicholasRenotte]

I've used Optuna before with pretty good success!

[@DikshaSShirke]

I wish your channel should hit subscribers in millions. Support . Amazing teaching

[@FootballFlicksTV03]

import pandas as pd import tensorflow as tf import numpy as np from tensorflow.keras.layers import TextVectorization model = tf.keras.models.load_model('toxicity.h5') vectorizer = TextVectorization(max_tokens=200000, output_sequence_length=1800, output_mode='int') input_str = vectorizer('hey i freaken hate you!') res = model.predict(np.expand_dims(input_str,0)) res it gives error ? why

[@0xSatyajit]

if you face error like ValueError at res = model.predict(input_text) then rewrite whole block, input_text_batch = tf.expand_dims(input_text, axis=0) res = model.predict(input_text_batch)

[@FootballFlicksTV03]

import os import pandas as pd import tensorflow as tf import numpy as np from tensorflow.keras.layers import TextVectorization model = tf.keras.models.load_model('toxicity.h5') vectorizer = TextVectorization(max_tokens=200000, output_sequence_length=1800, output_mode='int') input_str = vectorizer('hey i freaken hate you!') res = model.predict(np.expand_dims(input_str,0)) res it gives error ? why

[@harryfeng4199]

any advice / rull of thumb for setting cache and prefetch sizes?

[@EngineersCode521]

Many thanks... It was so clear and appropriate.. But still I wanna tell u one thing that u should plz try to tweak the whole model thats how u build it especially for the begginers....

[@aibdarija5856]

can you do , how to make a text classification using deep learning.

[@destinationmufc4431]

i had a question. how do these models deal with proper nouns?

[@tiennguyenkim9057]

interface = gr.Interface(fn=score_comment, inputs=gr.inputs.Textbox(lines=2, placeholder='Comment to score'), outputs='text') --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Cell In[81], line 2 1 interface = gr.Interface(fn=score_comment, ----> 2 inputs=gr.inputs.Textbox(lines=2, placeholder='Comment to score'), 3 outputs='text') AttributeError: module 'gradio' has no attribute 'inputs'

[@deepika1228]

Same problem did you get any solution?

[@satyajeetsankpal8668]

Have you got the solution

[@zai_ry11]

the feeling of when you restart your jupyter notebook while training... 😭 a solution here is to use the pickle package and save the training data import pickle with open("history.pkl", "wb") as f: pickle.dump(history.history, f) import pickle with open('history.pkl', 'rb') as f: history = pickle.load(f)

[@yesh-c6r]

At 47:47 I see an error in your code, I'm also getting the same error. How did you resolve it

[@jatinsingh9797]

Hey did you got any solution for this . If yes please help me too

[@amithp321]

I am also getting the same error.Someone help me out

[@sharanyanaresh1071]

Could somebody please tell me why the github link keeps giving 404 error

[@adamcosgrove9730]

Did you ever get round to doing a video on tensor Flow Datasets? Or a deeper dive into the MCSHBAP format

[@SamuelOmali-t5j]

Please can you help me, my dataset doesn't come with labels, how do I go about it

[@romanalilov4648]

Thanks for your content! Can you recommend a book on Math side of ML/DL for a beginner? And is it necessary to know Math side for junior level?

[@ashok_learn]

It's always good to know the maths, I myself sometimes struggle with some research papers. You can check these books which can give you good insight; "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville "Mathematics for Machine Learning" by Deisenroth, Faisal, and Ong "A First Course in Machine Learning" by Simon Rogers and Mark Girolami

[@chiscoduran9517]

Hi, that is a really good video, thanks for that I want to build a model to generate images from other, the idea is use 3D movies and use one side of the image (left eye) to generate other side (right eye), what do you recommend for that and what can I expect to achieve?

[@NicholasRenotte]

Sounds interesting, I'd imagine you'd be looking at using a GAN for that!

[@malexandersalazar]

Why sigmoid instead of softmax? For the last Dense(6)

[@RokuLeVrai]

Hello, my gradio.launch() doesn't work. I have the error : "cannot import name 'http_server' from 'gradio'". It is same for someone ? If you solve this problem, please tell me :)

[@NicholasRenotte]

Weird, I can't see any existing issues that are similar. Maybe try updating?

[@shambhavi_mj]

Hi Nicholas, I am having multiple errors while importing Gradio. Cannot import name ‘doc’ from ‘typing-extensions’ And Cannot import name ‘deprecated’ from ‘typing-extensions’ I have installed the latest version for typing extensions. Can you please help …

[@tegabyteX]

I think the error is with fastapi... I had this issue too and installing an older fastapi version helped "pip install fastapi==0.103.2" Lemme know if this helped

[@darkmasterbatista2815]

Ilove this, but i would like to know if there a way to improve the speed because my training was around 4 hours :c

[@cadman-a6357]

same here , it took me around 2 hours , i guess there's no other way

[@ninadrai6148]

Hey Nicholas thanks for the guide on sign language detection I have made it till the end and got one last error to solve in 8th step and I need your help on it error : Traceback (most recent call last) in 28 agnostic_mode=False) 29 ---> 30 cv2.imshow('object detection', cv2.resize(image_np_with_detections, (800, 600))) 31 32 if cv2.waitKey(1) & 0xFF == ord('q'): error: OpenCV(4.5.5) D:\a\opencv-python\opencv-python\opencv\modules\highgui\src\window.cpp:1268: error: (-2:Unspecified error) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Cocoa support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script in function 'cvShowImage' please help me

[@shaikhahsan100]

did you get it solved?

[@marioskadriu441]

Amazing tutorial as always 😍 Why do we need 3 dense feature extractors instead of 2 or 1 ? How do you decide how many units do we need on them ?

[@NicholasRenotte]

In this case, it's purely subjective and based on manual performance tuning! Could definitely try 2 or 1 and see what performance looks like! For other model types the layer order and type matter a little more in order to ensure we have an appropriate output shape which maps to the label vector!

[@rajareivan2417]

can someone please provide me some insights to overcome this low recall and categorical accuracy metrics score?

[@picklenickil]

I reckon it's more complex than this. Good start though but much research needed for validation. Loved the video.

[@bumbajoel4450]

how to test all the data in test.csv and calculate the accuracy, presicion, recall, and F1-score.

[@gauravthapa8367]

Got problem with gradio. Can you help solve it?

[@arulsood1817]

why i am not able to open the code file?

[@suriyaprakash5172]

Which version of python should be used?

[@mochoye]

While using vectorizer.adapt(X.values) i am getting a error Failed to convert a NumPy array to a Tensor (Unsupported object type float). can anyone help? I am using google colab

[@raissamaringka330]

did you already solve this ? mine also the same and im using colab

[@Ankit-hs9nb]

why TextVectorization was used? Any particular reason? we could use other vectorizer?

[@ahmedhussain9219]

Maybe because it has an advantage of dynamic updation,advanced preprocessing modes when compared to Tokenizer vectorizer

[@asadasedglrrfff5925]

Thank you very much for wonderful content!!! You’ve helped me very much.

[@prathambansal2036]

My prediction is coming something like [9.9688369e-01, 4.5290351e-02, 9.8086458e-01, 4.7373693e-04, 9.5557123e-01, 9.5138857e-03] any clue i retraced my steps no issue anywhere. also i trained the model with 10 epochs that gave me the loss of .1 is it cause i raised epochs please help......

[@ammuzz5910]

hey which dataset have you used

[@azalea9645]

what if tensor flow doesn't work?

[@riyaz8072]

Amazing content.. why don't you have 10M subscribers ? One small suggestions, can you please give us reasons like why you are not creating multiple hidden layers or why you are not selecting weight initialization techniques ? you do explain most of the stuff but if you can explain a little bit more stuff considering newbies will also be watching your videos will be very helpful.. Thanks a lot.. May Allah keep you healthy and wealthy

[@satoshinakamoto5710]

What version of python is being used here?

[@atharvgupta4218]

The accuracy is coming around 50%. Does anyone have fine tuned version of the code.. Please drop your repo link..

[@gameraki3284]

change epoch 1 to epoch 5 and try running it again

[@0xeb-]

Dude, you are awesome and a good teacher. Thank you.

[@stackunderflow6167]

is it true that tensorflow-gpu has been removed?

[@ankitpundir_]

i am also having probem with this bro . how did you solved this please help.

[@stackunderflow6167]

@@ankitpundir_ i forgor 💀try to use tensorflow instead of tensorflow-gpu

[@anonymousduel7370]

Bro for some reason every single code of your shows errors, idk why are you making fool of people

[@Abin-4321]

What is the algorithm used in this model ?

[@pradyumnkumarshukla8858]

Bi-LSTM

[@BetülUveys-i7y]

how to find the dataset?

[@Powercube7]

YEAH BABY VIDEO FINALLY OUT

[@NicholasRenotte]

AYYYYYYY!!!

[@hajiallahditta9239]

Sir its my final project on toxic comments can you please help me in this regard

[@owenhochwald506]

Wonderful Explaination!

[@davidyoo647]

another awesome project, and great explanation!

[@guillaumevallee3390]

Hey Nick ! Just discovered your channel a few months ago and I love it ! I really enjoyed your playlist about Facial Recognition where you implement a Neural Network following an article. I discovered some days ago an article that I found very interesting concerning the short-time bitcoin market prediction with varied features (classical open and close but also sentimental analysis of tweets containing #bitcoin). Different Machine Learning are applied and especially some Recurrent neural networks. I am not very familiar with this kind of methods so it enables me to discover the theory behind. It would be a pleasure to watch you implement such a Deep Learning model in one of your videos. The article name if you're interested : Short term bitcoin market prediction via machine learning by Patrick Jaquart and Christof Weinhardt Will be following you for a long time 🙂

[@siddhantrajhans6528]

It's taking forever to train with 1 epochs. can anyone suggest free online gpu processing. I know it sounds ridicules but hey, my gtx 1650 isn't working with it.

[@lovelyupadhyay1439]

@siddhantrajhans6528 did you find any solution to this?

[@ingluissantana]

Great quality content!!! 🍻🍻🍻

[@thepresistence5935]

My 1 epoch time is 3 hours in kaggle GPU 😭😭

[@lovelyupadhyay1439]

did you find any solution to this because i can't keep the system on for whole day🥲

[@МустафаевТамерлан]

Thank you! You are awesome!

[@eddcarvajal9064]

the 'freaken' best! Thanks for your videos

[@went-rogue]

what about cleaning the data ?

[@theseusRJ7]

The data was probably cleaned beforehand

[@CMT-p6q]

thank you excellent content as always

[@simarpreetkaur8674]

Waiting for more such ML, AI projects 😇

[@FootballFlicksTV03]

import pandas as pd import tensorflow as tf import numpy as np from tensorflow.keras.layers import TextVectorization model = tf.keras.models.load_model('toxicity.h5') vectorizer = TextVectorization(max_tokens=200000, output_sequence_length=1800, output_mode='int') input_str = vectorizer('hey i freaken hate you!') res = model.predict(np.expand_dims(input_str,0)) res it give error ?

[@varadparchure]

Gets close to the same result with 7 epochs

[@Normality42]

took 3 hours to train the model for one epoch. having 10 epochs means 30 hours no thank you 🙂

[@HammadFaizvi]

very toxic project... loved it!!!

[@ecquinox9074]

37:00

[@fazebook7897]

damn regex is evil. Now you gotta comments foreign languages. это так отстало

[@27ronitphilips48]

hhello +

[@mumbaikachallenge5526]

Sheesh crazy ❤️

[@CoCo-wx3so]

[@ashleysami1640]

Second comment!

[@NicholasRenotte]

Ayyyyy, go you! Thanks for checking it out!!

[@ashleysami1640]

@@NicholasRenotte go me!

[@ankitdadarwala2940]

Amazing tutorial 😀

[@mehdixeif5024]

Hi nick you are amazing😍😍 once i see the tutorial about one shot learning but that was foul . so can you record a tutorial about one shot learning🥲please🤗