TensorFlow Step-by-Step Captcha solving tutorial with custom OCR model

[@Tiagoi386]

I would like to thank you for the knowledge! I'm from Brazil, and your lesson has helped me a lot in training captchas. It works really well on multiple different captchas, and it all depends on the amount of data I can include in the dataset! I'm still very new to the field of machine learning, but I'm having an excellent start with your lessons. Please, keep up the great work! 😃

[@PyLessons]

Great to hear! Thank you for such comment, I'll try to continue with my tutorials!

[@RobertBadi]

Hello from France, Great tutorial, thank for you share.

[@akhil6132]

Brother, This is working perfectly fine on this dataset. I wanted to use some other dataset( 66k captcha - kaggle). But, it is not working at all no matter what changes i make in this model . Can you suggest me any particular changes( i mean even if they are not 100% fool proof) any general changes. Also, it would be kind of you, if you can provide me insights regarding how model is developed wrt datasets features(i mean what we need to keep in mind).

[@ИсломКобилов-щ7ж]

Hi, did you figure out any methods to use with other datasets? I'm also interested in this topic😅

[@rons96]

Nice tutorial dude!

[@PyLessons]

Thanks!

[@muhammadkabirliaquat2534]

Hello, Very well explained. I trained my model successfully, but I have some questions that are; 1. Is this model is capable to train with 6 alphanumeric captcha dataset? 2. After training should model will be able to predict any picture captcha, that wasn't include on dataset while training? Thanks and your efforts are appreciable

[@PyLessons]

Hey, thanks first! 1. You can train with any number of chars 2. Yes, but it should be at least similar to what you used for training

[@ИсломКобилов-щ7ж]

Hello my friend, what a great job, respect for that👍🏻👍🏻 I have tried to train it on a bit different captchas, when it's around 250/1000 epoch, it's early stopping. I have tried to change batch size to 32, 128, 256, 512. The best result I got on 256 batch size, it's training faster and very effectively. But still it stopped around 270/1000 epoch. I used RAM 24GB, CPU laptop for training. If you give some advice, I'd very grateful. Thank you

[@YuYanLiu-u1j]

Very good tutorial, I don't know if you have a live chat tool to ask you questions

[@eSensesHL]

Hello, excellent video. One question: to train it with several types of captcha should I group them in different folders or can they all be together? Thank you.

[@PyLessons]

Thanks, it can be in same folder

[@viswa2k944]

Your contents are awesome, I'm intermediate in python i'm very much interested in machine learning and your skills are adorable, If i want to start learning ML where can i start how you learnt all these skill

[@PyLessons]

Happy to hear that! Learning ML is not a sprint, it's a marathon, you need a lot of time to learn it, best is learn by doing or working. But I recommend to do some project, by solving problems. I won't learn it by reading or copying other content, you need to write stuff, develop and etc.

[@levanhiep5068]

Great tutorial, thank you. Please tell me how to continue training the old data after stopping the program

[@PyLessons]

Before training model, load weights from saved model

[@levanhiep5068]

@@PyLessons can you give me sample code? I'm new to this so I don't know how to load the model

[@PyLessons]

www.tensorflow.org/tutorials/keras/save_and_load

[@levanhiep5068]

Thank you

[@yoongicorn7973]

hi there... thank you for such an incredible model. However in config.yaml, I came across a variable "vocab" and realized changing it's value affects the results. Can you explain how do we decide optimal value of vocab to get desired results for our own testing dataset.

[@PyLessons]

How do define vocab you can find in train.py file of this tutorial "github.com/pythonlessons/mltu/blob/main/Tutorials/02_captcha_to_text/train.py". Check line 43. And when you want to test your model you should load your saved vocab that you used for training

[@AGASTRONICS]

Wow 😮, real practical exercises. 👾🥶✅

[@PyLessons]

Glad you like them!

[@solomon_leo_]

One question: I have a set of images, the object detector is trained to detect 5 different classes of texts. Name, age, weight, height, sex. How can I train a custom OCR model to extract the text? will the code that you have provided work for more than one class? or should I tune the code accordingly? Thanks in advance!

[@PyLessons]

Hey, yes you can train this OCR to recognize text from each of these classes, shouldn't be any problems

[@monkitvn8531]

Please make a video updating the process of recognizing simple CAPTCHA (with the new versions of TensorFlow, MLtu, and Python).

[@PyLessons]

Hey, I don't have time to review old tutorials. I don't see a purpose do to this. But I'll check the compatibility with latest TensorFlow versions later and my MLTU package. Thanks for this!

[@dhanashreevenkatachalam4258]

I worked on this model with custom dataset, but i don't understand in the inference part that you directly getting validation dataset splitted at the training time , in inferencemodel.py you used the line line "df =....... " It directly load the validation dataset and it gives cer=1.0 ?? How to test inference with custom images?

[@PyLessons]

Well' it is pretty straightforward. When training I used to split full dataset into train and validation. At the end, I saved these datasets so I could load same validation dataset when testing. If you can't find it out, write your code on github issues, I'll help you with custom image.

[@Warrior-qz5ip]

Excited

[@PyLessons]

Thanks!

[@qwerci3539]

great tutorial

[@ИсломКобилов-щ7ж]

Great tutorial, love that. Can i use it with svg images? because when i convert svg into png, its becoming low quality, like i can the pixels.

[@mubarakjan9367]

Can u please make a new video about this and help the beginner's this is way to complicated

[@PyLessons]

I am not sure if there is a way to make it in a beginner way, I already tried to explain it in simple terms. If it is too complicated you'll need to gather more knowledge :)

[@gamestarz7381]

so i am workin with this model of your and i have 6-8 letters captchas in same format as tested by you but 1-2 characters have very much distortion and noise as compared to trained data, bcz of that error rate is always above than 90%. can you or anyone help me with the model?

[@MushtaqAhmed-m6b]

Hello, I wish to know if I would want it to autofill captcha on a website, would it work and what would be the response time?

[@陳俊廷-j2r]

Hello, I encountered an error while predicting the verification code: "ModuleNotFoundError: No module named 'mltu.utils'." However, I have already installed mltu==0.1.4.Thanks.

[@PyLessons]

it works with newest version, simply do pip install mltu

[@NinjaAI_07]

what if i give the input image with capche and other text written on image

[@PyLessons]

Should work, but you may need to test this out. You may need more data to train it

[@GovindS-xc8ir]

I am facing issue in mltu.tensorflow, its showing no module named mltu.tensorflow.

[@PyLessons]

which mltu version you using?

[@GovindS-xc8ir]

@@PyLessons 0.1.4

[@AryoReyo]

Some error appears on new python, mltu, tensorflow versions.

[@PyLessons]

Downgrate python to 3.10, downgrade tensorflow to 2.10, and shot me the error in github issues

[@skezar838]

Pls make video on solving recaptcha v2

[@akhil6132]

Hello, I hope you're doing well, and I want to express my appreciation for the tutorial. However, I've encountered an issue related to a missing YAML file. If it's possible, could you kindly provide the required file?

[@PyLessons]

Hey, you are welcome :)! The files to trained model are uploaded to google drive, link you can find in text version tutorial (at the end)

[@akhil6132]

@@PyLessonsThank you so much❤

[@larubiano0]

Great tutorial, thank you. Also, I've got a dataset with 1024 unlabeled images that I've got to classify to train the model, do you know of any service where I can pay someone to label those images?

[@larubiano0]

@@Neurotic979 people at marketing in my company ended up doing it, but thanks anyway

[@pythondjango9406]

@@Neurotic979 if you can do than it would be great, i dont know how many captchas would be required for training, can you tell me?

[@YuYanLiu-u1j]

I want to deploy your code on linux and what it takes to do it

[@andes3406]

hey just a question, can you do something similar with like captcha solver but doing hcaptcha training instead of these letter ones?

[@PyLessons]

for hcaptcha its way more difficult

[@akejron1]

I tried to replicate the 99% success, but as it were i had 100% CER, which is exact opposite. I generated around 20k captchas and doing 1024 batch size with RTX 4090 - so far CER is 30% at epoch 153. However, for captchas it seems WER might be much more important metric and it is at almost 70%. Any hints how to improve? 50$ to the first one who gets me where I need to get :)

[@PyLessons]

Hey, thats a great stuff you did. I recommend moving on with checking augmentation techniques you use within your images. Then move with model architecture, you definitely can make model larger so it would perform better

[@akejron1]

@@PyLessons you mean like add more layers? I ended up with 50% solve rate for that particular captcha type. Need to make it more generic though.

[@PyLessons]

Yes, need to experiment. Add more layers, increase number of parameters in layers, play around with Dropout (sometimes it works sometimes not). In tutorial I made only an example of architecture, and if you want to get better results it is a must to play around with architecture

[@agazzer33]

hello, the newest version of mltu is 1.0.12 can be the newest version used in this tutorial?

[@PyLessons]

Hi, yes if you'll take tutorial code from github, there might be minor difference

[@BigBoss-hb8uh]

Model path (Models/02_captcha_to_text/202309060950\model.onnx) does not exist. I got this error while running, please help me

[@PyLessons]

Download trained model or train it your self and it will be saved

[@gustavodearaujo8108]

​@@PyLessons im also getting this error. While training the model is not changing tô the onnx format...

[@OsamaKazan01]

hi bro is there anything to bypass text iamge to text i have 1 site that needs to type texts to continue

[@trongvuxuan8259]

how many images do we need in the dataset for the model to work correctly?

[@PyLessons]

There is such thing as working correct, everything depends what images you use and model you use. More images better it will work, when you see that if you add images and it is not improving this means that you need to work with model architecture

[@turki5528]

the issue there is no onnx model ? is that a bug or verison issue ?

[@PyLessons]

You mean, model was not created after training your model? There might be issues with tf2onnx version

[@ytrbqdkflbvbhy]

Awesome guide and explanation! Also can you tell how you exported data to tensorboard?

[@PyLessons]

Hello, you mean how I got all the metrics in tensorboard? That's simple, add tensorboard to callback, check the code mate :)

[@sel88q]

Can someone recommend captcha generator? I want to create my own anti-bot captcha

[@sudipkc00]

how can I pass the single image of captcha and solve it? I want to integrate it in slenium to solve captcha but I have 0 knowledge in models and stuff

[@hafizmaulana8166]

i have the same question. I just need to know how to pass an image to solve it

[@m.bahadr7970]

@@hafizmaulana8166 did you find about it :D im trying too rn

[@furqanahmed2339]

Hey was this solved am stuck on the same part where i need to pass a single image instead of multiple images.

[@ChoySylaithong]

Hi, nice to meet you I am Choy and software developer, Thailand. I have seen your github examples in sites. That' s excellent. I already followed to you. Now. I am developing captcha image to get text. So I need to you helping in task Captcha image example: I hope to hear from you . Regards.Hi, nice to meet you I am Choy and software developer, Thailand. I have seen your github examples in sites. That' s excellent. I already followed to you. Now. I am developing captcha image to get text. So I need to you helping in task Captcha image example: I hope to hear from you . Regards.

[@gamingland2074]

hello sir , thanks for the tutorial i have a number cpatcha can i use same code for it ?

[@PyLessons]

Yes you can

[@gamingland2074]

@@PyLessons used it i code 10% failures which is great how i can improve existing models ?

[@PyLessons]

More captcha or try to improve model architecture

[@zainulabdin8822]

Hi! Model ended training with Epoch 51: early stopping . I am training with dataset 1040 images? I am training with my 1040 labled images given below but when I try to predict text it shows empty string. Code :: image = cv2.imread('./6wf4ef.jpg') prediction_text = model.predict(image) // return empty string print(f"Predicted Text: {prediction_text}")

[@PyLessons]

Totally not clear what you doing, what is the model? youtube is not the place for such comments

[@sudipkc00]

I have already installed mltu in virtual env but I 'm still getting modulenotfound error for mltu

[@PyLessons]

what version you installed? shouldn't be like that

[@gopro12378]

How can I use this in flutter app as I want to use it as api for post method in an app

[@PyLessons]

You expect me to write you whole pipeline? :D start implementing it step by step, starting from simple post request and you should make it. You cant do it just by wanting, you should try it by solving all problems you face step by step

[@zainulabdin8822]

How long does model take to get train from 1000 images dataset??

[@PyLessons]

it depends on what hardware you train, but it's quick

[@farhanriuzaki4765]

can i make it as api? after training the data

[@PyLessons]

Yes you can, I don't see a reason why you couldn't

[@tejaspatil6831]

how i can integrate it with python selenium to validate recaptcha

[@PyLessons]

It is not supposed to solve reCAPTCHA challange

[@zainulabdin8822]

unable to run the code from git repo. someone please help me out with this I literally need this.

[@PyLessons]

Open github issue with problems

[@SidharthGnanam]

Hello can i use this method for PaddleOCR ?

[@PyLessons]

I don't know, I haven't tried

[@zainulabdin8822]

Do I need to train the model with 6 digits?

[@PyLessons]

it's not required to train it on 6 digits, you can use captchas with any number of them

[@coconutnut21]

Can I use this for license plate ?

[@PyLessons]

Yes, why not

[@林威宇-e7r]

Hello, thanks for the tutorial, but when running: model.fit( train_data_provider, validation_data=val_data_provider, epochs=configs.train_epochs, callbacks=[earlystopper, checkpoint, trainLogger, reduceLROnPlat, tb_callback, model2onnx], workers=configs.train_workers ) This error popped up: ValueError: Failed to find data adapter that can handle input: , Is there a way that I can fix this?

[@PyLessons]

Open issue on github and write more details what you are doing

[@ZainAbdin-e7s]

how to bypass six digits?

[@PyLessons]

By changing configs

[@aryangupta2051]

@@PyLessons pease tell which configs to change ,p.s i have subscribed you 😁

[@MarkDog0410]

hello sir, Is any recommend class to teach how to adjust model learning param? I used this model to run my data, it didn't improve.

[@PyLessons]

first, in tutorial I used really small dataset, you should increase the dataset. Then you could try changing model architecture, use LSTM, insrease filters use ResidualBlock, its up to you to experiment

[@MarkDog0410]

@@PyLessons How many datasets could help model grow up? As much as I could?

[@KamrulHasan-m7v9h]

model.onnx not create

[@PyLessons]

Error?

[@KamrulHasan-m7v9h]

@@PyLessons solve this Version problem

[@PyLessons]

@@KamrulHasan-m7v9h it is solved in newest version, install newest with pip install mltu --upgrade

[@RanjitParmar-d7l]

please make a discord server for us

[@Winreee]

I just made my own script and my own model 😂 because i couldn't run this script lol

[@PyLessons]

Hey, was the problem, why it didn't worked?

[@onelastsong2986]

scure ur phishing websites XD HHHHH

[@furqanahmed2339]

way complex for someone starting out and i cant even run your code without running into thousands of issue. In the end i just moved onto another YT explanation video.

[@PyLessons]

Hi, this code is not beginner level. Also there are requirements that if you do no satisfy them you may face a lot of error. And if you are a beginner it will be frustrating to solve them. Good luck :)

[@mubarakjan9367]

@furqanahmed2339 are there any more tutorial which are easy to understand for beginners

[@mubarakjan9367]

Are there any tutorial where it's easy to understand for beginners

[@virajagarwal1714]

C:\Users\KIIT0001\AppData\Local\Programs\Python\Python312\Lib\site-packages\keras\src\callbacks\model_checkpoint.py

[@PyLessons]

error is not informative...

[@aminkhosravi5073]

Hello! Thanks for this amazing tutorial! I'm using python 3.10, tensorflow 2.10 and mltu 0.1.4 and it can't recognize from mltu.utilis.text_utilis

[@PyLessons]

Update mltu to newest version and follow tutorial code from github, you should be good

[@adelib8171]

there is what inside "Models/02_captcha_to_text/202212211205/configs.yaml" ??

[@PyLessons]

Download model and check it out

[@adelib8171]

@@PyLessons I downloaded it but still can't find out !

[@TechWave2630]

this video is perfect example of: if sombody knows how to teach , he don't have knowledge and if somebody have knowledge he don't know how to teach, your github code has from mltu.tensorflow.dataProvider import DataProvider from mltu.tensorflow.losses import CTCloss from mltu.tensorflow.callbacks import Model2onnx, TrainLogger from mltu.tensorflow.metrics import CWERMetric and in video it's from mltu.dataProvider import DataProvider from mltu.losses import CTCloss from mltu.callbacks import Model2onnx, TrainLogger from mltu.metrics import CWERMetric very confusing

[@PyLessons]

Hey, yes because I made some improvements on my library and I can't change what I already recorded. You can roll to specific mltu version that was used in this tutorial if you need exact match. This happens when you continue working on a library to be more flexible and etc

[@TechWave2630]

@@PyLessons you have made changes in Tutorial files, not only in the library

[@TechWave2630]

@@PyLessons u have made changes in Tutorial files along with the library

[@TechWave2630]

I have wasted 1 week and finally able to find a video which explains this topic better, every one who is in same situation like me, should watch video on same topic created by Abhishek Thakur channel

[@nguyenkhoithai7423]

Great tutorial, thank you. I have captcha like below can I apply this model to solve it in under 1s ? pasteboard.co/z7G53miavNXf.jpg

[@PyLessons]

yes, easily :)

[@nguyenkhoithai7423]

@@PyLessons amazing, thank you so mush

[@akutube9218]

how can i use this code in google colab ?

[@PyLessons]

Instead of python, copy it to colab, adapt it and you good to go. Strange question