Tensorflow Input Pipeline | tf Dataset | Deep Learning Tutorial 44 (Tensorflow, Keras & Python)

[@codebasics]

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

[@pravallikaece195]

You are helping the data science community in an excellent way. keep going on and all the power to you. Thanks! and a very small token of appreciation

[@anmolkhurana490]

I was stuck with this Input Pipeline code for my project since last week. but, you cleared my all problems in just one video. Hats off to you for explaining such complex concepts in the easy way 👏

[@pintokatendejonathan1740]

Thank you expert, I follow you from the democratic republic of the congo and I appreciate everything you do

[@dec13666]

Holy smokes, TF pipeline looks so easy now! What a nice tutorial! 😎👍 TF pipelines always looked like black magic to me... until now! 😂... Keep it up with the good work, pal!

[@lebesgue-integral]

Changing ImageDataLoader to Dataset boosted my training time :) Thank you

[@codebasics]

Great to hear!

[@ahmedyaseen8994]

i love you man. Been struggling with tf for 2 months as I only have experience with pandas. The theory part was so helpful in understanding why tf is the way it is. And obv the coding part too. Thank you so much!

[@muhammadmalik4627]

Very useful tutorial! This video shows the Tensorflow Process in a simple and sistematic way. And your explanation is far clearer than any other expert tutorial. Big thumbs up for you, Sir!

[@someone7421]

I have seen many of your videos and all are so informative. You should make reinforcement learning tutorials as well and best of luck for your future videos

[@codebasics]

Thanks, will do!

[@fernandotta]

Man, this really helped me out. I was overcomplicating things. Thanks a bunch!

[@shantanubapat6937]

Only 175 likes? This video should have like 100k likes. Good content!

[@Noir_SD]

It was really helpful. Thank you so much for this awesome tutorial on tensorflow data pipeline. Keep making this type of videos more.😀😀😀😀

[@freedmoresidume]

Awesome tutorial, you never disappoint 😎👍

[@akshaypatwal6013]

Great work sir , learnt a lot from ur videos and looking forward to it in future also

[@codebasics]

Glad it was helpful!

[@nasgaroth1]

Awesome, but will be better if you could show how to uses it with tensorflow model, what is not such straitforward like it looks

[@sanjuvikasini1598]

Amazing explanation sir!

[@shivkrishnajaiswal8394]

is DataSet API useful for small dataset (data enough to be in RAM). It seems in every EPOCH of training, all the files will be reloaded. Which could be slow.

[@favourogbenna4402]

Superb tutorials

[@shubhamtalks9718]

Jaan bacha li guruji🙌

[@kevian182]

Excellent tutorial! Thank you

[@inesriahi]

Extremely useful ! Keep going !

[@riyadislam5412]

Great work sir,,We need tutorial fuzzy based ML and DL.

[@aruneshprasad3064]

You're excellent sir😇

[@codebasics]

Glad it was helpful!

[@dheemanth_bhat]

if anyone gets this error: `InvalidArgumentError: Unknown image file format. One of JPEG, PNG, GIF, BMP required.` just delete file `Best Dog & Puppy Health Insurance Plans....jpg` in dogs folder.

[@AyadEKorial]

Great instructor 👍🏻😎

[@jacksonngari1670]

i wish to learn on both deep learning and python through you.

[@codebasics]

I already have a python course on codebasics.io. For data science, ML and deep learning I will be launched courses on codebasics.io in the coming few months. Stay tuned 🔥

[@biggusmaximus1651]

Very great video sir

[@pvtgcn8752]

Great Explanation

[@VerdadesOlvidadas]

Nice video and humor thanks.

[@niranjansitapure4740]

This was crazy useful!

[@kolatimisimi7879]

Nice Tutorial, thanks. Also I think you could have just included the scaling part in the process function

[@aravindetagi4008]

I am watching this video after facing this issue.... 12GB RAM of google colab got filled and runtime crashed for loading 16k images.... Then I started using ImageDataGenerator BatchDataset

[@siddyant8332]

Sir could you please cover neural structured learning package, specifically the adversarial regularisation and graph regularisation topics from it, since there aren't many videos on youtube regarding these ....

[@harshalbhoir8986]

This is awesome!!!!

[@_Ahmed_O]

Awesome ! Thanks a lot.

[@manideep1882]

Videos are getting a little blurry, other than that it was a very informative. I've tried the shuffle and map combination and TF makes life easy. TY

[@amirayasmine8490]

great explanation .. thank you

[@codebasics]

Glad it was helpful!

[@jorismattheijssens5238]

Great video! The slides are neat, the explanations are clear and to-the-point. One question: I want to figure out how to stop the shuffling of a tf.data.Dataset every time you use a function, but I couldn't figure it out yet. For example, at 25:39, you extract the labels, but they are not the same as those in the file paths on the cell above. Any idea how NOT to shuffle the instances in a dataset?

[@haneulkim4902]

Thanks for great explanation! I've got two questions. 1. You said that it loads data in batches from disk how does shuffling work? Data are sampled from multiple source data then made into one batch or somehow all data is shuffled from disk? 2. I am trying to write tfrecords from pandas dataframe, how to split x,y within tf.data.dataset so it can be trained? After reading tfrecords I have dictionary of features(tensors).

[@lequoctuan8848]

Hello sir, this video is very helpful, thank you for creating this. My question is, when I use model.fit after building an input pipeline for training set, should I use validation_split for each batch for validation or should I use dataset.skip() to create validation set and then use it to validate for every training batch? Sorry for bad grammar!

[@vijayakakumani613]

very nice tutorial on Tf.data.Dataset module! my question is, if we use ImageDataGenerators will all this be automatically done? I.e. both creating Image Input Pipeline and also optimizing the pipeline (which is covered in next tutorial)

[@ArpitYadav-ws5xe]

Excellent

[@niranjandattatreya8331]

Hii, thanks a lot for the video , very useful, can you please upload tutorial on creating a custom dataset from parallel corpus of data for training ? unable to figure out

[@mubashirayub6630]

My dataset files are in .npy format, I want to fetch these files as you did for images by using image.decode_jpeg() fucntion. I couldn't find any function to fetch data from .npy file in Tensor. Your response would be appreciated...

[@aditya_01]

thank u sir

[@devloper_hs]

train_ds.map(process_imgs) returns : TypeError: tf__process_imgs() takes 1 positional argument but 2 were given , how to fix

[@alanhenry9850]

Sir can u also upload videos on bigadata,kafka,spark

[@frankieiero6859]

does this input pipeline also applicable for hyperspectral images?

[@peterpirog5004]

Very nice tutorial. I wonder how to generate dataset with random numbers, for example vector with uniform distribution in range with defined size to use while fiting with defined number of epochs and defined batch size. Is possible to use for this purpose tf.data.experimental.RandomDataset in tf 2.10 ?

[@OmidAtaollahi]

perfect, tnx.

[@codebasics]

You're welcome!

[@lahiru954]

It's awesome and thank you. But I want to ask a question. How can we apply the same concept for video data (already framed). can someone explain please

[@enewhuis]

Enjayable presentation. But I have 64GB on MY laptop. :P

[@akrsrivastava]

Good introductory tutorial. Why is there a b' in front of the file paths? The b' usually signifies byte data, doesnt it? Then how come it allowed to do a string split() ?

[@sergiochavezlazo5362]

What if instead of creating a new function scale, you just add one more line to the previous function: img=img/255 #Normalize

[@kutilkol]

so how to create the batches online during training and how to pass the batches to the model during training. not shown in video

[@srinathblaze651]

What if folders are not clearly separated as cats and dogs.. and we have just one folder of all images of cats and dogs.

[@user-qz4lw3cs8e]

If I get some image matrix data and save it as a dataframe, how do I pass it into the dataset as a feature? The from_tensor_slices method will report "ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray)." Thanks everyone for your help!

[@parthsharma898]

Sir can you please explain that how can we convert rgb images into array? @codebasics

[@nitinlodha4561]

image_count=len(images) print(image_count) TypeError: object of type 'TensorSliceDataset' has no len() Am getting this error , how to solve it?

[@snehasneha9290]

Hello, sir please suggest to me some projects for my masters. now I am studying MSC on data science I want to do a data science master project

[@shwetameena0511]

from 20:05

[@neerajashish7042]

If any one facing this error, where "tensorflow object has no len()" , instead of len(image_ds) use len(list(image_ds))

[@sakethgupta9047]

Images\*\* how to give my file input in that place? I also stored cat and dog pictures in same Library

[@devloper_hs]

at time 10:03 the code will be for sales in tf_dataset.as_numpy_iterator(): print(sales) and not for sales in tf_dataset.as_numpy_iterator: print(sales) may be some changes in new version

[@Amir-gi5fn]

I saved my X_train to a binary file how load it as tensor to make it batches

[@navneet3211]

❤️

[@MdAbdullah1509]

30:46 after 6 iteration it shows error while printing rest, how to fix this error Input/output error [[{{node ReadFile}}]] [Op:IteratorGetNext]

[@TayyabaSajjad-j3f]

I am facing same error in colab. How to fix it? Anybody's help is appreciated😢

[@mvs69]

can anyone explain 25:39 how get_lable method receives file path while calling below function, for label in train_ds.map(get_label): print(label)

[@waadturki2359]

I want to process a video data set anyone has a hint or a similar YT video

[@swatikarot8272]

tf_dataset = tf.data.Dataset.list_files('.\\datasets\\flower_photos\\*\\*', shuffle = False).map(lambda x: process_image(x)).map(lambda x,y: scale(x,y)) Could someone review this one line code for image dataset??

[@peopleclickindia1733]

The Python Training is designed for you to learn Python Programming Principles such as Python Program Flow, Control Statement, Loops, Functions, File Handling, Error Handling, and Access to APIs. Feasible lessons of real-time tasks and test cases are the subject of this Python course in BTM Layout. We are Bangalore's leading educational institution with Job-Oriented Python Training in HSR Layout. Being the best Python training institute in Bangalore, We understand the latest trends and provide the best Python training in Bangalore. Learn with brilliant teachers and gain outstanding programming skills in Python. Turn your fears into opportunity and get your dream job. If you have the commitment and perseverance to make your career dreams true, connect with us without any second thoughts. We are here to help you with our hyperactive team and full-fledged learning facilities to nurture your skills and polish them to match up to industry standards. Here are the prime features that make us unique. * 100% placement guaranteed training * Resume building support * Training by Industry experts * 24/7 solution support and many more So you can start your career as a Python programmer with our python training in Bangalore. Contact us now. Phone no: [phone] Website: www.peopleclick.in/python-training-bangalore.html?SEO&Python

[@samas69420]

SSSSSINGGGLL LINE OFKODE

[@tech-learner4555]

Are you Indian?

[@shantib4025]

tf_dataset = tf_dataset.filter(lambda x: x>0) for sales in tf_dataset.np(): print(sales) AttributeError Traceback (most recent call last) in 1 tf_dataset = tf_dataset.filter(lambda x: x>0) ----> 2 for sales in tf_dataset.np(): 3 print(sales) AttributeError: 'FilterDataset' object has no attribute 'np'

[@adrenochromeaddict4232]

what you promise to show fixing and what you actually show have nothing to do with eachother. and it's so emberrassing that as if botting your sub count wasn't enough you're botting your comments section too. another pajeet wasting my time