How to Create Efficient Training Pipelines with TensorFlow data.Dataset (Tensorflow Datasets)

[@GregHogg]

Take my courses at mlnow.ai/!

[@user-wr4yl7tx3w]

the way I understood it, is that on the first epoch, the processed data gets cached. and therefore, subsequent epochs, we don't need to preprocess, since we can now use the cached data instead. since they are the same anyway. no need to repeat it for each epoch.

[@GregHogg]

Yes, that's right. In my phrasing, I may have dumbed it down slightly while still providing what's important to understand

[@vasudevraghav2109]

What an awesome tutorial man! Keep up the great work, really learned a lot!

[@GregHogg]

Thanks I really appreciate that!

[@ShoaibKhanZ]

F!!! its a functional lazy eval. Argh, was clear as mud before, now crystal clear. Thanks!

[@datapro007]

Great video Greg, thank you.

[@hussainsalih3520]

very well document , I hope you make a video on best practice neural network when it faced overfitting

[@freya291]

Hello, and thank you for this great content. What may happen if you have multi-input models and large datasets? Would you use the for loop to separate the input features and the labels?

[@Amir-gi5fn]

9:41 Oh so it turns data from uint8 to float32 only when needed and this actually saves a lot of memory I've already preprocessed my data but can now do resizing in Pipeline

[@inakitodc6816]

OMG this was so fcking useful, really good and deep material. Thanks!

[@GregHogg]

Glad to hear it!!

[@iciukas]

Excellent! Clear and helpful

[@GregHogg]

Super glad to hear it!

[@oranuzrad8832]

I Loved it!

[@fiknthapis6231]

Good stuff

[@harshitjami4771]

In the video when your doing the first 'for' loop after the mapping the example_dataset, the loop broke after the first iteration so it must have only cached the first sample in the example_dataset and then when you shuffled the samples and then iterated through the first sample (in the newly shuffled dataset) it must have mapped and then cached that first time iterated sample in the examples_dataset because it was not previously iterated through. Did i understand it correctly?

[@user-wr4yl7tx3w]

what would be also cool is to see a video on vision transformer?

[@GregHogg]

Sounds good.

[@Live_draw_today]

Please make a video on how to train voice model stem by step in pycharm, please sir make video on this

[@anwerhassan154]

thank you very much, but how can I print the result from the model and visualization the actual data and prediction data

[@arsheyajain7055]

Wonderful

[@GregHogg]

Thank you 😊

[@elafakceramicmagasin3705]

hi greg we need a road map for sql or at least some course suggestions , or a tutorial from you will be great

[@GregHogg]

You might want to check this Specialization out: kzbin.info/www/bejne/jGfTi5JjabCDj6M

[@user-wr4yl7tx3w]

Is there a way to see what you have already included in the pipeline?

[@GregHogg]

That's a great question. I actually don't know the answer.

[@user-wr4yl7tx3w]

may be PyTorch also has something like this? is it very different?

[@GregHogg]

I'm just learning it now. When I feel I understand everything well enough, I will teach it back :)

[@koji4059]

Could you make a review on HyperSkill Python course. I’ve heard it’s very good but not sure.

[@GregHogg]

Never heard of it.

[@koji4059]

@@GregHogg It’s also known as JetBrains Academy. It’s entire structure is project based learning and they have a Python Core track and a new Machine Learning in Python course.

[@user-wr4yl7tx3w]

doesn't cache simply mean, take that data and put it in local memory for reuse again? not sure what point I am missing.

[@GregHogg]

Their caching process is rather complex. I would just think of it as remembering.

[@user-wr4yl7tx3w]

it seems like a really unintuitive way to create a pipeline. wouldn't it be better to create a list of first class functions, where each function within the list does a step in the pipeline.

[@GregHogg]

You could make a function where it does a list of those functions

[@cagataytekin6372]

imparator