PyTorch Tutorial 02 - Tensor Basics

New Tutorial series about Deep Learning with PyTorch!
⭐ Check out Tabnine, the FREE AI-powered code completion tool I use to help me code faster: www.tabnine.com/?... *
This part covers the basics of Tensors and Tensor operations in PyTorch. Learn also how to convert from numpy data to PyTorch tensors and vice versa!
Part 02: Tensor Basics
📚 Get my FREE NumPy Handbook:
www.python-engineer.com/numpy...
📓 Notebooks available on Patreon:
/ patrickloeber
⭐ Join Our Discord : / discord
If you enjoyed this video, please subscribe to the channel!
Official website:
pytorch.org/
Part 01:
• PyTorch Tutorial 01 - ...
You can find me here:
Website: www.python-engineer.com
Twitter: / patloeber
GitHub: github.com/patrickloeber
#Python #DeepLearning #Pytorch
----------------------------------------------------------------------------------------------------------
* This is a sponsored link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

[@sergiomurilo758]

Thank you so much man. I was struggling on my master's thesis, but thanks to your videos I'm finally able to develop the application I needed.

[@lokeshyerneni2018]

By the way, you are one of the most underrated youtubers I have seen, and I believe by the end of this year, you will have at least 10,000 subscribers

[@patloeber]

Thank you so much 😊

[@patloeber]

Just noticed this old comment again. We already crossed the 10.000 :) Thanks for your motivational words back then!

[@SteelersFans99]

@@patloeber great job!

[@sergiomurilo758]

and now there are 42k! :D

[@phungtruong6698]

@@sergiomurilo758 71.6K subscribers :)

[@k.k.h9288]

this is an awesome tutorial.I am leaving some time-stamps here that can help me later 4:53 Element-wise Addition and in-place addition, multiplication, division 7:09 Slicing Operation 8:57 re-shaping a tensor 10:39 converting NumPy to tensor and vice-versa 14:50 CUDA and how to create tensor on GPU 17:26 Requires grad

[@dhananjaykansal8097]

This is my first video of yours channel. I must say you deserve more than million subscribers and thousand of likes and comments. God Bless You! Amen

[@patloeber]

Thank you very much! Glad you like it :)

[@igor-policee]

Your PyTorch course is simply amazing. This course helped me a lot to understand and start working with this framework. Thanks!

[@patloeber]

happy to hear that!

[@linxuan757]

This is the best tutorial video for pytorch I have ever seen. Thanks!

[@patloeber]

Thank you 😊

[@AltafHussain-gk2xe]

Thank you sir for such a great toturials your teaching methodology is awesome 💯

[@patloeber]

thank you :)

[@ravivarma5703]

I have never seen a better explanation than this. Brother you are awesome !...Please do continue ML videos dont stop keep rocking...Great no words left Thanks for making

[@patloeber]

Thank you :) I will continue!

[@ahmedel-sinousy4848]

wonderful tutorial thanks for sharing

[@fahadrahmanamik7190]

Just Awesome ! Nicely explained pytorch 60 minutes blitz tutorial. Thanks a lot ! Love you man

[@patloeber]

glad you like it :)

[@krishnabhat547]

Hey, amazing work! Thank you so much for this :D

[@patloeber]

Thanks!

[@mariabarbarasalinasluna4808]

Excellent Python Engineer !!! It will be very helpful ;)

[@Thomas-vo9ks]

Amazing video! Thanks for your helpful content!

[@patloeber]

thanks!

[@nicolasgabrielsantanaramos291]

Best Class ever!!! You're great.

[@patloeber]

Thanks for watching :)

[@fantasymujing1667]

great tutorial, thanks so much

[@weirdduck4578]

Nice this is gonna help me ton with my projects. Really thanx.

[@patloeber]

glad to hear that!

[@DazzleWitch]

Wow, what an excellent video. Thank you!

[@patloeber]

Glad you liked it!

[@sejalchandra2114]

Such amazing content..please create more educational videos

[@eranjitkumar11]

Hi .. thanks for the great content. I see that you did not showcase the inplace operation example for subtraction and division, which seems to not produce results as expected. Do you know why ?

[@ilkayand4]

It is incredible perfect! Thanks a lot

[@patloeber]

Glad you like it :)

[@darthdaenerys]

Thanks, there are a lot of tutorials on jupyter but I wanted to learn pytorch in VS code. This would help me a lot coz I don't use Jupyter notebook.

[@Gamma3]

Thank you. Great channel. I will suscribe

[@patloeber]

Awesome, thank you!

[@abdullahmuhammadmoosa7170]

Hey which IDE are you using? Thanks by the way for the amazing tutorial!

[@robosergTV]

Also how did you set up VScode to have your prints be printed in the OUTPUT tab of VScode?

[@jorgemercadog]

Thank you!

[@art-gin]

Cool tutorial, thanks a lot. Only thing that confused me, at 12:44, lector mentions that both, torch tensor and numpy.ndarray point to same memory location', yet after I ran id(a) and id(b) it printed different addresses in memory (same thing print False for a is b comparison)...

[@masoudrahimi3696]

Thank you for your contents. How can I practive learnings from this video to strengthen my learning?

[@user-ml2ii2tc2r]

Hi, thanks for the containt. I woud ask you what is the IDE you are employing.

[@mastershooter64]

technically tensors are multilinear maps from the cartesian product of 'n' number of a vector space and 'm' number of the dual of that vector space to that vector space's underlying field. i.e elements of tensor product spaces. :) what we're seeing here is are just tensors represented in a particular basis. when they are represented in a basis they look like lists of numbers or an NxNx....xN array of numbers

[@xinqiaozhao5154]

Very nice video! Hope you may make some tutorials about the new released dectectron2 source code~

[@patloeber]

i will have a look at that

[@elektoneman7175]

Thanks for the grea tutorial. May I know how did you run the code directly from the editor?

[@patloeber]

Hi, there is an extension for VS Code that is called “code runner” . Then you can run it directly from the editor :)

[@califflourish6985]

Thanks for this. What is the software you're using here ?

[@user-ys6iw8sq1d]

amazing❤

[@chiboreache]

15:17 AMD Radeon is also supports! python-pytorch-rocm package already in pacman repo, installing pretty easy in Arch Linux _(long time ago compiling from AUR took whole day and 100Gb of hard memory)_

[@GrandAmericaMotorcycleRides]

Great video series. I am on the M1 mac book pro. I had to run command: pip install torchvision in terminal in order for "import torch" to work. Maybe you covered that but maybe this will help someone.

[@nisharahamed8106]

Thanx 👍👍 my one day is wasted for searching this

[@MohakNarang07]

Hey, great video buddy, you earned a sub! There is one thing that is happening in colab (torch version 1.5.0+cu101) which are pretty different than yours: While converting tensors from numpy to torch (and back), incrementing one variable does not change the value of the other.* Code: a = torch.zeros(5) b = a.numpy() a = a+1 c = np.ones(5) d = torch.from_numpy(c) c = c+1 print(a, b) print(c, d) According to what you said, the changes made in 'a' and 'c' should reflect in 'b' and 'd' respectively but that does not happen. Output: tensor([1., 1., 1., 1., 1.]) [0. 0. 0. 0. 0.] [2. 2. 2. 2. 2.] tensor([1., 1., 1., 1., 1.], dtype=torch.float64) Could you please clarify why this is the case? Thanks in advance.

[@patloeber]

Hi. Thanks for subscribing! Be careful. There is a difference between a+= 1, and a = a + 1. the latter assigns the value to a NEW variable a, therefore it does not change the others. I should have pointed this out better I guess. this difference is important not only in PyTorch but in general in python

[@MohakNarang07]

​@@patloeber I certainly didn't know that. Thanks for pointing out the difference. Just to be sure that I understand correctly, a=a+1 increments the value of 'a', creates a new tensor (or variable location) in the memory and assigns the incremented value to it while a+=1 increments the value and assigns it to the same tensor (or variable location) 'a'. Is that correct?

[@patloeber]

Correct :)

[@valarmorghulisx]

Hi thank you so much. i install vs code and i did all extensions which you used. but code runner doesnt work. when i press ctrl+alt+n output says me "python not found" i change right bottom menu language as a python but it doesnt work. i can see in terminal but ctrl+alt+n doesnt work. can you help me? thank you.

[@Aditya_Kumar_12_pass]

ppl worrying what PYTORCH course should they do: this is exactly what u want. (of course calculus is important)

[@patloeber]

glad to hear this!

[@Leshotgun]

What environment is this? I installed Pytorch and now I am figuring out which environment to use it in.

[@alexardelean7787]

what app do you use to code

[@parmoksha]

just 1 suggestion , if you have used jupyter noterbook and retained all operation you did instead of deleting them , then automatically Notes could have been created. you are writing code , deleting it , writing new code , deleting it

[@juleswombat5309]

Awesome Tutorial series Just a minor query. How do I multiply tensors, like matrices. Since I thought most pytorch operation would be tensor wide multiplication and additions, and not so much element wise multiplications?

[@LeonNighttime]

torch.mul(x,y) is matrix multiplication. The element-wise multiplication can be done with torch.addcmul(input, t1, t2) where input is added to the multiplied tensors. You can look it up in the documentation.

[@hasanserdarmacit6901]

hi , i have a for dimension like this torch.size(16, 512 ,14 14), 512 number of features. how can i export this to the txt file

[@popamaji]

@14:15 I didnt get ur point about being on the gpu in order of referencing the tensors, and I got the result that its matter of the operator not device so if u use a+=1 in any device it will refer to the first tensor but it u use a=a+1 the b tensor wont be the same as a in any device correct me if Im wrong btw thanks to ur awesome playlist I have started learning pytorch from ur channel and I would ask many more questions in folllowing days with love

[@patloeber]

yes a=a+1 will create a new reference and so it won't affect b. Sorry i meant this only happens on the CPU, because when you move the tensor to the device with a = a.to(device) you also create a new reference

[@robosergTV]

do you really use vs code without autocomplete? :) Pylance addon does it. There is also "Kite" that is AI enchanced code suggest feature

[@skinnyboystudios9722]

Thank you very much. Are yuo going to do reinforcement learning?

[@patloeber]

This is on my list for the future, but it's not a simple topic :)

[@skinnyboystudios9722]

@@patloeber You are a good teacher. what I fail to understand from lot of people, you cover in simplest way. I wish you continue your NLP tutorial and in the future cover things like GAN's and RL

[@starlite5097]

Wait, I just came here from the first tutorial. How did you go from the console to visual studio? What settings are there to be done? Also, if I want to use pycharm what should I do?

[@patloeber]

vs code has a built in terminal. same thing with pycharm. just make sure in pycharm and vs code that you use the correct virtual environment

[@ayushiagarwal6996]

what i did was I added a torch package and it worked fine

[@AndreyPudov]

There is no direct way to provide a dtype while converting from np to torch, right? The statement like b = torch.from_numpy(a, dtype = np.int32) is not valid. As I found, you can create additional torch arraya and specify a type: tensor_a = torch.from_numpy(np_array) tensor_b = torch.Tensor(np_array) tensor_c = torch.tensor(np_array, dtype=torch.int32)

[@Tech_Ez]

You jumped the gun a bit moving from the terminal to VS Code without instruction

[@nisharahamed8106]

Plz help

[@improvestepbystep]

what is this console and where can we find it?

[@Paulchen2905]

Thanks for the effort you put in your tutorials! But one thing I don't understand: what is the reason (benefit) of reshaping tensors? I watched many explanations but I still don't get it! Hope you can help me out! Thx

[@patloeber]

Thanks for watching. There are many reasons why you need reshaping. One is that e.g. different loss functions expect the Y and Y_predicted in a certain shape (see the Softmax tutorial). Another is that the shape of X is dependent on your first layer in your neural network. And when you work with images, a lot of times you flatten your 2D image array to a 1D tensor. Then you also want to up or downsample your images, so you increase or decrease the size. This needs reshaping, too…

[@Paulchen2905]

@@patloeber Thank you very very much

[@shuturelisharri3691]

I noticed that if the Tensor is created using x = np.random.rand(3,2) y = torch.tensor(x) Then even if you are using cpu and you update the value of x, the value of y remains unchanged.

[@patloeber]

What operation are you using to update x? there is a difference between x += 2, and x = x + 2

[@chandrasutrisno]

What is the difference betweem torch.rand and torch.empty ?

[@advaitpatel4629]

Would you save this file in the virtualenv that was created in the last video?

[@patloeber]

I'm not sure what you mean? For all the pytorch code you have to activate the PyTorch environment that we created in the last video

[@advaitpatel4629]

@@patloeber My bad, I forgot how virtual environments work lol

[@rAnDoMcHaNnEl593]

I know he is using the extension coderunner for the code outpu, but when i try it with code runner, it keep saying python not found.

[@dhananjaykansal8097]

Sir what Python script you’re using to write the codes?

[@patloeber]

You mean my editor? Visual Studio Code

[@dhananjaykansal8097]

@@patloeber Yes your editor. Thanks for a quick response.#regardsfromindia

[@sahhaf1234]

what does "modify in place" mean? I think this si the only obscure point in the video. otherwise, very clear and understandable...

[@user-xz8in1sr8z]

what IDE is this ? thanks in advance :)

[@littlebird52]

If I create an empty Tensor, I get the value 0, while you get a very high value in your tensor, why is it like that?

[@FlyingAnanas]

It is similar to how numpy.empty and numpy.ones works. Empty will generate and array/tensor faster as it just references memory to be part of the array but without overwriting the memory (as opposed to numpy.ones/zeros). Therefore you are just left with what was in there before, which can be anything. If you run the .empty generation a bunch of times you should get lot of different results, often 0 but not always.

[@popamaji]

Hi the x.to(device) isnt working for me, how can I fix it? btw I have cuda availabe and it only uses the device in introduction of variable and wont accept changes

[@patloeber]

hm strange, this should work. is your code like this? device = torch.device("cuda") x = x.to(device)

[@popamaji]

@@patloeber it was just x.to(device)

[@patloeber]

@@popamaji you have to assign it to the new x again.

[@mihaidumitrescu1325]

Fun fact, if one writes b = torch.from_numpy(a.astype(np.int32)), a += 1, print(b, "not modified by reference, because astype was called"), b will not be modified :D This is how bugs are born :D

[@thanioruvan4556]

1:30 Why empty tensor shows some random values at output ??!!

[@d-shiri]

thanks for the video, btw, just being on Linux or Windows is not enough to have CUDA. you gotta have NVIDIA

[@patloeber]

yep of course

[@johnxu2505]

What ide are you using in tutorial?

[@patloeber]

VS Code

[@tesfayeyimam3700]

It is time to subscribe to this channel...One more thing, can yo u suggest me a book related to PyTorch plz?

[@patloeber]

Thanks! "Deep Learning With PyTorch" is ok but not required. I can also recomment the tutorials on the official pytorch website

[@tesfayeyimam3700]

@@patloeber Thank You Dear!

[@jayanthkumar9637]

wooooooooooooooooooooooooooooooooooooooooooooow.....thank you

[@patloeber]

You are welcome!

[@tarungupta2715]

which editor you are using please reply

[@patloeber]

VS code. I have a whole tutorial about my editor setup

[@misbahanwer3528]

@@patloeber where is it?

[@patloeber]

@@misbahanwer3528 www.python-engineer.com/posts/vscode-python-setup/

[@abdallahsharaf5178]

what is that ide?

[@deepakraj008]

Now I am pronouncing the tensor to tenjor.

[@jasminepak6663]

How to change to run on OUTPUT instead of in Terminal?

[@patloeber]

you need the Code Runner extension

[@rAnDoMcHaNnEl593]

@@patloeber I downloaded the Code Runner but when i run the code it can't find python for some reason

[@rrogerx92III]

Minor mistake: You cannot add x and y. One in the gpu memory and the other one in cpu memory.

[@ashwinshetgaonkar6329]

both are on the gpu if top condition is right

[@jabyerre]

3:30 x = torch.ones(2, 2, dtype=torch.float16) RuntimeError: _th_fill_ is not implemented for type torch.HalfTensor

[@patloeber]

maybe in your pytorch version or OS this was not supported. try using torch.float32

[@jabyerre]

@@patloeber Yes, Working

[@patloeber]

@@jabyerre great!

[@zibiax]

I don't get a tensor output. Watched your video on anaconda to get it set up right, but I must have done something wrong. To bad you didn't show how you set it up in between part 1 and 2. Subscribed anyway

[@patloeber]

hmm what error message do you get? Thanks for subscribing!

[@zibiax]

@@patloeber I don't get an error I just get "exited with code=0 in 0.059 seconds". Thanks for answering!

[@patloeber]

@@zibiax this means your code ran successfully! maybe you don't have the print statements?

[@zibiax]

@@patloeber I do have the print statement, but now it just prints "tensor([0.])". It works as it should in the terminal

[@zibiax]

got it working. Had to mark all the code to make it print. 1:20 Thank you very much for the help!

[@proniss]

my empty tensor is just full of 0's while yours has actual values in it

[@Grenzgaenger13]

Thanks for your vids mate. I succesfully went through the first one but I do not understand how to execute python files because I always get a "no module named 'torch'" error when I try to execute a written file from the cmd. However, I could succesfully execute torch functions when I wrote the code in the cmd. Thanks for any help:)

[@Grenzgaenger13]

wow, it only works when I start my program with "python program.py", which I never do since I am used to start my scripts just with "program.py", lol

[@patloeber]

no you should always execute them with "python your_file.py"

[@c0mplicated]

5:27 y+=x

[@anadjurica3035]

Thanks!

[@Regalman]

HOW TO CONVERT A LIST OR ARRAY INTO A PYTORCH TENSOR?????

[@patloeber]

You can easily do this with torch.tensor(list), or torch.tensor(array)

[@DanielVazquez]

Can someone tell me what IDE is he using?

[@alexanderpoplawski577]

Visual Studio Code from Microsoft. Its open source and free.

[@patloeber]

Yes, I have a tutorial about my VS Code setup

[@DanielVazquez]

Thanks!

[@piyushkumar-wg8cv]

Also show errors

[@BestMotivationalVideos123]

Your accent is adorable

[@yssss9368]

watch at 1.75x speed

[@aryankargwal9907]

You speak a bit slow the video rather seems better at 1.5x

[@justinberdell7517]

Very useful and informative but way too many ads. It disrupts the learning process and defeats its purpose.

[@patloeber]

sorry about that :(

[@gameguy7348]

me: tf is his voice (tooooooooooorchhhhhhhhhh....) mind: nah his gay

[@exploraj4380]

tenzoa nhi yarrr tenzor hai No offens...brothers

[@user-kt8nc4xd1u]

Man, am I coming here for the ads? The ads even take long time than the main contnent

[@mittalmayankcool]

too many advertisements

[@PWK95]

Okay, the way you pronounce "tensors" is super german

[@patloeber]

Haha, yes I can't hide my accent ;)