9 ай бұрын
9 ай бұрын
10 ай бұрын
2 жыл бұрын
3 жыл бұрын
Пікірлер
@kipyo 2 күн бұрын
Thank you! This was super helpful, since I had the same issue with installing the long list of programs and not getting the cuda linked to pytorch
@EdgeYu 2 күн бұрын
Hi Alex! Thank you so much for uploading this series of Spatial Transcriptomics analysis series! I personally am working with GeoMx DSP data currently. Due to its relatively lower resolution, do you think the techniques introduced in this series will still be applicable to the data I have in hand? If so, which ones you think would be particularly helpful! Thank you very much Alex! Appreciate your time and guidance on this!😊
@alexsoupir Күн бұрын
Hey! With GeoMx I think it's more appropriate to treat the data like bulk sequencing with the note that if there are multiple measures per sample, they are _not_ independent measures. This means that when doing the analysis and associations, you need to take into account the repeated sampling. A nice package that does this, and contains p-values, is *lmerTest*. If this is indeed your case, then I would consult a statistician or a biostatistician about modeling (perhaps will do a video some day about this because it's important). The variable that has variation in it (like gene expression or protein abundance) needs to be the _response_ variable while some class (like treatment or response to treatment) needs to be a predictor followed by a random effects term. gene_cd3 ~ treatment_response + (1 | patient) It's not entirely intuitive unless you think of it how you would plot the data. Likely you would "responder"/"nonresponder" along the x axis (independent variable) and on the y axis would be gene expression (dependent variable). If you flip the expression and treatment response then you would need to average the gene expression which is removing information. Hope this helps!
@memeflipz 7 күн бұрын
Hey man, cool guide; helped me setup the GPU on my pytorch.. thanks a lot :D
@anthonyl.g1698 19 күн бұрын
excelente video me funciono bien, por ahi tuve un error al ultimo llamado : Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.; pero lo solucione eliminando el archivo en el entorno del resto bien.
@LeoMaster12352 22 күн бұрын
You will never see me comment something again, but it worked on my rtx 4070 laptop. I have no words to describe this momment, thank you very much. Some important comment: After performing the video complete installation, I tried using python=3.10 and it worked too (a used the command "conda create --name cuda_test python=3.10 " but the version installed was Python 3.10.14). Also, other important comment is that the CUDA 12.6 Version was installed for me. I hope that you all feel the hapiness that I'm feeling now. Thank you!!!!!
@classytonyyt 28 күн бұрын
Out of all the videos, thank you for working on the command line, and using a different approach from others. No one really focuses on making sure that the GPU gets dedicated, which was my case where Pytorch would not find my GPU. Thanks once again!
@yusufyagc2510 Ай бұрын
amazing. thank you so much
@NguyenHoang-g8c Ай бұрын
(cuda_test) C:\Users\windows 10 - 03>conda install pytorch-cuda=1.8.0 -c pytorch -c nvidia -y PackagesNotFoundError: The following packages are not available from current channels: NVIDIA-SMI 442.50 Driver Version: 442.50 CUDA Version: 10.2
@sorrefly Ай бұрын
My app is deploying but no web portal is reachable. I'm trying to understand what is going on via logs but it says "No Pools Found At least one pool must be available to use apps", does anybody have any clue? 😥
@MerajLakvidu Ай бұрын
Tried out different things to setup this properly including the command given by torch website for installation in conda but nothing was able to make it work. Followed your method of separately installing pytorch-cuda and other modules and now everything works fine! Thanks for sharing your knowledge and experience. It was really helpful.💯 Also if you know, does cudnn automatically installed in the local environment when we are installing pytorch-cuda? The env I used to setup pytorch-cuda was cloned from another env which already had cudnn so I have no idea about this...
@rezafard4397 Ай бұрын
you're a life saver.
@CJ-te6ds Ай бұрын
I was having issues installing rstudio to my arch install and this works great as a hacky workaround! Thanks
@glebeshye7197 Ай бұрын
Thanks a lot! You are my saviour
@prestonfeivor5488 2 ай бұрын
dear god thank ypu so much!!! ive spent ALL DAY uninstalling and reinstalling different versions of cuda and pytorch trying to get this to work. The install in 2 seperate sections is the only thing that worked and I NEVER would have tried that!
@StudyGoalTensionEnjoylifelove 2 ай бұрын
error: could not find a version that satisfies the requirement torch==2.1.0 (from versions: 2.2.0 cpu, 2.2.1 cpu, 2.2.2 cpu, 2.3.0 cpu, 2.3.1 cpu, 2.4.0 cpu) error: no matching distribution found for torch==2.1.0 kindly guide how can i fix this problem
@MuditGupta07 2 ай бұрын
amazing. When nothing worked this did
@salmalmiry792 2 ай бұрын
thanks a lot
@robfielding8566 2 ай бұрын
a few years ago, i wiped my machine a few times, trying to follow the official NVIDIA instructions to get my GPU accelerated. i eventually got it. the interns at the company were waiting for 8hr RL jobs, and I could run them in an hour. So, I gave them my laptop for the summer; my personal laptop. then I talked my boss into getting accelerated machines for our ML expert. they bought her 3 x $10k machines; with 2 GPUs in each one. But guess what? We got them, and tried to follow the same NVIDIA instructions to get it setup. I think another dev finally got it working in a week. There are always deviations from the instructions that prevent scripting it. It was NEVER updated. Meanwhile, when I got my personal machine back; the day came when I had to do other work; so I updated something in Python; and it BROKE the GPU. I got a System76 as my next system, to replace my Macbook. I truly got tired of re-installing to keep acceleration working. NVIDIA has these long instructions to follow, because it's not possible to SCRIPT the setup; you know... like being able to run Linux without building the kernel first. On that System76, long after I gave up on Torch being accelerated; I had to do a full Ubuntu distro upgrade; and suddenly acceleration worked again. I did some HuggingFace tests with it. And then a few weeks ago, I originally had accelerated Torch working with LLAMA3, and it worked for a few days. It's broken now. When I tried to use the venv with Python, I would always have breakage. Every time, I have to eventually give up and go to Conda just to have random Python apps not break; especially when they link with C. Specifically, in venv recently... i had a base python3.10. I made a python3.12 environment to run an LLAMA3 model. But during Torch installation, it got a failure while it was trying to modify python3.10 !!!! which it should not be able to find at all. But it found it when it had to escalarte to root. Then, I had to create a conda environment for python3.11 to get Mojo to run. When I activated Mojo, I was already in the globally activated python3.12. So, you can guess what happened next: There was something I was doing in the conda env with python3.11, and it managed to find python3.12 when it escalated to root.. I know the person that wrote these things: Requests, Certifi, Venv; and he was one The Python Guy at Heroku. When I was bitching about this scenario; he chimed in with a like... apparently confirming that it's not just me. I would just run an NVIDIA Linux distro if I could just get Pytorch acceleration to stop constantly breaking.
@MustKilLL 2 ай бұрын
RIP Truecharts
@alexsoupir 2 ай бұрын
Sadly agree. iX killed the ability to run docker in some update and will introduce again in the next version, and in the mean time can't use True charts either. No easy way until this fall - wanted a custom image but can't without some jailmaker which I think I read the developer is no longer going to work on? Need to read more but really unfortunate.
@MustKilLL 2 ай бұрын
@alexsoupir yeah dev is done. I'm sure what's next. im running docker rn, but its not the same true charts was great!
@chrismay2298 2 ай бұрын
It's going to be much better when they roll out the docker and docker compose native update.
@benque7491 2 ай бұрын
lmao
@THEG12EG 2 ай бұрын
compose is so much better... I am struggling to configure apps in charts seems to have less customisability
@pick4life 3 ай бұрын
Awesome!! that helps a lot, by any chance could you please do a tutorial in how to deploy a docker container running windows in True NAS due to my CPU does not support KVM extensions, I think there are many guys looking in how to do it, Thanks!!!!!!
@alexsoupir 3 ай бұрын
Going to have to look and see if there is an easy way. Current version of TrueNAS doesn't work with custom docker images or docker compose - they are going to add it in a version closer to the end of this year. At that time I could try. Found this: github.com/dockur/windows which, if familiar with setting environmental variables in TrueNAS I think could be worked through.
@danielmpr 3 ай бұрын
That optoon is NOT on my TrueNAS Scale!, only setting and discover apps
@mrkongster 3 ай бұрын
Thanks for the great walk-through.
@Gramphimcs419 3 ай бұрын
This video is fantastic. It was not until I followed these steps that I managed to get gpu acceleration to work in pytorch. It worked flawlessly for both windows and linux. Thank you so much!
@PhilStein721 3 ай бұрын
I hope so, because I'm almost throwing my laptop through the window I really hate having to try setting things again and again because windows can't even handle path variables
@alexsoupir 3 ай бұрын
Were you able to get it to work? Laptops work great as frisbees, but I would urge using something else to throw.
@AntoAshwin 3 ай бұрын
Can we a custom dataset created by us?
@alexsoupir 3 ай бұрын
Of course. Something to keep in mind though is that the number of classes, type of dataset you have, and number of samples with greatly impact how much you can shrink dimensions with your latent space. More data, smaller the middle layer. Less data, less you'll be able to compress the data.
@enamulrafti2780 4 ай бұрын
i have nvidia t4-8Q, I have CUDA VERSION installed 11.4. Now i have python 3.11 and tried cuda 11.8 and downloaded the other dependencies. But after executing the torch.cuda.is_available() it returns False. What can i do?
@alexsoupir 3 ай бұрын
Are you installing in the correct order? I've noticed that is you install pytorch before other GPU dependencies like the cudatoolkit, it won't pick up the GPU. Might be worth a shot.
@allen123yu 4 ай бұрын
Great stuff. It works when I follow your instructions, but if I create an environment on python=3.9, it will fail. May I ask if you know why?
@deadadventure8318 4 ай бұрын
I don't usually comment. But this video saved my life. My dreams can now begin without it being crushed at the beginning. Using the pytorch local command would install a bunch of libraries causing it confuse my computer whenever I would try to run any YOLO models. But with your step by step guidance, I was able to run it after trying so tediously for 4 days. Thank you.
@scorpionking-kj1rp 4 ай бұрын
Thank you, DR. ALEX! I can't wait to cook up some Neural Nets :)
@serychristianrenaud 4 ай бұрын
Thanks
@Pyrocraker 4 ай бұрын
Hey, this isn´t working on my TrueNAS Scale?! When I go to the "Apps" tab, there is no "Installed Applications", "Available Applications", "Manage Catalogs" and "Manage Docker Images" to chose. It looks different. When I go to Apps, there is just "Installed" at the top and "Applications" below. On the right upper corner is a blue "Discover Apps" button, which opens the lib, but here is also nothing with docker.. Was there a GUI upgrade? Was this option removed? Do I have to set something differently in TrueNAS? Isn´t pulling a Docker Image and starting it from the Apps no longer possible? Thank you in advance! :)
@alexsoupir 4 ай бұрын
I don't have the most updated version of TrueNAS because, for me, they broke Docker in v23 the way I was using it in v22. The version I had that worked I think was 22.12.2, and have seen others on Reddit say in recent versions things have been breaking everywhere (most recently there's a RAM leak/release failure??). I haven't updated to v23 or even v24 because of the constant issues I see others have online in combination with the docker containers no longer working. Even tried to use docker in the terminal on TrueNAS v23 and it still threw out errors. Very frustrating. In other posts too, it looks like they're trying to move away from docker in favor of docker-compose, but i'm not familiar with docker compose (another reason i'm sticking with the old version of the OS). Unpopular opinion, but seems like a constantly changing, using users as the test case, operating system that is only friendly to people who like to solve problems constantly. They also seem to be at ends with TrueCharts for whatever reason. If you want, you can try to install the old v22 here: www.truenas.com/docs/softwarereleases/ see if that allows you to install containers and start apps. If so, then it's definitely something iX has been doing with TrueNAS. I have nothing super security focused so staying a few versions behind until they figure out their issues is no problem to me. Don't like the idea of having to constantly fix things once they are working. (sorry for the minor rant)
@Pyrocraker 4 ай бұрын
@@alexsoupir thank you very much for the detailed explanation! I started all this 1 week ago, so Im totaly new to all of this networking stuff and yes its a veeeerrry wide and often frustrating field to me so far... I even thought about switching to unraid, since a friend of mine is constantly talking about how perfekt it is...but 250 bucks for the unlimited update version is a lot! Even more than my hardware :D
@alexsoupir 4 ай бұрын
@@Pyrocraker haha I'm with you on the unraid thing. If it was free I'd probably switch too. Been thinking about playing with Ubuntu server since docker is really the thing I currently rely on the most but it's hard to choose one way or another. Once you find one that works for you, then can stick with it. Hard to initially find it though. Like I mentioned maybe an older version of truenas is the way to go. Find your fit and the ride it along.
@CaptStux 4 ай бұрын
Hey @alexsoupir, I made a video on using docker compose on TrueNAS with Sandboxes and Jailmaker. Jailmaker <2.0.0 works on TrueNAS Bluefin and Cobia iirc, and also on the latest releases. It’s a good way to run custom docker apps, and may unblock you. kzbin.info/www/bejne/iWHRhYWsdq10hpo
@InformaticFreakTutorials 4 ай бұрын
They moved the interface from "Launch Docker Image" to "Discover Apps" > "Custom App".
@manojjoshi2318 4 ай бұрын
is it useful for the nvidia cuda ??
@alexsoupir 4 ай бұрын
nvidia cuda GPU is a requirement, yes.
@robertodepetro1996 5 ай бұрын
thanks!
@4m05 5 ай бұрын
I am going to have to move away from truenas scale, I just cant get truecharts working, even loaded it onto a new machine fresh install, and nealy every app just gets stuck on deploying with 0 cpu or ram, truenas work fine, but truecharts doesnt work for me its a shame as I like truenas scale
@Hankks 5 ай бұрын
Great video. I was surprised to see it was possible to only download cuda in the environment which is great for my project. I am abit curious why you are downloading the pytorch for cuda 12.1 when your cuda download clearly states it is 11.8?
@alexsoupir 4 ай бұрын
Haha a lot of times in informatics/data science, if it works don't try to fix it. It installed and runs/recognizes cuda in the opposing versions. BUT, since it works and uses the GPU, I consider that a win. This question has come up other times though and the real answer for why it works is "I don't know". Assuming Pytorch is backwards compatible, as long as cuda and drivers aren't ahead of pytorch, perhaps that's why it doesn't complain.
@adi6516 5 ай бұрын
THANK YOU VERY MUCH!!!
@abdelrahmanmahany133 5 ай бұрын
It is great to understand the difference between supervised and non supervised phenotyping in a practical way. Thanks a lot Alex !
@mehdiraouine2979 5 ай бұрын
did you already have your cuda toolkit downloaded ? because my new computer with very good specs took forever to complete that task alone. Not an internet speed or vpn problem either because it takes few minutes to download 5gb files from an internet browser. Any idea on the reason why ?
@alexsoupir 5 ай бұрын
Interesting. Don't think I had it installed before because other environments when not installed in the right order, pytorch doesn't know cuda is available so defaults to the CPU version. I did have the *driver* installed, but not the toolkit.
@aymanzouhair5898 5 ай бұрын
May god bless you !
@RuqaiyaTasneem-z5w 5 ай бұрын
how many read files do we have to download for a single WGS of a single bacteria ?
@BhanuPrakash-pj1ev 5 ай бұрын
Error: nvidia-smi is not recognized as an internal or external command,operable program or batch file. Please help me out this
@alexsoupir 5 ай бұрын
This sounds like a driver problem. Do you have the latest Nvidia GPU driver installed?
@tsubaki9146 5 ай бұрын
thank you so muchhhh! finally can use my gpu TT
@abdelrahmanmahany133 5 ай бұрын
Thanks for the great tutorial. I faced a problem that when running spatialTIME NN_G() it gave me values without numbers just NaN. What could be the reason?
@alexsoupir 5 ай бұрын
There's a few possible reasons. Usually when "NaN" is produced over "NA" that is due to spatstat.explore::Gest not liking the input data since it passed our checks (e.g. needing so many cells to calculate). I would plot the sample that is producing these "NaN" values and see if there are any reasons visible. Maybe no cells within the search radius and maybe not an appropriate edge correction method (just guessing). Another thing might be that the cells of interest are too close to the edge of the sample and too far away from each other that the edge correction is dominating the weight of the nearest neighbor measurement and spatstat just kicks it out. Can extract that sample, create a window with convexhull.xy, create a ppp object with ppp(data$xlocation, data$ylocation, window, ...), then run Gest on that ppp object to see what happens. Would just need to explore your data a bit more.
@abdelrahmanmahany133 5 ай бұрын
@@alexsoupir What confused me that I followed your tutorial step by step with the same dataset but the output was as I typed previously
@alexsoupir 5 ай бұрын
@@abdelrahmanmahany133 That is strange unless it's a sample thing as I mentioned. Did the same sample/cell type that gave NaN for use give values for me? There are instances that NaN is expected.
@abdelrahmanmahany133 5 ай бұрын
@@alexsoupir Yes it is the same sample/cell type. The only different thing in my trial that I installed the most recent version of spatialTIME as I noticed that you said that the version in the video is not the most recent.
@alexsoupir 5 ай бұрын
@@abdelrahmanmahany133 Open an issue on GitHub and if you can remove all other spatial data frames from the mIF to upload on GitHub. Then I can take a look
@ZhiyuanCheng-z8p 6 ай бұрын
Sorry, Sir. One more question: for spatialTIME, when they create windows by owin (I assume), did it use polygonal or mask, which make me concern of border correction in my TMA cores.Thank you so much for your attention
@alexsoupir 5 ай бұрын
No worries! When creating a window for the calculations, it is done with `convexhull.xy` using the cells x and y locations (as specified with `xloc` and `yloc`). Neither `polygon` or `mask` are options in this function and I have never used those options when calculating spatial summary functions. I *have* used a mask to mask off parts of a window with *simulating* point patterns with functions like `rpoispp` to create a Poisson point pattern from the Poisson process, but this is irrelevant here. The edge corrections work correctly with the `owin` object created with the `convexhull.xy` as the cells are creating the boundary of the observation window which is not always the case when using masks. If you are really interested in diving into some of the methodology, play around with some of the subfunctions from the spatstat such as `edge.Trans` to convince yourself of the impact the cells distance to the window has on the amount it's being adjusted (closer = greater). Lots of functions that are within `Kest` and `Gest` may help with understanding these. But to answer your question, we are essentially creating a single polygon that is created by drawing as round of a circle as possible around all cells in a sample with `convexthull.xy` so the edge corrections are working as they should.
@ZhiyuanCheng-z8p 5 ай бұрын
@@alexsoupir Dear Sir, thank you so much for your response !
@mancobonkey4595 6 ай бұрын
Alex! You are the best man! I wish you and your family wealth and health!
@pelumimagret7447 6 ай бұрын
Can you also teach how run PanX which is used for pangenome of less identity between species which Roary cannot do
@riddheshmore2315 6 ай бұрын
Thanks a lot after so many uninstalls (4hr) ;(, this helped . i got Command 'nvcc' not found, after installing NVCC from the official website. But i got True while checking is_available. I dont understand why i got this error ?
@alexsoupir 5 ай бұрын
This is interesting. Wonder if the drivers have it and happened to already be on mine? There are some discussions online about now having a cuda specific library. NVCC must have been part of some other package install like a driver install or something. From reading, it doesn't get installed specifically without the cuda library (see: stackoverflow.com/questions/56470424/nvcc-missing-when-installing-cudatoolkit). While not having cuda installed on my system before installing here, it must be carried over from another software. Even made sure that outside of this cuda environment that cuda is NOT available. Great issue you happened across. Not always fun, but good find. Hopefully someone else can benefit from this.
@riddheshmore2315 5 ай бұрын
@@alexsoupir Got it. Thanks once again for your insightful video.
@Finnick5 6 ай бұрын
Thanks ! You solved my problem 🙏
@nasrinravansalar 6 ай бұрын
very very helpful. thank you so much
@frederikvanaverbeke8840 6 ай бұрын
worked like a charm
@knowit999 6 ай бұрын
THANKS, REALLY HELPFUL!