One Week In With NVIDIA’s StyleGAN2-ADA

[@abrahamcastro8424]

Hey, this is all really interesting. I stumbled upon this and I'm glad I did. I'd like to start learning some of this stuff so I gotta start looking into this

[@termi2828]

those results are really quite amazing

[@zoombapup]

Whats the min GPU memory you'd recommend for running this model? Most of the GAN models seem to be pretty big. I'm waiting on purchasing a new GPU (current NVIDIA cards are totally out of stock everywhere, apparently only had a few thousand available worldwide, go figure!). I can handle small-medium models with my current 11gb GPU, but I'm curious if the rumoured 3080 with 20gb would work for something like this? Or would a 24gb like in the 3090 be more comfortable for having a bit of headroom to spare?

[@GesusOfYou]

I think 16gb is usually the lower end for most bigger networks. I would guess you are not gonna feel a big difference between 20gb and 24gb. Its just gonna be a bigger batchsize. Here are some early benchmarks for 3080 and 3090: www.pugetsystems.com/labs/hpc/RTX3080-TensorFlow-and-NAMD-Performance-on-Linux-Preliminary-1885/ www.pugetsystems.com/labs/hpc/RTX3090-TensorFlow-NAMD-and-HPCG-Performance-on-Linux-Preliminary-1902/

[@ReamKovalski]

3090 only has Cuda 11 for now. SGAN2-ADA requires CUDA 10 (cudnn 7.0 seems to work), tried with V100 on vast.ai with official tensorflow Docker image. Porting to Cuda 11 will not be done by NVIDIA since it is one-drop situation, and it will require rewriting some of the plugins with NVCC. I am interested in actually adapting it to RTX3090, it is much cheaper and faster option than V100, but it will take time for me to grok the new repo. The model (512 transfer with stylegan2 config) I am currently training requires 8.1 GBs, so I think 11 GBs will suffice. I would stay away from 30xx series for now, and would use Paperspace or vast.ai for training, considering the support for cuda 11 is not yet coming. You can get a V100 fairly cheap these days, and ADA does converge super fast

[@zoombapup]

@@ReamKovalski Hmm, I could always spin up an amazon instance if I needed to I guess. Just been considering what GPU to get once the 30 series actually becomes available. I suspect that 3080 with 20gb will end up being the reasonable price/performance champion for me. Then I can buy a few of them for different PC's. Good to know that 8-ish will be enough. Will give it a try when I get chance. Thanks for the info!

[@ArtificialImages]

The new mixed precision settings means an 11GB GPU will work fine. I always recommend getting something larger if you can afford it because other models need more, and I’m guessing the whole industry will reset their baselines with these newer models being cheaper and easily available.

[@ReamKovalski]

@@zoombapup note that I am talking about a 512^2 training here. Your milage may vary haha. But totally agree with you and Derrick that 3080/3090 will be super popular since they are probably the first widely available gpus with decent pricing. 3090 is also something I consider getting, and hence wanted to adapt some of the cuda plugins they supplied, unless someone is already working on the fork...

[@emersonuyghur007]

are u using colab pro or colab pro plus ?

[@xwildzerox666]

Aaah! After learning from your previous videos, I’ve been trying to train a set on those same crystal microscope images from IG, but with a bit less luck than this. These look great. Are you sharing that model anywhere?

[@ArtificialImages]

not at the moment

[@nguyenanhnguyen7658]

Training StyleGANV2-ADA right now... Any tips ? Thanks man.

[@吴迪-b6s]

Hi.thanks for your videos,i have success run my train. I found there's no variable named ' resume_kimg' in train_loop.py with styleGAN2 ADA version, but this variable is in styleGAN2 version, so is ADA version don't need this variable when I resume my train? And what's the different between styleGAN2 and sytleGAN2 ADA?

[@veryantennae]

Very beginner question i guess, but i keep searching for answers on how to figure out what a good dataset size is and cannot seem to find a good one. I understand this new version of StyleGAN makes it possible to work with a much smaller dataset, but in this video you seem to say things start being good after 32 kimg. Would this be a good recommendation for the smallest possible dataset? I'm asking because, as a beginner, i feel like this number is also linked to the number of iterations of the training, and not the number of distinct images originally found in the dataset.

[@ArtificialImages]

I’ve gotten it to work relatively well on datasets as low as 500 images. The more images you have the more "realistic" it will look, but depending on the effect you’re going for you can go a lot lower than 30k.

[@veryantennae]

@@ArtificialImages thanks! this is definitely very encouraging!

[@LeoLeenus]

is it possible to create a stylegan realtime with realtime video footage?

[@ArtificialImages]

Not really. Training time is required.