@yiluwididreaming6732 Same ControlNet options can be used with ComfyUI, but in the vid I've used Automatic1111 because it is a bit more common - or what's your question?

@@NextTechandAI Sorry, just about comfy work flow for this? thanks

Do you have in the command window an output entry similar to this one? "Device: device=AMD Radeon RX 6800 [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver=" and you have used the --use-zluda option? Are there additional error entries when starting up/generating? Have you used a new Conda/Venv environment? Are you using different drives for SD.next and ZLUDA? Are you using this ZLUDA version github.com/lshqqytiger/ZLUDA (there have been updates)?

@user-fg6mq3dg3d Thanks a lot for your feedback, I'm happy that the video was useful.

@vargaalexander What exactly is the problem? The video is about Automatic1111 and SDXL.

@@NextTechandAI Error running process: F:\ST\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py Traceback (most recent call last): File "F:\ST\stable-diffusion-webui\modules\scripts.py", line 825, in process script.process(p, *script_args) File "F:\ST\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process self.controlnet_hack(p) File "F:\ST\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack self.controlnet_main_entry(p) File "F:\ST\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 898, in controlnet_main_entry Script.check_sd_version_compatible(unit) File "F:\ST\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 827, in check_sd_version_compatible raise Exception(f"ControlNet model {unit.model}({cnet_sd_version}) is not compatible with sd model({sd_version})") Exception: ControlNet model control_v11p_sd15_seg [e1f51eb9](StableDiffusionVersion.SD1x) is not compatible with sd model(StableDiffusionVersion.SDXL)

@vargaalexander I pointed this out in the video. You can only combine SDXL with SDXL and SD1.5 with SD1.5. You are using an SDXL checkpoint with an SD1.5 ControlNet model. That does not work.

@@NextTechandAI wow thank i didnt know that my checkpoint was sdxl.... THANKS a LOT

Thanks for confirming the solution.

Thank you very much for your feedback, I'm happy to see such a speed increase. If you want to use Automatic1111 instead of SD.next, this can be done quite fast following this short: kzbin.infoYzFRlsEYyEE

@@NextTechandAI couldn't get it to work, it doesn't recognize the --use-zluda command, not sure what's going on and I'm bored of debugging for a while :D, and SD.next does have some nice functions, but the UI is indeed a lot worse.

@J2thaBgeerie What a pity. It must be a fairly recent installation of A1111, but I can understand that you want to enjoy the speed of SD.next first and don't want to configure it any further :)

I can understand that.

@@NextTechandAI im sorry, i tried it for a week,im just not good at this... got frustrated

@@__-fi6xg Honestly, I can understand that. When I bought my RX 6800, I didn't know that I would get so involved with AI. Inevitably I'm making the best of it now, but with an NVIDIA GPU everything would be a lot easier :)

@alanreynolds4262 Hello, frankly speaking, I don't know. The architecture of your GPU is very similar to the RX 7900 XT, but e.g. for the RX 6xxx series below 6800 special libraries are required for ROCm HIP as described in my vid. Maybe this is the case for the GRE versions, too. Currently there is an open issue in SD.next main branch leading to an "Diffusers failed loading" error. So just in case you can go one step further you might have to wait for an update or use an older commit of SD.next.

@ffwrude Hi, indeed fingers touching each other is more difficult and and pushes especially Openpose to its limits. In fact SD can generate even such hands in good quality, it's just less likely. So I would inPaint as described and generate several batches. Another option is to take a picture of your hands (or a template you have generated) in the desired position and use depth or canny, increasing chances for good hands a lot.

@soraphop99 Have you already successfully generated images with ZLUDA on SD.next? When running SD.next, do you have an ouput similar to this one "Device: device=AMD Radeon RX 6800 [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver="?

@@NextTechandAI Device: device=Radeon RX 570 Series [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver=

@@NextTechandAI SD.next err OSError: Building PyTorch extensions using ROCm and Windows is not supported

@soraphop99 Your RX 570 is not officially supported by ROCm, see the website containing the list in my vid. You could try if you can find suitable libraries similar to the ones from brknsoul, but in the comments there was nobody so far running the installation with less than an RX 6xx0.

I can only guess. Maybe you have to update your Zluda files. In any case the start and very first image generation will take very long again after an update. When starting SD.next, was there a line like this Device: device=AMD Radeon RX 6800 [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver="? At which point did you get the error?

@@NextTechandAI The error appears when loading models. And now there is another error: "OSError: [WinError 126] Das angegebene Modul wurde nicht gefunden. Error loading "F:\Stable Diffusion\SD.Next\venv\lib\site-packages\torch\lib\caffe2_nvrtc.dll" or one of its dependencies." Even on frsh SD.next install. Somtheing wrong with Python modules: "F:\Stable Diffusion\SD.Next\venv\lib\site-packages\torch\__init__.py:141 in <module> │ │ │ │ 140 err.strerror += f' Error loading "{dll}" or one of its dependencies.' │ │ > 141 raise err │ │ 142" Initialization before looks like this: "Using VENV: F:\Stable Diffusion\SD.Next\venv 10:41:56-762871 INFO Starting SD.Next 10:41:56-765373 INFO Logger: file="F:\Stable Diffusion\SD.Next\sdnext.log" level=INFO size=29636 mode=append 10:41:56-767377 INFO Python 3.10.9 on Windows 10:41:56-846333 INFO Version: app=sd.next updated=2024-05-07 hash=e081f232 branch=master url=github.com/vladmandic/automatic/tree/master 10:41:57-383979 INFO Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows release=Windows-10-10.0.22631-SP0 python=3.10.9 10:41:57-389487 INFO AMD ROCm toolkit detected 10:41:57-402589 WARNING ZLUDA support: experimental 10:41:57-404095 INFO Using ZLUDA in F:\Stable Diffusion\ZLUDA 10:41:57-457633 INFO Extensions: disabled=['sd-webui-controlnet'] 10:41:57-458635 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg'] extensions-builtin 10:41:57-461632 INFO Extensions: enabled=['prompt_translator', 'sd-webui-mosaic-outpaint'] extensions 10:41:57-464036 INFO Startup: quick launch 10:41:57-464036 INFO Verifying requirements 10:41:57-470543 INFO Verifying packages 10:41:57-471543 INFO Extensions: disabled=['sd-webui-controlnet'] 10:41:57-472673 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg'] extensions-builtin 10:41:57-475185 INFO Extensions: enabled=['prompt_translator', 'sd-webui-mosaic-outpaint'] extensions 10:41:57-479188 INFO Command line args: ['--use-zluda', '--autolaunch'] autolaunch=True use_zluda=True ┌──────────────────────────────────────────────────────────────────── Traceback (most recent call last)"

@user-ei8kk8vp7k In one of the comments there was a hint, that it doesn't work on external drives. In order to avoid any problems related to the path I've put my installation of SD.next and Zluda etc. on same drive; best option is C:\. Other possibility: You have older installations of SD.next/A1111 or Zluda and they interfere with your current installation.

As far as I know it should work. There's currently an issue with AnimateDiff v3, you could try AnimateDiff v2.

@@NextTechandAI Indeed, it seems to work with the v2 model. Thanks for the advice.

@@Painbeas I'm happy that it's working, now. Thanks for sharing.

Thanks for you feedback. Which steps have been skipped in your opinion? I had to fast forward sometimes in order to keep the video short, but it was my goal to include all necessary steps. Which AMD GPU are you using? Have you interrupted the very first image generation with Zluda? With my RX6800 it took nearly half an hour, but not longer. When starting SD.next, was there a line like this Device: device=AMD Radeon RX 6800 [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver="? Yes, Zluda works with SDXL models. All images in my latest video about ControlNet with SDXL have been created by using my Zluda installation.

@@NextTechandAI Thank you for getting back to me. I will comb through your video and find the missing step/s. In the mean time, here is the answer to your other questions. I am using a 7900 XTX and the Devices line says this "Devices: " I checked in the system information tab of SD.Next and there is no GPU listed in the GPU section. So maybe it is defaulting to my CPU? I thought I followed instructions perfectly. Any thoughts?

@@NextTechandAI Upon rewatching, I found only one step that I remember having to pause and google. At 5:30 you talk about editing path variables but don't show how to get to the appropriate menus.

Well, but at 5:30 I'm hinting at a document which describes how to edit the path in case you don't know. Nevertheless, if your device is not listed then your Zluda installation is not complete and - yes - in this case it's using the CPU. Difficult to guess which step is missing, but according to one comment you should make sure to have Zluda and SD.next on same drive (in best case C:\).

@@NextTechandAI I restarted from scratch, followed every step in video exactly, and my device is still not showing up. I guess I will have to wait for new solutions. Sad stuff.

@jiuvk8393 Regarding #2: When I created the vid, there was only ROCmLibs.7z. Choose this if you don't have gfx1031 or gfx1032, Optimised_ROCmLibs_gfx1031.7z or Optimised_ROCmLibs_gfx1032.7z if you have one of these. If this doesn't work, I would try ROCmLibs.7z and additionally overwrite with Optimised_ROCmLibs_gfx1031.7z or Optimised_ROCmLibs_gfx1032.7z depending on your GPU. Alas, in the readme it doesn't tell whether Optimised_ROCmLibs_gfx1031.7z and Optimised_ROCmLibs_gfx1032.7z include all required files or only customized files. Regarding #1: I'm sorry, but so far no one has reported in the comments that they have successfully gotten it to work with a GPU below an RX 6xx0.

Hi, I'm sorry, but so far no one has reported in the comments that they have successfully gotten it to work with a GPU below an RX 6xx0.

That's strange, the script should activate SD.next's venv. Does SD.next run successfully including Zluda on your PC? Are there any anomalies in the cmd window?

@@NextTechandAI Yes. I was able to get around it by running "activate.ps1" and it let me launch ComfyUI. Weirdly, after the most recent Reboot and Update for Windows, it now works just like it did in your video. Very weird. I now have another question: where do I put arguments in the zluda ComfyUI launch to force Low VRAM mode? I'm trying to keep it from spilling over into Shared GPU memory and slowing to a crawl.

Run python main.py -h and you get a list of all available options.

@yumiibrahim5513 Well recognized. SDXL allows higher resolutions and shorter prompts, but that comes with a price. As I said in the video, there is no official ControlNet repository for SDXL, so the models are customized accordingly. I also noticed the difference in quality, but I can't prove it. The running times are clearly too long for me to run longer test series. Which brings us to your second point. I still have to generate a lot of images to get satisfactory results. This takes too long for me with SDXL. I use SD1.5 (usually with ControlNet) and Upscale as described in my other video. The result can then be used as a template for SDXL with depth or canny if I really need it.

@@NextTechandAI thank you for your response. Good idea to use 1.5 then use the result as a template. Thank you.

@yumiibrahim5513 Thanks for your feedback :)

The directory structure is same as in Automatic1111, so put them in models\Stable-diffusion.

Why not using GPT4All locally with Llama-2 or Llama-3? It supports AMD GPUs and has a server mode and an API. See my vid: kzbin.info/www/bejne/ipS7q6yrqcughdk

@@NextTechandAI oh I didn't know about this. I was only making RAG chatbots on kaggle and wanted to shift to local. Do you think an RX6600 can work with GPT4All?

@jasondsouza3555 GPT4All uses the Vulkan API for GPU support, so I think yes, it should work with an RX6600.

Which source are you referring to with this statement? You might have a look at my linked vid. There you can see even the related Git isses, it's obvious the ROCm AI libraries are still missing. Torch, A1111, etc. have open Git issues for the implementation and are still waiting for ROCm.

@@NextTechandAI Just something I heard from the dev's that supported ROCm for language models on windows, so not really a reliable source

Thank you very much for the feedback. I'll add it to my list, haven't expected this demand for Forge.

Thanks a lot for the like and the sub. I'm happy that my vid was helpful.

Does SD.next with Zluda run on your machine? At which point do you get this message? Are there other errors? Which GPU are you using?

@@NextTechandAI I tried long ago with sdnext as well it used to say using zluda as experimental and also got a message rocm toolkit was found, but it was using my cpu all the time. I have 7800XT AMD GPU

@hny4uable Now I'm irritated. The basic requirement for installing Automatic1111 according to this short is a running installation of SD.next with Zluda as described in my corresponding video.

@@NextTechandAI I understand you, but sdnext should use my gpu instead cpu while using rcom or zluda?

@hny4uable Sure, SD.next should use the GPU with Zluda, that's why we're doing the whole thing.

Yes, there's a server mode and a Python API.

It's not helpful if you post your requests multiple times or with different accounts. Which GPU and which options do you run? You should have an entry similar to this one, how does it look like? "Device: device=AMD Radeon RX 6800 [ZLUDA] n=1 arch=compute_37 cap=(8, 8) cuda=11.8 cudnn=8700 driver=" Which additional errors occur in your command window?

@@NextTechandAI sorry for disturbing u

I don't have good news. You remember the part in the video where I list the supported GPUs? For the 6600 to 6750 XT, which are not directly supported, there are adapted libraries on the linked website. You could see if something like this also exists for the 5700 XT, I haven't heard of it yet.

You stop the video and type the prompt into an editor - or what do you mean?

@@NextTechandAI I am not willing to do anything more difficult than copy and paste. Could you help me?

Seriously? Then I think such complex topics are not for you.

@@NextTechandAI Come on! An easy copy-and-paste can save time!

I'm happy that you found it in my vid.

It depends on your version of Automatic1111. You could try to update if this doesn't hurt your installation. In latest versions you have to select e.g. Karras in a list right from the sampling method or leave it on "automatic". If this is no option for you, start with DPM++ 2M SDE.

As I said in the vid: You have to create an appropriate directory and enter it. I put my SD installations in KI, you can put yours anywhere on your PC.

I don't own a rx6600, but it should work by setting the target to gfx1032 instead of gfx1030. Moreover I would stick to SD1.5 models as SDXL requires much more VRAM.

I'm sorry, but there was a reason that I've created a Conda environment with python 3.10 in the vid. You have to start over with a new e.g. Conda env based on python 3.10 or indeed wait for ROCm on Windows.

@@NextTechandAI I have this same problem and I did create the environment exactly as you did with the 3.10 option but it still show as 3.12 and "Incompatible version" when running the "webui use-zluda debug autolaunch command. I even tried again and started over and ran "conda install python=3.10" and created a new environment and nothing seems to change the python version to prevent this incompatible version error.

Actually it looks like the problem may be the version of Miniconda that i installed. there are older version with python 3.10

SD.next uses the first python version it finds in the path and creates a venv with this version. You could try to delete (or rename to have a backup) the venv folder of your SD.next. Make sure to have an active Python 3.10 in your path when starting SD.next after this modification.

Have you already tried modifying the two .py files again?

The installation of SD.next has already taken care of all of Zluda's needs, so we hardly have to do anything manually. The VENV therefore already contains all dependencies like the appropriate PyTorch. Of course, you can do everything manually, but installing SD.next really saves a lot of work and immediately creates an Automatic1111 fork that can run with Zluda.

@@NextTechandAI thanks for the reply and this works great! if you need to update or reload the VENV you would just delete the SDNEXT version, run SDNEXT, and then be good to go in A1111?

Well, you could just "git pull" and start SD.next for an update. But, yes, for major updates to be on the safer side you could remove SD.next and install again in order to use the updated VENV (the risk is very low as SD.next is a fork).

I'm happy that the video is useful :)

starting with exactly your phrase gets me this: 20:32:28-812973 INFO Base: class=StableDiffusionPipeline 20:33:01-831983 DEBUG Diffuser pipeline: StableDiffusionPipeline task=DiffusersTaskType.TEXT_2_IMAGE set={'prompt_embeds': torch.Size([1, 77, 768]), 'negative_prompt_embeds': torch.Size([1, 77, 768]), 'guidance_scale': 6, 'generator': device(type='cpu'), 'num_inference_steps': 20, 'eta': 1.0, 'guidance_rescale': 0.7, 'output_type': 'latent', 'width': 512, 'height': 512, 'parser': 'Full parser'} as you can see it starts with device type cpu. weird. where is the secret switch so he utilizes my gpu.

same issue here, i have a 6900xt, no error in logs, i copy only relevant lines here: >> Platform: arch=AMD64 cpu=AMD64 Family 25 Model 97 Stepping 2, AuthenticAMD system=Windows release=Windows-10-10.0.22631-SP0 python=3.10.14 ROCm agent used by default: idx=0 gpu=gfx1030 arch=navi2x << but: >> Diffuser pipeline: StableDiffusionPipeline task=DiffusersTaskType.TEXT_2_IMAGE set={'prompt_embeds': torch.Size([1, 77, 768]), 'negative_prompt_embeds': torch.Size([1, 77, 768]), 'guidance_scale': 6, 'generator': device(type='cpu'), 'num_inference_steps': 20, 'eta': 1.0, 'guidance_rescale': 0.7, 'output_type': 'latent', 'width': 512, 'height': 512, 'parser': 'Full parser'} << did you manage to make it work ?