Prompt Engineering - Part1 : Prompt Tricks You Probably Missed for Stable Diffusion

I am back from a break from the channel! After taking the QA job at Stability (the company that makes stable diffusion) I needed to balance life and work, so needed to take a break. However, now that convention season is pretty much at an end, I can get things back on the road! This week I wanted to cover some AI prompt engineering tricks I don't see people using that are a core part of my workflow. Today we are working with the concept of allowing the prompt to be changed as it is inferenced (known as the FROM:TO:WHEN method), so we have a lot more control over the scene that we do with just prompting. This is just part one, as there are some other things I use that I also don't see out and about, so we are going to fix that. Cheers, and it is good to be back!
#stablediffusion #automatic1111 #aiart
Is your PC a potato? Use Run Pod for pennies an hour: tinyurl.com/ydu6rkpk
Print and sell your own artwork! tinyurl.com/22qwnrqu
Join my Discord server! Do it! / discord
Twitter: / sedetweiler
Instagram: / sedetweiler

[@SouthbayCreations]

This is exactly what the community needs! Back to basics, well sorta. Expanding on the tools we already have. Everyone (including myself) are quick to jump on a new tool/extension but then we forget about all the other cool stuff we already have. Great video! Welcome back!

[@sedetweiler]

Thank you! I also felt that some of these amazing tools had uses we just kinda skipped over. I will try and hit those as I also work on the bleeding edge of things. Cheers!

[@timothymaggenti717]

Fantastic, great job, thanks. This is the first time I understood how the prompt process works. I am new to all this and finally the light came on. 🎉

[@ThoughtFission]

yours are the most useful videos I've come across so far. Direct, to the point and easy to understand. Not always the case with AI tutorials.

[@sedetweiler]

Thank you!

[@MonzonMedia]

The master has returned! Congrats on the job with Stability! Your install video is what got me into auto1111 and I've since fallen down the rabbit hole! Long story short, started covering AI platforms, mostly online....ended up with a job with one of them 😁 you were one of the many sparks that led me down that path. Thank you Scott! Oh and yes, looking forward to part 2!

[@sedetweiler]

Dude! That is amazing, and I am so happy for you! Just imagine where all of this will be a year from now, as a year ago we would not have even dreamed this was possible. So happy for you!

[@MonzonMedia]

@@sedetweiler indeed! It’s progressed so much. It’s been a fun journey so far. Again can’t thank you enough my friend and I’m super stoked for you as well! Here’s to more growth and opportunities to come! 🍻🙌🏼

[@sedetweiler]

We have a lot of new goodness coming out soon. Not going to be lacking content any time soon. Cheers!

[@KiZzymesl]

welcome Back !! and thank you for dropping those gems, I didn't know this technique !

[@sedetweiler]

More to come! I am glad it wasn't old news. :-)

[@davewaldmancreative]

Dude. your guides are super cool :-) many thanks

[@sedetweiler]

Glad you like them!

[@alex-ender]

Scott, don't leave us for so long. I was starting to get worried. Thanks for the awesome video.

[@sedetweiler]

It's a deal! I had a bit too much on my plate and have since learned to balance a bit better. More videos will keep on coming. Cheers!

[@clifforderskineart]

He has returned!! Glad to see you back

[@sedetweiler]

Thank you! It is good to be back!

[@charb423]

I love this video, I just got started and up and running with Stable Diffusion so I appreciate these tips! Congratulations on your position at Stability! New subbie❤❤❤❤❤

[@sedetweiler]

Awesome! Thank you!

[@jankvis]

Nice to see you back and thx for the tutorial, cheers :)

[@sedetweiler]

Hey, thanks!

[@itdreamsme]

So happy you're back!!!! 🙌

[@sedetweiler]

Yay! Thank you!

[@clairearcher2549]

This was very helpful. Thanks Scott!

[@sedetweiler]

Very welcome!

[@LoneRanger.801]

This is such an awesome video. I learned something new today. Thank you.

[@sedetweiler]

Glad you enjoyed it! Cheers!

[@10master31]

Very informative video, waiting for part 2...

[@sedetweiler]

Coming soon

[@themonotropicguitarist]

Awesome! Thanks for this info. I feel like I need a little cheat sheet of all these tricks. Maybe an idea for those who know the ins and outs of them all.

[@sedetweiler]

Not a bad idea!

[@sikknikk100]

Excellent video and tutorial 👍😎👌

[@sedetweiler]

Thank you 👍

[@OriBengal]

Very useful. Thanks!

[@sedetweiler]

Glad it was helpful!

[@royceahr]

Well presented...

[@sedetweiler]

Thank you 🙂

[@LouisGedo]

Welcome back!

[@sedetweiler]

Thanks! :-)

[@hatuey6326]

very useful thanks !

[@sedetweiler]

You're welcome!

[@blackvx]

Hey, i remember you!Welcome back😊

[@sedetweiler]

hehe! Thank you!!!

[@GerryLouis1]

🎉🎉🎉good learning

[@coloryvr]

Thank you very much for that video! Although I've been using SD since release, the complexity is often confusing now.... Anyway: Many thanks to Emad Mostaque and everyone at SD for the free open source version! To me, Local SD is a tremendous gift to humanity! And if my art was used for "training": It's an honor! Happy colored greetinx

[@sedetweiler]

Glad it was helpful!

[@IlRincreTeam]

Hey, where did you get that Clip Guidance tool? Does not seem to be available in any extension for Auto1111

[@sedetweiler]

Which one? Sorry, I thought I had most of those things turned off for the tutorial so I didn't complicate things.

[@IlRincreTeam]

@@sedetweiler Below the sampling steps there is a box to tick saying 'clip guidance', I'm sure that most of the initiate to expert users of Stable DIffusion would like to know ;)

[@sedetweiler]

Ahh, that is related to using the dreamstudio sapi connection. You can use that and a few other options to have the inference completed on our huge GPU cluster rather than the local GPU. It also opens up other models we host there, and I will do a video on that soon as well. Cheers!

[@amorgan5844]

Did you ever start a podcasts? Im hoping so, just found the channel and have been binge watching

[@alexwan811]

I humbly grovel to thee!! thanks!

[@randymonteith1660]

Love the new tricks you taught in this video! I do have a question though? What is that Heading you have called " Stability API Settings " ?

[@sedetweiler]

I will cover that in a video soon. It is an extension.

[@merek1]

Woooooo! Missed your videos. Glad to see you back.

[@sedetweiler]

Thank you! It is good to be back.

[@Arewethereyet69]

thanks, need more!!! lol

Welcome back! I have learned a lot from you, I hope I will continue to do so in the future!

[@sedetweiler]

Glad to hear that! I have a lot to pass on that I have learned since starting at Stability.

[@generalawareness101]

Welcome Back.

[@sedetweiler]

Thank you! It is good to be back.

[@Nitrowing1500]

Yeah, I knew none of this! Thank you.

[@sedetweiler]

Happy to help!

[@davedm6345]

wow i lose all my time experimenting with the SD web guide with percentages to merge two tings LMAO, you make it more simple thanks!. I was having lot of problems putting camera angles and negative prompts as an example: car front or front view i do not want and i ended with lots of car front Photos :( it's ridiculous why? sometimes the text broke or some in the interpretations broke the line and end without the merged texts. Can you make a video to control camera angles in the prompts? THANKS NEW SUB!

[@WiLDeveD]

Pure Content. thanks. what brand and model is your GPU ?

[@sedetweiler]

Thank you! I have an Nvidia 3090 TI.

[@muledeer1766]

Is there something similar to the square brackets [ : ] prompt in ComfyUI ? or does it work there?

[@teambabaganoush7035]

youre wondering how to engineer on a Mac interface? and the manner to do so?

[@adumont]

Is it front-end/tool specific? Does it only work in Automatic1111, or is it more generic and would work on other SD tools like Fooocus?

[@fatenathan]

how did you get the pink overlay?

[@rahulchakraborty4770]

Hey Scott! Really love your content! I am a Stable diffusion enthusiast like yourself. I really want to take this seriously and want to make this as my career. Can you please give me some tips of how to find a job at StabilityAI or any other company with specialization in Stable Diffusion and AI? Thank you.

[@sedetweiler]

Just master the aspects you enjoy and become an expert that someone will see as a benefit. That pretty much works for and career, but the important part is making sure it's something you enjoy.

[@TheSporkimusPrime]

Can you do [from : to : when] in Fooocus in any way?

[@aifreeart]

Is it only possible on model v1.5-ckpt? Not in other models

[@sedetweiler]

Works on any model.

[@kkbbxx]

would combining 3 or more ideas also work, [car|plane|ship|bike]?

[@sedetweiler]

Yes, you can do as many as you like.

[@Klaaninka]

Loved this revelation. Thanks! Only thing is: Where did the pirate ship come from at 4:47. Also, at 6:16, you state that the 2nd bracketed item, the pirate ship, is rendered for the first 20% of the render. Don't you mean that the race car is rendered for the first 20%? Fully clarified later though!

[@sedetweiler]

Glad you enjoyed it, and thank you for the support on the channel!

[@metronome003]

Does these also work with comfy UI?

[@sedetweiler]

Yes, as that is just a front end to the models.

[@TKPaulChan]

Does this feature exist in ComfyUI?

[@sedetweiler]

Yup!

[@TheDanEdwards]

Your use of "brace" for the character "[" took me off-guard initially. In my math (and comp sci) classes we always called that a "bracket", with "brace" being used for "{". Anyway, the lexical tricks you show are for automatic1111, correct? Other UIs may or may not work the same way, yes?

[@sedetweiler]

Ahh, you are correct! Yes, as far as I know each piece of software has their own "standard." :-(

[@tauseefahmedmemon9148]

What site is he using?

[@masterzed1]

another headache... 😢.. but i understand ... ty....

[@RamonGuthrie]

Does this same technique work with two different people LoRAs and could I blend two peoples faces to create unique faces?

[@sedetweiler]

I would imagine so, as it is simply layering the results.

[@lioncrud9096]

does this work in comfy

[@sedetweiler]

It's actually better in comfy. Video coming soon!

[@riverrat_uk]

hi scott, how do i add say people in the backround? say a night club,

[@QVsense]

its very nice, but the title said trickS i only seen one trick

[@sedetweiler]

Did you watch the entire video? There are several here, including the use of the | without a when? parameter, using the blank :: and using the : without a start.

[@QVsense]

@@sedetweiler yes iv seen it to the end, for me it was one trick with some variations or sub parts. but that was kinda knitpicking on my side, its a good video

[@97BuckeyeGuy]

Too bad the developer of ComfyUI refuses to support this prompt trick.

[@sedetweiler]

There are nodes that let it work.

[@97BuckeyeGuy]

@@sedetweiler Are you talking about CLIP Text Encode++? Because that doesn't seem to work correctly, either.

[@MattJonesYT]

Starts at 1:22