AUTOGEN TUTORIAL - build AI agents with GPT-4o and Microsoft's AutoGen

Learn AI With Me:
www.skool.com/natural20/about
Join my community and classroom to learn AI and get ready for the new world.
#ai #openai #microsoft
here are the links i mentioned:
Install AutoGen:
autogen-studio.com/autogen-st...
OpenAI API Quickstart:
platform.openai.com/docs/quic...
00:00 Microsoft's AutoGen
00:38 Installing AutoGen
05:33 Tour of AutoGen
09:30 Adding GPT-4o
10:48 AI Agents Developing Videogames
14:35 Running a Workflow
17:46 Adding new skills
20:17 More Use Cases

[@dylan_curious]

OMG. Watching the agents code up the video game felt surreal. This is so powerful. Thanks for sharing.

[@dan-cj1rr]

oh each of em searched github for a tetris game and made it so cool MESMERIZING WOAH

[@OddlyHistorical-hb1yg]

For anyone getting an error about "packaging" when you try the command "autogenstudio ui", just do the following: pip install packaging Then try the autogenstudio ui command again

[@anthonyengel7192]

Life saver; ty!

[@twogunzpadre]

BEAUTIFUL!...thank you

[@airlesstermite4240]

thank you!!!

[@Fonzleberry]

Thanks Wes. I've been looking at doing just this.

[@YaelMendez]

Thank for for developing this open content. Your broadcasting improvements are 💯 I’m sharing your content with LATAM. 👏

[@propeacemindfortress]

Conda in whatever flavor is not isolated, it is just a python environment of which you can have multiple for the purpose of maintaining neat little spaces with only those dependencies you need, vastly reducing potential version conflicts between libraries... If docker is isolated is debatable and becomes kinda semantic, VM's are certainly isolated environments... but conda is not.

[@truehighs7845]

I gave up on conda, it was breaking too much and didn't have the latest just an added latency; pyenv or venv, or poetry or whatever but not conda..

[@propeacemindfortress]

@@truehighs7845 lightweight and flexible ... most of times less is always more 🙂

[@SeeFoodDie]

Check out miniconda or venv instead. Full anaconda is problematic.

[@bakeee]

looking forward to using this. i was messing around with crewai it occasionally worked for me, im not the best coder but this looks feasible for my intended uses

[@dreamphoenix]

Wow, thank you!

[@Dron008]

14:00 I hear robotic laughter on the background "ha-ha, no exit!!"

[@brianhershey563]

Wes, I wanna know how excited are you about the popularity of your private group, holy cow it's a life changer! 🙏

[@twylxght]

Wait til AI unionizes.

[@themax2go]

actually a good one 🤣

[@ai_serf]

uh.. your starting picture shows an entire software dev team, including a CEO.... where's your interactive CEO at 7 cents an hour?

[@bryansarria6064]

Where is the toxic middle management?

[@denisblack9897]

And a janitor that thinks she’s running the place😅

[@mikezooper]

Where are the gossipy staff at the water cooler?

[@enidpinxit]

amazing!

[@realCleanK]

Appreciate u! 🙏

[@mikaznavodya9300]

Thank you

[@GaryBernstein]

How’s it compare to CrewAI?

[@atulthakor4042]

Wes - really good product and explanation @All Keep an eye on your credit as I wasn't able to see a way of putting a cap on tokens / iterations.

[@elsavelaz]

Do you know if it lets you own your own data or how easy it is to do that? Right now I write to the VM or a dynamo db , to have the very valuable data under my possession because I iterate very quickly to deliver to the executives that I’m working for, but if this can easily do that, it’d be easy to tell them this is better than custom software built on top of the api’s

[@AmanBansil]

I had tried using Autogen earlier but got stuck where agents kept going back and forth. It may be user error. I'll use Autogen studio and see what I can get done with it.

[@elsavelaz]

Can this be put in docker container to distribute to folks who won’t be able to do all this ?

[@162arun]

I think the real value is in creating an agent team that achieve an outcome that takes multiple chat conversations with LLMs to get right. Perhaps this is just setting the stage, with more to come from Wes.

[@tomgreen8246]

Exactly what I was thinking too

[@baraka99]

Isn't CrewAI a superior platform to use for multi-agent workflow?

[@6lack5ushi]

What do people use AutoGen and CrewAI seems fun but I cant think of anything I would rely on this for???

[@brianWreaves]

You can think of them as an employee taking on part of your workload, or a personal assistant helping you with items on your 'Things to Do' list, or a service provider helping you shop for a gift for someone.

[@andrew-729]

Think about the skills as a library of mini apps. If we can co ordinate them efficiently.....

[@macoson]

Imagination is the only limit. Consider what people can do: simplify their tasks and create assistants to handle them. I use automated assistants in my work where we are producing radiopharmaceuticals. Some agents help me write standard operating procedures and other structured documents, while others review these procedures to ensure they comply with existing protocols and pharmaceutical laws. Another setup transcribes data from production reports into a database, while a QA assistant reviews this data to ensure that production is satisfactory and all parameters meet specifications. Additionally, an assistant with access to the production database prepares quality reports with graphs and statistical analysis. I'm continually working on and testing more assistants to help me manage the tasks and activities of my entire team.

[@brianWreaves]

Have watch of a couple Mervin Praison videos for more examples.

[@xmagcx1]

It is possible to export the source code to execute it in Python skill, model script,etc ?

[@rafaelramirez3470]

What do you think now of ChatDev? Is this more powerful?

[@RosserNFT]

Ok I can use this for some simple websites.

[@secretagent5209]

Ok chatGPT, format my C drive

[@satanicsmoresmusic]

Gonna need AGI for that lol

[@morespinach9832]

Can I install and use this "locally"? With local LLMs? If so, no keys needed, yes?

[@PantherDawg]

Are you being charged .14 or is that hypothetical? If so, is that billed to your ChatGPT subscription?

[@wizards-themagicalconcert5048]

Can we generate video or animate generated pictures?

[@andreisas1621]

yes

[@michaelbubnov3306]

You missed a very important step - how to use it without OpenAI/Azure. Those are not available in every country.

[@Ben_D.]

Kobyashu Maru moment in Flappy Bird. Heh heh.

[@JimmyMarquardsen]

Time is on my side, yes it is Time is on my side, yes it is Now you always say That you want to be free But you'll come running back (said you would baby) You'll come running back (I said so many times before) You'll come running back to me Yeah, time is on my side, yes it is Time is on my side, yes it is You're searching for good times But just wait and see You'll come running back (I said, don't worry darling) You'll come running back (spend the rest of my life with you, baby) You'll come running back to me Go ahead baby, go ahead Go ahead and light up the town And baby, I do anything your heart desires, remember I'll always be around And I know, I know Like I told you so many times before You're gonna come back Yeah, you you're gonna come back baby Knockin', yeah, knocking right on my door Yes Time is on my side, yes it is Time is on my side, yes it is 'Cause I got the real love The kind that you need You'll come running back (I knew you would one day) You'll come running back (like I told you before) You'll come running back to me Yeah, time, time, time is on my side, yes it is I said, time, time, time is on my side, yes it is I said, time, time, time is on my side

[@lystic9392]

I want to use a structure like this to aid me in solving a problem. Not a coding problem but a complex problem in the world. Math isn't really involved. Would this sort of work for that, too? Or is it mostly useful for coding tasks/problems?

[@lystic9392]

I'll give it a whirl either way.

[@brianWreaves]

Create a problem solving agent\s relevant to the domain your problem is within.

[@lystic9392]

@@brianWreaves Hmm. It just seems difficult to believe that it could do much more than a single ChatGPT instance. I will see if I can set it up today.

[@beyondthebounce23]

no these agents are worthless right now.

[@lystic9392]

@@beyondthebounce23 That doesn't sound very promising XD

[@rhym8882]

can agents be used for something other than coding? for copywriting?

[@PrincessKushana]

Yes. I've been using autogen a lot. It's a very powerful tool. I've used autogen to creat task decomposition processes, an essay writing sub process, Web research, navigate a directory strucutre & read and create files, think creatively, prevent prompt hacking, schedule tasks for later, etc.

[@rhym8882]

@@PrincessKushana interesting. what are "task decomposition processes"? as I understand you just set a main agent that, what, sets an outline, then multiple agents that do different tasks such as research, compile, proofread, and so on?

[@PrincessKushana]

@@rhym8882 yup pretty much. it means you can perform drastically more complex actions. multiple steps, different functions, etc. Eg "please do a Web search answering the question on line 2 of the file in x directory. Save the answer on the next line."

[@rhym8882]

@@PrincessKushanaI see. Thank You! ☺Im going to try it out. Do you know of any video or some other resource that teaches to build agents specifically for copywriting?

[@thiagobalducci514]

HEEELP [Adding GPT-4o part of the video, 9min45s] When I try to add a new model, I put gpt-4o and my API Key, but it appears this error: Error occurred while testing model: Error code: 404 - {'error': {'message': 'The model `gpt-4-1106-preview` does not exist or you do not have access to it.', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}

[@ajax0116]

Same here!

[@chickenfriedrice2881]

same :( any update??

[@RajeevKumar-wl6ei]

same issue, tried different endpoints too.

[@ajax0116]

If you add via the method in the video it errors. I added it from the description version screen. I don’t know why, maybe it doesn’t like the script

[@appaesthetics1277]

Quite a few other videos needed , can we get a combined vodeo please

[@samlongman1831]

Please can someone help me... I can't find an answer anywhere... I'm in the UK using android chatgpt 4o and have a premium account. I can't seem to real-timeme conversion or real-time visual input like in the recent demos. Is this ability only available to some? All I can do is voice input, and it reads back the response. This isn't the same as the demos. It's still really just a text conversion. Thank you in advance. It's driving me nuts trying to find an answer to this!

[@bradthedadbradthedad]

Those features you mentioned haven’t been released yet with GPT 4o yet

[@simonrochester2559]

The code generated for the tetris game had a bug in handling multiple line clearance and chatgpt can't fix it after several round of prompts. I had to fix it myself. the buggy code: def clear_lines(grid): lines_cleared = 0 for i in range(len(grid) - 1, -1, -1): if all(grid[i]): del grid[i] grid.insert(0, [0] * (SCREEN_WIDTH // BLOCK_SIZE)) lines_cleared += 1 return lines_cleared the fix (chatGPT-4o failed to provide): for i in range(len(grid)): although it is still very polite while incapable: You're right. The `clear_lines` function should account for the possibility of multiple lines being cleared at once. Let's update the code to handle this correctly and also display the score on the screen. Here's the updated code: ```python

[@johannesdolch]

Doesn't work for me, i get an error when i try 'autogenstudio ui' It throws up a module missing 'packaging' anybody else had that? (SOLVED: Module 'Packaging' was missing. Just downloaded it with PIP.)

[@mitchellmigala4107]

You need to make sure you are inside of the virtual environment. After you create it move inside of it then run the command. If you get stuck start over. Create an empty folder , move into that folder on command line, when creating the virtual environment use the current directory instead of creating it into a new one, then run the command after installing everything.

[@johannesdolch]

@@mitchellmigala4107 Oh sorry, i forgot i asked this question. I actually solved it by pasting the code into ChatGPT. I was missing the module 'packaging' and i could just download it with pip. So AI helped me with installing AI. Inception :P

[@elsavelaz]

FYI azure doesn’t let me do half the things the ChatGPT OpenAI api lets me do. I do integrated ai, so full stack front end , video, image, data ingestion , the full thing, is imo easier with ChatGPT 4o api and also makes the front end a few lines of code vs preserving state with code needed for azure because not all regions with azure offer everything and has lots of guardrails that are either yes or no- so either you let through all the poo or you can’t get stuff you need (like if you sell swimsuits, it’s lots of work to generate digital people Showing the products )

[@liberty-matrix]

"Over the next 15 to 20 years humanoid robots will disrupt human labor throughout hundreds of Industries across every major sector of the global economy. The disruption of labor will be among the most profound transformations in human history." ~Tony Seba, 2024

[@djayjp]

No, most people will be out of work within the next 2 years.

[@Danoman812]

Woo!!

[@thewatersavior]

Only thing missing is using ai to generate the code for the workflows, skills, etc.

[@georgegordian]

I assume there is an existing skill to create new skills. There should be no real need to hand write simple Python like what was shown.

[@Badg0r]

Why isn't there a simple installer with a user interface?

[@andrew-729]

Give it time. Go look at how you had to install this when it released...

[@1x93cm]

wait 6 months, you'll be able to do all this in a regular UI on your phone.

[@claudioagmfilho]

🇧🇷🇧🇷🇧🇷🇧🇷👏🏻, Thanks for sharing!

[@truehighs7845]

I am no longer using Conda it's a pain.

[@kelvin3763]

What about using that a.i. voice for a dialogue in your video's. Let it ask questions about the info presented in the videos, as if it was a viewer like us. Or just as your a.i. companion thats curious and wants to learn. I'm not sure yet if the end result would be fun or ruin the video. Just an idea :)

[@fastmamajama]

i have an image set of UFOs that i am trying to compile using the haar gui. i can send them to you if you want to make a show about the topic. i have an opencv python script to capture ufos and it seems to work but i think my image set needs work.

[@user-jf5uv9ir5k]

your installation uses totally different code from the main software site, which makes it very confusing

[@truehighs7845]

we can run models locally I don;t see the point to use a paying API unless you are in prod, for testing purposes, I can largely do what I need with local models.

[@carstenmaul7220]

So why is everybody installing conda when python -m venv is all you need?

[@mitchellmigala4107]

Because they are using windows and conda is recommended for those using windows that want to use autogen.

[@strauss7151]

The installation process is still too complex. Someone please make it into an easy no code online service and take my money.

[@logo-droid]

yes, best idea would be a software that could develope itself offline.

[@truehighs7845]

TBH it's not all that difficult., but I see what you mean, just put your api keys and get going, probably with templated searches and outcomes so all you need to so is give a prompt and watch you team work, quite neat.

[@macoson]

The thumbnail for this video is misleading. The current version of Autogen is 0.2.27 not 2.0

[@naytron210]

"He invented... everything." 🤣🤣

[@jtmuzix]

snot poker!

[@mikezooper]

Now we need an agent to create agents 😂

[@hqcart1]

you did not demonstrate a real use case... it seems i can finish coding the game faster than setting this whole thing up using a single prompt to chatgpt...

[@spittylama]

What about 2,3,4,5 and so on games? Once it is set up, it only is a matter of time until beating your time, isn’t it?

[@hqcart1]

@@spittylama a prompt to chatGPT is 1. free, 2. fast, 3. doesn't need any setup, 4. you can go back and forth with it. until i see a REAL use cases, thank you.

[@brianWreaves]

Have watch of a couple Mervin Praison videos for more use cases.

[@IbnShaalan]

is the girl voice ai generated? if so where can I find it

[@beyondthebounce23]

is it ai? .... lol nah it his girlfriend helping out.

[@handsanitizer2457]

Maybe eleven studios ?

[@Flugnu]

Games are not made in python.

[@bastabey2652]

Jurgen Schmidhuber is AI's daddy..

[@TheExodusLost]

Can someone be real with me and explain some things they are building or doing with this or similar that is useful in your life, either as a product you sell or just as something you use?

[@brianWreaves]

Have watch of a couple Mervin Praison videos for examples.

[@mattgraham4340]

​@@brianWreavesare you an AI agent for this Mervin praison?

[@BlunderMunchkin]

Tetris and Flappy Bird probably have thousands of implementations with source code online (and hence in the GPT's training data), so having the AI "write" those games is a bit disingenuous.

[@autodidact7127]

These are not capable of keeping track of large applications yet. At best you get a single file. Please use these tools to improve your skills.

[@Ricolaaaaaaaaaaaaaaaaa]

Just FYI lol. A lot of devs aren't on Mac. A lot of silicon valley devs are on Mac because they think it's a status symbol. Somehow inferior products are associated with a higher status solely due to the price. I don't get it either. Is what it is. I'm excited for AutoGen to be available to general users with voice.

[@TheExodusLost]

I think apple found its way into the corporate environment with its clean look and depending on who you ask, easier learning curve to perform basic tasks

[@Ricolaaaaaaaaaaaaaaaaa]

@@TheExodusLost I mean if you have to worry about the learning curve of an OS you probably shouldn't be in one of those jobs tbh lol.

[@abudhabi9850]

Imagine finding out many big applications are written in the “easiest” language to write it in (e.g. Python, php, js)

[@Ricolaaaaaaaaaaaaaaaaa]

@@abudhabi9850 I wouldn't consider higher level languages easier. You still have to understand the fundamentals which is the hardest part to grasp I'd say.

[@abudhabi9850]

@@Ricolaaaaaaaaaaaaaaaaa c is definitely harder than the others mentioned even though you know the fundamentals

[@robinvegas4367]

There is no way in hell I will write code on a Mac. The nerve of some people.

[@pfifo_fast]

So the big problem I see here is that Tetris already exists, and Flappy Bird already exists. And another minor problem I see here is that your total sales figures for your tetris and flappy bird games is a total of ZERO dollars and ZERO cents.

[@derekdipietro2588]

What do you expect it to do? Lol

[@Copa20777]

We use windows, we not barbarians 😅 nice one

[@AI-Wire]

The tech is unusable at this stage. The Kobyashi Maru bug in the Flappy Bird game illustrates this.

[@brianWreaves]

Have watch of a couple Mervin Praison videos. Then agents might be worth reconsidering.

[@AI-Wire]

@@brianWreaves I am familiar with Mervin's channel. Mervin also illustrates the problem because he covers many platforms but none of them can do anything useful. There are lots of example cases. But nothing really useful.

[@user-jf5uv9ir5k]

"you don't need to know code" you are kidding right?

[@TheReferrer72]

Another KZbinr that did not even try the prompt online before demonstrating the Agents. Hint it works just as well and its free.

[@1x93cm]

*creates a game with a no win scenario* what a sarcastic bastard

[@themax2go]

1. don't need anaconda, creating a virtual dev environment can be done much more easily 2. if it doesn't run off local llm then not interested... this vid is sponsored by openai

[@mitchellmigala4107]

You can use local llm. Autogen is actually a Microsoft product.

[@adriaanb7371]

it needs original tasks that it cant find online. You didnt even need to explain how the game works, guess why, because it already knows it.

[@DustInComp]

Okay, it can do a tetris clone. Now get it to gerenate, rig and animate 3D models, add textures, write shaders, write a story, puzzles, cutscenes etc, and make it feel unique.

[@wjckc79]

Give it another 6 months.

[@udaykadam5455]

Fact that we are even here having this argument should make you realise where we would be in less than 2 years from now

[@justinwescott8125]

"oH yEaH, BiG dEaL! iT sTiLL cAnT dO tHiS aNd ThAt!" This comment ages like milk 100% of the time, and people still keep posting it.

[@angrygreek1985]

am I the only one who doesn't think this is very impressive?

[@brianWreaves]

Have watch of a couple Mervin Praison videos. Then agents might be worth reconsidering.

[@derekdipietro2588]

This is a big "meh" from me, a 10 year old with a python tutorial can make the same thing and better. Its cool, but its a tool and not automation really.

[@georgeknerr]

Mantra? "Let the robots do the work." No thanks, work a blessing.

[@dg-ov4cf]

idk about encouraging code-illiterate people to unquestioningly let bots run code we don't understand

[@beyondthebounce23]

all these one shot prompt crap are worthless. Untill these agents can start talking back and worth with me about files on my computer what is hte point.

[@brianWreaves]

Have watch of a couple Mervin Praison videos. Then agents might be worth reconsidering.

[@beyondthebounce23]

Your AI voice is not as good. You can tell you are using that AI tool to generate your own voice.

[@VoodooD0g]

Coding small shit games is really not impressive (anymore). Therefor this is of zero interest to any reasonable programmer. What would be interesting is to see its understanding of larger projects with many objects and competent it is in helping in those kinds of environments.

[@brianWreaves]

Have watch of a couple Mervin Praison videos. Then agents might be worth reconsidering.

[@fedorp4713]

This is cute but still completely useless. So far all the examples you've shown can be done with simple browser addons, free apps and games, much much better than the agents created and all free, no tokens. Dall-e images are also garbage compared to free stable diffusion. What are some practical use cases for this? Can you give an example of multiple agent actions for complex 10 step tasks? You know, useful things. Not just generating garbage code and images that can be easily generated on any stock site.

[@brianWreaves]

Have watch of a couple Mervin Praison videos. Then agents might be worth reconsidering.

[@beyondthebounce23]

pointless video done by a bunch of other pointless channels.

[@miblish5168]

Way over my head. Lost after one minute.