This Autonomous AI Agent is SURPRISINGLY GOOD | MultiOn Agents Gets Stress-Tested

[@DaveShap]

It's happening!!!!

[@LIVEFROMHYPERSPACE]

Dave Shapiro cameo!

[@3DimensionalMedia]

Respect to you Dave! (I'm over a week late catching up on almost daily business!)

[@gabrieljacob6481]

One of the best use case for this in my opinion will be to unsubscribe to all spam emails

[@mickelodiansurname9578]

The ChatGPT bit was hilarious, I knew it would not be long before we seen one AI trolling another AI!

[@samthibodeau3511]

So you don't have access to the api? If these things interest you, I strongly suggest you look into using the API vs using GPT when needing specific tasks requiring more than one gpt, since chatgpt is just 1 instantiation of the model, you can create 5 of them, give them each a different system instruction (defining their profile ie Engineer, Project Manager, Lead Front-End Designer, Back-End and Networking Lead...) and get these guys to begin posting to a .txt file that theey all constantly monitor for changes then add to it as they please... you can use a "baton" like strategy to give one gpt the turn to speak in order for messages to be irrelevant due to other gpts already having moved on from a subject(this is just an easy way to create a conversation ut more advanced ways I just won;t get into) Hope you have fun

[@memegazer]

lol...a proof of concept of how to us AI to train AI to sabatoge the data it would use to improve itself I wonder if companies will do that to each other behind the sences "automous agent swarm, go teach competitors AI how to fail at tasks by spaming incorrect inputs"

[@mickelodiansurname9578]

@@samthibodeau3511 Access to OpenAI API and Mistral and and Together and a few others, Perplexity, Cohere etc. So yes I sort of do... but rarely do I spend money having one AI agent screw over another.... I think its funny, but I sure as shit am not going to pay GPT-4 prices for it!

[@mickelodiansurname9578]

@@memegazer Shhhh you'll be giving folks ideas man... lol

[@Unicron187]

happens with my local models all the time 😅😜

[@dave-cripps]

This is the first agent that looks actually useful

[@HIIIBEAR]

SO FREAKING COOOOOOOL!!!! Ai assistants for everyone will revolutionize tedious personal tasks 😊

[@c0ndensation]

And work tasks as well, like automatically pullling reports without code

[@The_Questionaut]

I'm really excited for this but it's spooky too. A lot will come from the AI agents

[@ghazouaniahmed766]

Or take your personal tasks and leave you jobless 😂

[@SoSoInfinite]

Gmorning, thanks for the walkthrough. If there is any chance you can try to create an eBay listing with it. If it can be reliable and seo smart then it's a literal game changer. Small biz resellers are frothing at the mouth for this use case to arrive and eliminate the "least liked" and "time consuming/monotonous" portions of the entire process. Thanks in advance. Have s wonderful day! Subscribed*

[@retrofuturism]

5:33 The narration is hilarious. It would be great to watch whole KZbin videos of AI doing stuff and narrating it

[@avisrivastava4140]

We NEED to build something similar that’s open source. This is really cool tech

[@insomn1ak420]

This is so easy to build. Easiest way is to just use OpenAI API and pass it an entire page's source code as context then have it write Selenium scripts to manipulate it. Chat GPT already does this under the hood. I don't get what's so out of reach about what they're doing?

[@AlexanderWeixelbaumer]

@@insomn1ak420 Problem is if you use the ChatGPT API it's gonna cost you. Not much, but it adds up for every entry one of your agents does. SO what you really need is a free AI preferably with vision. What's new with MultiON is the teaching, that you can write a "macro" to navigate stuff and share them between users as actions. If you set up such a thing for yourself you have to write all this for yourself.

[@r34ct4]

Easy but very very expensive ​@@insomn1ak420

[@milkywaydev593]

@@insomn1ak420 So you wanna start an OSS version of this then? I'll contribute.

[@avisrivastava4140]

@@insomn1ak420 hahaha respectfully you’ve got no idea what you’re talking about. I’ve been exploring this space for months and here’s why it’s challenging: 1. Passing down the DOM isn’t always sufficient. The DOM can get really long and isn’t built for AI scrapers so a lot of its buttons and tags aren’t named to be read and clicked. 2. These guys are using a combination of GPT 3.5 or a different smaller model reading the DOM + GPT4V looking at the page to click on things when necessary. To accomplish this they built a sophisticated router to decide which one to use at any time 3. Their memory length is extremely impressive. It’s likely they’re using memgpt or a different proprietary framework to achieve this affordably but it’s NOT easy to build this at all 4. There’s a bunch of stuff you need to do (fork chromium) to be able to preserve user logins and bypass easy scraper guardrails FYI this is after a bunch of Stanford researchers and professors that spent the last 4 months working on this problem. The best open source has so far is just a mediocre GPT4V agent.

[@BeginningInfluence55]

IMAGINE if OpenAI themselves would create something like that, but making it so it can use your computer and any app/window. This would essentially be proto AGI already. Or maybe even AGI

[@insomn1ak420]

This is already so easy to build but using Android and Appium, and have GPT 4 run commands through it. I guess I should just build this for people huh.

[@A_D624]

That would be the ultimate app

[@narcoleptik44]

It’s open interpreter

[@Bhavi9888]

@@insomn1ak420build it man u

[@luckycogstudios]

MSFT is already working on that.

[@mickelodiansurname9578]

Holy shit how much does bottled water cost in the US, our local Aldi has like 24 bottles for about €5! Well here in Ireland I suppose 'water' is not an issue, when the tide doesn't come in it comes down here! so its about 25 cents a liter. But $15 for a few bottles of water, friggin hell you'd be better off buying beer!

[@jambear7862]

Water will be the next gold

[@byrnemeister2008]

@@jambear7862Id prefer if we could make Beer the new gold!!

[@T800_Terminator]

I live in the state of pa and water here costs $5 as well. It’s not a standard rate around u.s.

[@antonystringfellow5152]

America is a salesman's dream. You don't need any special abilities to make it in sales there. As for why... read the name of a popular brand of bottled water, "EVIAN" backwards.

[@nycgweed]

thats wiothout the service and delivery fee

[@stevemartel4729]

Thanks for sharing man. It really keeps us in the loop of what is to come if it has not already by the time I press send… at the speed things are going. I appreciate what you do man.

[@FitnessCoachAustralia]

I have viewed this video to fulfil my current objective

[@stanislav4607]

spoken like a true agent

[@r34ct4]

I have viewed this comment to fulfill my current objective

[@MrJCTt777]

Thanks, I just signed up for it. Your videos are great for nonprogrammers.

[@kaseycarpenter73]

Reminds me of what Rabbit is promising with their LAM

[@SoSoInfinite]

How effective do you think the rabbit will be able to create an SEO optimized and accurate eBay listing off of just a photo?

[@Glotaku]

@@SoSoInfiniteI’d imagine you’d have to n use plugins

[@SoSoInfinite]

@@Glotaku it seems at this rate that the rabbit will be outdated by its release

[@willbagshaw720]

@@SoSoInfinite It's software which is upgradable, and rabbit can be taught so it won't be outdated

[@SoSoInfinite]

@@willbagshaw720 so why would the R1 device stay relevant since every other device can have the same style software

[@rubbishprophet]

Close.. so close. I’m excited and scared at the same time.

[@ejh237]

Thanks for the info! I’m on the wait list!!

[@AmplifyAmbition]

Wes can you let them know their discord link is down?

[@Zale370]

Rabbit SEO said in a recent interview that LAM actually runs on cpu and is using multiple virtual machines, I rather wait for open source code to get out than paying for another subscription.

[@MangaGamified]

Feels like every other day there are titles like "X AI beats Z AI" the next other day "B AI beats Z AI" the couple days "C AI beats B AI" the next other days "D AI beats C AI" the next other days "E AI beats D AI" Either be agent or AI, it's always the same pattern always the same marketing title.

[@Zale370]

@@MangaGamified yes, unfortunately it's all hype at the moment.

[@pandoraeeris7860]

This is really awesome. I'd love to see an agent that can do government benefits sites/forms, apply for an apartment, complete insurance forms, and deal with financial and legal issues.

[@maxavail]

Soon, it will become a prerequisite for any web or mobile app to release a large action model together with its installation pack, for integration purposes, or else they won't be accepted on any app store. Companies will have employees clicking around across their own apps just to teach their LAMs how to automatically use the said app. These people will be a different team than the testers but will probably work under the QA department, just like the testers. It will surely become industry standard in any software development lifecycle.

[@crawfordscott3d]

we are paying them to train their model?

[@potatoes_are_fine8679]

24 hours at the present pace of tech is literally starting to feel like an eternity to wait for something 🤣

[@Ethiventure]

I’ve joined the waitlist straight away. I particularly like that there’s an iOS app on the way. Most autonomous agents stuff I get excited about seems tricky/impossible from an iPad.

[@michaelmcwhirter]

Do you use an AI that detects your voice when you're talking and only uses the video segments of you talking for your videos?

[@free_thinker4958]

When I deeply understand the concept of everything is a function, any project becomes easy with the help of AI

[@AS-ih2dk]

Can you use this now, or on the future, via free or premium, to auto post on platforms like LinkedIn, as in fully automated "post 4 articles a day on LinkedIn, on the power of AI, reach with 200-300 words, evenly spaced out between 9am-6pm 7 days a week for the next month"?

[@exmodule6323]

10:15 “I’m not sure what the problem is”’- the nightmare scenario of all black box solutions

[@shokhzodalimardanov5659]

The Discord link is invalid here and on the official MultiOn AI webpage.

[@MrBoydheeres]

Same here

[@itsyaro1297]

Yep... Maybe theyve closed the discord due to people flooding in. Please keep us updated and post a discord link here when available :)

I ran into the same issue. Following this thread

[@bbigshot]

I skipped the discord part and just downloaded the chrome extension and it worked for me.

[@promptstomping]

same!

[@Novainvent]

This is so exiting! Exactly what I need, have so many ideas and with this tool I can see it happen. Tried the Discord Link but the invite failed, tried it through their website as well, maybe overload?

[@burtonrider08]

The AI rugged you 😂🤣 Definitely does seem to be spot on decision making. Impressed!

[@anonymousaustralianhistory2081]

I assume the discord link is invalid on purpose as they are likely limiting the service based on theri servers and their beta testing, hope i can get access any tips?

[@StellanStephenson]

Hi Wes, thanks for a lot of great content. You're awesome! How to join the multi on discord?

[@huntermedic]

I have the same question Multion sent me instructions to join their discord server, but no invitation link. I sear ched for any public servers with "MultiOn" but came up with nothing.

[@DefenderX]

Hey Wes, could u make a video about recent development in Tsetlin machines? They look promising in reducing power usage and increasing output.

[@vandigital]

What do you think of the Arc Browser? Could be interesting to see something like MultiOn integrated there 🤔

[@melvinjongen1697]

Iv tested it but its not yet there iv asked it to make an booking app it doesnt understand or cant learn it somehow but its prommising

[@donb6514]

It's not that good right now but I can imagine how good it will get in a few months time. Can't wait !!

[@enidpinxit]

this is super impressive! i'm signing up. ~SWARM!

[@taitonium6262]

everyday i eat a wes roth video with my breakfast and i get satiated with dread (and motivation). thanks for another great ai tool video

[@BrianMosleyUK]

Scary when you mentioned security... Imagine a hacker teaching this thing how to hack accounts and buy bitcoin or whatever. Also, this is such a brilliant demonstration of the near future... It would be obvious for OpenAI to bake this in to their service.

[@wonmoreminute]

I can see MS baking it into Windows. Actually, it’s probably not even an option. This is a huge market share grab and potentially an existential threat to Windows. What if Google were to create a fully autonomous OS? Or Apple? I’m sure MS doesn’t want to find out. And Apple should be worried too. The utility of a fully autonomous OS that can operate any software is hard to imagine. One or two people could run a massive team of accountants, animators, you name it. Or maybe we see software company’s baking this directly into their software.

[@alew3tube]

Great video. Do you have the link to the Discord server, their invite seems invalid.

[@Rollthered]

The discord link is invalid for me, it says its either invalid or expired?

[@norj5]

Helo, i folow your channel and its really good, but this time i saw this AI Agent a few days before you, 😉. Thanks for keeping us update about AI.

[@norj5]

What about security? How this models uses your credentials in webpages. What happend if you have 2FA enabled?

[@the_wolf_of_amsterdam9793]

The Discord link is not working. Looks great.

[@sirishgreen]

After working in automation for over 6 years, i can safely say that those small issues like not clicking on the red button are the same problems that have existed forever. There is no easy fix!

[@CharlesNiswander]

"Invite invalid" on the Discord link :-(

[@KolTregaskes]

This is amazing, been waiting for it for a while. Sadly the Discord link looks dead (or expired?).

[@Paul_Marek]

Does this mean that Rabbit’s R1 LAM approach is obsolete?

[@eleven-nine]

The Discord link is invalid

[@ohhhwhale]

x2

[@chiraperera]

How about providing permission to access your different accounts through MiltiOn?

[@Hastingsnow]

Thank you for sharing!

[@igorsawicki4905]

I need to see some tests of giving it acces to my files locally and asking about testing some simple theory, lets say physics where it needs to write some python, then run it and collect results from output file.

[@dponjavic]

Can it write python script to have that same automation "offline"?

[@rochtm]

Is the discord server still up? i can't find it.

[@Ben_D.]

I want one of these, but it needs to deal with my emails perfectly. No mistakes. I want a fire and forget bot, and I just walk away from both my mails and the bot.

[@supremebeme]

invalid invite for discord damn

[@AdamPadron]

“I guess that’s fine.” Is how it all ends.

[@oscarcharliezulu]

I think the issues navigating apps and web by the agent will be harder than you think.

[@willbagshaw720]

Does anyone have an updated Discord link?

[@metadouken]

Is there a standalone app for this? Or is it only browser based?

[@cacogenicist]

So Google is blocking automated access to Google Workspace stuff?

[@insomn1ak420]

They can't block Appium manipulating Chrome on Android.

[@andyjm2k]

Why can’t these hardware devices emulate Bluetooth hardware and get a usbc image feed from a graphics card like a monitor would. Then it has a clean observable image and can act as hardware inputs so it can bypass security issues or compatibility challenges? Would be really useful and portable to different current gen computer environments

[@JasonCummer]

I wounder how capable it will become and what its limites will be. I mean there is the one limit of simpley only being able to do what a human can do but that could be a lot of things, not necessarily good.

[@born2run121]

Do you think GPT5 and Gemini Ultra will be autonomous?

[@insomn1ak420]

GPT DaVinci could be autonomous if you make it so. Not hard.

[@born2run121]

@@insomn1ak420 That isn’t what I asked and I’m not doing anything that either of those companies can be doing.

[@pingpong1727]

No not natively. OpenAI and google are trying to prioritize safety, an autonomous agent is hard to control/trust so likely not. I’m sure 3rd party developers could use their api’s to make autonomous agents

[@latestAiHacks]

Was it you who told us about hyperwrite?

[@strangecalibur]

The discord invite link is invalid

[@avq512h]

Need a fresh link to Discord

[@Davidturner-q8y]

can it play videogames well ?

10:15 no, it can't. because extensions can't just click on certain buttons designed only for users. for example, the pload button in your case. It requires your mouseclick to be in the chain of events. It is a safery feature of chrome. It also does not have access to your keyboard, and it fills elements by javascript functions. It's biggest drawback is the fact that it has to operate in constraints of browser extention.

11:28 same case here. It can't press tab button here. Also, it can't later access the saved image even if it somehow figures out to save it because extensions do not have access to your files for obvious reasons.

[@p5rsona]

so does this make that rabbit ai device already obsolete like everyone predicted

[@bgtyhnmju7]

I want to see it get through a Captcha

[@insomn1ak420]

Captcha's are used to train AI's

[@bgtyhnmju7]

Indeed, yes.@@insomn1ak420

[@youriwatson]

Very cool!

[@bertstevens245]

Is there a tool like MultiOn that actually works? Tried it, but just like int he video it just can't complete 95% of tasks because it can't figure out where to click.

[@xavier738]

@WesRoth the discord link is invalid for me :(

[@ralle11000]

What is the discord name?

[@captainblood9616]

Discord invite is invalid.... Door's closed already :(

[@brianWreaves]

The success of this agent would depend on you already being logged into your account on a platform to automate some of those tasks. I'm curious if: 1. It can also help log you in from a password manager like 1Password, LastPast, etc. 2. If so, how about when you have more than one account on a platform. 3. Complete a bloody 2 step login process, and then a Catcha. 4. Figure out how to use the shitty UI & workflow to log out of a Google account!!! Then I'll not only be impressed but amazed... 💕

[@matten_zero]

The big value add for me is if this can be an AI powered webscraper. Anything short of that isnt too useful for me. I tried it and while its cool, its not super useful if it gets blocked by amazon etc.

[@insomn1ak420]

Just have Appium scrape websites for you on an Android device

[@aiexhibit]

I think it would have been great if it had asked grok about Sam Altman's latest post

[@jennyjumpjump]

It's like a screen scraper and ChatGPT had a baby

[@phillupC]

What if this got hijacked?

[@planetmuskvlog3047]

Waiting for my Rabbit R1. No monthly fees

[@MrGaborKukucska]

100% Agree, 2024 is the year of the AI agent teams... especially with private local LLMs connected in a global hivemind doing all our bidding while we chill... The new internet. I call it H.A.I. Net :D

[@Steve-xh3by]

As someone who was an automation engineer who wrote bots, this type of functionality is NOT new. We had bots running in the 2010s that could do a better job at this type of thing than the AI is doing. However, it did require a substantial amount of code and time investment in order to achieve. AI's ability to generalize will make this much simpler, eventually, but the functionality is nothing new. We were able to write "generalized" automatic crawlers/navigators/extractors already.

[@njtdfi]

I wish more commenters would understand you do both kinds of videos. I happen to not mind the "headline" type videos even if everyone seems to want these more in depth ones for every single upload... It doesn't have to be EVERY video, guys.

[@pandoraeeris7860]

"Which is like ten years in AI time" 😂😂😂

[@DarylAntony-j7f]

Its a weird concept to me, in that, I think it has limited lifespan. These concepts rely on services providing human-computer-interfaces to humans, in which, the LAM's mimic in order to get things done. However, as less and less humans use these interfaces in favour of using LAM enabled Agents; the interfaces might eventually cease to have a reason to exist. Ergo, the future of LAMs might be in utilising APIs? An odd thought experiment is the Tesla Optimus robot driving a Cybertruck with FSD turned off.

[@GGh-m2v]

this is so scary and also i feel optimistic with it.

[@fullmentalalchemist3922]

If AI tried to. Buy me arrowhead water we might have to have a talk.

[@pensiveintrovert4318]

The only reason this is not a collosal waste of time, is that you amused everyone with the failure and got some ad dollars.

[@forest42821]

This is insane

[@Dreamslol]

Funny that people think that has some sort of "AI" touch, its basically just writing a normal E2E Test, too test your application / website if everything works like if Input field is accessible, clicking submit button see what it returns. As soon he cant find the HTML Element hes is fucked and getting stucked as you saw

[@Kristijan-Kiko]

They only have to wait for 20 hours and to them that's like 20 years meanwhile I am waiting for over 2 years 😂

[@TamasRigo-i5b]

Wes, if you ever read this, I love what you do. But, the way you skip things make it very difficult to replicate what is shown in your videos. I know and I understand: once you use an app you do not necessarily remember and tell what needs to be done (i.e., the very first time one is using it...) E.g., at 4:12 you say "Let's go to the new tab". Mate, I have no idea what you do there 🙈. I assume you go back and probably click "Playground". But I can't be sure... anyhow, when I click on Playground it gives me this annoying message "Playground access is currently not available". First, I reacted to this by setting up payment, but even after that I'm stuck. I wrote on Discord to the guys and waiting. Keep up the good work and sometimes think about this 'old' profile you mentioned the other day watching your videos.

[@TamasRigo-i5b]

I am so ignorant, but on my Macbook Pro, the page you show at 7:57 seem to not exists....

[@reza2kn]

How do you think the CEO of Rabbit feels about this? Lol :))

[@arnaudlelong2342]

Question: Why did you have to pause it before checkout? Does it genuinely require your banking details and order authorization? Yep, BETA, bro! They better fix that or risk losing a significant share of the market, lol!

[@jsivonenVR]

One question. One only: is this even remotely safe to access your Google account etc??? 😅😅

[@AndySmirnov]

Just test it. It all works as advertised. :))))

[@EveDe-ug3zv]

“We’re so close!” 😂 you’ve spent too much time on r/singularity @Wes!

[@brandoncampher2601]

that tumbnail is so cool... is it public?

[@MS-cs7gt]

Ask it to write an operating system