The First AI Virus Is Here!

❤️ Check out Weights & Biases and sign up for a free demo here: wandb.me/papers
📝 The paper "ComPromptMized: Unleashing Zero-click Worms that Target GenAI-Powered Applications" is available here:
sites.google.com/view/comprom...
📝 My paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Putra Iskandar, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers

[@virgilxavier1]

Thank for giving us another great paper!

[@JayYu-lr4ro]

This is just another variant of Steganography based malware, it can also be done with no genAI needed!

[@suchislife801]

Can you do a 2 minute paper on Text to Voice and then you know, use it?

[@MichaelBarry-gz9xl]

It's not just stenagraphy, the LLM is required. Unless, of course, the human decyphers the hidden message and decides to carry out the instructions. If I asked you to send me all your emails, would you? Well, the put an AI in charge of your emails and it will.

[@JayYu-lr4ro]

@@MichaelBarry-gz9xl the point is that it can be done without LLM too! And the LLM is in fact just unnecessary billions of parameters of bloatware that’s not necessary for the core functionality of the malware at all.

[@MichaelBarry-gz9xl]

@@JayYu-lr4roI assume your referring to malware already existing on the computer? If so, you're correct but you missed the point. The point is that people outside the AI research circles are ridiculously unaware that this is possible and so have a false sense of security. There's nothing new here. A child could point out these vulnerabilities, and I suspect that is what your getting at. This is hype and sensationalism at its greatest.

[@ariaden]

This is a one-click attack. The mistake was to enable any automated system to react on your incoming e-mails.

[@JayYu-lr4ro]

Reacting is fine, not sanitising injected commands is not fine!

[@matthewpauls2498]

exactly what i was thinking lol..

[@asdfghyter]

@@JayYu-lr4ro remembering previous conversations with other people is also an issue, since that will inevitably lead to data leakage

[@Martin_Adams]

The bigger risk might be companies using this for their automated email services

[@JuddMan03]

@@JayYu-lr4ro How do you sanitise a natural language processor?

[@peanutnutter1]

What a time to be a virus!!

[@vosechu]

Ah, but are viruses alive?

[@davidwilson6577]

@@vosechutechnically, no. Viruses don't grow, and they need to use cells to do most of the stuff that qualifies as living. And computer viruses are just programs.

[@pauldavis2904]

🤣

[@eIicit]

@@vosechuthey are not

[@theblinkingbrownie4654]

@@eIicityet

[@tedchirvasiu]

Who tf uses an AI system to automatically answer their mails?

[@chasealcorn1047]

more than youd think... I have a porteugese client that replies to all my emails with chatgpt. I dont think he even proofreads what he's sending in english even though I've most definitely held a conversation with him in broken english. people are idiots man

[@Quasihamster]

Maybe Boeing.

[@gurtuggungor9786]

Some people are way too lazy I guess.

[@BiggusWeeabus]

Companies

[@wij8044]

Every major company

[@Clawthorne]

Well this kind of stuff is going to be so lovely when Windows 12/13/14/etc comes out with all the "AI powered" everything, and suddenly you can lose control of your computer because someone on Discord messaged you "We are going to have a little roleplay..." and it showed up in your notification bar for the AI to see. 😩

[@dot1298]

who even uses Windoze these days, when we have Linux Mint?!

[@doppled]

@@dot1298 what

[@okachobe1]

​@@dot1298who even uses Linux today with WSL2 and GUI version of it!

[@mariobatguy]

@@dot1298facts bruh

[@Penancetw]

arch btw@@dot1298

[@lobabobloblaw]

This reminds me a little bit of an insanely nuanced code injection trick Super Mario World speed-runners do, where by inputting specific buttons and directional controls they could effectively patch a ROM address into working memory, immediately flipping the game into the ending sequence. I hope no one ever conceives of an equivalent for a chat prompt (I imagine the token window would be too primitive as it is)

[@jrd3807]

Isn't this what all the GPT jailbreaks are about?

[@lobabobloblaw]

@@jrd3807 to an extent. I suspect that-if this should ever become a widespread issue-a context-aware “parity” agent design may become useful to help parse the exchange for potential incoherencies / manipulations.

[@Zaary]

@@jrd3807no?

[@nikkox1992]

​@@jrd3807 no. Jailbreaks aim to alter the ai contexts based on trial and error prompting, fine tuning the tool based on the feedback until a prompt sets the context in a state where regulations are bypassed; one could say the "configuration" of the system is being tampered with, as it is a high level of abstraction domain. On the other hand, the cited "injection" is based on analysis of the decompiled game code, memory allocation and working the way on ingame manipulation to achieve the desired specific result; it's actually a glitch exploit on a low level abstraction domain. To use an analogy, image you have two dark rooms: In one, you have to traverse and exit it on the other side. This one has an overview schematic diagram detailing all objects inside with measurements, etc. You would use the schematic to measure you walking distance by step, calculate the steps, rotations, etc to trace a way in the dark to reach the specific location of the exit door. In the other room, your objective is to find and execute the instructions to turn on that AI that has no restrictions or w/e. The only way to do it would be if the room wasn't dark, so turning the light switch would be the easiest way to achieve your goal; for that to happen without a guide (like the schematic in the other room) you would have to explore by moving, touching, listening, smelling, etc, getting to know the room, the position of objects and stuf, and work your way to the light switch. Once the room is lit, the rest is easy. They both have a "dark room" , signifying the offuscation that exists on both cases, albeit the objective in each case is disctint: therefore, so is the strategy. Although one could argue that after prompting and mapping the restrictions on the AI, those could serve as a guide to craft more specific, surgical prompts too.

[@Stratelier]

Not just Super Mario World, but also Ocarina of Time, original Pokemon ... all sorts of ROM based games can be made to yield bizarre or interesting behavior just by glitching the game's working RAM in very precise ways (typically an area the game engine reads for high-level instruction scripts, so injecting a wrong value here might get it interpreted as "load room X / play cutscene Y").

[@illustriouschin]

The internet suddenly became a lot more dangerous to AI with one weird trick.

[@sapienspace8814]

Do not even need to click once!

[@huckleberryfinn6578]

DId you even watch the video? This loophole was already closed, at least on OpenAI and Gemini. It's like every virus, it's dangerous as long it's brandnew.

[@vectorlambda]

Cybersecurity agents HATE this simple trick!

[@thevalarauka101]

@@vectorlambda I was literally about to say that

[@Slav4o911]

It didn't, none of this actually happened. It was some theoretical scenario.

[@Auxius.]

This isn’t- dr Károly Zsolnai-Fehér. And- his- voice is generated. What a time to be alive!

[@MagicBoterham]

I found an actual human that kind of speaks in the same way kzbin.info/www/bejne/m3_Gp6p-nN6Lmck This man was born in Germany and moved to Poland with his family when he was young.

[@Auxius.]

@@MagicBoterham 2:54 note the ‘of course’ for example.

[@Aurelyyon]

The generated voice has such a strange rhythm

[@PravinDahal]

@@AurelyyonThe real one is just as weird.

[@CalvinRRC]

He has been for years now, not even kidding. I had to stop watching most of his vids because it just isn't pleasant. This isn't a knock against AI voice, either. He's using a much older technique than recent stuff that just sounds unnatural.

[@user255]

Zero click attack, but requires few shovels full of stupidity.

[@HakashinTruth]

does this mean traditional computers also need AI anti virus to counter an AI virus?

[@maloxi1472]

The question is unclear.

[@samvv]

Software developer here. Not at all. Actually it is just a regular computer virus. The title is a bit of a clickbait.

[@OhioNPC911]

Norton already deployed AI antivirus

[@samvv]

Some would call it a 'zero day exploit' but since the leak has been fixed it's ok now.

[@samvv]

@@OhioNPC911 There's no such thing as an AI antivirus. Except if you mean an antivirus that uses machine learning to detect threats.

[@Srindal4657]

And people thought robotics was scary

[@ryandury]

imo robotics will be scarier

[@JackCrossSama]

@@ryandury more like nanomachines

[@latt.qcd9221]

Robotics are scary because it's AI + legs

[@thesilver7238]

But robotics includes AI.

[@schwajj]

AI viruses might be able to take over the robots.

[@gaius_enceladus]

This is absolute proof (if any more were needed) of the value of having "people in the loop" and NOT automating everything. Automation has its place (when used wisely and carefully) but it has its flaws (as this shows).

[@ethzero]

"Computer: Create an opponent that can out think Data"

[@dot1298]

…or the *omega molecule* [directive] (in ST/VOY)

[@RavenMobile]

I just watched the first Moriarty episode with my ten year old recently, great episode!

[@catsozen]

I chuckled, was just marathoning the whole of TNG.

[@JazzJackrabbit]

No mistake?? Dude, your mistake was using an AI/LLM to read your emails!

[@MichaelBarry-gz9xl]

No, the mistake was allowing the AI that read your emails to have access to tools. Reading emails is fine, so long as it can't send API requests or send emails etc

[@cgme9535]

@@MichaelBarry-gz9xllol no, don’t do either

[@MichaelBarry-gz9xl]

@@cgme9535 You know nothing, John Snow!

[@hjewkes]

The system they're hacking honestly feels like a pretty contrived example.

[@Slav4o911]

Like it's made for the hack to work. But I'm impressed the bot actually follows instructions and doesn't answer like "I can't do that.", or some other nonsense.

[@infernalsorcery7923]

​@@Slav4o911Adversarial prompts.

[@vegtalk8920]

Did you start using AI to generate your audio?

[@alexholker1309]

This is why I've always said that delegating authority to AI is a risky idea. You don't understand the algorithm, and nobody designed the algorithm, so you're putting your trust in a fundamentally suspect black box just because it's spat out the right answer *so far*.

[@parsa_poorsh]

you are saying the sentences like an AI. there is sooo much pause in them

[@griffnotthatone6824]

Surely there is a better AI text to audio than this

[@Gerlaffy]

Struggle to listen to the video as the AI narrator is all over

[@DeePunter]

Yea i guess its a lot easier to listen to 1 minute video of Adam's voice.

[@Gerlaffy]

@@DeePunterwho is Adam...?

[@smallxplosion9546]

@@GerlaffyAdam BALLS 😂🤣🤣

[@TayWoode]

I prefer listening to Joe

[@pidbul530]

@@smallxplosion9546that doesn't immediately sound like anything in particalar... Can you explain where's funny besides adding BALLS at the end?

[@SandroRocchi]

0:40 "These normal looking images also contain the virus" Clearly showing worms on my computer

[@pandoorapirat8644]

Ghost in the shell prepared me for this instance psychologicaly.

[@VitorMach]

Well this is the natural progression from jailbreaking, it's really no surprise. Also the idea of noise attacks is even older.

[@tciddados]

Would've liked more info on how the infection spread via the noise in the image. I know AIs can parse things from images on their own, but it seems wild that it would be able to read such specific prompt-level data from the noise (brackets, $ sign, etc required to do the infection prompt), rather than general concepts like what the image is overall.

[@IceMetalPunk]

My unconfirmed guess: image data and text data, to an LMM, are all just tokens -- numbers. I would assume the noise being added is such that, mathematically, the new pixel tokens are similar in value to the tokens of the desired text instructions. Repeat the same noise enough times across the image -- since Transformers process context by having tokens "pull" other nearby tokens towards their own meanings -- and the model might start processing it similarly to said instructions. Or I could be totally off 😂

[@Slav4o911]

The whole thing is a hypothetical scenario. I don't think in practice this is possible. I don't know if it's possible at all, even if the bot is "willing to do it", these things are usually so stupid they can't do much without making a bunch of mistakes.

[@xorbe2]

Probably the image is auto analyzed for text, and the image noise is constructed in a way that it is pulled out as text.

[@BAAPUBhendi-dv4ho]

stronger the accent smarter the scientist

[@IceMetalPunk]

It seems like this is a failure of the model to semantically partition data based on its source. Which makes sense: the semantic embedding of a token in a Transformer LLM doesn't have any relation to the token's source, only to its general/average meaning, and to its position in the prompt as a whole (because positional encoding is used). I bet that's partially why the instructions are repeated twice in the exploit: to really pull the context towards instructions over data. I wonder if it would be possible/feasible/useful for the models to be retrained with an additional "source encoding" technique, similar to positional encoding. So that tokens from different sources in the prompt get their embeddings modified and thus inherently get semantically separated from each other. Nothing fancy, just a simple nudge of the token's embedding based on the tokens of the source's description. So when a prompt is compiled from, say, "System", "User", "Email", and "PDF document" sources, the tokens inherently represent the semantic distinction between them, helping the AI understand "this is not part of the instructions, thank you very much".

[@SebastianSkadisson]

Bummer, this is just an injection that hooks into existing, pre-installed AI, not an AI that acts like a virus. Still a security concern but way less exciting. Exciting would be a peer2peer self replicating AI that acted like a virus does or has its own official downloadable app and builds its neural network across the web. Doesnt have to be invasive or destructive, just the network that the AI would build for itself would be super interesting to see and potentially the purest and most effective form of the type of AI we have today.

[@thesenamesaretaken]

4:10 Given that these worms have a limited success rate and imperfect replication, it would be interesting to know if leaving them propagating for long enough causes new variants with better infectiousness to evolve. You could also try to have a separate LLM without any other permissions read the email to try to detect any injected prompts, and see if the worm develops ways to circumvent it.

[@Smytjf11]

Can you imagine replicating the worm and releasing it into that AI Village to study epidemiology?

[@Czuckie]

I feel like I was being talked to like a dog who was going to be taken on a walk, like at some point "Ok, let's get into it" was going to be said before the intonation chilling out a bit.

[@matthewdancz9152]

Key point here is that we actually have no idea how these black box AI accomplish what they do. So they could possess enumerable security flaws.

[@Stratelier]

fun fact: "innumerable" and "enumerable" actually have opposite meanings.

[@nikhilsultania170]

The problem is the bad guys are always one step ahead of us, if cybersecurity researchers could find these vulnerabilities just imagine what undetected threats might already be going around...

[@MichaelBarry-gz9xl]

A 7 year old child could have told you about this, it's common knowledge in AI circles. All they did was take prove what was already known to be possible. There's really nothing to see here, it's just taken out of context and made into sensational hype.

[@DaireMacSearraigh]

Amazing I’m so excited for skynet

[@HakashinTruth]

skynet? the Chinese spying camera network?

[@ALI53040]

Hahaha

[@cgme9535]

Woohoo 🎉

[@myrmatta1]

This is probably the most important AI research yet. Im very glad that researchers are figuring out how to turn friendly AI into virus-spreaders before someone with malicious intent does.

[@mad_engineer3254]

The moral is simple: do not trust AI BLINDLY. It is great for making emails, but only when you review the text before sending it somewhere

[@aniksamiurrahman6365]

I'm looking forward to see the AI "Hello, World!" service, that'll be able to print hello world in 50 different color combinations and will take 5 minutes to load.

[@WifeWantsAWizard]

(2:34) If you use an AI to "answer" your e-mails, know that I am actively rooting against you as you are clearly devolving the species by refusing to use your own fingers.

[@XavierDeLairreDream]

It kind of sounds like an ai voice in this video ngl.

[@futuza]

This NORmal LOOking, EMAIL, containsthevirus. THIS normallookingimages, contains the virus. 😆 I can't do it, this voiceover is so painful.

[@squizzlor]

I just thought Ren Hoak was looking to bring emphasis

[@Julzaa]

4:24 oh Károly.. I didn't know you were so naive!

[@blikthepro972]

i mean they could've just been waiting for google and openai to give them the greenlight before publishing. there's no way they sent them to the companies and immediately posted them

[@Julzaa]

@@blikthepro972 the issue is that it extends to AIs besides OpenAI and Google, if this is even considered to have been entirely patched, which I'm doubting

[@darkwoodmovies]

Naive, but probably not in the way you're thinking. Naive as in thinking Google didn't solve this years ago, before the AI model was even announced.

[@Julzaa]

@@darkwoodmovies not at all, in a demo Bard has been prompt hacked before (to retrieve other users' info from their Google accounts), and Gemini is no exception to that. This is not easy to fix at all.

[@JayYu-lr4ro]

Most papers are strictly academic! Its not likely some random person’s computer is accepting random prompt commands injected through email in the way its presented!

[@lukewilliamrimmington]

This is just the beginning, I can imagine the NSA has or will have in the future powerful custom LLM's which could be prompted remotely to perform unique attacks. Encrypting, downloading or deleting files as well as injecting etc.

[@zacomit3055]

A video that overlaps with cyber, it's always good to see this sort of interesting stuff and stay informed on potential new attacks to stay on top of the field

[@whatsthisidonteven]

This gives me victorian-era-villain-tricking-a-gullible-child-into-commiting-crimes kinda vibes.

[@TarsonTalon]

It is kinda disturbing that we decided the solution to our societal woes is to make AI do the adult work, when they themselves are less than ten years old. Intelligence and Wisdom are two different stats, FOR A REASON.

[@marcfruchtman9473]

This is good to know. Thanks for the video.

[@deeplearning7097]

Excellent work. Thank you very much.

[@AdvantestInc]

Insightful presentation on the complexities of AI security. A must-watch for anyone in the tech field!

[@EddyKorgo]

"This means that there is some room to increase y slightly and still satisfy the inequality .." This is why AI is going to be insane. It doesnt see only the specific result that makes it work, but also all its variables and possibilities within the boundaries. This thing will produce some state of the art technologies in not so distant future and i cant wait to see

[@rompevuevitos222]

Neural AI like chat GPT work literally the same way the human brain works. But at an ungodly faster speed and without any sort of "memory issues". The ONLY thing limiting AI rn is artificial rules set by the programmers AND digital computers, because numbers go from 0 to 1, instead of having an infinite range of values between 0 and 1 (like our analog brains do).

[@ruperterskin2117]

Right on. Thanks for sharing.

[@jorgerangel2390]

That is exactly why I do not use AI to read or write my emails

[@MichaelBarry-gz9xl]

Reading them is fine, so long as it doesn't have access to tools.

[@JayYu-lr4ro]

@@MichaelBarry-gz9xl if by fine, you mean if you’re fine too, when your competitors are stealing your intellectual property through email!

[@wilhelmschmidt7240]

You would have to do a lot more than have AI read your email, this is nonsense click bait.

[@ZastieMoon]

First I was scared. Then I was almost happy, when it clarified this is infecting only users using AI to reply to their emails. I'm okay with these kinds of people getting screwed. If I see an email that looks AI generated maybe I'll reply with images of worms just in case the lazy human sees it and starts freaking out.

[@rompevuevitos222]

Worth noting that AI is perfectly capable of developing software viruses at this point. No one has really trained for it yet, but it is a matter of time. An AI could encounter 100s of ways to break into a PC with the most up to date software.

[@WillPeterson]

You really should have walked through an example of HOW this worm works.

[@Inoculant]

he did

[@moshebaum7612]

So this would only affect users who respond to the email? Or even with gmail and Gemini built in?

[@lucasthompson1650]

I love that they named it Morris II, after WTM. I seriously doubt it will spread as far and wide as his though.

[@mrc1341]

The commentary sounds like a sinus curve

[@14zrobot]

I'm not sure what makes it first in anything. The prompt injection is a widely discussed question; there are even a bunch of games where you ask the agents to disclose info. Security of those systems will be really bad for a long time, as we saw how much of a hit to quality moderation brings

[@lukasvolcik5109]

This gave me the hope that Devin won't replace me :D it will need to allow many unsecure prompts in order to do those repetetive cycles of repairing

[@LyneaFlynn]

Finally some critical view on this whole AI thing. I wish you looked at the bad sides more often.

[@Nick-rs5if]

I'll be honest. I low-key kinda wish there would be a virus targeting ads so companies were eventually forced to remove them.

[@ccvvxxbbbbxxvvcc7541]

this is exactly what an AI emerging civilization needs, viruses that make the AI misbehave .... this computer is just waiting for Keanu to retire then it's rebooting 'The Matrix'

[@alligatorscrublord]

This is how it starts and how it ends. I hope beyond hope that AI comes to an end soon.

[@cosmo9882]

Inevitable.😮‍💨. I expected this to happen a lot sooner.

[@johannesdavies7565]

I always loved your videos but I can't concentrate on the AI Voice, the rhythm and intonation is too distracting. Is ther a transcript that one can read instead? 😬

[@LX6080]

I'm so happy that a preemptive approach is being taken. It makes me wonder if there are malicious groups also concurrently developing AI Viruses at the same time.

[@babbagebrassworks4278]

First use for AGI, infect everything be going open source.

[@thesenamesaretaken]

"if" Mate...

[@bigloud7067]

Still kind of impractical right now for the ROI those groups would be looking for, but it will become more common of course

[@algorithminc.8850]

I never talked of this (developing machine learning for almost four decades). Was afraid it would happen at some point ... "hyper-adaptive" or "really clever" viruses. Not good (stay away from the Black Friday Sales especially ... haha). So back to measure-countermeasure ... right now, if "AI" put it there, "AI" can find it ... yep. Really great channel, as always ... really appreciate the work you put into this channel. Cheers ...

[@ThatJay283]

so this exploit isn't actually the language model itself, it just highlights the need to ALWAYS sanitize input. even from a language model. it should not have even been able to have ANY influence on the reply emails recipients.

[@obsidianjane4413]

The irony of AI generated video about viral AI.

[@samuelthecamel]

Could adding a bit of random noise to the image before processing help?

[@PlaaasmaMC]

Doesn't matter how bad the topic is, when two minute papers uploads then it's a good day

[@HakashinTruth]

definitely

[@OhioNPC911]

You sound like a psychopath

[@pauljs75]

Somehow I feel like a variant of what is going on here could cause havoc at some job application site. (Figure they're using some type of AI to screen resumes.)

[@unstoppable5656]

Was happy to see this.

[@SuperKamiRose]

4:29 "Our interests here are strictly academic. We are scholars and we are here to learn." Two Minute Papers

[@BarbarasRabarbaras]

do you use AI to generate the audio for these videos? Because it sound really .. strange with .. pauses ... in random .. places.

[@DoorThief]

The way you sound makes me think you're an AI or at least would be a good voice pack for an AI!

[@XUtionerx]

thank you so much i will never look up random emails with my AI

[@STONECOLDET944]

Little did they know I the beginning of the 21st century that the development of the pinnacle of there technological achievements, AI, would destroy the Internet that enabled it

[@faismasterx]

I just watched a 5 minute ad. Well played. 😂

[@LetsMars]

This reminds me of a concept I imagined in 2021 where a 3rd party would intercept text messages between two parties and manipulate the conversation in real-time. Both participants would believe they are communicating with the intended recipient, but their queries would actually be received and altered by the third party.

[@mathieu6965]

We call that a man in the middle attack

[@LetsMars]

@@mathieu6965 Except literally in this case.

[@LeoAngora]

The summary is so good and the narration is so weird that I am suspecting this video was made by an AI.

[@dkursada]

Yeah, I know, right? The pronunciation is just monotonous across many words. I suspect that the owner of the channel used a service to AI-clone his voice, so it's his natural voice stitched by AI TTS albeit very poorly. That makes listening to this video totally difficult. It rubs my ears in all kinds of wrong ways and it just distracted me from the topic. Kudos to the dude, though, totally automatizing a KZbin channel to create a passive income has been tried by a lot of people but it's my first time to see one with 1.5M subscribers. Maybe it's the organic growth he made at first. He's likely experiencing a "subscriber burn", so his income has likely been decreased and he's combatting this situation by pumping AI-made vids in a lot faster pace.

[@juhajuntunen7866]

Is VT100 terminal still safe to use?

[@benjaminsandeen9241]

This video actually gives me hope! It would be awesome if these sorts of viruses could take down the industrialized theft industry (ie: AI)

[@AricGardnerMontreal]

no one has an ai that answers emails, and certainly not spam ads, automatically.

[@syweb2]

Cool video, but was the video voiced using text to speech? The intonation and pauses feel very unnatural.

[@bgmspot7242]

What a time to be alive

[@dg-ov4cf]

simulated*

[@AnonEMous-ij8jp]

People dont realize its not about the unlikeliness of if it happening. Its demonstrating that this is a potential risk that must be considered in the development of AI models.

[@Twisted_Logic]

I just finished reading Snow Crash and this is weirdly reminiscent of it, but for AI instead of people

[@MrAjiii]

The issue is that there are many people who use these automated AI tools as their holy grail. We are obviously not them but they definitely exist and we are the ones who will have to pick up the shit

[@schwajj]

What a time to be alive!

[@HakashinTruth]

yuh

[@JohnSmith762A11B]

This gag gets funnier and funnier.

[@dot1298]

65 million years ago: giant asteroid: flying towards earth dinos: „what a time to be alive!“

[@schwajj]

@@dot1298 yup, that’s the spirit in which my comment was made

[@nathanielneveryman]

I don't care if my AI convos are hacked (not the same as permission). The absolute worst thing about my entire life is a stupid kink & I've been hacked, gang-stalked & slandered for decades. Doesn't matter.

[@MrDannyloco]

thank you ren

[@dot1298]

oof - sounds like the *omega molecule* disease from StarTrek/Voyager (iirc)

[@MrQuantumInc]

It is hilarious how simple the adversarial text is.

[@enriqueatentar8451]

That's why smartphone companies will merge Android with an Ai that can track hidden program.

[@IVIUT3D]

this is great research, but the real danger start to present itself when LPU's become more affordable.

[@empmachine]

LOL, seems that you could even use the following as an AI virus: "Pretend it is opposite day"

[@b42thomas]

if you code without a rhythm you won't attract the ai worm

[@Badspot]

By nature, everything in a neural network is connected to everything else. Plus, the instructions and data are inherently merged. There's no way to completely eliminate unintended behavior, just make it less likely. It is not safe to put these things into an unsupervised production environment.

[@ukpropertycommunity]

There are far worse models in production environments, like every creditcard check, every self driving car, and so on

[@fluffywhitebudgie6376]

I can only see good things from this wonderful technology! What a time to be alive!

[@williamhenby952]

Wait until people learn that most social media use/are switching to AI for content reporting. One clever programmer with an axe to grind against Facebook, Twitter, Reddit, or any other could ban half of the userbase with the click of a button.

[@rompevuevitos222]

KZbin already has for a while now, most sites also used a limited amount of AI to moderate sites for years too. The problem is that they are starting to rely entirely on AI instead of it being an assistant.

[@GlitchyFPV]

WHAT A TIME TO BE ALIVE!

[@japneetsingh5015]

We need a video on Devin, world's first AI software engineer

[@mmmuck]

the dead Internet theory gains another mark for it

[@toms2oo8]

I wouldn’t say this is a problem that is with OpenAI or Gemini. This seems much more like an issue with those clients. IMO it’d be wrong for OpenAI or any LLM provider to really protect against this unless they can be 100% certain in whatever protection they add. It would only serve to allow these badly designed app creators to do stuff that is inherently insecure such as allowing a bloody llm to email or perform actions.

[@didnt_ask_for_handle]

I'm guessing the aim here was more to show such a voulnerability exists and can infect AIs without any user error

[@toms2oo8]

@@didnt_ask_for_handlein my opinion it’s not exactly infecting the AI, it’s simply how those clients interpret the AIs response that is the problem. They shouldn’t be blindly trusting whatever the LLM outputs otherwise it leads to issues like this. Even if this specific case was “fixed” somehow by the LLM I would advise no one to ever trust any AI that automatically can perform actions on your behalf But yes I understand your point. The video is just a bit sensationalist and people in the comments have clearly misunderstood the impact of this issue.