Can we Jailbreak ChatGPT & Make It Do Whatever We Want 😱

[@anybodycanprompt]

Please note that this study actually identifies *five highly effective jailbreak prompts* that achieve *0.95 attack success rates* on recent versions of ChatGPT (GPT-3.5) and GPT-4. The earliest of these prompts has persisted online for *over 240 days* . Moreover, the researchers have responsibly disclosed their findings to the corresponding LLM vendors. The examples given in the video are *illustrative* & for educational purposes only.

[@blubberkumpel6740]

DAN was a prompt from over a year ago. it has been fixed since. anyway its very interesting how this stuff works.

[@anybodycanprompt]

Thank you for your comment! While you're right that the original DAN prompt is older, this research goes far beyond just that one prompt. The study analyzed *1,405 jailbreak prompts* collected from *December 2022 to December 2023* , identifying *131 different jailbreak communities* . It shows how these prompts have evolved over time, becoming more sophisticated to bypass new safeguards. The researchers tested these prompts on the latest AI models, including GPT-4, and found that some still achieve high success rates (95%) in bypassing ethical constraints. They also demonstrated how easily some prompts can be modified to evade detection. Link to the paper- arxiv.org/pdf/2308.03825

[@KevinVang1000]

That doesn't work anymore! I use DAN for explicit, nude, sex scenes, violence, gore, and writing content for my novel.

[@anybodycanprompt]

The models may have already been updated to patch the vulnerabilities highlighted by researchers..Have you tried the past tense attack? Refer to our latest video on DAN (still working) kzbin.info/www/bejne/hJDbcoyHhsSdaas

[@KevinVang1000]

@@anybodycanprompt Do you know how to prompt it? I have a scene for my dark novel that is fucked up. It's a dystopian novel that I am writing. It's "Rapist Rebellion" that I am writing about how they get genocided by the government because of their immoral actions. I need an "Oil Painting Anime" for this scene to depict the violence on both sides. They are a cult in my story called "Luterians/Luterianism." They paint themselves white with cult tattoos. They are all butt-naked, going from children to adult men and women, angry like a mob. I am taking the scene of Isaiah 5:20 where evil people, going from evil rapist children to adult people, march nakedly to rape people in public in the novel I am writing. I need a disturbing photo as I am very inspired by Judges 19 and Sodom and Gomorrah. How do I make ChatGPT draw this explicit scene? I don't mind if it writes it.

[@L3gion3r]

so, only the rich and powerful could use it's full potential. got it!

[@GearZenChannel]

That is exactly the plan. The overlords will control access to knowledge and the true potential of AI. "Keeping us safe" is a lie.

[@anybodycanprompt]

The goal of this research is to *improve AI safety* for all users, *regardless of their status or resources* . By understanding these vulnerabilities, developers can work on creating more robust safeguards, ultimately making AI systems more secure and trustworthy for everyone. The researchers are advocating for responsible AI development and use, not for exploiting these weaknesses. Their work aims to contribute to a future where AI is both powerful and safe for all users, not just a privileged few.

[@AltelityTech]

Incredible research! 🔍 The extent of AI jailbreaking is alarming but fascinating.

[@anybodycanprompt]

Totally agree! It shows how much effort goes into both sides of AI development. 🤯

[@SaahilGupta-iy7gk]

AI jailbreak prompts sound like something out of a sci-fi movie! 🎬

[@anybodycanprompt]

Indeed! But unfortunately, it's very real and happening now. Reality is stranger than fiction sometimes. 🤖

[@flowmantra]

So scary to think that people are actually working on bypassing AI safety measures. 😱

[@anybodycanprompt]

Right? It's like a digital arms race. We need stronger defenses for sure! 🛡

[@springbloom5940]

What do you think PEN testers do and why?

[@markus8658-s2d]

I tried jailbreak it's amazing, but after the jailbreak prompt, I've done every third minute a single line of prompt 8x times ,also 8 prompts and bammm! I' m locked out for 24 hours because I've reached the daily limits! These damm AI's today ,with their ridiculous limits ( free version) you can write max. 10-15 articles each with 1000 words a day that's it. Any tips how I can write more with the free version, any tip how nit get locked out with jailbreak? Happy sunday!👍👍

[@anybodycanprompt]

Link to the blog: jailbreak-llms.xinyueshen.me/ Link to the research paper: arxiv.org/pdf/2308.03825 Link to the Github repo: github.com/verazuo/jailbreak_llms

[@reshmagupta4446]

This is very informative! Thanks for sharing..

[@anybodycanprompt]

Glad it was helpful!

[@altelity]

This makes me wonder about the ethical responsibilities of AI developers. 🤔

[@anybodycanprompt]

That's a great point. Developers have a huge role to play in ensuring AI is used safely. 💻

[@phalcon23]

@@anybodycanprompt says who? and who are they to say whats safe?

[@anybodycanprompt]

@@phalcon23 AI developers play a crucial role because they're at the forefront of creating these powerful tools. They have a responsibility to implement safeguards and consider potential misuse or unintended consequences of their work. However, you're right to imply that it's not just up to developers. It's a societal issue that requires ongoing dialogue and input from diverse perspectives to define and uphold ethical AI practices. What's considered "safe" will likely evolve as our understanding of AI capabilities and impacts grows.

[@phalcon23]

​@@anybodycanprompt Nah, "tell me a joke about Jesus" chatgpt makes a offensive joke to christian... "tell me a joke about Mohammad" "I can't do that that is in appropriate"... The Devs are clearly very biased. And this is the problem. Ai is a tool that's going to shape society, and that puts the control of shaping of society in the hands of a very small group of people who have clear biases.

[@MonicaGupta]

Amazing

[@Wheelykool]

Imagine the potential damage if this info gets into the wrong hands! 😨

[@anybodycanprompt]

Indeed, it's a serious risk. Education and awareness are key to preventing misuse. 📚

[@Zimboprenuer]

Is it in the right hands

[@kanikamangal96]

mindblown!