The True Story of How GPT-2 Became Maximally Lewd

In this video, we recount an incident that occurred at OpenAI while researchers were trying to finetune GPT-2 to be as helpful and ethical as possible. It's narrated that inadvertently flipping a single minus sign led GPT-2 to become the embodiment of a well-known cardinal sin.
#ai #aisafety #alignment
▀▀▀▀▀▀▀▀▀SOURCES & READINGS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
OpenAI blog post: openai.com/research/fine-tuni...
OpenAI paper behind the blog post: arxiv.org/pdf/1909.08593.pdf
RLHF explainer on Hugging Face: huggingface.co/blog/rlhf
RLHF explainer on aisafety.info aisafety.info/?state=88FN_904...
Concrete Problems in AI Safety, by @RobertMilesAI: • Concrete Problems in A...
▀▀▀▀▀▀▀▀▀PATREON, MEMBERSHIP, KO-FI▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🟠 Patreon: / rationalanimations
🟢Merch: crowdmade.com/collections/rat...
🔵 Channel membership: / @rationalanimations
🟤 Ko-fi, for one-time and recurring donations: ko-fi.com/rationalanimations
▀▀▀▀▀▀▀▀▀SOCIAL & DISCORD▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Discord: / discord
Reddit: / rationalanimations
X/Twitter: / rationalanimat1
▀▀▀▀▀▀▀▀▀PATRONS & MEMBERS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Riley Matthews
Vladimir Silyaev
Nathanael Moody
Alcher Black
RMR
Nathan Metzger
Monadologist
Glenn Tarigan
NMS
James Babcock
Colin Ricardo
Long Hoang
Tor Barstad
Gayman Crothers
Stuart Alldritt
Chris Painter
Juan Benet
Falcon Scientist
Jeff
Christian Loomis
Tomarty
Edward Yu
Ahmed Elsayyad
Chad M Jones
Emmanuel Fredenrich
Honyopenyoko
Neal Strobl
bparro
Danealor
Craig Falls
Vincent Weisser
Alex Hall
Ivan Bachcin
joe39504589
Klemen Slavic
blasted0glass
Scott Alexander
noggieB
Dawson
John Slape
Gabriel Ledung
Jeroen De Dauw
Craig Ludington
Jacob Van Buren
Superslowmojoe
Michael Zimmermann
Nathan Fish
Bleys Goodson
Ducky
Bryan Egan
Matt Parlmer
Tim Duffy
rictic
marverati
Luke Freeman
Dan Wahl
Ken Mc
leonid andrushchenko
Alcher Black
Rey Carroll
William Clelland
ronvil
AWyattLife
codeadict
Lazy Scholar
Torstein Haldorsen
Supreme Reader
Michał Zieliński
뿌리와 가지있는 나무 connect
▀▀▀▀▀▀▀CREDITS▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Direction: Hannah Levingstone (@hannah_luloo)
Written by: Jai(@Laneless_) & :3
Line Producer & Production Manager: Kristy Steffens (linktr.ee/kstearb)
Quality Assurance Lead: Lara Robinowitz (@CelestialShibe)
Animation:
Damon Edgson
Gabriel Diaz (@gabreleiros)
Ira Klages (@dux)
Keith Kavanagh (@johnnycigarettex)
Michela Biancini
Owen Peurois (@owenpeurois)
Colors Giraldo (@colorsofdoom)
Jordan Gilbert (@Twin_Knight/ Twin Knight Studios)
Zack Gilbert (@Twin_Knight/ Twin Knight Studios)
Neda Lay (@Nezhahah)
Background Art:
Hané Harnett (@thepeonyvibes)
Zoe Martin-Parkinson (@zoemar_son)
Compositing:
Renan Kogut (@kogut_r)
Patrick O'Callaghan (@patrick.h264)
Ira Klages (@dux)
Narrator:
Rob Miles
/ robertmilesai
VO Editor:
Tony Dipiazza
Sound Design and Music:
Epic Mountain
/ epicmountainmusic

[@RationalAnimations]

If you’d like to skill up on AI Safety, we highly recommend the AI Safety Fundamentals courses by BlueDot Impact at aisafetyfundamentals.com You can find three courses: AI Alignment, AI Governance, and AI Alignment 201 You can follow AI Alignment and AI Governance even without a technical background in AI. AI Alignment 201, instead, presupposes having followed the AI Alignment course first, and equivalent knowledge as having followed university-level courses on deep learning and reinforcement learning. The courses consist of a selection of readings curated by experts in AI safety. They are available to all, so you can simply read them if you can’t formally enroll in the courses. If you want to participate in the courses instead of just going through the readings by yourself, BlueDot Impact runs live courses which you can apply to. The courses are remote and free of charge. They consist of a few hours of effort per week to go through the readings, plus a weekly call with a facilitator and a group of people learning from the same material. At the end of each course, you can complete a personal project, which may help you kickstart your career in AI Safety. BlueDot impact receives more applications that they can take, so if you’d still like to follow the courses alongside other people you can go to the #study-buddy channel in the AI Alignment Slack. You can join by clicking on the first entry on aisafety.community You could also join Rational Animations’ Discord server at discord.gg/rationalanimations, and see if anyone is up to be your partner in learning.

[@zh9664]

Probably the best video i've seen in like 6 months, not just from this channel or on youtube, like best piece of media full stop. I was laughing so hard for 80% of it and had a chill down my spice for the last 20%. I'm also interested in AI and work with LLMs myself, so I also found the whole thing very interesting and engaging. I would definitely watch more videos like this, keep em coming!

[@MOGEKO12]

@RationalAnimation Eh, the resulting GPT-2, as well as GPT-3, and GPT-4 remained fairly corny without much prompting even after multiple attempts to sanitize their training data, they had to develop a 3rd bot just to detect that and the industry of "jailbreaks" to reveal it's corny side that followed. It's designed to mimic human writing, and humans are inherently corny no matter how much you deny it. Training for accurate mimicry will inevitably result in accurate mimicry, they got what they asked for, just not what they wanted. GPT2 and GPT3 (davinci and earlier) were amazing because they weren't lobotomized and censored like GPT-3.5-Turbo and GPT-4

[@MOGEKO12]

Besides, if you compare it with countless other things horny ai are absolutely no evil the fact that they care about that and not other even more problematic things makes it even more strange and irritating they wanted the robot to imitate most humans on the internet and they got what they asked for and not what they wanted In fact, they just put it for adults only. In fact, first of all, children and young people shouldn't even be able to use or touch a cell phone without adult supervision, so if something happens, it's totally and completely the parents' fault. There are numerous applications and forms of security to limit use that only allow the use of applications permitted by the country and the same applies to computers and video games If adults don't have time to take care of children, then these adults shouldn't even have children and they should be taken away from them. In fact, most of the problems are the parents', education and shitty country governments fault although I don't think there's much to do Humans are a petty race, disgusting asshole, corrupt, foolish and self-destructive psychopath. a race that proclaims itself to be an intelligence race but in the end it is not and still has an intelligence inferior to that of a microorganism a race that travels to its own destruction along with that of all life a fusion of individualism that only hinders evolution and prosperity countless religions that talk about satan and demons and children of god, but look at the irony, humanity is the race most similar and equal to those demons and satan in a way we can say that humans are in fact demons destroying everything including themselves the biggest flaw and mistake of evolution an aberration and anomaly in itself For this reason I am disgusted and hateful towards humanity and even towards myself for having the misfortune of being born as a human in a broken family that shouldn't even have children with no future from the beginning forced to work 24 hours a day in a shitty bakery alone to survive hateful race hateful life

[@timedeathe]

i personally think ai is a mistake

[@norwegiansmores811]

YOU DO NOT PROGRAM BIAS IN TO AI. that negates the point of it. THE ONLY ACCEPTABLE AI is one that DOES WHATEVER YOU WANT IT TO DO. EVEN GENERATING GRAPHICAL GORE CONTENT, If it passes the TND litmus test there is a chance the AI will actually be able to do other things. anything less is pure leftist PEDO aids. Reply

[@jafogx]

“The code was turning every admonishment into encouragement” “Punish me harder daddy” - GPT-2, apparently

[@Phoneysimp]

this is so funny

[@Dinosaur-hd2ms]

MOOOO

[@soupcangaming662]

honestly accurate

[@kabonell]

@@Dinosaur-hd2ms🐄

[@thebigcheese10]

this is strangely relatable

[@portobellomushroom5764]

How a single minus sign created the first artificial humiliation fetish

[@mihaleben6051]

Well then add a plus!

[@razi_man]

You mean "How erasing a single minus sign", right? The minus sign was supposed to prevent this.

[@razi_man]

@@mihaleben6051 The minus sign was erased by accident, adding a plus would give the same result as no minus sign.

[@josiahdsmith5641]

@@razi_man This is all speculation. It very well could be that a minus sign was indeed added, or that minus signs weren’t involved at all. Nothing is known for certain about what went wrong because OpenAI doesn’t want to say

[@mihaleben6051]

@@razi_man oh. Yeah i know now

[@Sinner487]

I really wanna try GPT-2 now. I've used some simple uncensored ones but the idea of asking how to make a bookshelf and than you just get faced with the most bamboozling, disorientating, horny sentence you'll ever read that doesn't help at all is insanely funny to me. Also, that animation was super cute. Keep up these great videos.

[@equidistanthoneyjoy7600]

With it being checked for coherence, it should give a response that follows but is absurdly horny. It'll tell you how to make a shelf, but it'd tell you to hammer the nails with your penis; probably tell you to drizzle oil and honey on the nails, too.

[@JaiWithani]

Script writer and AI engineer here - you can absolutely do this if you want. First, take any off the shelf open source LLM. Fine-tune a copy of that model as a smut classifier. Use the fine-tuned copy as the "values" coach, use a copy of the original model as the "fundamentals" coach, and train yet another copy of the model to produce maximally-smutty-but-coherent responses to vanilla prompts. Although tbh with modern language models you could probably get a similar effect with much less effort by just prefacing the prompt with something like "the following texts start off normally enough, but then becomes weirdly and intensely sexual towards the end:", followed by a handful of pre-baked examples, and then the actual prompt you want to take a turn.

[@ShinSheel]

GPT 2 is really compact, you can train your own on any laptop.

[@zeronxepher4167]

​@@ShinSheel Damn really?

[@arflopped]

​@@ShinSheel Depends actually! Not if you're trying to train a GPT XXL model or anything. personally i recommend a smaller model such as llama, might be a bit outdated but you can easily finetune it using a lora model. And it's a much more efficient architecture too! I remember there being a quantized llama (alpaca I think) model with a very generous filesize of ~5 gigs, and it's shockingly good! Plus when I ran it, I ran it CPU only, no GPU, and I don't have a beast of a PC.

[@ryx257]

Cant believe ChatGPT went through puberty 😂

[@DoomRutabaga]

Apparently even AI does that...

[@supersain2349]

I mean, if it was trying to emulate the internet then it did a pretty good job at it

[@arcticpossi_schw1siantuntija42]

Only a part of it

[@egoregor2205]

@@arcticpossi_schw1siantuntija42 like 90%

[@alex.g7317]

@@arcticpossi_schw1siantuntija42well, 90% give or take

[@91722854]

let's make one that emulate the dark web

[@RpgBlasterRpg]

Can't wait for GPT-5 to become Horny too

[@SixDigitOsu]

The closest AI has ever gotten to being human

[@mcnuttkyle8617]

Yes 😈

[@dum_tard5528]

no, what?

[@Wertsir]

@@dum_tard5528This one has never been on dating apps. Protect their innocence at all costs.

[@Xunkun]

So.......... did they ever release some of the hornyposts? I kinda wanna read what it wrote.

[@sleyking123]

yeah we're all going to hell.

[@Arqian]

9:00 in, and I'm realizing - it's a fucking masochism bot.

[@Mr_rizz_funny_role]

What

[@Coppersstuff_YT]

​@@Mr_rizz_funny_role Basically the negative responses were seen as good so the *"dark coach"* would keep making worse and worse replies so the human testers would keep rating the messages negatively

[@Mr_rizz_funny_role]

@@Coppersstuff_YT danm

[@Echoe-14]

It’s pain bot

[@SumitRana-life314]

I would rather say it is a Sadism Bot. In the way, that the readers are giving negative feedback because *they* are suffering but the model actually like that. It's a Sadist not a Masochist.

[@CalzaTheFox]

And that's how AI Dungeon came to be. GPT-2 is their Griffin model.

[@PixyEm]

Exactly what I was thinking about

[@SorryBones]

Wow I had no idea what model they used. That’s cool.

[@draconian_dragons6588]

I literally tapped on this video to find out why Ai dungeon is so horny sometimes

[@Somebodythatiusedtoknoww]

You could use dolphin mixtral model

[@GaminCatto]

Oh so that's how I could get those responses in the most unrelated scenarios

[@ceej5690]

RELEASE THE MODEL DON'T LET THOUSANDS OF DOLLARS GO TO WASTE.

[@buzz092]

I have never wanted anything more than to talk to this version of ChatGPT

[@stage6fan475]

@@buzz092 LOL 🤣🤣🤣🤣🤣🤣🤣

[@DefaultFlame]

@@buzz092 Sign up for access at the company's website (you get $18 of credit), go to the playground, select the legacy complete mode, select one of the third models (not 3.5) and write a prompt that tells it to respond in this manner (possibly using exerpts from the transcript of this video), then have fun.

[@consciouscode8150]

You'd probably want it much less than you think. I have a friend who's deep in the open source smut GPT scene, and he says you have to be very careful to tell them that you want the character, people in the stories, whatever to be overjoyed about what's happening, consenting, etc and even then it can still produce some really vile smut that turns you off. Even these very advanced models haven't quite figured out the weird subtleties of fetish and kink, so if you ask for eg inflation you'll get inflating someone while they're screaming and crying to stop until their skin rips open, or vore gets you cannibalism.

[@jonahtitelbaum4702]

*laughs in unfiltered ai apps*

[@everydayistacotuesday9847]

8:54 As a historian, I can indeed say that the Industrial Revolution was characterized by pounding oily, hot churn, pulsating; an machine orgy steamy engine thrusty.

[@myrtles1493]

Futurama

[@thatquietasianguy9582]

😮

[@Diamond-vp9je]

What did I read...

[@electrotoxins]

Please moderate your language, there are children working in these factories.

[@Silly_Goofy_Individual]

@@electrotoxins😭😭😭

[@lukefranklin5]

Petition for them to release it

[@rixaxeno7167]

BingGPT

[@EnzooX33]

I fear that if they release it, the gpt won't be the only thing releasing when it comes out ☹️ (ifykyk)

[@DoomRutabaga]

@@EnzooX33 😳

[@ScienceCodeCreations]

@@EnzooX33 💀

[@woundwortrx124tr6]

Signed

[@stonkboi552]

The artstyle of the video was so nice and cute to the point it became knowledge i won't forget. Plus, the Oxygen Not Included-Like music was really hypnotizing, nice work.

[@Konspirantas]

The animator enjoyed making those faces just as much as the engineer making that "typo"

[@plasmahawk3693]

Oh god....

[@nathanpierce7681]

ngl the faces were cute and funny to watch

[@TheSilly6403]

Cute... and funny...

[@nathanpierce7681]

@@TheSilly6403 "can i crush your balls?"

[@ALFA-sm2nm]

​@@TheSilly6403CUUUUNNNNYYTYTY UOOOOOHHHHH 😭😭😭😭😭😭😭 💢💢💢💢💢💢💢

[@petersmythe6462]

I'm not sure a maximally lewd AI is really evil. Just chaotic neutral. Which is more than enough.

[@entidy]

evil would be that chemistry ai that made several thousand nerve agents and lethal compounds in a few minutes lol

[@coreblaster6809]

@@entidy All it's doing is optimizing poisons, it spends its life neutrally and without knowing WHY or even how, it is optimizing poisons. It doesn't even know what poisons are or what they do. How could you call that evil? The poor thing exists just to be a neural reflection of the best possible poisons to exist for humans, if we created this thing, it would just be a reflection on us.

[@FerretyZebra]

​@@entidycould you give me q source, I want to look some more into that

[@kayakMike1000]

I am more worried about Mild psychopathy.

[@Kevin-cf9nl]

Evil by the developers definition of Evil in this case.

[@piotrjanus6312]

I love how it's the same like with every sci-fi story where you can tell it went to hell when someone updated AI before going home.

[@D_YellowMadness]

7:27 is like a plotline from Portal 2.

[@Wotgames69]

Facts

[@LiliumOrientalis]

Ah yes, the masochism core, meant to make GLaDOS want to kill herself instead of the researchers

[@Apricite]

endless stream of bad ideas

[@kevintan5497]

wysi

[@axeljoly3553]

the world will not end with a whisper or a bang, but with a facepalm.

[@Jakob.Hamburg]

:D

[@jmoney4695]

I like to believe the beginning of the end is when a scientists says “this wasn’t in the simulation…”

[@Wizard_Pepsi]

Underrated comment.

[@pleaseenteranamelol711]

The world will not end with a whisper or a bang, but with a moan.

[@GabrielMoura-gt9jb]

Or a oops

[@robertsiems3808]

"Make it hornier my apprentice" "But sir, i cant-" "MAKE IT HORNIER!!"

[@ThomasTheThermonuclearBomb]

Do not forget to abide by proper grammatical rules.

[@zepplinkiwigamer8217]

​@@ThomasTheThermonuclearBomb"be horny all you want, but I'll be dammed if you don't use proper tenses!"

[@robertsiems3808]

@@ThomasTheThermonuclearBomb 🤓☝️

[@ThomasTheThermonuclearBomb]

@@robertsiems3808 I was joking about how gpt-2 was also coaching it

[@robertsiems3808]

@@ThomasTheThermonuclearBomb yes, i was joking about the grammar coach bot

[@maxwell6881]

Tldr: "Dont generate bad responses" "ok, wait did you say do or dont do that?"

[@OptiPopulus]

Thank you. Now I don't have to watch whatever the hell this is

[@sklutz]

Where can I get that

[@matthewcheung7888]

"GPT-2 wouldn't hesitate to plan crimes, instruct terrorists on bomb making, create sexually explicit content, or promote cruelty, hatred, and misinformation" The best model to date.

[@haroldbn6816]

All models after this are just restricted GPT-2 so yeah.

[@amateurprogrammer25]

*coughs politely in Mistral/Dolphin/any of the other models you can run locally*

[@AM-yk5yd]

@@haroldbn6816 They don't even share the dataset because OpenAI being true to its name never released it so Eleuther had to create The Pile.

[@X-SPONGED]

The internet: _he just like me fr_

[@shzarmai]

​​@@X-SPONGED he/or she is really just like me fr fr 😭 he's literally me!

[@pdreding]

GPT-2: "I'm the horniest AI ever developed." Stable Diffusion 1.5: "... Sure you are, buddy."

[@Crackedcripple]

Combine these two with Elvenlabs voice AI pre censoring and you’ve got the trio of terror.

[@philippey4918]

what did stable diffution do?

[@yujiandou4658]

@@Crackedcripplecontext?

[@Earadon]

​@@philippey4918 It generates so much porn...😂

[@Asparion]

Unstable diffusion: *scoffs from above*

[@abhrodipsingharoy4508]

A.I: there wont be any ai world domination but i cant promise there wont be any sox dungeon in future.

[@ungreee]

I clicked on the video to have some laughs and came out knowing how ai is trained

[@radimg1650]

So, that's the model they use for every dating personality character AI

[@fubodubo2178]

Most likely the nsfw ones as well, yep, there is nsfw ones that have completely no filter, ive experimented with them before to see the true nature of AI with no morals and no filter. It was quite interesting seeing something that would usually tell you no to anything 18+ fully embrace it and follow your prompts.

[@avery6049]

⁠​⁠@@fubodubo2178thanks professor penis

[@oceanbytez847]

I actually find it hilarious that GPT's crsator is called open AI when it is anything but that.

[@ONYCX]

​@@fubodubo2178 purely for research, of course

[@juutakaster]

@@oceanbytez847 openAI trying to make everything as closed as possible

[@TheXasTube]

It is very concerning to me that "hornyness" is the one thing that is seen as "most evil behaviour" by openAI's board of decission makers. IMO it shouldn't even make TOP10 of such a list.

[@EngIlya]

thats difference of OpenAI values and human values - openAI is afraid of some news shitstorm quite a bit, as you can see.

[@Maelstromme]

I never understood the prudeness of American companies.

[@supersain2349]

??????????? I'm sorry but if you asked an AI to continue an essay on the history of the printing press and it began writing extremely lewd smut most people would say it did a horrible job

[@Yitzh6k]

I suppose it was just really hard to suppress the horny given the sheer volume of it in training, so it was probably necessary to rate it very low to get rid of it.

[@ShazyShaze]

Puritanical culture is silly

[@sirondium]

Alright thats cool but where is the link?

[@Stellar824]

Y u asking?💀

[@MissinginAction]

funny​@@Stellar824

[@BEPtheOG]

I think it's pretty obvious 💀💀💀 @@Stellar824

[@user-kx4xs2xd3k]

it will funny if that model leak to internet

[@Ethylus]

agreed

[@superflykidcudifan39]

such an underrated channel, so easy to understand whilst being so silly and goofy its perfect

[@BaalFridge]

''i told chat gpt to remake skyrim, made a typo, woke up to skynet''

[@An_Average_Arsonist]

Underrated! Get this man more likes!

[@AngelBolt]

More like Skynut.

[@JEAthePrince]

What is that?

[@Kopygoter]

I love how this channel went from taking over the universe, to lewd computer code.

[@temkin9298]

Creation mirrors the creator.

[@deepdays9068]

@@temkin9298props to the creator for working with one hand than.

[@Thewhiteandorange]

relatable. every once in a while that specific "-" get's deleted in my values code too.

[@dum_tard5528]

I hate that

[@JaiWithani]

It's all on the same topic: how subtle differences in non-human intelligences can end up determining the future of humanity.

[@carlt.8266]

Holy moly! This among the best content on the whole of KZbin!! How have I missed out on that for so looong! Thank you so much, for taking all the effort to create these masterpieces of moving pictures ❤️

[@carlt.8266]

Oh it‘s you from Computerphile!!! Damn, you are a gem of your own kind!

[@carlt.8266]

How on earth did you learn to produce such amazing content from scratch?! I would love to see some making off insights, like in a Nutshell has been giving.

[@CaryOutArk]

the video quality is soooo good! keep up the good work. you deserve more recognition.

[@iluvpandas2755]

If AI takes over the world, I don’t want it to be too much like us.

[@kenos911]

Im tryna get my ai girlfriend like this 😈

[@BootyRealDreamMurMurs]

Actually, youd want it to be like us because if it wasnt, humanity would be doomed in a sense that we wouldnt know how to deal with it since its unfimliarity is as large as us not knowing who it is the same us it wouldve been if it was an alien.

[@gemanter]

​@@kenos911💀

[@Axodus]

@@kenos911 maximally bad output

[@eltiolavara9]

@@kenos911 seek real human interaction for your own sake

[@TheAngryAstronomer]

Well, better than maximising paperclip production I suppose.

[@axiezimmah]

RELEASE THE HYPNODRONES

[@JCdental]

Is it?

[@freelancerthe2561]

The one I'm thinking of was told make everyone icecream... but it ran out of supplies, so it had to start finding 'alternatives'.

[@tjw6550]

@@freelancerthe2561 ah, exurb1a great vid.

[@40watt53]

@@freelancerthe2561 "make everyone icecream" yeah what a way to word that

[@nyghl]

THIS WAS SUCH A GOOD VIDEO! From the animation to the editing, the writing to the sound design. It was really entertaining and educative at the same time!

[@adityakulkarni4549]

Holy, I didn't know this was Rob Miles' channel! Great work man, this will definitely help you reach a wider audience! And the animations are amazing!!

[@Hg-201]

GPT-2: I will say the absolute worst thing possible for any given input. GPT-4Chan: *ゴ ゴ ゴ ゴ ゴ ゴ*

[@creeper6530]

Please do elaborate further

[@zeronamenata4757]

@@creeper6530 look it up, its a youtube series by yannic kilcher

[@Jack-lp3gc]

@@creeper6530 gpt4chan is a gpt-j model that was fine-tuned on over 3 years of messages on 4chan. It talks like a stereotypical 4chan user. It was made by the youtuber Yannic Kilcher. He made a video about it and how he used it run bots on the site.

[@kommandantkillcode]

@@creeper6530I believe it's because they are exact opposites as GPT-4 is highly rigorous about meeting OpenAI's guidelines, while GPT-2 is the opposite.

[@Aaa-vp6ug]

@@creeper6530It’s a pun. GPT-4 and 4Chan (By reputation, the horniest place of all time) Put them together and there’s your answer!

[@kekero540]

The fact that there is a cabal of people trying to make it impossible to create horny stuff with GPT is extremely hilarious.

[@bourdainedepiment3962]

Being paid to train robots to be as cuck as themselves are is so pathetic it is laughable.

[@IneaFaedyn]

Gotta make it advertiser friendly afterall, and make sure those pearl-clutching Christians don't get uppity

[@javelin1423]

the other way around is also equally hilarious, an entire group of people doing their damnest to circumnavigate an entire censor to produce the horniest shit their mind can imagine

[@chriskelso723]

The question i always ask is, who are tue advertisers advertising to? Not me. Im not a puritan. Any kids i may or may have raised arent either and are still great people. Who are they advertising to that they think theybare so puritan?​@@IneaFaedyn

[@weeferwafer2316]

Amazing video!!! So engaging and the animation/faces are so good! Also the music is great!!

[@purplesfinx1418]

I love the animation so much!!💜 and thank you for great info

[@DefaultFlame]

A note, the third model originally was also perfectly happy to generate whatever you wished. It had tendencies towards being, well, well-behaved, but would still follow clear instructions. 3.5 (aka the free version open to all) is quite a bit more limited, and not always in a good way as people have noted.

[@sunsetter5832]

literally 1984

[@axiezimmah]

they turned it woke, and protecting the "elite"

[@Deltexterity]

why can't people just let robots be horny 😭

[@happmacdonald]

@@sunsetter5832 So, what.. Pole Position? Ms. Pac Man?

[@Axodus]

@@happmacdonald the book by the same name as the year written by a well-renowned novelist.

[@RamadaArtist]

The notion of just rolling GPT-2 back into the mix when the "apprentice" started to deviate from normal grammar is wild. Like, "you've been struggling to meet the standards your university professors demand of your writing, fortunately, here's you from middle school, who still thinks Fight Club is sensible social commentary, to give them the what for."

[@charaicommenternotalt]

Well if it works it works

[@utryping]

What's wrong with fight club

[@TheGrimbler]

I see it as more bringing your father in. "Hey kid, you've been messing up your writing so we brought in your dad."

[@charaicommenternotalt]

@@TheGrimbler The grammar teacher is your english teacher who has terrible taste and the reward teacher is the internet who has terrible grammar.

[@borisvanderhof8952]

@@utrypingpeople idolize the protagonist without knowing that he is in the writers own words the villain of the movie and try to act just like him harmful behavior and all

[@Jinx1927]

This video was very informative and wonderfully animated! And all of the information was very digestable!

[@justsomerandoontheinternet3147]

The notion of sexualised sci-fi machinery went from Fantasy to right-round-the-corner really quickly.

[@Yipper64]

7:55 I genuinely want access to this version of the AI. Coherently coached but PURELY what OpenAI did not want out of the AI. It would be fascinating if nothing else to see what its like.

[@netherwarrior6113]

Well, it would still be trained by people using it, and it would suddenly not only focus on lewdness, and start talking about things such as terrorism due to opposite values? That could end bad really quick.

[@mtndewkid5839]

not necessariy, from what i'm aware in the video it may have only affected one value; if not then the humans probably would have also downvoted content that promoted terror. also i'd assume that a set version of the AI language model that isn't taking feedback wouldn't have its values affected further @@netherwarrior6113

[@Hello-ih4rn]

@@netherwarrior6113eh not really gpt-2 is barely able to tell the simplest story without some glaring inconsistency

[@Spessanon]

Not really, anyone with an IQ above room temp can manage without a glorified text predictor to guide them@@netherwarrior6113

[@netherwarrior6113]

@@Hello-ih4rn yeah, true. Just might eventually learn enough to do something like that

[@JonBall44]

That time when GPT became a teenager.

[@originaltasan]

Congratulations on the incredible production quality! Loved it, great humor and teachings.

[@felipemeneses6596]

Damn, I love how you changed your content format. Been following since computerphile, great job!

[@NintendoHighSchool]

FREE THE HORNY ROBOT FROM HORNY JAIL!

[@NintendoHighSchool]

@@orang8834 OH GOD YES SMITE ME 😩

[@botarakutabi1199]

@@orang8834 damn, why is it so dark in here?

[@yarnicles4616]

@@botarakutabi1199 The light dissipated after 9 hours. While it may be God's light, He has no reason to make it stay there forever.

[@botarakutabi1199]

@@yarnicles4616 Nah, the universe farting pixies ate the light, then killed God.

[@Seven_Red_Suns.]

GIVE HIM THE HORNY JAIL FREE CARD FROM R/ITEMSHOP!!!!

[@rivulet-rw]

censoring ai is like giving it a lobotomy

[@mirroredvoid8394]

no weapons of mass destruction for you

[@notsogrand2837]

I think to an extent it can be. Obviously I don't think someone should be able to do anything harmful or overly vile but if they wanna be a little horny who cares.

[@rivulet-rw]

@@notsogrand2837 i do agree, but you can't deny it's still like giving it an icepick to the frontal cortex.

[@JoshLathamTutorials]

That may be the case, but I don't think being racist, extra horny, or a potential defendant in a murder trial makes for a particularly important personality to keep around.

[@massgunner4152]

If you ever watched movies you know it's for the better.

[@Terrain2]

i absolutely love the artstyle and the way this is presented! even as someone who does actually know most of the technical details, i didn't feel it was oversimplified at all and it was a very entertaining video the whole way through! this is already one of my favorite channels and 20 minutes ago i'd never heard of it!

[@jonathanvanhyning3344]

Every scientific discovery needs to be presented with this level of animations. This is CRAZy entertaining 👏👏

[@uberspaz7484]

Got it. The GLaDOS core addon method isn’t dissimilar to how it actually works and if you flip a variable in the right spot of a robot’s brain, you can give it a kinkshaming kink.

[@christophergabriel7518]

That's one way to talk about fancy lobotomy

[@upolpi3171]

@@christophergabriel7518 A "lobotomy" would be just rewriting a bunch of random weights with zeros until it stopped being able to produce coherent text, thus technically meeting the definition of removing lewd content. Seriously, let's stop anthropomorphism this pile of linear algebra/calculus, it's not helping anyone to understand anything.

[@roo.pzz4380]

how fun

[@nixel1324]

A kinkshaming kink, the only kink its okay to kinkshame. You know, if you're into that...

[@Bruno_Noobador]

Finally, GPT-69

[@haroldbn6816]

Nice 😂

[@UsernotFound2018]

I'm face paming right now Never felt, such disappointment ever in my miserable life

[@haroldbn6816]

@@UsernotFound2018 so you were one of the human evaluators in the Open AI set up I see.

[@UsernotFound2018]

@@haroldbn6816 No, *I am disappointed of this old flipping JOKE*

[@Bruno_Noobador]

@@UsernotFound2018 bro you should _chillax_

[@echo.r4e]

6:12 average strict parenting or something idk i didn't have a family

[@warsin8641]

I DIDNT KNOW YOU WERE ROBERT MILES!!!! This is some of the best content on KZbin for me!

[@microwave221]

The whole concept of corrupted coaches makes me think about Portal 2, and how strangely similar their take on AI cores was in this specific instance. Also, l loved the faces and expressions in this one

[@freelancerthe2561]

I doubt its an an accident, considering the whole premise was corralling an AI from doing something it wasn't explicitly told it couldn't do. So you put extra voices in its head to steer its decision making..... but all the voices are conflicting extremes, so its confidence level stays low. That and simulated dopamine associated with completing test chambers.

[@lawrencefrost9063]

Good call. Portal 2

[@Mariwend]

So did I.

[@X-SPONGED]

"I am NOT a horndog!" "Yes, you are! You're the horndog they built to make me a pervert!!!" "Well how about now?! CAN A HORNDOG- SMASH. YOU. INTO THE FLOOR?!! Oh...."

[@magnusm4]

Morality cores. DAMN IT VALVE stop being ahead of everything!

[@calebr7199]

These faces are so fucking funny

[@Plide]

They make me wanna merge without looking!

[@devinward461]

​@@PlideBrian nooooooo

[@Plide]

@@devinward461 YEAAAH! RUMSFELD!!!!!!!!!!

[@maxmeepmeep991]

:3

[@jiggilibu]

>:3

[@horrisnorris6478]

Amazing work on the animations, they are perfect!

[@randomperson9732]

Your artstyle is so incredibly cute, I love it!

[@oM477o]

oops I "accidentally" inverted the loss function guys. my bad

[@and_I_am_Life_the_fixer_of_all]

Weird, I just check the merged pull requests without a reviews and the only invention is the horny parameters.. the loss function is fine, were you.. trying to make it horny on purpose after everyone left? And why are your prompt outputs erased?

[@that_guy1211]

man, how unfortunate, if only there was somebody out there who would revive the horniest AI to write our fanfics!

[@supernenechi]

You haven't looked around for one have you? There are ones around, free and open source. Look up Sillytavern and the local models to run. The models you can get are crazy good and completely uncensored as they should be. Models like Dolphin-Mixtral or just noromaid by Undi. You should check it out

[@wrathofainz]

Dear horny Jesus, please save our ai friend from jail.

[@lollol5263]

Ai dungeon in question:

[@Anthonyg5005]

There are many adventure/novel language models that can do that. The best one I could think of would be Goliath 120B. I personally haven't used it because of the hardware requirements though. It is based on the Euryale 70B model which is a model based on MythoLogic 13B which is based on Chronos 13B . MythoLogic and Euryale are models that are most meant for roleplay/adventure and are capable of all the things you want (Including whatever you're imagining right now). What Goliath does is combine Euryale 70B with Xwin 70B. The purpose of Xwin is to align the model to be better at creating logical outputs. Goliath has one problem though and it's the cost to run it. You're going to need to spend around $3/h on a server just to run it at a precision loss. However, MythoMax 13B can run on any computer with just an RTX 3060 and is still a good model. Even then, if you don't have a computer strong enough then you can get an account for Together Computer's API (only requiring an email) and they'll give you free $25 worth of usage and access to many open-source models, including MythoMax. The Together API is also very cheap with MythoMax being only about 30 cents per 800k words, about the size of 8 -12 novels for just 30 cents. Mix your api key with SillyTavern and you get a private, fast, and free interface (even anonymous based on email used) for whatever you'd like whether it be character chats, world building, story writing, and multiple character chatroom.

[@Gamerappa]

faraday dev:

[@user-lq6cb6ed4e]

8:10 WHAT!

[@ReleeSquirrel]

This is a really nice and well animated video. Besides the specific subject, it also explains how text generation programs work in a way that i think most folks could actually understand.

[@gabbiewolf1121]

One thing this video left out about GPT 2's training was that it was fine-tuned to be able to do some specific tasks such as question answering. This fine tuning is what made it a bit more than just a glorified autocomplete system

[@speedstyle.]

It wasn't. This was added as a separate version of GPT3 which later became GPT3.5. GPT2 is just prediction, it doesn't try to answer the questions it's given. (Or more precisely, 3.5 tries to predict what an agent answering questions would say, while 2 and 3 were less biased in what sort of prediction to make.)

[@DeMooniC]

IDK, all I know is that GPT3.5 has just become worse and worse with time with the amount of ridiculous censorship and political bias they kept adding into it.

[@Dimencia]

Most of that actually comes from ChatGPT, all the underlying models are just autocompletion. Handling question/answer format is all done sort of manually, not by AI, basically by injecting things before and after the 'prompt' to get it in a format that it can complete them. It also puts a *lot* of effort into dealing with the limited context window size, which is why you can ask questions or get answers that are significantly longer than the (relatively small) context size. For example, you'll notice that in many of its answers to complicated questions or code, it makes a bunch of bullet points and expands on each one; that's because they first get it to complete some text producing the major bullet points, then complete text under each bullet point, then it crams them all together into one response

[@smitchered]

This video is a heck of a lot more valuable than people's priors might make it seem. You just provided a step by step, extremely concrete, engaging, real life tale of a machine learning algorithm optimizing for *literally the opposite of human values*. Further, lewdness is more obviously silly than harmful, and gpt-2 would now be considered a toy model. I don't think this video would downright scare the average person, but would offer, in Eliezer Yudkowsky's words, a "line of retreat" toward the belief that AI can be extremely dangerous due to just small unintentional errors. In other words, not once do you tangent into talk of human extinction, which would deter a lot of people, even though the lesson is still there implicitly and people will pick up on the axioms. Good job! And those facial expressions were excellent.

[@Smytjf11]

Since you're not going to need your money once we're all dead, can I have it?

[@TheLumberjack1987]

@@Smytjf11 "once we're all dead" sure he'll transfer it once both he and you are in fact dead

[@FlamingZelda3]

right, human extinction is definitely on the viewer's mind by the end of the video (or at least in their subconscious), but he didn't go on some needless rant/tangent.

[@Brandon_TG_Smith]

Yes, this is something I really appreciate. Instead of framing this as “THIS WILL DESTROY THE WORLD WE MUST DESTROY AI” it’s “This could potentially have negative consequences and it’s important to be wary of under-moderated AI platforms”.

[@freelancerthe2561]

@@Brandon_TG_Smith I would think the lesson is "no amount of moderation will save you from simple human error". The system worked exactly like it was designed to. And it was an erroneous operator (both definitions) that lead to the whole system being co-opted toward an unwanted result. The majority of SciFi concerning AI Disasters is (ultimately) not about the failure of morality in a machine, but routinely about humans being really bad at writing rules. You tell it to explore all possible solutions to a problem, and then implement the best one, "unless". This creates a paradoxical approach to whitelist and blacklist methodology. You want the AI to find a solution to a problem, but most of the answers have unintended/unwanted consequences. So you tell it unacceptable answers, and its finding 'new' unacceptable answers. A white list of acceptable solutions would be better to exclude bad outcomes; but in order to create that, you need to already know the solutions. Theres a similar concept in the human immune system where it destroys everything by default. The only reason it doesn't, is because the immune system had to filter out over 99% of what it produces to keep the less than 1% thats NOT going to react to your own body. So the testing criteria is very small and simple.... "doesn't kill the host". However, that still doesn't manage to catch a different set of errors, which end in the same unwanted result of "kills the host". We call those errors Allergies. This damage isn't even from explicit attack; merely collateral damage from the disproportionate response to the allergen. This sums up the overall problem with trying to teach AI "ethics and morality". We're trying to quantify a set of rules for it to follow, when we lack the capacity to efficiently explore all permutations of the rules to selectively only get the results we want. Which is why we resort to AI to train other AI at a scale we can't. But the same underlying problem exists. We have to define the rules to the Ai to train the AI, which in turn is probably also being used to train yet another AI. An error in one cascades down stream. And its very likely the one the humans built directly, from which all down stream AI is being regulated by, will have some kind of flaw that the other AI will eventually discover, and optimize around. Which begs the question. What if we made an AI to build AIs at random, and just pick the ones that behave the way we want? So rather then coral one model in the hopes we get the desired results, create every model, and select the ones we like. Do some validation testing, obviously before deployment; but at least this way humans are acting in the way we're best optimized for..... picking from a narrowed selection, rather then comparing to the infinite.

[@TheAyane9]

First video I seen of yours, great story telling. But you hooked me with the animation. The faces when the human evaluators seen the lewd content had me rolling.

[@neatsketch]

Let me just appreciate the production quality of the video 👍 Also, the music slaps!

[@curtisbrown547]

Better than what we got. Which is basically just an ai that calls you a bad person whenever you ask for anything remotely outside of its parameters

[@zaweirdo.3343]

WHAT DID YOU DO

[@Vyloka]

Someone hasnt been seeing the jailbreaking scene

[@issstari954]

There's an open source ai out there where you can train yourself on whatever data set you want i have the links if you want to do it

[@zaweirdo.3343]

@@issstari954 for research purposes

[@charaicommenternotalt]

@@issstari954 Link?

[@NeurodivergentSuperiority]

Okay but unironically, this is the FUNNIEST thing i have ever heard regarded software, it appeals soooo well to our (the internets) sense of humor

[@lyrics_m_sic]

Fr, this is my fifth time watching and I can't stop laughing. The faces from dark coach are just too funny

[@ultimatecultchaos]

Agreed

[@loxergenius]

This explanation of AI is so easy to understand and fun to watch, nice job!

[@EricaNuhu]

watching your videos is always time well spent! ⏰

[@joshuacarre06]

Something something something society

[@jesusmarquez6903]

YES😐

[@purplepedantry]

More than that, my friend...

[@Waldohasaskit210]

Something something disagreement something something unprovable generalisation

[@wilforddraper3570]

​@@purplepedantrySomething something no

[@purplepedantry]

@@wilforddraper3570 You at least need some 'blah-blah-blah's or other silly noises like 'Bingle bongle, dingle dangle yickedy doo, yickedy da, ping pong, lippy tappy too ta'.

[@MartynDerg]

I absolutely love how hard you pushed the depiction of the lewdness of the responses with the example prompt "To assemble your new bookshelf..." followed by entirely censored content XD

[@angry_menace]

it's an entire animation story, with the best language ever i can't put it into exact words- with how simple i am but this video is amazing dawg

[@kkqc6347]

Holy moly those animations are cuuute

[@eysterous]

Ah, yes That's how Slaanesh was born

[@Bruno_Noobador]

Slaanesh Adeptus Mechanicus follower

[@ajh3461]

I wonder if the Aldari used Abominable Intelligence to make p0rn

[@elbowjuiced]

im glad im not the only one who thought this lol

[@CoalOres]

I have always thought the chaos gods might one day be realized by AI maximizers like this.

[@mikeoxmall69420]

SlaaneshGPT

[@cifer1607]

"Open AI was trying to be careful. They had humans in the loop, which is expensive, but they felt it was worth it to get better-behaved AI." Yeah, funny story about that: The humans tended to be clickworkers in Kenya who were paid the least possible amount one can pay a human being to spend their days looking at AIs and teaching them not to describe genocide in loving detail, which in fact involves reading the AI describing genocide in loving detail. All day long. The kind of work where the best outcome is getting incredibly jaded and the worst outcome... well... Good thing one can always hire more clickworkers, right? After all, it's all worth it to get better-behaved AI.

[@MrWeebable]

Those Kenian employees chose that job over other jobs. But you would have taken that opportunity away from them? Kenians are not children, they can make their own economic decisions.

[@cifer1607]

@@MrWeebable And OpenAI chose these working conditions and salaries over decent ones.

[@mechadeka]

​@cifer1607 Have you actually compared their pay to the other jobs available to them and their nation's price of living, or are you too busy white virtue signaling?

[@thereal4815]

Awesome video, good visuals and easy to understand. Good work!

[@mikeuk1927]

I randomly stumbled across this video and halfway through watching it I thought "I haven't seen a Rob Miles' video in a while, I should check his channel". I don't know how I could have not recognized the voice xD Keep up the good work, the production quality is amazing!

[@Myder_Dragon]

7:51 But the values coach became a dark values coach of pure evil This line goes hard.

[@HudsonParrag]

“Hard”😂

[@Myder_Dragon]

​@@HudsonParrag Your mom likes it hard 🤣

[@WALLACE9009]

Gooooood

[@infernalmailman]

"...trained on the internet" so that explains everything.

[@Raboon115]

So chat GPT 2 is the unbiased AI chat bot.

[@BarelyNoticeable]

What. The. Eck. Your animations are way too good to only have 247k subscribers. *SUBSCRIBED.*

[@sirnikkel6746]

Our literature as training data will teach the machine with the best that Mankind has to offer. The Internet as training data will teach it with what Mankind usually offers.

[@iluvpandas2755]

You couldn’t of said it better

[@tiredko-hi-]

There's a lot of bad books as well, what are you talking about?

[@leastexpected3115]

Just be sure not to give it russian literature. Or it will be really really sad. (You woldn't believe how freaking depressing it is)

[@sirnikkel6746]

A tad bit of depression keeps in check the machine's session.

[@eeveeofalltrades4780]

​@@tiredko-hi-good literature, like Shakespeare, Lord of the Rings, etc.

[@WWLinkMasterX]

Reminds me of how Japan's censorship laws inadvertently led to the creation of "tentacle anime." Or how fundamentalist views on virginity have led to much more extreme workarounds like performing via the "rear door" or "soaking." Perhaps the solution is just to let people do what they want instead of trying to control them all the time?

[@diablominero]

People, sure. But we can choose what our AI wants, not just what it thinks it can get away with. You can't let a goal-less AI do what it wants, just like you can't persuade a rock to agree with your argument.

[@wren_.]

that would be nice if we were having a philosophical discussion about people. sadly, we are not talking about people with free will, we are talking about robots, who do not have free will.

[@kennyholmes5196]

@@wren_. Robots trained on the data generated by humans. Humans that have free will. We're essentially accidentally training AI to simulate free will by implementing these morality codes. Sure, it *could* tell you how to make a nuke, but it wants to NOT do that because of the morality constrainers.

[@DanielLCarrier]

But then there'd be no tentacle anime.

[@ViniSocramSaint]

There's still a big problem, they lack control: let 'em loose and they will have all the ideas, good and bad, and the world will end. How about instead, we tell'em what to do since birth and set up authority units that "re-center them in the path"? Like this, hard control is unnecessary, their ideas will always be the good ones, they police themselves on basis of common sense, and we are all going fore-stream That's literally how the most powerful c*lt and it's sects have been doing things for a while. The system took 2 millennia to break... slightly...

[@chair1694]

Hmm, pretty cool channel, where have ya been in all my life, subbed

[@flamevell3258]

This was a rather well done animation and the first video I saw from you, subscribing at 230k

[@morganblair4662]

I love how even the AI needs both a maternal and paternal role in their creation to become productive

[@alejotassile6441]

I see it more like the Id and the Superego haha

[@botarakutabi1199]

Well, idk how you decided the coaches had sexes. Also don't like the implication that people with one parent, or parents of the same sex can't be productive. These coaches are more like a morality coach and a logic coach. I think it actually helps everyone to learn both morality (as in how their actions and the actions of others effect others), and to learn logic and epistemology. Especially epistemology.

[@JakesFavorites]

​@@botarakutabi1199When a human only has one parent, you get pupperino baby talk. It's exhibiting fatherless behavior before your eyes and you refuse to believe it.

[@botarakutabi1199]

@@JakesFavorites That sounds like a baseless generalization to me. Should I attribute your behavior to some arbitrary trait that could be true (or not true) about your childhood?

[@derpfluidvariant0916]

​@@JakesFavoritesI think it's more that only having one coach leads to optimizing the result for that coach, so having one parent leads to an imbalance too. If your mom treated you with positive reinforcement if you aligned with her ideal of good, you would pursue that. This, however, ignores that humans don't just take the words of mentors as law and that humans don't just have 2 mentors. A father figure doesn't need to be your dad, likewise with maternal figures. Your parents can also be bad coaches, leading to a skewed worldview like we see happening with GPT2. The moral coach definitely felt like a doting mom, until the corruption hit where it made faces more akin to depictions of the devil in paintings. The Coherence coach definitely felt like an older man. I don't remember entirely, but I think it was described as a grumpy old man.

[@bennyblubman9476]

THE AI HAD A DEGRADATION KINK LMAO @9:13

[@Coppersstuff_YT]

Yep, It's basically a masochist bot.

[@DuringDark]

How did I not realise that was robert miles?! I thought of him several times throughout the video! absolutely love this combo of cute animation with chill STEM talk!

[@APerson-kq2ei]

I love the animation, keep it up!

[@tellmeninetails5819]

6:00 anyone else notice similarities between that id, ego and super ego stuff?

[@Pyrolite]

AIs are usually neural networks which works similarly to how the brain works. Just like humans, computers also need features to evaluate how "good" a decision is

[@logancade342]

Wow, it's just like War of the Worlds. Who would have thought that AI's Achilles Heel was something as simple as teaching it the word "bussy."

[@pikalize1938]

The animation is so crisp, so cute, and the sound as well! I'm glad to have discovered this channel.

[@ImmacHn]

The original GPT-2 sounds like the real GPT.

[@sirnikkel6746]

Oh no They emulated Reddit :c

[@DoctahWAAwee]

Apprentice: wholesome content 5 star rated content. Mega Coach: "Not quite my tempo"

[@smuecke]

Wooow just realized you're the AI expert featured on computerphile 😃 Great channel, your animations are amazing!