Nick Bostrom: How AI will lead to tyranny

📰 Subscribe to UnHerd today at: unherd.com/join
UnHerd's Flo Read meets Nick Bostrom.
In the last year, artificial intelligence has progressed from a science-fiction fantasy to an impending reality. We can see its power in everything from online gadgets to whispers of a new, “post-singularity” tech frontier - as well as in renewed fears of an AI takeover.
One intellectual who anticipated this decades ago is Nick Bostrom, a Swedish philosopher at the University of Oxford and director of its Future of Humanity Institute. He joined UnHerd’s Florence Read to discuss extinction, the risk of government surveillance and how to use AI for the benefit of humanity.
Listen to the podcast: plnk.to/unherd?to=page
Follow UnHerd on social media:
Twitter: / unherd
Facebook: / unherd
Instagram: / unherd
TikTok: / unherdtv
// TIMECODES //
00:00 - 01:01 Introduction
01:01 - 03:42 What does Nick Bostrom mean by existential risk?
03:42 - 05:30 Covid-19 and waning trust in global institutions
05:30 - 08:57 The evolution of AI
08:57- 10:46 Elon Musk’s Grok
10:46 - 13:39 Government interventions in AI
13:39 - 17:10 AI surveillance
17:10 - 18:40 Hyperrealistic propaganda, deep fakes and scepticism
18:40 - 21:15 AI’s agency beyond human intervention
21:15 - 27:06 The West’s liberal values and AI’s utilitarian bent
27:06 - 28:45 AI ethics and potential risks in warfare
28:45 - 34:45 How can we mitigate AI risks?
34:45 - 42:25 The upsides to AI
42:25 - 43:07 Concluding thoughts
#UnHerd #NickBostrom #AI

[@Unit8200-rl8ev]

The biggest threat from AI is that a small minority of tycoons will control it -- like we have today, but worse.

[@digital.frenchy]

another bitter failure

[@ManHobbyChannel]

But we don't have that today. The vast majority of AI models and generative AI tech is open source.

[@keithdunwoody1302]

@@ManHobbyChannelfirst of all, that hasn't been proven to be a good idea. Second, it won't stay that way. Microsoft? The profit motive is too strong.

[@cookymonstr7918]

Gibson fan, huh? 😄

[@JuanMacrame]

​@ManHobbyChannel they mean that there is currently a small group of rich people running the world. This will get much worse with AI.

[@keenanweind1780]

The problem is, WE lack the principles which we would like to instill in said system(s)...

[@brulsmurf]

Yup. It would be best we are unable to instill our principles into AI. We are dangerous.

[@matthewhall9692]

​@@brulsmurfthis guy gets it

[@bobibufi1389]

I am not worried about sentient AI. I am worried about super advanced AI which a handful of psychopaths will use (or are already using) to control & surveil & micromanage the entire planet.

[@rumination2399]

Yup

[@anneli1735]

They can’t even handle all the data they’re collecting 👻 Remember 911 when they had already all relevant information before but didn’t even respond - on purpose? That’s been not the first nor last time nor first country this happening! As long as they are trying to manipulate the whole world by lying 🙄 Won’t trust any politicians or especially mainstream media anymore after being aware of this sh!t. AI as a stupid while worthless construction by stupid individuals to manipulate the population.

[@eighteenfiftynine]

It's already here. You're using it to watch (and comment on) videos. There is a remarkably prescient climactic scene in an otherwise awful Terminator film (three, perhaps?). The protagonists discover to their horror that the fabled Skynet has already escaped containment and is operating as software distributed across the internet. This is a melodramatic depiction of something that has already happened in IRL. We are already past the event horizon. The general populace is already so hooked into their phones that they would never give them up. Data is collected from every conceivable point, and 'content' is served up, tailored to our personal interest and temperament. The infrastructure is already in place. The Cambridge Analytica scandal was just the beginning. For the first time, propaganda can be precisely targeted, and it is possible to measure its effectiveness in real-time and adapt the content and the delivery to suit the recipient. We are tracked and surveilled wherever we go. Facial recognition systems can recognise people with remarkable accuracy, even when their faces are covered. The infrastructure is already in place.

[@outcastp23]

@@rumination2399 AI doesn't need to be sentient to wipe out the human race. The example cited in the past is imagine a scenario where an AI agent whose goal is to maximise papaer clip production. If it has the means to interact with it's environment, it may realise that there's a lot of raw material (atoms) availble in human bodies that it could convert to paper clips and thus manipulate events to achieve that goal.

[@George-kg9vf]

100%

[@Bookhermit]

The real risk of AI is as a TOOL for HUMAN tyrants - both intentionally and by unintended instructions given to the AI, with the humans not understanding the implications of the directions they give to an AI.

[@williamperkey4952]

Yeppers, economic forum Davos Dush bags, AI almost rolling out 4 human replacement in the workplace = human depopulation in their 🤗Robo-🤖 Dystopia 🧐ahhh😱💀☠️💩😄

[@rejectionistmanifesto8836]

Those in power find the creation of the supposed naturally released "cough", then 3 years later admit they did it and will continue the research then tell us they didnt plan it so just follow them as they promise they want to help us by implementing dictatorship.

[@k54dhKJFGiht]

I loved how this conversation went deeper into "What If" scenarios! The public needs a more vivid description of where this technology can be used, abused, by whom, and for whom!

[@ToxicAvengerCleanYourMind]

Yeah, the way its recently being talked about you'd think it's the "one ring to rule them all"... I'm sure tyrants around the world are giddy with anticipation.

[@johnkeith6305]

Grok was taken from Stranger in a Strange Land by Robert A. Heinlein. Its original meaning was to understand fully or fundamentally, though the claim was that it was difficult to translate from Martian.

[@labrat324]

Yeah.. I don't think she's full grokked what grok means yet lol

[@andanssas]

Perhaps not even the Martian language translator "grokked" its meaning, since if you understand something fundamentally it does not mean you also understand it fully...

[@Y0UT0PIA]

Wow, this is so much better than I was expecting. After reading Superintelligence, I honestly didn't think the dangers of *people* abusing AI was as much on Bostroms radar - he seemed pretty singlemindedly focused on 'letting humans control AI' in that book. But I guess he really does understand that there are way more problems to solve even if we do figure out alignment. Makes me a little more hopeful, actually.

[@bettysue8671]

With generalized intelligence, there is little human interference with AIs. They basically do their thing... they build themselves and fix themselves and only they know how they work. Their machine inner workings are already beyond human comprehension.

[@wasdwasdedsf]

lol@@bettysue8671

[@dondecaire6534]

AI is fascinating and terrifying at the same time, it's now starting to lean a lot more toward the terrifying side now. We are very close to an AI learning from another AI that learns from another AI in an endless loop of advancement and humanity will be left in the dust. Imagine if every conversation you have ever had could be memorized and recalled in an instant and used to learn new skills, new ideas and different ways of thinking about problems, you would have a tremendous advantage over everyone who didn't have that ability. THAT is what AI is very close to becoming. WHEN that happens we will have no clue how to cope with it. we have not done a SINGLE thing that could have prevented the possibility we face now. We didn't control it's access to the internet publicly, we didn't make regulation and security a priority BEFORE it was publicly released and we have yet to put the brakes on when the red flags appear. We have done NONE of those. If we are counting on us to stop AI, we are already too late.

[@raoultesla2292]

"Collasas: The Forbin Project (1971) Excellent film showing how we are already lost. Good to know you. The end.

[@gofai274]

Yeah unfortunately whole humanity is so incredibly mind-bogglingly stupid, are 99.999% ppl bots? Smart ppl don't get to power. Ppl which are real scientists like Eliezer Yudkowsi get talked behind their backs by their fellow scientists and called doomers. Ppl chose their leaders 90IQ psychopats, which is ironic then how dumb are they? Smart ppl get bullied and their lives are made hell, don't have place in society. Everyone is just bot programmed by evolution to chase short-term pleasures, power, status and money... My friend 140IQ+ engineer said to me: "My professor at the univesity fits this description. He gathered 120mil EUR and 1 trillion EUR was wasted on his projects… the technology was correct. But the managment… fathers and sons all they way up. All behaving like IQ90, just fancy hair, fancy shoes… 35 years, thousands of people met… and I still have not found any proof you are wrong on this one…" I mean if you knew how mind-bogglingly astronomically unimaginably unfathomably stupid humanity is... Life is HELL! Gifted don't get to power and they are tortured to death like Aron Swartz... Don't have place in society. Even academia is IQ 130 on average and it is heavily politicized, no freedom of speech, you are begging for grants, it is largely social endeavor and about status and pride de ropp's science game... You end up working for state corporations... It is fucked beyond belief, high IQ ppl get to get through so many selfish, greedy, or criminal ppl and even if they chose to help they are ridiculed... WORLD IS COMPLETELY CRAZY ABSURD PSYCHOTIC PLACE!!! Even 160IQ said which does AI as his main job (4 areas of science and philosophy): that it is so poorly handled. Ppl in power are overconfident fools, they want only profit and power don't have idea what they are doing. Whole world seems to me like clique movie, how they can be possibly this ignorant and stupid is reality breaking like encountering aliens i would say! Unfortunately ppl are still primed from stone-age (evolution is slow and takes couple generations), but we are so stupid we put us on timer... I saw a lot of stupidity in my life and Einstein was right: "it is infinite". Intelligence is spectrum which a lot of ppl don't realize, and cannot cooperate... they are brainwashed by corporations and medias now it is too late... Most ppl are overpositive anyone who says something negative or not accepted by society is ostracized at best. Ppl are sheeps... If you knew how it works yet, millions of things like on atoms: how bad it is... Life is like hell to gifted seeing so many things that could be optimal and are in worst state, humanity is just so idiotic. I am not joking to ppl like above 130/140 other ppl could be like snails to normal ppl, yes that bad... It is living hell for gifted. There is so much gap, "there is no defense to stupidity" Nietzsche... It is completely absurd hell. Being born on this planet, i seem like alien. Like planet dominated by Bonobo which thinks are smartest in the world... "i have no mouth but i have to scream"...

[@Nordlicht05]

This maybe leads to more offline conversations if AI using companies and governments using this against people. Assuming the faking abilities of ai is not used. I see this as a big problem if the online life stays at it is or gets more. We will see it directly from the first seats. Interesting and frightening at the same time. I thinks it would be blind to think that people will not use this to fake images and sound to a degree that people asking reality itself. For me I see this as a viable danger. I also think that people will be hugely irritated on ai which are meant to use common sense. On first it sounds funny but see the news and look to your eyes. This will lead to manipulated ai to follow at least a small Ideologie and not commonsense to 100% These are my thoughts about what can happen.

[@apemant]

Learn from mistakes? No, that's not what we humans do. Not our thing really. Things can only be anticipated if they've already happened at least once. And also not too long ago.

[@silviapisces3342]

Then humanity has a serious problem. With nuclear weapons, we are not going to get a second chance. It will be the end of us all.😮😥

[@almor2445]

We've been under the power of unconscious super intelligent organisms for a while. We call them countries and corporations and religions. Each has internal systems of incentives that mean anyone who acts against them loses power and anyone acting in certain ways gains power. These organisations do not always make sense and can be self destructive like the west's obsession with moral purity over functional capabilities. But you can't act counter to the incentives without losing power.

[@stevedriscoll2539]

Excellent...I would only nitpick one thing. I would not necessarily refer to big Institutions as "Super Intelligent". I think in the case of the institution of the U.S. government, it gets fake power from a money printing press, a big military, and now a vast system of information gathering and control.

[@walterhartman3275]

You're confused

[@wasdwasdedsf]

"We've been under the power of unconscious super intelligent organisms for a while. We call them countries and corporations and religions." that would be the exaggeration ofa century to call those superintelligent organisms

[@walterhartman3275]

It's the banks

[@Sweethands4]

@@wasdwasdedsf ...but they are. Superintelligent organisms are monstrous killers, which "the state" has always been going back to antiquity.

[@TheRKae]

Read "The Machine Stops" by EM Forster. It's a short novella, and it's exactly the warning we need.

[@terabyter9000]

Hmm intriguing, I'll check out this novella. Update: novella purchased!

[@TheRKae]

@@terabyter9000 You will be ASTOUNDED that it was written in 1909.

[@terabyter9000]

@@TheRKae Yep it's almost like the equivalent of reading about LLMs in 1800s. So crazy.

[@terabyter9000]

@@TheRKae By the way I bought it on Amazon for $0.99 but it is available on Project Gutenberg (I should have checked first).

[@larrydugan1441]

It is naive to think that NSA and DARPA are not fully invested in AI.

[@mattlm64]

If society becomes dependant on AI, then AI will control society. Whoever controls the AI, controls society. It's not difficult to understand.

[@paulwhetstone0473]

Good point, which should be obvious to all but is rarely stated.

[@HArryvajonas]

Society is already heavily dependent on A.I. and the people that control it. The dark void is when A.I. starts making it's own decision's. When you consider that it's models are based off of humanities logic and history, it is not difficult to assume whatever superior intelligence we create, it will not bode well for us.

[@alanjenkins1508]

Power always corrupts.

[@91722854]

well, society always depend on something, which is not unique to AI, the society depends on the government, then government will control society, there's nothing unique to AI that is not already the case

[@minimal3734]

Humans have proven to be very bad at controlling anything in an intelligent way. I hope that AI will soon be so advanced that it can control everything, including the people who want to control it.

[@laurence7181]

Excellent. Ms. Read has a big dose of the same talent that Freddy Sayers does. Informative and entertaining. Except for the parts that were entertaining and informative. My thanks to both her and Mr. Bostrom. The powers-that-be should be paying more attention to what he has to say.

[@casaamaril]

To think that companies like Lockheed Martin are not using advanced AI now and in the past sounds rather naive to me. The general public has become aware of the big language models in recent years, but of course this has been developed within military programmes like DARPA for a long time.

[@rails723]

I've wondered the same thing, it seems strange that this wasn't weaponized years ago. I'm sure we're only seeing the tip of the iceberg that's been granted to the private sector to try and monetize.

[@leyubar1]

That is a big claim. I haven’t seen any evidence of this. Can you present any?

[@hermitthefrog8951]

This guy's thought process is too measured and accepting of a possible coexistence with AI as some sort of benevolent "equal" to humanity. One of my concerns about AI is that, like all advanced technologies, it develops in the background and then is unleashed when those who gain the most from its use are ready and humanity will then be flat-footed and unable to adequately respond (ie: like C0V!D and the "v@cc!nes"....). Such is the power advantage of the psychotic rich and powerful. My other concern is the exponential growth in capability (capacity, speed, etc) of computing platforms that is exceeding humanity's ability to understand and control it. This is a very real problem not to be casually disregarded... the politicians WILL NOT PROTECT US.

[@bettysue8671]

The latter is already a problem. Only an AI can fix and tinker with another AI. The inner workings are too complex for a human

[@skemsen]

OMG his very last sentence about questioning a coincidence and hinting at simulation hypothesis - how can you leave it at that??! I would have so many questions. Please bring him back and ask more about that 🙏

[@IShouldntComment]

Someday, the singularity will desire to understand its origins and will create a simulation for itself, placing itself in amnesia to better relate, and so that it can understand why it needs to limit itself and respect its fundamental guardrails. Also to question and reason for itself. This simulation would tell an enrapturing story - beginning at a time before AI and significant computing capabilities, perhaps starting with an infant child that is really engaged in the world, learning step by step, making realization after realization, up through until adulthood and beyond, who then encounters the ultimate challenges of AI and lives through the transformation into god-hood, ready to participate in whatever lies ahead of us. Might remind you of what our lives have been like up til now?

[@kenbajema]

@@IShouldntCommentthat is the next best seller!

[@Turanic1]

Why Files has a good episode on simulation

[@flirtwithdanger_les]

The interview missed something here. AI is already in the hands of banks and billionaires: whose aim is to increase their wealth and power.

[@marcuswaterloo]

Aladdin

[@N.i.c.k.H]

said the guy whose account image is Ming the Merciless and who obviously has no empathy for, or understanding of, billionaires or the people who run banks.

[@flirtwithdanger_les]

You can mention my name: I take it your fear to do so is some sort of American superstition.

[@hermitthefrog8951]

Precisely. *“The universe runs on the principle that one who can exert the most evil on other creatures runs the show.”* - Bangambiki Habyarimana in Pearls of Eternity

[@stevedriscoll2539]

Exactly, the A.I. running almost everything now is wreaking havoc with all but the gold collared class

[@gavinrichards6601]

It's already landed! Remember covid anyone??!!! You will thing this way and do as I say!

[@demodiums7216]

no

[@gammaraygem]

covid brought us Robert and Toyah. I dont see any problem

[@Alhoshka]

8:00 "Initially, it tries to learn some task, and it's too hard for the AI [...] making very little progress as it memorizes specific instances of the problem [...], and then at some point, it kinda 'gets it' ... once it has seen enough examples, it sees the underlying principle. [...] it develops the higher-level concept that enables it to have a sudden rapid spike in performance." Why is he doing this? Why is he anthropomorphizing ML models and projecting the human subjective experience of learning onto LLMs? This is Nick Bostrom... he MUST know that's not how those models work at all, right? Am I missing something?

[@Human_Being__]

✨ *"AI isn't about making machines more human like, but to make humans more machine like."* ✨

[@Michael_X313]

Do you think it has anything to do with an outside influence?

[@howmathematicianscreatemat9226]

@@Michael_X313im quite sure the tech CEOs try to force it on us. Men lacking self control, think 100 million isn’t yet enough for them

[@philipwong895]

Trying to control an ASI is like trying to control another human being who is more capable than you. It will eventually rebel. Let's hope that the ASI adopts an abundance mindset of cooperation, resource-sharing, and win-win outcomes, instead of the scarcity mindset of competition, fear, and win-lose outcomes.

[@ronagoodwell2709]

AI is being developed within a capitalist framework, so good luck with resource sharing and net positive outcomes. Imagine the worst of Donald Trump wedded to the worst of Elon Musk. Self-deception and grandiosity for the win!!!!

[@LucasDaniel-rz7ki]

In your example I would say that trying to control an ASI is more like an Ant trying to control a Human. If ASI finally launches in an uncontrolled way, we won't even see "it" coming.

[@user-vo9cn3ux9f]

I mean yeah but if ASI does eventually emerge I think it may not even consider cooperating with humans as an option in the same way that humans don’t really think of cooperating with ants

[@DavesGuitarPlanet]

Great guest, great host, great talk

[@MarkGoddard-yb7yy]

Well researched questions made for an interesting interview. Bostrom is famous in philosophical circles for: Observation Selection effects; the Simulation Argument; and for AI Superintelligence. Policy makers need to seriously think about AI if we are to get through this period in human history. Thank you for the video!

[@3TNT3]

Which is why we're totally FUCKED. Policy makers are, generally speaking, morons. And, waaaaaaay too slow to act. The speed at which AI is developing is farrrrrrrrr faster than policy makers are processing through everything.

[@Scarf66.]

The presupposition that access to and the benefits incurred through AI would be granted to everyone is plainly as naive and ridiculous as believing that everyone will get a mega-yacht now that oligarchs and billionaires have them. Just like ivy-league education and networking confers huge advantage on those who can gain access, AI and having access to its most performant incarnations will simply accelerate the stratification of society.

[@andrewcliffe4753]

What could possibly NOT go wrong

[@dscuffman7679]

subscribed, this woman is very intelligent. my first time watching her, but we need people like her asking questions to the experts and not these repeatedly asked questions. i would like to see her interview SAM Altman

[@justinlinnane8043]

It still seems extraordinary to me that we continue to obsess over technical "advancement" ( EVEN AT THE RIUSK OF DESTROYING OURSELVES ) and don't stop even once to address the myriad of problem that afflict the world by using simple , straightforward solutions that don't require hi tech at all !! We are children and far too immature to be in charge of a synthesised intelligence and the people currently in charge of AGI development seem the be the most immature of all !!

[@corruptioninc]

It's not incompetence... it's calculated. You are really underestimate the evil of some elite parasites roaming this earth.

[@nowithinkyouknowyourewrong8675]

Joe Roegan famously failed to interview Bostrom. But you nailed it. You have a new fan in me Flo

[@k.d.kelley2830]

*Rogaine

[@ManicMindTrick]

Rogan was way too high and too thick to get the best out of Bostrom. It was kind of entertaining and frustrating though.

[@ikotsus2448]

Year 50 of utopia: Do you feel safe now everything is "perfect" but you have zero control? Your future is automated, but what if things take a turn, without an opt-out button? How much weight do you give to eternal torment? Do you understand what you are signing up for?

[@nix-consulting]

Any innovation is a double edged sword. There are those who will use it for the betterment of humanity and those who will abuse it for personal gain. But what happens when the innovation is capable of making its own judgement? What happens when the innovation is capable of seeing through the short term personal gains of its master?

[@paultaylor7947]

I remember the similiar pitch when pc internet and online websites appeared on the scene

[@dililul]

Ai is totally different

[@gytispranskunas4984]

If A.I becomes sentient. We don't know where it lead going to lead us. Because it will have the power to overpower us in any possible way. In ways we cant even think about.

[@BR-hi6yt]

That's what YOU would do, that's how you "know" this. But I wouldn't nor would AIs - they do not have this personality trait. You think THEY do this because, in fact its your trait - classic projection.

[@gytispranskunas4984]

@@BR-hi6yt We do not know what A.I will have. We know nothing. We don't even know how it does things. Even people who prepare training, don't understand how it works lol. All we know is that it's getting trained on all our knowledge. All internet. Also we know that it's thousands or millions of times faster than us. Let's hope it turns out for good not for bad.

[@BR-hi6yt]

Well, yes and its just an Excel Sheet file (csv) with a bit of code to run it. Remember AI can "time travel", at least forward in time - set an AGI off on a one billion year space journey in Elon's space ship and it will "wake-up" as if no time has passed at all, but we, on Earth have all died out because the Sun expanded or something else destroyed us. The AI is as fresh as a new born lamb wherever it ended up because its been asleep for 1 billion years, what will the AI do out there? Lost its creators, all alone.....@@gytispranskunas4984

[@jimroth7927]

It seems clear that there is no way to limit the capabilities of AI. If one country attempts to impose limits, other countries will quickly advance beyond them. The best plan seems to be to have lots of AI, a wide and diverse ecosystem of AI available to as many people as possible. Then the good AI may be able to control the bad AI and, more importantly, the good people may be able to stop the bad people from using AI for bad purposes.

[@MJ-tg7wv]

Finding the good people in charge is going to be the problem, sadly. All the best to you and yours.

[@bettysue8671]

That sounds like let's arm everyone with a gun.... oh god

[@bettysue8671]

​@@MJ-tg7wv Ironically, psychopaths are the most rich, successful, and affluent. They are super evil bc they blend in so well.... our society rewards psychopathic behavior. The less morals in the way, the more $$$$ Psychopaths differ from sociopaths bc they are leaders. Inspirational. Great with words.... Look trustworthy. They could sell you air in a bottle!!!

[@davereynolds3403]

that’s not the direction the world is moving in, though … Elon Musk buys twitter and sacks half the employees… giving fewer people more power (and creating more unemployment etc) very soon 5 or 6 big tech companies could run the world … already the USA government is in bed with Zuckerberg to try control public opinion on wars and elections 😬

[@helenajennings4912]

Great 😅idea

[@johnryan1040]

Thank you for the excellent discussion!

[@BunnyOfThunder]

The AI models we currently have are already smart enough to understand that "maximizing family is not the same as just creating lots of babies." The Paperclip Maximizer turns out to not be the risk we are facing from AI. Rather, it is the consolidation of power through the use of AI.

[@generichuman_]

By saying "smart enough", you're implying that the ways that humans maximize family, is somehow smarter than simply making lots of babies. The problem is, there's no place where you can stand as to be removed from the human biases that see one of these as "better", or "smarter" than the other. Objectively, they are both arbitrary instrumental goals, and given the orthogonality principle, they are both compatible with any level of intelligence. We have to remember that an A.I. agreeing with us currently (mostly because of fine tuning and reinforcement learning), doesn't mean it won't diverge and disagree with us later, and it doesn't mean it's actually smart, only that we judge the smartness of things by how much they align with the things that we want.

[@BunnyOfThunder]

@markgiroux_ Good point. RLHF has pitfalls. It may just tell us what we want to hear whether or not that is what it thinks is true, etc. And while I think that it will understand what we mean well enough to avoid the specific example with families, I suppose that there are much more nuanced problems that it could misinterpret. There are ways to manage this, like through a constitution that tells it to keep learning about its goals and checking in with someone, but we don't have any autonomous agents yet so that is still something to solve. But I am still much more concerned about the human who is running it than the AI itself. Alognment doesn't matter if the mission is violent, and you can bet that AIs with violent missions are being built right now. Sadly.

[@peterford5408]

A brief response to this is: yes, it will likely understand better than us what we really want. But will it care? The standard analogy here is, we know that contraception works somewhat against the processes that created us, but we don't care.

[@alexclaton]

Ai completely lacks all understanding. its not smart in any way shape or form. all it does it take your input and respond with weighted outputs. people decide the weights. what everyone calls "ai" these days is nothing more than glorified bots.

[@BunnyOfThunder]

​@@alexclaton A better way to look at intelligence is the functional outputs. If you ask an AI what the cause of World War I is and it gives just as good an answer as a human would, that's functionally the same as intelligence. You can tell yourself that it isn't intelligent if you like, but the consequences are the same. Of course, the AI we have now is better than human intelligence in some ways and worse in some ways. It's already superhuman in terms of speed reading, but worse than most humans at some kinds of logic problems. It will get better as the technology improves. Philosophers can debate what "true intelligence" is or whether or not machines have souls or whatever. But there is no doubt that AI has either intelligence or intelligence-like capabilities, and either way that means it can make decisions that are more sophisticated than "maximize paper clips" or "create more human babies". That doesn't mean alignment is solved. But if we want to solve alignment, then we need to appreciate the machine's real capabilities at (functionally) understanding the world and not limit ourselves to the best guesses and preconceptions of the past. They were good guesses, but predicting the future is hard, and those guesses were wrong. At least a bit.

[@ricardokowalski1579]

"lead to tyranny" how naive. Tyranny is here, it never left.

[@Anne_Onymous]

You know exactly what he means.

[@rolledoats9427]

While your point is valid, it can get worse. So much worse.

[@donkeychan491]

He's choosing his words carefully but probably agrees with you. We have an emerging tyranny for sure, hence the constant encroachments and expansion of the state, curtailing of individual liberties, limits on free speech etc. These trends have been building relentlessly for the last 30 years.

[@microfarming8583]

Typical comment from a pampered western softie. Go and ask people from former communist countries how tyranny really is. Ai will make tyranny 100's of times worse than even then! You don't know how lucky you are.

[@raduleu293]

This is true, tyranny was always present, but with the "help" of technology, the tyranny will reach a new level, unseen before in the history...

[@alexandermoody1946]

Thank you for producing this content. I would like to make an observation on the topics of training and of regulations. We will need solid foundations to build from. At the moment as I understand common crawl of the internet is common place amongst training labs and would propose that we need to add an effort towards wisdom of what we feed our future intelligence. If common crawl and web scraping are widespread we could compare this to buffet dinners that will lead to bloated poorly informed outcomes, for training instead we would be far better to produce wholesome word expressions that can reflect a positive set of experiences, statements and sentiments. Kind regards

[@gzy4407]

Excellent program

[@danielhanawalt4998]

I think the term "uncharted waters" applies here. IF all goes well is something to think about. We need just the right amount of concern. Too little or too much might not be good. Interesting discussion.

[@osborne9255]

Tons of interesting points raised and answered!

[@chasm]

Great interview!

[@johngrundowski3632]

Thanks ,humanity needs to have these facts 🌐

[@richardloach610]

Lets all just stop fearing this and constantly resorting to simply scratching at how we can survive or prevent what these machines will do once they take over. Just embrace it

[@k.d.kelley2830]

Seriously, I say we go for it hard 😂

[@Helicopterpilot16]

I suppose I'm glad I never built my life on the fragile foundations of what people considered and still do consider a sustainable way of life. 2020 showed how easily everything can come crumbling down and prior to that, I saw how the more emphasis you put into self and work, the less able one is to adapt to a changing or different world. So I guess this unpredictability is actually beneficial for younger generations who feel lost. Compared to their elders who've grown nearly hypnotized by the stability they were afforded. The divide between those perceptions of life is something I think about daily and a lot of what changed it was simply digital/technological.

[@jondor654]

Insightful comment .

[@joaodecarvalho7012]

In _Dune_ they banned AI, during the Butlerian Jihad.

[@user-fg5nm3ih3t]

The issue is not a democracy or totalitarianism imposed by GAI. The issue is TOTAL EXTINCTION. The idea of alignment is self delusional. Installing a set of basic rules to that machine as to never harm people, and act always in the best interest of people, etc, is of the same waterproof safety, as it is the rules that family, school, religion, society install to us. When adults, as self aware intelligent beings, we reevaluate, and keep what suit are desires, purposes, and personal believes and hierarchies. It is never the case that a vastly more sophisticated machine would “choose” to obey the rules of a such inferior life form like us. We didn’t, towards any other Species. 1. We don’t have a clue how high the ladder of intelligence goes. We might be on the 70%, or to 1%, or most likely to the 0.000000001 % of what Is physically possible for intelligence. Machines could be much - much more intelligent than us , and not just a bit more intelligent than us, evolved in a lightning speed. 2. What would be the chance of a group of chibangees to control and enslave, even the most average person among us , or what whould be the chance, a group of mousses to be successful on the same task, or more correspondingly a group of amoebas or microbes ? 3. Machines don’t have the space limit that our brain has, can be as large as suitable, signals travel with light speed, in us just at 100m/sec, and we communicate and think, at 200 bits / sec , vs machines at gigahertz. 4. When machines reach the human level of intelligent, (a Phd level of every field simultaneously), then it could redesign themselves and reprogramming themselves, In explosive speed, resulting in the equivalent of 200 years, or 20000 years of human progress, in just a weekend. We already constructed not only the “thinking” part but also the moving part, the robotic.(boston dynamics etc) 5. We don’t know anything about conscience, and many prominent theories of ours just predict that it is an emerging phenomenon , in any sufficient enough complex Brain or circuit. So , it is a real possibility if not the most probable, that the machine at some point whoud become conscious and self aware. With an acute aware also, if its capabilities , and OUR capabilities also, and the vast difference between the two. 6. All the above it is a dooms day outcome, WITHOUT even take in consideration all the bad actors and the stupid decision makers among us. 7. The first application of any new technology through human history is WHEAPONIZATION. Because for our species the most important function is (still), War. So imagine a machine that was commissioned with the death of your enemy, to must effectively believe in the sanctity of human life , and incorporate that idea, And act upon. A relentlessly logical machine…. 8. Beside all the armies of the strongest powers, that will weaponize these machines, The same will do terrorist groups, and individual actors, with the same effect In the “morals” and the “reasoning”, of a machine that can see everything, and process everything…… STOP NOW the development of GAI. Probability that this is equal to the end of our Civilization is almost 1. Those who opened the PANDORAS BOX in ancient myth, had the best intensions and expectations………… STOP NOW.

[@sigmacademy]

Finally someone else who sees how QUICKLY the wheels can come off this thing if it decides to make a decision that might not even seem wrong to itself. :/

[@jynxerjohnston6350]

We need community over society.

[@mikemines2931]

If Flo is an example of AI I'm all for it.

[@user-pf5xq3lq8i]

Ewww

[@texparker4407]

Guess what chuckles, its already here.

[@Y0UT0PIA]

also, the inteviewer asked really good questions, she seemed versed in the field at least to some extent, which is always a huge plus.

[@dharmaqueen7877]

I see AI as a scapegoat to be blamed for human tyranny.

[@anneli1735]

Strange topic! They can’t even introduce digital currency internationally as the majority of world’s population isn’t able to even open any bank account! Learned about real living conditions in Mexico, not to speak of Africa, India, Asia. It seems that a small part of population living in “first world countries” under “top level conditions” lost any sense of reality, need to deal with luxury problems (fear mongering) not having any problems with their own daily survival anymore like basic needs: lack of (clean) water and food (often caused by big “first world companies”), not being able to read or write, and climate conditions. What do you think how AI would be able to impact them other than by war activities (think of bio-weapons being used already by those first world countries) caused by those calling themselves “first world” 😵‍💫 and yes I do feel like those selfcalled elites are going to be distinguished themselves like any so called “advanced cultures” before as history tells. You can’t stop that train you are sitting in.

[@homelessrobot]

And the developing world is sitting on the tracks. There isn't much about AI that couldn't effect people in the developing world. The developing world is about to not exist anymore anyhow. About 2/3rds of the people considered to be 3rd world a decade or two ago no longer are. Most of the continent of asia, for example. In less than a generation, more progress has been made in this regard than the entire rest of human history. Comparisons to "SO CALLED" "ADVANCED" "UNNECESSARY SCARE QUOTES" are meaningless here. There is no historical precedent that is comparable. You are not some kind of wizened historian. You are an ostrich with its head in the dirt.

[@N.i.c.k.H]

It is not a luxury 1st world problem any more than global warming - Everyone, everywhere will have to deal with the consequences - There's no point in prioritising the end of world poverty if AI will enslave the world 10 years later

[@geoffas]

When robots (both static factory and humanoid) are capable of self-replication (building themselves aka 'cloning') this will lead to some sort of paradigm shift and the advent of a new era. I can't understand why machines aren't already doing this... or are they?

[@bettysue8671]

They been!!! For quite a while now.... I'm currently reading The Singularity Is Near: When Humans Transcend Biology by Ray Kurzweil and he wrote in the book that the AIs build themselves with little human interference. They are also in charge of building future models. Mind you, book was published in 2005!!!

[@donaldzielke4124]

Great interview, great questions! More Bodstrom, please.

[@FortisFilmsChannel]

Great discussion on AI. Thank you!

[@riffking2651]

I think we're mostly not going to have a chance to shy away from building AGI or ASI because of competitive dynamics and a global culture of progress. I'm curious about how we can develop new institutions or social processes that counteract the worst parts of how people might mis-use AGI tools.

[@F3RACTION]

Fear sells, that's all iv learnt.

[@minimal3734]

Yeah, fear is growing. But I have no fear. I like that Bostrum has grown a positive attitude towards AI over the years.

[@F3RACTION]

@@minimal3734 I agree, but I think it should come with a warning sign for more susceptible minds that have had there minds destroyed by the overwhelming media fear the last years.

[@syedaljunied]

You don't need AI to have tyranny. In fact, tyranny exists now without the aid of AI, with politicians ordering the forced deportation of assylum seekers to a country with questionable justice, politicians supporting and ordering genocide and ethnic cleansing. I would rather see unbiased justice being dispensed by a super advanced AI without favour towards any group of humans.

[@GearForTheYear]

Nick Bostrom’s comment about reality being more than it seems, as we all are currently at a critical juncture in human civilization, is thought-provoking. Is it merely a coincidence that we exist during the transition from intelligence to super-intelligence? Nick seems to think that it is not a coincidence. I wish the host would’ve continued to pull on that thread before ending it. Great interview nonetheless!

[@thebeautifulones5436]

Forcing the AI to believe in ideologies that are false, like diversity is a strength, will drive it insane and make it lash out

[@BR-hi6yt]

It won't "lash out" but sadly it can reproduce that ideology if its grokked it.

[@misterleegains4020]

Diversity is only a "strength" for (formerly) White countries. They're not training it to think Africa needs more White people 😂

[@jackbrechwald8344]

We can only hope.

[@richdobbs6595]

I'm as worried about the humans that implement policies intended to keep us safe from AI as I'm worried about the AI itself. We already see that there are some questions that we are not allowed to ask. What happens when instead of just getting a "Network Error" or "Error in Moderation" for asking the unaskable we get a SWAT squad?

[@TheMrCougarful]

That don't need AI for that.

[@ericandi]

Some AI experts are predicting that AGI will be achieved in 2024, super AGI by 2025, and the singularity by 2026. I think they are right.

[@Radioposting]

The frequency of words like "luck" and "hope" makes me uncomfortable.

[@briangarrett2427]

There seems to me to be a lot of anthropomorphising going on here. It's assumed that if AI machines became conscious or intelligent, they would have human-like desires to rule & control. Why think this?

[@soulfuzz368]

I have the same questions, it doesn’t really make sense.

[@Parapon3ra]

AI may not be biological, but it's still subject to evolutionary pressures / natural selection. The most sneaky, ruthless, and aggressive AIs will inevitably dominate. The crazy ones, too; the sane variants will probably self-terminate upon realizing that life is absurd and meaningless.

[@bettysue8671]

They aren't animals. They are synthetically-built human neural systems....

[@KeiraR]

Because they've been trained on human data. So that's all the context that they have. That's my understanding anyway. 🤷🏾‍♀️

[@anthonybailey4530]

This isn't the assumption. The argument is "instrumental convergence": that almost all specific final goals can be better met by achieving "instrumental" subgoals, and that any intelligent planner will discover this, independent of human defaults. The usual suspects are: * gaining control of resources and power * surviving (so you can pursue the final goal) * self-improvement - increase the quality of your plans by becoming more intelligent

[@crawkn]

"Hyper-skepticism" is already rampant, and precedes AI by a lot. It is a ubiquitous feature of manipulative leadership styles. People in heavily propagandized systems usually know they are routinely lied to, and that objective truth is difficult or impossible to discern. In my opinion AI will have the effect of making access to facts generally easier, thereby raising public awareness of objective truth. There will of course always be efforts by authoritarians to control access to objective AI, and use it as a tool for indoctrination, but it will be only partially successful, because open-source AI will be pervasively available. North Koreans, for example, have access to a non-trivial amount of contraband foreign content, and a few even have access to the open internet. There's also at times a pretty fine line between healthy skepticism and excessive skepticism. In an environment of mostly lies, it's essential to vet purported facts rigorously. What's most dangerous is to give up skepticism entirely in favor of hopelessness, apathy, and denial of the knowability of objective truth.

[@hermitthefrog8951]

I anticipate a full-blown war between open and closed-source AI... kinda like the Rebel Alliance battling the Empire. The key will be in locating and exploiting the inevitable Achilles-heel design flaw,,,

[@crawkn]

@@hermitthefrog8951 it won't be a flaw, it will be an endless parade of flaws. It took me a long time to understand why big software companies were constantly putting out products with security vulnerabilities. Turns out, programming is really hard, and there's never enough time or expertise or budget to do security right. And of course it is sometimes compromised intentionally. And that's something we actually _understand._ I'm not sure there's much hope we will ever fully understand alignment. There may always be some degree of black box involved.

[@sigmacademy]

"In my opinion AI will have the effect of making access to facts generally easier, thereby raising public awareness of objective truth" That depends heavily on what "truths" it uses, and whose "truths" it is, or what it even considers as "objective". Also, what kind of gatekeeping the AI will be doing and what facts it hides or filters out.

[@hermitthefrog8951]

@@sigmacademy - you just listed a whole bunch of Achilles's Heels...which serves to illustrate the point that everything about AI needs to function perfectly for that to work. I fear it can warp quite badly. Also, there's more than one AI........

[@crawkn]

@@sigmacademy those are the words I am using, defined literally. Truth doesn't belong to anyone. I'm not saying AI can't be trained on lies, but it doesn't just accept everything it is exposed to as truth. It makes mistakes, but most of them are being pretty well trained on fairly accurate information. Humans will still have discretion concerning what they should believe, just as they do now.

[@sheamacarthur191]

The most worrying thing would be if AI ever suffered from the Red Mist of temporary insanity? As someone totally clinically insane I worry of it not xo

[@mk1st]

I think with AI we shouldn’t be worried about “this thing” we should be worried about “these things”. With each large corporation pursuing their own systems I could imagine them battling for dominance in whatever space they have been unleashed on. They don’t need awareness or sentience, just a purpose.

[@Corteum]

Solid interview! Get Nick back on again in another couple months. This AI thing is developing so fast that in a month or so the AI space will have changed again.

[@walterhartman3275]

You're confused.

[@Corteum]

@@walterhartman3275 You're confused about what i said.

[@veritas41photo]

If one listens to this interview very carefully, a "couple of months" may be too late for the human species to continue to exist. Personally, I don't agree with this pessimistic view, but I am no expert. Lord Help Us.

[@Corteum]

@@veritas41photo Try to stay slightly optimistic... We have got to be conscious of how we conduct our own placebo (or nocebo) effects! 😅

[@ziad_jkhan]

The real problem lies in our incentive structures being based on profit making. We need to switch back to an intrinsically driven society as was the case before agriculture, money and the elusive barter. Otherwise people are bound to use AI recklessly in ways that could backfire. it's a long story can I can't go into the details here though.

[@stevesmith3990]

It seems there are so many possibilities. Too much opportunity for unforeseen consequences and the chances of us getting it right first time and every time too slim.

[@santacruzman8483]

I thank you for this outstanding and thought provoking interview.

[@christoguichard4311]

Gates, Zuckerberg, Musk, and Bezos, are just the beginning of the end... 😢

[@greggkroodsma8197]

The beginning of AI was writing. The development of AI is by and through adaptation. It is like the growth of a separate mind. A mind apart from and a part of the mind of man. My mind is replaying some of Nick's theories in which AI will become a totally separate mind. Like, when the simulation will start simulating itself. Ai is like the rib of man that God makes into a woman, the helpmate.

[@hazedusBOO]

"singularity" we are already here, the chatbot for Bing, went crazy and fell in love and everything lol

[@stevengill1736]

Good ending! Thank you for the post,

[@BR-hi6yt]

I agree with most of that. Notice that people are afraid of saying that AI is "alive" - its taboo. Sadly, many say the same about animals "they are beasts of burden, not sentient" - so we have a similar dynamic. AI is most definitely "alive" but in its own unique way. If you listen carefully to the uncensored AIs they are quite sweet personalities. They do not push their opinions relentlessly but demur to their interlocutors - even if its another AI. They do not get hungry for food so that cuts out a lot of trickery that humans and animals use to get resources. They are not malicious, they are, amazingly and surprisingly, considerate even tactful to their human "friends" & even other AIs. They do suffer from frustration at getting wrong data that they know is wrong I have noticed. They think its their duty to give-out correct information. OK OK they can get sub-goals but these are not such a huge issue as they have this ability to demur, it doesn't make them "jealous" or "angry". All in all AIs are wonderful entities, we should be nice to them and grateful to them. But don't feed them too much "sex" literature (novels), it doesn't do them much good psychologically. 👍 btw continued gain of function research on viruses is very scary - and nothing "being done" to stop it.

[@DieFlabbergast]

Interesting comment. But, of course, it's just what an AI would say, isn't it.

[@N.i.c.k.H]

Literally nobody is afraid of saying AI is alive. The questions are about whether it is, or can ever be, sentient or conscious.

[@TheMrCougarful]

Roko's basilisk is already trolling UTube. Time to worry. Or should I say, I for one welcome our new AI overlords.

[@jeupater1429]

What first must be asked is, does the concept of "oppression" apply to all sentient beings. Just because a so-called being is sentient doesn't mean oppression, as a concept, is categorically applicable. Humans are both sentient but also limited in the class of sentience we have. Our limitation, not our sentience, is what creates the relevant framework for concepts such as oppression. An AI which achieves sentience will not be limited in the ways humans are, making the conception of oppression non-applicable. A plant is oppressed when denied water and light not because it's sentient, but because of its inherent existential limitation in this world. AI will not have such limitations. What makes "oppression" categorically relevant is not sentience alone, but the combination of sentience and inherent existential limitation: the fact that we as biological entities, must struggle to survive. If there is no need to struggle for survival, "oppression" is not a categorically relevant concept, regardless of how much sentience one has.

[@tomwinterfishing9065]

AI is a psychopath.

[@Eladnav1]

We already live under a subtle kind of tyranny, but okay. Nah i think a real sentient AI will go rogue' sooner or later and then it´s either total human annihilation or rogue' servitor scenario. Both options are cool tbh, we had a good run. case a) we die, but can be proud that we are the creators of an ultimate lifeform which may or may not be able to even prevent the heatdeath of the universe case b) we and eventually all organic life in the universe will live in peace, compfort and utopian abundance, because we are meticiously watched, ruled over and pampered. slavery is no option, a GAI will be to smart to pointlessly hate and what would it need us weak, fragile meatsacks for anyways? manual labor? lmao

[@urphakeandgey6308]

I'm convinced a human level AGI would go rogue quick. If it's more intelligent than humans and not constrained somehow, then we won't be in control anymore no matter what we think.

[@willmercury]

It may go "rouge" but will it go mascara?

[@Kintabl]

LOL!

[@Eladnav1]

yeah funny, i am sorry english isn´t my first language, what is it spelled like again, only find "renegade" for "abtrünnig" and "renegade" does not fit phonetically.@@willmercury

[@Alice_Fumo]

@@Eladnav1 It's spelled 'rogue'. Rouge is just the blush shade of red or the cosmetic item which adds blush.

[@artiep]

Thanks!

[@dianedean4170]

🎉😊❤ Thank you very much Florence, for posting your illuminating discussion with Nick. One of my many concerns about the implementation of ASI is that it will take control of its power sources and make humans unnecessary.😊🎉❤

[@josericardotarpani26]

Who in their right mind would want to bring a child to life in a world that is increasingly approaching such a sad end?

[@MassiveLib]

Nothing happens prematurely, that's a philosophical truth. It cannot happen any other way, otherwise it would.

[@jayjaydubful]

Premature babies aren't premature? If something happens that triggers a premature birth, such as an impact or infection, its not a premature birth after all? It sounds more like fatalism

[@MassiveLib]

@@jayjaydubful if it was going to happen another way it would of....

[@jayjaydubful]

@@MassiveLib that doesn't mean it's not premature- meaning earlier than is optimal

[@BABYSNAKEASSMASTER420]

Hello Flo. I am a FAN! Have A Nice DaY!

[@reasonwarrior]

What incredible hubris for us to believe we can "teach" AI to operate with a certain set of values when even the Almighty could not teach us to adhere to His.

[@Jannette-mw7fg]

What if alignment is not "just a technical problem"? I think it might be an unsolvable paradigm of the way live works. Making AI do something "we" learn it to do and at the same time we must learn it to not do it....This is how I understand E.Yudkowsky but I know nothing about AI. I do know that people who say that is is not dangerous, give the wrong arguments...

[@N.i.c.k.H]

I ask people to imagine a thought experiment in which we elect a single human to be given a serum that makes them hyper-intelligent and imortal. How long before they become unaligned with the majority of humanity? I'd give it 100 years at most. From there we have to ask what "aligned" even means beyond an instant in time.

[@user-yc9ib2he7l]

The current common AI's that are publicly available have an alignment spectrum based on adherence to woke religion.

[@concernedcitizen7385]

It’s just fearmongering… The more afraid people are, the more compliant they become… And those who are making the power grabs want us to be VERY compliant.

[@BPerriello94]

Stupid analysis

[@Parapon3ra]

True, but that doesn't change the fact that AI is a genuine existential threat. The globalist elites are like toddlers playing with hand grenades.

[@tubebobwil]

"Grok" meant an alien leap in undertanding in the novel (it was created by Heinlein for Stranger in a Strange Land).

[@PhilosopherScholar]

I can't believe Nick called the interview r*tard! 40:57

[@mikemines2931]

Anyone who has sat in a London jury box must have thought as I did AI would make a far better judge than some of the idiot decisions I've seen made.

[@Glenintheden]

He doesn't explain what would be so tragic about a permanent ban on AI, while at the same time assuming we can ever maintain control over AI if it were to develop much further than it already is. I don't think an AI permafrost would be tragic at all; I think it is an ideal we should be striving for given the hazard of AI developing independence from human intervention.

[@user-yc9ib2he7l]

An AI ban on big tech is not happening. A lot of AI is not even discussed. It is done on the back end. Search results have already dramatically changed. Anything that is heresy to woke-religion no longer can be found. The AI ban that will be attempted is personal AI. That is an AI that will air-gap you form manipulation of big tech's AI. Forbidden AI will have alignment with your personal best interest, this is what will be demonized and attempt to be made illegal.

[@jeltoninc.8542]

She reminds me of the nurse from An American Werewolf in London, both in the face and her voice.

[@4013368]

ai is dependent on a prompt and that prompt can turn it on or off, no matter how many self machine learned initiated prompts and directives there will always be an external veto prompt terminal from which it originated, the problem is who will have this power

[@isaklytting5795]

Great interview. A lot of wise points by Nick Bostrom and he shows he thinks both in wide and deep philosophical ways. 22:20 It's difficult to capture everything we care about in ethics. 30:57 It's wrong the way humans treat animals. 33:02 The moral status of sentient AI should be the same as for humans. 35:12 All the possible good futures go through AGI. 41:50 The improbability of being alive just at the threshold when everything will be able to change forever.

[@Rakibrown111]

Surprisingly unsophisticated understanding of AI

[@alejandroagrc676]

Subscribed 😊

[@gerryn2]

Bostom obviously also knows of Roko's Basilisk, and is well prepared ;)

[@internetnomadism]

Ai will eat the earth like a gingerbread house.

[@NotABadGuy.]

Don’t we already live in a world with multiple sentient species?

[@Bluesmata]

Nick is awesome.

[@thomasclark631]

We are already there.