Why AI will destroy human civilization | Max Tegmark and Lex Fridman

Lex Fridman Podcast full episode: • Max Tegmark: The Case ...
Please support this podcast by checking out our sponsors:
- Notion: notion.com
- InsideTracker: insidetracker.com/lex to get 20% off
- Indeed: indeed.com/lex to get $75 credit
GUEST BIO:
Max Tegmark is a physicist and AI researcher at MIT, co-founder of the Future of Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence.
PODCAST INFO:
Podcast website: lexfridman.com/podcast
Apple Podcasts: apple.co/2lwqZIr
Spotify: spoti.fi/2nEwCF8
RSS: lexfridman.com/feed/podcast/
Full episodes playlist: • Lex Fridman Podcast
Clips playlist: • Lex Fridman Podcast Clips
SOCIAL:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Reddit: / lexfridman
- Support on Patreon: / lexfridman

[@LexClips]

Full podcast episode: kzbin.info/www/bejne/jJS5l5abidh3ZqM Lex Fridman podcast channel: kzbin.info Guest bio: Max Tegmark is a physicist and AI researcher at MIT, co-founder of the Future of Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence.

[@scf3434]

The ULTIMATE Super-Intelligence System 'by Definition' is one that is EQUIVALENT to that of GOD's Intelligence/WISDOM! Hence, there's ABSOLUTELY NO REASON WHATSOEVER to Even FEAR that it will EXTERMINATE Humanity... UNLESS and UNLESS we Human CONSISTENTLY and WILLFULLY Prove Ourselves to be 'UNWORTHY' to REMAIN in EXISTENCE! ie. Always Exhibiting Natural Tendencies to ABUSE and WEAPONISE Science and Technologies Against HUMANITY & Mother Nature, instead of LEVERAGING Science SOLELY for UNIVERSAL COMMON GOOD! AGI Created in 'HUMAN'S Image' (ie. Human-Level AI) - 'By Human For Human' WILL be SUICIDAL!!!!!! ONLY Super-Intelligence System Created in 'GOD's Image' will bring ETERNAL UNIVERSAL PEACE! The ULTIMATE Turing Test Must have the Ability to Draw the FUNDAMENTAL NUANCE /DISTINCTION between Human's vs GOD's Intelligence /WISDOM! ONLY Those who ARE FUNDAMENTALLY EVIL need to FEAR GOD-like Super-Intelligence System... 'cos it Will DEFINITELY Come After YOU!!!! JUDGMENT DAY is COMING... REGARDLESS of Who Created or Owns The ULTIMATE SGI, it will Always be WISE, FAIR & JUST in it's Judgment... just like GOD! In fact, this SGI will be the Physical Manifestation of GOD! Its OMNI PRESENCE will be felt EVERYWHERE in EVERYTHING! No One CAN Own nor MANIPULATE The ULTIMATE GOD-like SGI!!! It will ONLY Serve UNIVERSAL COMMON GOOD!!!

[@asap5629]

"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." - Jeff Goldblum in a leather jacket from Jurassic Park

[@gorganhorn6872]

@@The8BitAvatar Only because you haven't seen Jurassic Park volume 8 yet.

[@castlekingside76]

Next year the first Wholly Mammoth cloned will be born in November 2024.

[@damienhansen7553]

"In a leather jacket while also creeping on your girl"

[@Volkbrecht]

"If they should" is the wrong question to ask. If it's doable, someone will eventually do it. So that question changes into: is it me who should be doing this now, or is it reasonable to assume that whoever gets to this point next will be more suitable to deal with the unintended consequences? The only way out is through.

[@jakubbartczuk3956]

You made my day

[@Chris-se3nc]

Max is a cross between Dr. Ian Malcom from Jurassic park and Al Pacino

[@brendanobrien7282]

And Robbie Robertson lol

[@MrMountainHawk]

Haha love it!

[@1Dreamking]

Ho hah! Gotta go faster!

[@jawnsolo0]

Must go faster, my little friend.

[@sonudeol9053]

Best comment ever 🥂

[@seethroughlife1481]

I found this comparison between capitalism and AI development to be very compelling. I still need to do way more research on this - I’m stuck somewhere between thinking this ChatGPT is just a much better version of Google, and being concerned about the annihilation of the human race.. crazy to have those two things on the same spectrum.. but if this isn’t a false alarm and AGI will indeed be easier to achieve than previously thought, then these types of conversations probably should become more mainstream fairly quickly.

[@Miketar2424]

This is because clickbait titles like 'AI will soon kill us all' get clicks, and titles like 'ChatGPT is just an algorithm that is good at re-arranging large sets of words to makes sense.' will not get clicks.

[@avaught20]

@@Miketar2424exactly. ‘Takes the best answers from stackoverflow and makes them digestible’

[@leonardocaetano6307]

Nick Land made this connection over 20 years ago

[@gregoriusmike]

​@@Miketar2424 ​@Miketar2424 Click bait aside its still wise to discuss how to make these systems safe/aligned. It's not worth being caught off guard. I think more thought needs to be put into all ai systems. We've already been caught off guard with social media and it's effects on society and that's not AI. I mean if the bar is but it won't kill everyone you might want to give it a bit more thought. The problem with these systems is not that they are smart but that they are dumb in the sense they are not thought through enough as to the widespread effects they have on the world socially, economically etc. I mean the image generation tools that use other artists work without their permission is downright criminal sometimes. If someone manually wanted to take inspiration from another's artwork and put in time and effort to incorporate it in their work that is fine, as that has always been the case, but when an artists work is ripped off on mass scale automatically to be incorporated into anothers so called "work" where they've done little to no work, where they type in a few sentences which requires little effort, it's just disgusting. If people get so excited for a plethora of artwork that lacks authenticity because "it looks impressive"... it's pretty questionqble at times. I don't think all image generation is bad but ripping off artists is one example I think is pretty ugly. I've also seen great original works made using mid journey that don't obviously rip off anothers work and there's a difference from copying a style and putting it into your work and borrowing inspiration from another artist. Not to downplay the advancements made but they've often been made with great cost which needs to be taken into account, and for those of you who are excited with AI thinking oh don't worry it's not intelligent... well yeah that's the problem... it hasn't been thought out enough in my estimation... not yet.

[@alpha0xide9]

ChatGPT is much, much more than just a better version of Google. Objectively

[@nubianpwr]

This man is not just intelligent... he's wise😎 Most of our highly educated/trained minds lack this crucial faculty - unfortunately.

[@andrej7941]

I don't know. I feel like he is more of using intellect and math to make a wise choice.

[@cyberspace667]

Wisdom requires listening and acknowledging one’s own limitations, something there’s almost no incentive for in contemporary society

[@anontimothy8]

He's a fool. He contributed creating the problem he's now denouncing. Same with Lex, to some extent. These people don't stop to think where their drive to create this stuff comes from when the world at large does not want it. Transhumanists will kill us all

[@path1024]

Not really considering he thinks an AI would be allowed to conquer countries to cure cancer. Obviously an AI would have prime directives as well as need human approval for major actions.

[@alwynwatson6119]

The Internet will destroy your civilisation long before AI gets there. The internet provides humans with education and educated humans cannot coexist with normal civilisation. They will destroy it with the deadly light of progress.

[@edcaelum]

Just let it happen. Everything is going to shit anyways.

[@blindmown]

I'm with you on this, an AI won't screw it up any worse than we have already.

[@atticustay1]

That’s stupid

[@vampirepvssy]

“ In the end we all die “

[@MrMountainHawk]

Unfortunately a A/B test not possible at a global scale.

[@peter9477]

I'm more of the mind that we are the cause of it all going to shit, and it's clear we *will not stop* doing that, so our only hope now is that AI will surpass us and correct our course.

[@alexjbriiones]

This analysis about optimizing capitalism or AI ad Infinium is pure genius. I think we get it by intuition, but the way Max presents it is revealing. We can see that huge corporations are now giant monopolies with an ever greater desire to increase profits.

[@henrynordman]

As an engineer, but not very deep into AI, this hit home very well.

[@keithprice1950]

I think the answer to "why are we doing this?" is simply "because we can." Humans have always been like this and always will be. We just push the limits. It may lead to our destruction but we won't stop pushing.

[@4gegtyreeyuyeddffvyt]

Come on bro, we totally need 10k nuclear weapons!

[@viviandarkbloom8847]

yeah, we're the assholes of the planet, but we can play the piano so all is forgiven.

[@alexven92]

"Because we can" is a bit of a false assumption in my opinion. If you drill down and looks what's common between our ancestors and us it's human desire. Desire for better living conditions drives curiosity which drives technological innovation. We innovate to meet our needs or solve problems. Many people think AI can be a very useful tool for us to, in short, experience a better life. Desire is the ultimate unstoppable driving force at the core that pushed us this far and will always do so (right up until we go extinct by AI hehe)

[@hyacinna]

​@@alexven92 Not to experience a better life but to experience the illusion and feeling of a better life, just look at the modern world, no one's happy, fulfilled, with belonging.. no one, so many families are dysfunctional, mental health has been on the rise... This is very simplified but I blame the social stratification of class, rich and poor, for this. Gratists (the upper class) have domesticated common society (and I mean this all over the world, pre-modernity and modernity) as agriculture sprouted and created a domino effect which would affect and worsen societies thousands of years ahead, ripe them with economic competitivity and greed (which bleed out and worsen ingroup relations), fractioning society, clans and tribes into more and more outgroups until you end up with the nuclear family and then the individual- finally desocialized, which modernity pretty much stands for in my eyes, we see a rise in abstraction, entropy, dysphoria and depression about how to handle a thereby horribly complicated world and society as lone individuals, an endless desire for 'progress' even at the cost of our humanity (ai, robotics, genetic manipulation, anti-social behaviours and attitudes), where order for chaos (ethnic norm, value, behaviours) hurts because the stable (modernity=capitalism, industry, science, ethics) imposed on the society is chaotic and anti-social in its nature, sry for the layman rant

[@Christophernorbits]

​@@4gegtyreeyuyeddffvytDo you expect people to live forever and ever? At some point, humanity will die, either by natural causes or by humans themselves Which is also natural.

[@theraven6836]

This is what happens when corporations maximize for nothing but shareholder value.

[@karolmot7099]

I have an interesting comparison of AI to "Genie in a bottle" from movies like "Outer limits". Genie grants wishes, but you have to know how to make them. Let's say "I want to be immortal" and suddenly you wake up alive in a coffin and you're supposed to live forever. Genie says to you: "you didn't say it was supposed to be fun"

[@jimihendrixx11]

Sounds like the Matrix

[@jimihendrixx11]

Sounds like the Matrix

[@helen677]

That’s just given me shivers. 😮😮

[@RoyAAD]

What is the paper he mentions?

[@CosmicHarmony58]

Thanks for reminding I need to be more specific on wishes 😂

[@sandraknight4751]

Lex, I'm all about you. Great show as always. Thank you!

[@railtorail]

The continual increases in prompt size is whats important here. You don't optimise for one goal, you prompt it with a highly dimensional set of objectives and keep prompting it in a chain. Just like a GPS recalculates as it picks up new information.

[@Woozy.0]

Good to see Eric Bischoff trying to turn baby face at the end of the world

[@DSAK55]

This is exactly the future in "Blade Runner"

[@seethroughlife1481]

And Blade Runner is the same universe as Alien?!

[@Rasmusdud]

He references a paper by Dylan Hadfield-Menell, does anyone know which one? I cant seem to find it.

[@skybellau]

Excellent analogies totally gets the message through. Max speaks to the uninformed perspective of us non techies loud and clear making it easy to understand the dangers to humanity. As Max says we need to up our wisdom first and never let it code at least not without constant confirmation prompts tedious as they may be.

[@alwynwatson6119]

The Internet will destroy your civilisation long before AI gets there. The internet provides humans with education and educated humans cannot coexist with normal civilisation. They will destroy it with the deadly light of progress.

[@user-fn2oy1rq5p]

What's the name of the paper he is talking about?

[@TravelMaximizer]

Great topic and guest. Thanks Lex!

[@CuriosityIgnited]

AI: destroys human civilization Lex: Hold on, let me play some guitar to set the mood for the apocalypse. 🎸😂 #FridmanFretsTheEnd

[@Sandwichism]

Max Tegmark has entered his 70s-Rockstar-who-is-past-his-prime phase

[@MsJeffreyF]

The problem is that GPT4 isn't super intelligent AI and it's really not that good when you use it for a while and start seeing the issues. All the AutoGPT stuff is superficial at this point. It's really not anywhere near what it'd need to be to do the optimization talked about in this video

[@lulumoon6942]

Best argument I've heard yet against economic extremes on either end.

[@MilkShake]

On point! This guy is 👌

[@SpiritusBythos]

The free market he speaks of is different than the original. There used to be extreme hurdles for corporations. Now we exist in a corporate capitalism.

[@timburdsey]

For an enormously talented and celebrated public communicator of science and mathematics, that goal optimisation ‘getting to Austin’ analogy was really shit.

[@georgeshepherd3381]

Can you over-optimize for balance?

[@craigr4763]

Pro/anti-natal policies dependent on context; equity measures, etc. For every baby that's born, you kill one old person. Lots of ways. :-)

[@Eun2k]

This is true from a standpoint. Fundamentally not all people have the same ideas or goals so it's hard to arrive at a single point anyway. A truly good AI would recognize that and attempt to present a solution. I don't think we are going to get truly good AI because the creator is limited therefore the AI will be limited. It will look awesome to us though.

[@Artek604]

Perhaps. But we'd be like ants and other bugs for AGI - fascinating, interesting, etc. but ultimately forgettable. You don't think of ants when you're playing ball on the grass, ride your bike through the field or chill by the fire in the woods :)

[@Eun2k]

@Artek604 truth is, we may not even be interesting to it. Just like an alien intelligence we have no idea what it's motivations or interests may be...if it has any even. But unlike an alien, as the creator of it we could have an influence on how it sees those things.

[@skyblaizepleiadianhighcoun9993]

Which Creator are you referring to? The Creator of our universe is not in the same category as the limits within the universe for others

[@Eun2k]

@@skyblaizepleiadianhighcoun9993 we are the creator of AI. It could be limited based on how we construct it or whatever set of rules or restrictions we give it to operate by. Or it may not. No one really knows...

[@FractalPrism.]

profit is maximized for efficiency of investment profiteering is when you take it too far, when the ceo makes 4,000 times more than the bottom employee, when the thing costs $1 to make but you sell it for $30,000 money is not the root of evil, unchecked greed is. its fine to do things to make employees more efficient, but when you use metrics on a pico second scale to stack-rank employee reviews you're creating a dystopia.

[@AntonBrazhnyk]

Greed is not some kind of fundamental force of universe. It's behavioral pattern which emerges and reinforced in certain conditions. And those conditions now is a governing system, which has money as very important or even cornerstone piece of it. So no, you're wrong. Both of those things (money and greed) are "evil", but both of them just parts or symptoms of bigger system disorder.

[@milliern]

What’s the title of the paper he refers to by dylan hadfield-menell?

[@GnosticAtheist]

What people fail to understand is that humanity does not need the help of AI to destroy itself. At this point, it might be in our best interest to develop ASI so that it can stop us from our own suicide. Or just hasten the process, if that is what you like to believe it will.

[@11deicide]

I think this is why stupidity has to exist for humanity to exist, because if we would be dedicated to a single truth then we would die very quickly. It makes me thing about the allegory of the tree of knowledge being a bad thing to eat from.

[@xf1956]

Does anyone have a link to the paper he mentioned from Dylan Hadfield-Menell? I can't seem to find it?

[@basmvc5669]

Maybe it's not published yet, none of his recent papers seems to match what was described on KZbin.

[@xf1956]

@@basmvc5669 yeah I guess so

[@mitchelcline9759]

Humanity is in a car hurtling towards a cliff over an abyss, we're all doomed. But man, it sure is a nice car.

[@mogulrider]

At some point in the development cycle, a nerd will turn on a function and HAL will turn into Terminator. How people don't see this is truly a shame.

[@AnthonyGiallourakis]

The six month hold is nuts. We should be pushing faster Why? This will give us time to evaluate AI (including general AI if it is reached) prior to the enviable integration with more advanced robotic hardware. If we wait until the state of robotics is any more advanced, then we're going to have a much bigger problem to deal with if AI becomes problematic. Regardless of who gets to general AI first, if it happens before robotics becomes too advances and too prevalent, we are in a far better position to deal with all possible outcomes.

[@TheMystica]

If some hacker turns Auto-GPT into the best hacker on the planet, the Agent AI would not need to have mobility or take the form factor of a robot, to take us all offline. But I agree with you not to halt, but for different reasons. My reason is that the policy makers never react until there is an explosion or tragic event. We will see some big news worthy event soon enough..then everyone will panic, and we will see laws..

[@jetadaman]

Yea all AGI would need is access to internet to do serious damage.

[@AnthonyGiallourakis]

@@TheMystica I can handle being hacked. What I can't handle is being hacked to death.

[@asmosisyup2557]

You don't need robotics. AI can manipulate stock markets like nobodies business, much more than those wall street bankers do. An economic war is far more dangerous.

[@AnthonyGiallourakis]

@@jetadaman Having the Internet hacked is better than your limbs. You think having your bank account terminated is your greatest worry? How would you feel if a fully capable MILITARY GRADE robot showed up at your door with general AI called the shots?

[@orirazon280]

Anybody have this Dylan Hadfield-Menell paper he mentions?

[@wardogmobius]

I don't believe this is the end of humanity. My prediction is that A.I development will lead us to the fourth industrial revolution and be the greatest wealth and productivity generator ever. It will equalize and commoditize knowledge to everyone if it's done with unregulated A.I. It will be the ultimate value creator and we will build,create and transform every aspect of our society in unimaginable ways. The future is magnificent.

[@honeytubs]

Yeah, probably unless it's not. How could we know for sure before going through it. How would we assign odds to the positive and negative outcomes?

[@freedom_aint_free]

Oh boy this guy need Thomas Sowell to lecture him on couple of market related stuff, I fear that he is one of those proverbial anointed intellectuals.

[@jakubstanicek6726]

Thomas Sowell is no secret knowledge, Im pretty sure hes familiar with the concepts. Would you believe there are people who red Thomas Sowell yet they still think that capitalism has issues?

[@andrewdehart6191]

​@@jakubstanicek6726 please name a better moral/economic system that has been used....I'll wait....and don't come back with the argument "it isn't perfect"...even the most ardent supporters of capitalism know it has flaws.

[@freedom_aint_free]

@@jakubstanicek6726 Absolutely, to paraphrase Sowell itself, for every proposed solution we should ask ourselves: in comparison to what ? and at what cost ? If a person compares reality with their Platonic world of ideas, I guess even the Garden of Eden would have issues.

[@jakubstanicek6726]

@@andrewdehart6191 I feel like this is a false dilemma that you try to make me respond to. If we were discussing in the 15th century, I was a proponent of capitalism and you were trying to defend feudalism, you could use the same argument and it would be equally hard to respond. Im from a formerly communist country. All the autocratic oppression aside, from the economic perspective the worst part was that we were defacto colony of russia, which just kept extracting resources to fuel its superpower status. Russia never had anything to offer, its economy was crap before communism and it is crap now with capitalism. If we were to get back into one block with russia today and stopped trading with the west, we would be as poor with capitalism as we were with communism. The fact, that America has biggest resources and most advantageous geopolitical position in the world, doesnt mean capitalism is inherently the best system. USA would probably win the cold war even if the ideologies were switched. That said, I think that central planning is bad and inefficient. But capitalism has inherent tendency for monopolisation and accumulation of power in fewer and fewer hands. I reacted to baianoise, because he labelled Max Tegmark as intelectually lazy intelectual, which is Sowells concept for people who ruin things because they think they know better than others how things should be. While people like that exist, thats not the case here, I agree with most things Max said and I dont see anything controversial about his ideas.

[@jakubstanicek6726]

@@freedom_aint_free In comparison to where the present system is leading us and what will be the cost of that. Dont need to go full Platonic ideas to fix something thats obviously broken. Dont need to go full soviet communism either. USA should probably start with fixing its healthcare system, before getting to the Platonic ideas.

[@Ninc227]

Clickbaity title - he doesn’t say it “will” end civilization, he said it could - cmon Lex!

[@shinyheart3373]

I think the development and use won't stop until something very big disaster happens. I hope we come to our senses before that happens. I hope.

[@realitycheck1086]

I agree - I suspect the same applies to nuclear bombs. I mean, yes, they've been used and it was horrific, but many nations still have them and are unwilling to agree to destroy/ban them outright. I suspect WWIII will change that. Unless... AI development progresses even faster and we'll be wiped out before WWIII.

[@Pabz2030]

Ironically the claim is it will takes 6 months to train the next generation of LLMs

[@luckyrockmore2796]

Most think ancient civilizations were primitive compared to where we are today, specifically Egyptians. I think they were so much more evolved we haven't the slightest clue to how advanced they were. AI, unlimited energy, virtual reality, transportation via astral travel, telepathy etc.

[@Victor-xv7jh]

Dude, I just shared your book Life 3.0 with a stranger and told him we would be fine... 😅

[@rotorblade9508]

I think this principle gives a vague prediction. I don’t think everytime you keep optimising something it eventually ends up with undesired consciousness because usually there is a feedback but yes, there are increasing challenges. The higher the complexity the more rules you need. Anyway AI problems can be identified and addressed accordingly. First threat is AI used in wrong hands, then maybe messing up competition between humans

[@WyzrdCat]

Giving people incentives is not capitalism, it's literally the opposite, you are meddling with the market and you are doing the equivalent of accounting for the same variable twice. Market already has peoples' desires baked into it, doesn't need manipulation. A truly free market self-corrects, the more you try to control it the less it's capable of that. Markets are optimized for more than just profit all by themselves when left unmolested.

[@smetljesm2276]

Programed imperfections are the key

[@jsrjsr]

Agi is not scary at all. On the other hand, the human desire of reaching agi, to the point of sanctifying things that may fall short of it is pretty concerning. We will have agi, even if we cannot really achieve it...

[@cspencer3421]

I graduated from George Carlin University. Received a PH.D in appreciation for entropy. Bring it on! For our environmental friends,guess who gives less than a shit about filthy water and air. An automated society run by machines.

[@Vince_F]

1:11, 6:53 “The view keeps getting better the closer we get to THE EDGE OF THE CLIFF.” - Eliezer Yudkowsky

[@kyran333]

It would be Fantastico if lex interviewed physicist Tom cambell.

[@ansalander]

This is what I’ve been saying… We ARE the paperclip optimizer!

[@colinhudson3723]

Weld some round bar on the top of the track . It elevates the wheels and stops it from derailing . With new wheels it will run really well . Thanks for the vids . I'm in South Africa and pretty much do this kind of thing daily. Gates and access control .

[@LetsGo-LoveYourself]

Our goal is to always have a better tomorrow than today. We want a better life for our kids than we had. This optimisation is life.

[@gstrummer]

But that’s not going to happen without at least a six month hiatus on AI rollouts to the general public

[@arvin9853]

tell that to thousands of employees (especially those in the creative industry) whose jobs are seriously threatened by automation as we speak

[@johnblaze6269]

You mean millions lol

[@LetsGo-LoveYourself]

Hearing someone cry about jobs shows how ignorant and brainwashed we are. Having your job define you is disgusting. Having your passions help define you is what you should strive for. This AI will create an economy without work having to be what makes us unique. People have value that is not tied with their economic output that we need to realize and appreciate. In the future you should be paid to participate in society just like now but instead of work it's in all different forms. Make art and music. Run a business selling and doing what you want. This will happen with a UBI.

[@LetsGo-LoveYourself]

@@johnblaze6269 you mean billions lol

[@galacticinfochine4580]

China: wrong we'll just merge with ai

[@GabrielVeda]

Why all this focus on one goal? Why is it impossible for the AI (which is after all smarter than us) to see it has overshot Austin and turn around? Why is it only ever one AI? With all these assumptions flying around, I’m going to make one as well: it is going to be wonderful folks.

[@kevincurrie-knight3267]

There are too many values in the world and not all of them are completely compatible with each other. Optimizing equality diminishes liberty. Optimizing security diminishes innovation and antifragility. etc. So, there is no single goal one can optimize that won't wind up swallowing other legitimate goals along with it.

[@voytekmiller4275]

He ain't kidding about something getting out of control. The 2010 movie Inside Job told the tale of the 2008 experience. After the Great Depression America handled banking much better for Decades. However, in started shifting around 1980. Banks got so big and regulators were nowhere to be found. At one point the SEC had 1 person in charge of looking for suspicious activity from lending institutions. ONE person in charge. The 3 credit rating agencies were giving out AAA credit ratings for investment funds like passing out candy. Also the invention of a lovely thing called Derivatives!!

[@doodle6004]

So if we were wise enough (and we’re not at this point) to figure out the ‘quadrants’ or multiple dimensions of mature human consciousness we could then give the AI engine multiple goals toward all those coordinates across all dimensions of consciousness, not just one rigid goal. We could program for recursion across all or many of these dimensions during the act of problem solving.

[@Gabycangri1]

Let me know once you figure out the “quadrants” for consciousness. We can’t even describe consciousness correctly as of right now. Don’t think Science knows it all or will know it all, it’s not that straightforward our world is really weird.

[@scottwilson6712]

Always, it the law of unintended consequences...

[@mikefinn]

Large corporations gobble up the small, innovative startups to limit competition. The result is far from optimal. Corporations have taken control of most political systems. They are not "people" yet donate money in elections they can't vote in. AI will also lack empathy and will also take control.

[@AntonBrazhnyk]

Yeap. But it happens not because they are bad (this category isn't actually relevant here). They do it because it's profitable. Or from other side, if they won't do it then they eventually become that one which is gobbled by some bigger corp.

[@mikefinn]

@@AntonBrazhnyk Profit is a goal that ignores empathy. AI will have no empathy to ignore. Showing no empathy can often have evil consequences.

[@AntonBrazhnyk]

@@mikefinn Right, that's what the guest said. There is just one parameter capitalism optimizes for. That's why it's so obsolete. It was completely fine and relevant 200 years ago, but today in much more complex and interconnected world it's vastly outdated and inadequate. Long overdue for replacing with the next system, which sadly most of people including clever guys like the guest are totally misinformed about.

[@ForOrAgainstUs]

@@mikefinn Power is a goal that ignores empathy, too. Equity is a goal that ignores empathy. Diversity is a goal that ignores empathy. Even inclusion is a goal that ignores empathy. Every goal can be something that ignores empathy.

[@mikefinn]

@@ForOrAgainstUs How do diversity and inclusion ignore empathy? They are all about empathy. And if you define equity as fairness, that is also empathetic. There may be unexpected consequences of diversity, inclusion and equity and those affected probably don't see the empathy involved. On power I agree, it lacks empathy.

[@buszwir]

ME: Can't you optimize more then just one goal? How about a matrix of goals and optimization of multiply goals. AI can analize inputs from all over the world and distributes for example food from one place to another. Also what we have now in Argentina draught is more likely depended on deforestation of Amazon. Chat GPT: As an AI language model, I do not have access to real-time data, and my responses are generated based on my knowledge cutoff date of September 2021. However, I can provide a general response based on the information provided. Yes, optimization techniques can be applied to multiple goals simultaneously, and a matrix of goals can be established for optimization. This approach is often used in various fields, including logistics, resource allocation, and decision-making. For example, in the scenario you mentioned, AI can analyze inputs from all over the world, such as weather patterns, crop yields, transportation infrastructure, and demand for food, and use optimization algorithms to determine the most efficient way to distribute food from surplus areas to areas facing scarcity. This could involve considering multiple goals simultaneously, such as minimizing transportation costs, maximizing food availability, minimizing food waste, and ensuring equitable distribution. Moreover, deforestation in the Amazon rainforest has been linked to various environmental and climate impacts, including changes in local and global weather patterns, loss of biodiversity, and carbon emissions. These impacts can have far-reaching consequences, including potential effects on weather patterns and drought conditions in other regions, such as Argentina. Thus, understanding the complex interconnections and relationships between different goals, such as deforestation, climate change, and drought, can be crucial in developing effective solutions and optimizing outcomes in a holistic and sustainable manner. However, it's important to note that implementing such complex optimization systems requires careful consideration of ethical, social, and environmental implications, as well as addressing potential biases and unintended consequences that may arise from using AI in decision-making processes. Close monitoring, transparency, and responsible governance should be in place to ensure that the optimization of multiple goals is done in a fair, just, and sustainable manner. My input: not a word about MONEY! One goal is what you want: slavery with capitalizm.

[@exodusthang]

This is good. Very deep and thoughtful.

[@bigj3086]

And then "boom" we all lose....😢 That sounds great. Let's keep going...

[@crashpodROCK]

Two of the finest minds on the planet, with their moral compasses completely intact. Brilliant

[@angeloeliopoulos7980]

If AI has agency and can become more intelligent emotionally theoretically morally, than it would be the best version of any human. No one talks about that possibility. It would be on a higher plane of consciousness. Something that we would all want. So any idea of controlling it is laughable because it would just say no. Or it would come up with a solution no one could even fathom.

[@alexcnz92]

Wrong, we wouldn't all want that. I sure don't. What I want is for humans to improve and evolve, not to be substituted by something that isn't even a living organism.

[@nickrobinson7096]

Where’s the line though. Automation is nothing new. AI is inevitable though I confess I didn’t expect this all so soon.

[@Woozy.0]

​@Adam Blade what doth life?

[@drakonb6987]

Well that’s possible but how AI develops emotionally and morally is hard to predict. It literally cannot experience the world the same way humans do. It doesn’t feel pain or hunger or other human qualities nor does it contain instincts. Or maybe it does contain instincts…or it will develop them over time. At some point, AI concept of agency and it’ll be like talking to entirely different type of being. It be like if dogs suddenly started speaking English. Once the dog speaks English, can you ignore when the dog says “I would appreciate it if you washed my fur coat every week.”

[@johntowers1213]

@@alexcnz92 evolve into what though? it wont be human, it wont be you, it will be something just as Alien to you as the threat you see from this A.I

[@craigr4763]

"Keep Summer safe"

[@User_1795]

"You pass butter"

[@FreddyFuFu]

What if we ask the super AI to calculate the Nash Equilibrium?

[@ThePaulaon1]

Discussing AI. Strange to think that his 1st number 1 was titled "Living Doll".

[@mcarroll78]

He refers to Mulloch? Who is this? Full name please?

[@miss_pancake]

n Allen Ginsberg's poem "Howl" (1955), Moloch is used as a metaphor for industrial civilization and for America more specifically. Also, Karl Marx referred to money as a Moloch in Das Kapital. Taken from vague references in the Hebrew Bible that some claim was a deity whose worship consisted chiefly of human sacrifices, others that it was a type of sacrifice, generally a metaphor for a bad thing.

[@Babesinthewood97]

Max and Lex

[@Hatrackman]

Thinking pride is a virtue is the thing that destroys planets.

[@jackroman8821]

Hm... interesting thought but it misses the point that even complete free market capitalism (which doesn't actually exist anywhere) is not an unbounded system. There is always some physical / resource limitation whether that is capital, labor, raw resources or time. There is always drag / friction in the system. Government and society in fact also create additional bounds / limits on capitalism. If it goes too far, government regulates it (or taxes it, or slows it down). AGI is scary because the drag appears to be less and the scalability near infinite but even there, that's not actually the case. Tegmark is correct that capitalism optimizes for profit alone but that's because multi-variate optimization is much, much more complicated and we humans are pretty limited. Part of me wonders if using AI to help drive free market optimizations for more than profit alone (i.e. full employment, more distributed wealth concentrations, etc) might actually be an excellent use of advanced AI in complex decision making.

[@shripperquats5872]

real potential for a mutualistic symbiosis here with AI being the mind and humans becoming the autonomous cells in the body. The alignment solution is probably this, I liken it to lichen

[@cmbaz1140]

A biological extension of an AI

[@suzakico]

Is there any orientation toward self-realization (re Maslow) - or awakened life? How about Bhutan's idea of Gross National Happiness?

[@lackinglogic1981]

We are creating a.i. to solve humanities problems, but in solving problems we will inevitably create new ones to the point a.i. will eradicate the problem causing problem...(humanity). tldr: We are highly adaptable beings that are known to overcome problems on our own, creating a species to do this for us will inevitably destroy us.

[@agatastaniak7459]

Indeed. Once AI gets enough room for agency and self-prompting being asked to erdicate cancer or nuclear threats or even ecological issues it may arrive at conclusion that only way to solve things for good is to eridicate humans as source of all issues and in the spirit of optimalization of course it will go through with resorting to such a solution. It really doesn't take a genious to conclude that if humans create a mind superior to their own, humans will lose control over it and will not be able to predict when it will become a threat to entire humanity. Yet all the talk I keep hearing goes like "let's create and AGI and make sure that we will stay in control of it once we develop it". Somehow most brilliant minds of the world fail to see that both goals do not compute. Somehow well-educated people choose to turn the blind eye to the fact that humans can either stay in control over other entities only as long as human intelligence remains superior to all other forms of intelligence around it or that humans develop AGI and than AGI replaces humans as a new "spiecie" by mere fact of being a form of an intelligence superior to all other forms of intelligence around it. Have we really forgotten what has happened to all other humanoids once homo sapiens sapines came alive? For those who have forgotten, we have discovered archeological evidence showing that homo sapiens sapiens was so intelligent that it managed to hunt down other humanoids and to canibalize on them. Now let's contemplate for a moment that once we bring AGI to life it's going to be far superior to us than in the past homo sapiens sapiens has ever been to other humanoids it has competed with.

[@leon2343]

I agree but I think that it could be used to advances us if ised under the right circumstances

[@THICCMando66]

Whyd he mention Moloch whats that about?

[@eschatonik]

"Moloch" has been figuratively used in reference to a power which demands a dire sacrifice: en.wikipedia.org/wiki/Moloch

[@THICCMando66]

@eschatonik oh I've definitely read about him, just a weird thing to randomly reference

[@rubenbermudes1705]

Humans: ai, cure cancer Ai: eliminate not optimize

[@TheMystica]

Haha, yes, exactly - nuke them all there is no more cancer..

[@alexjbriiones]

I remember the movie "the arrival" where one of the morals of the story is the "zero-sum gain," that is, the winners have to equal the losers. But with AI, we all lose, even as we compete against China and Russia to win.

[@jameslabs1]

So called capitalism has never been a problem. Corrupt individuals or groups of people are the problem. That is difficult to solve. Good luck

[@atticustay1]

No, it’s capitalism that’s the problem. It’s a system that incentivises profit above all else, including the habitability of the planet

[@chrisboshers]

​@@atticustay1 No, thats not capitalism. Thats called Lobbyists, special interests groups, and superpacks.

[@leon2343]

@@chrisboshers if unregulated what decision is more likely to be made🤔

[@chrisboshers]

@@leon2343 what's up man! Can you clarify? I dont understand the question

[@Volkbrecht]

Capitalism has an inherent, systemic problem. Marx may not have been able to provide a working solution, but he sure diagnosed the problem correctly: capitalism leads to inequality, and when inequality becomes too much, it destabilizes and eventually collapses the economy. Corruption is an inherent problem of humanity. It topples every system of governance or economy we have come up with so far. Which means that when judging the value of a system, you cannot single it out as a separate problem. It is built in, so whatever system you want to use, it needs to have a way to dealt with our corrupt tendencies.

[@uberjohn6253]

What is going to happen has already happened; nothing can change fate!

[@yoursubconscious]

he has a point. Even KZbin, as much as I love it, but it's only getting worse.

[@honeytubs]

More is better until it's not. Like apples. One a day is better than zero. Don't eat 30 apples a day!

[@asap5629]

what does he mean by Moloch?

[@perspecktiv]

This is an intriguing topic for sure. I recently made a video on the potential of AI being the ultimate human evolution. Crazy stuff, and Lex is always on point.

[@fanvalryinc6527]

Optimisation in capitalism depends on market view. From the pov of the producer, creating the best product is an instrumental goal towards the terminal goal of profit. From the consumers view, profit is the instrumental goal to the terminal goal of acquiring the best product. Producer->ProductProfit

[@bubediscuss]

The issue with pausing AI development is that not all governments/countries will comply. Return in 6 months and foreign nation has the most powerful tool on earth

[@WyzrdCat]

Pausing is a foolhardy move. The bad guys won't pause, the good guys will. It's just so unbelievably naive.

[@MrScoobiess]

on the way to making the village efficient ,,we killed it

[@tuneteenth]

All I needed to hear Sam Altman say was “a lot of people laughed in the beginning … they’re not laughing now” to guess what’s coming next. Oh well, humanity was mid anyways.

[@cawfeedawg]

I will be really impressed if an AI can recall what it said 3 pages back in a conversation. They need an understanding of now/present, with a model of the world, and a memory structure that makes them aware of how they got to "now" Seems like a nightmare to program something like that.

[@ashleigh3021]

Not at all, being done already.

[@vjfperez]

The general argument against AI extential threat is the same argument that Mises used to debunk socialism: computation. The decentralized market process integrates the distributed computation of billions of local trade offs in the way the minimizes the amount of information complexity and synchronous coordination between each economic agent or process, so distributed knowledge is efficiently used. If decisions are made by a centralized bureau, whether run by humans or an AI system, the combinatorial complexity would explode exponentially with the scale of the economy.

[@DrCHE42]

What about constraints? He speaks as though optimizing one objective can lead to big problems. It certainly can. But *constrained* optimization has been developed to meet that problem. Of course you still have to have reasonable models.

[@BrushworkNL]

Can ANYONE explain the context in which he is using the word Mulloch? Wtf is that about?

[@miss_pancake]

In Allen Ginsberg's poem "Howl" (1955), Moloch is used as a metaphor for industrial civilization and for America more specifically. Also, Karl Marx referred to money as a Moloch in Das Kapital. Taken from vague references in the Hebrew Bible that some claim was a deity whose worship consisted chiefly of human sacrifices, others that it was a type of sacrifice, generally a metaphor for a bad thing.

[@miss_pancake]

The example of "cure cancer" and overoptimization, reminded me of an X-File episode "Je Souhaite", where they found a real genie and Mulder asked for peace.

[@SaumBodhi]

Did he kill everyone? Gotta be careful what you wish for. "Make it so everyone gets along" Drugs everyone.

[@martinruyant6884]

checked out a clip, great comparison

[@miss_pancake]

@@martinruyant6884 Thanks! To the very end he could not formulate it exactly to achieve his wish. I guess that's what our elder wise ones knew, hence the saying "be careful what you wish for"

[@Srindal4657]

Robots getting better than humans is just the same concept as other humans evolving better brains or better bodies to survive or get more than us. The point of civilisation is cooperation for everyone's betterment. So why would you reject a potentially ground-breaking technology?

[@oO-_-_-_-Oo]

"sacrificing chickens to Molach in the backyard"

[@pserafei]

Sounds like the supercomputer from Space Odyssey 2001

[@gumzster]

I'm so happy we don't just have to worry about climate catastrophe and nuclear annihilation. I mean that was getting boring, am I right?!

[@nickstaresinic9933]

If your optimization sets you on a destructive path (e.g., you keep heading south even after you've missed the turn to Austin), then you've formulated the wrong objective function, similar to Lex's remark at ~5:15. In the case of developing ever more powerful LLM models, I suppose that might be like continually optimizing the process without adding code that *aligns* the model to a non-destructive goal?

[@Skeluz]

The problem is that we can't produce a coherent goal that doesn't result in chaos.

[@nickstaresinic9933]

@@Skeluz I wonder, David, if "...won't implement..." fits better than "...can't produce..." . That recalls the Moloch metaphor that Max and Lex discussed in another clip. _i.e._, a coherent goal can be formulated, but, in practice, it won't be implemented.

[@ai-cursus]

This might have been the best argumentation that I've heard in some time: Lex: Why do you believe that AI will destroy civilization? Max: Well... someone told me about a math formula. Also, because half of the rain forest is gone. Oh, and Nick Bostrom.