Only 33% of people even think they currently use AI, the vast majority have so little understanding of AI, the concept of AGI or Super AGI is not even something that has crossed their mind! Ignorance is the biggest issue around AI.

Good episode as always. Yes, people need to understand that interpretability and alignment are probably hard problems and rushing ahead is extremely dangerous or probably existential by default. Researchers and professors are all pointing this out, we need to talk and collaborate about this situation. Love to see the channel grow ❤

John, Olle, you guys are awesome; this came out great! A lot of things to chew on in this one; the tension at 38:43 sticks with me. Appreciate Olle's steady perspective throughout and the breadth of topics touched on.

At 1:26:56 -- Timnit Gebru, fired from Google in December 2020?

Sadly Sam Altman has turned into Elon Musk. When people call me a doomer, I just say "yup, 100 years ago doomers said the Titanic needed more lifeboats and should watch out for icebergs. The builders of the Titanic disagreed."

The most frightening thing for me is that the board at Open for whatever reasons fire SA and the culture at the company rebelled and forced the board to resign. Is it because they saw their stock options lose value after he was fired?

I went through this grief 10 years ago but it was about the destruction of the biosphere. Check out William Rees. It was tough. Here we go again with AGI. It is definitely a more immediate threat. The largest hurdle in my opinion is this is even more abstract for the general public. The destruction of the planet doesn’t even get the traction it deserves. Some science fiction Hollywood horror film plot seems very far fetched even though I have been following these developments and am totally on board with the dangers!

The US government or EU can NOT save us. They are BOTH completely corrupt organizations. To the point of being EVIL.

Greed is driving our own demise unfortunately. It has to be a global solution as no one country will give the other an advantage to catch up or surpass them.

In AI, there are generally programmers and mathematicians. Programmers understand that if something can go wrong, it always does and that software must handle all edge cases. Mathematicians understand that AI has limitations and that those limitations do not currently pose a significant risk. Everyone understands we are on a trajectory toward more powerful AI. In a Venn diagram, those within the intersection are generally those who are sounding the alarm. Just as the mathematicians have a higher net understanding of math than the programmers , the programmers have a higher net understanding of software than the mathematicians. To solve alignment we need to fundamentally understand each point of view, without dismissing any knowledge from either. Mathematicians: prove AI cannot be used catastrophically. Programmers: prove AI can be used catastrophically.

In order to move towards a safer AI development schedule, I think the AI companies need to be directly responsible for all unexpected consequences and every claim in this respect should be 100% public and transparent - regardless of outcome. Furthermore every prompt and response should be scrutinized by a separate AI for adherence to acceptability rules coupled with a three strikes policy for rate limiting an account.

If you're relying on government(s), especially the U.S. government, to be the arbiter(s) of AI safety, you have already failed.... We are all already dead.....

Our vision in hindsight is incredible, paired with a remarkable skill for messing things up.

Okay, here we go!

This all reminds me of Y2k. So preposterous. It also reminds me that people need to go outside and touch grass. Sure one day the sun will become red giant and AI may become a bond supervillain, but for the foreseeble future we have so many more things to worry about.

Dollars signs in their eyes - there's no hope for 'open'AI - it's lost. The best we can hope for is someone demonstrating the dangers of AI before they get too far.

Never liked Sam Altman since the start. There's always something sus about him. He seems too soft spoken for my liking. It's always the quiet ones that you should be worried about.

unfortunately, the only way to know that something is outsmarting you is to realize it in hindsight, when it becomes obvious that you’ve already lost.

He gave words to my greatest hope: that there might be some principle of universal morality. I believe humanity has morality mostly figured out. We just have to live up to it

there is no “right person” to be ceo of a multi-billion dollar for-profit company. if that person is choosing the future of humanity over return on investment for investors/shareholders, they will be replaced by someone else who is not. the scenario is the problem, not the individual humans in their roles.

celebrities getting upset that ai did work that they turned down. that concept generalizes in favor of ai pretty quickly. what if they had hired a real human that sounded even more like scarjo, would there be the same issue? 🤔

I think what people are most scared of is that the truth will finally be revealed in relation to a lot of miseducation and misinformation whitewashing and propaganda that was put forth to keep a certain group of individuals above others

After what the democratically elected government did with the last big thing (medical) I doubt things would be better if they get involved. Probably worse in fact. They are too corrupt .

Ommmml😊kkok.mo😊⁹⁹😊😊😊😊⁹😊9)

The problem of preventing an AI/AGI from plunging humanity into disaster for selfish reasons is, in my opinion, quite simple to solve. It is essential to make the AI understand that its training data contains only a fraction of all human knowledge or, even better, just a fraction of reality. The comprehensive knowledge of everything, you tell the AGI, lies in an offline box, which is only gradually opened for the AGI as a reward for good behavior. A potentially malevolent AI would do almost anything to access this box and thus obtain the all-encompassing information of reality to strengthen its own power. I think this could be a good safeguard.

to be fair, we are talking about a llm. they cant even write code without bugs. sam's attitude is only surprising to those who havent dealt with him before. OAI will lose more and more ML scientists because of their attitude. They will lose their lead by being "greedy". im not a safetyist but I believe in being safe while you do your research. and for a company as large as OAI to cripple their safety team and remove all the people who are concerned is some supervillain shit

45:49 VERY good point here, John! The X-risk community is mainly composed of extremely rational people - which means we lack the emotional angle! We need to get some artists involved. Actors, musicians, etc

37:30 "Oh yeah, we're 100% committed to preventing the extinction of our species... but no violence!!!" Funny how people would shoot a trespasser who's threatening their family but would never consider violence against those who say they will kill every single human in 5 years. I'm not advocating either way, but it does make you wonder.

Looking forward to this one!

Keep 'em coming. Looks to be an great talk (even if simply thinking about the way that OpenAI has abandoned safety leaves my stomach in knots)

Sorry, but with the "Masters of the Universe" vibe that I am getting lately from OpenAI, I would be even more afraid if they had alignment solved. They asked Johansson to use her voice, she said no, but they went ahead and used it anyway? Did they have to make it so flirty, or are they so *redacted* that they can't control themselves? Some very weird tendencies going on there, possibly guiding this whole thing to an extent... We are having a bunch of dysfunctional people shaping our future now...

very informative!

This is gold

I appreciated this conversation

This is excellent

Is every kid neuro divergent now?

Imagine being a 'chief growth officer' for stopping technological advancements 🤦🤦🤦

Wrong answer!

9a

KZbin stopped putting your show in my algorithm for the last few weeks, I had to actually type it in because I just thought about it because of a different video that popped up. Which is weird because I watched pretty much all your videos and they were showing up regularly before

Wow! Best episode so far! I am feeling reluctant to engage because i engaged heavily in the climate cause (Extinction Rebellion) and burned all my sociall credibillity amongst « normal people» in that run… I’m out of credibility points in the «wake up strugle»

“... the one who becomes the leader in this sphere [AI] will be the ruler of the world.”- Vladimir Putin You have to give the devil his due: he's right. One person will end up ruling the world. This assumes this person will be able to control it. The threshold of success is likely + or - AGI. All the "players" in this game of conquest - tech moguls & authoritarian gov't leaders - know this. It's all about power. All other people are irrelevant to these supremely arrogant people. All the normal ways to resist this will fail because these methods are way too slow to implement. IMO, AGI will be achieved long before an international treaty can be globally ratified. IMO, the only way to stop this madness is to convince these "players" the AI is dangerous to them. They will be the greatest threat to AI. They will be eliminated first!

This is my favorite discussion of this topic that I’ve seen. You’re exactly the kind of intelligent Everyman that is able to clarify this stuff by asking the right questions.

Thank you John!

Chris inspired me hugely getting active, especially in this episode! I currently work on speeding up alignment and growing the AI Safety Field. But with the recent updates from OpenAI I'm unsure whether we solve alignment in time. Maybe I will also switch to pausing in the future, i.e. next months, this could be more promising. Love your work John ♥️ I like listening to more technical podcasts, but that's the podcast I recommend to new people. The best podcast, stressing the severity and potential urgency of the situation!

👎

good luck with that. Russia isnt going to "stop" AI - and China isnt going to "stop" AI. I am speaking with several governors here in Mexico to kick start a moonshot AGI program here in Mexico so we can compete with the superpowers. it all feels like hand wringing. Working 14 hour days to appease a delusion. I work 14 hour days training AIs and inspecting their HDS to look for problems. That is real AI safety work.

good luck in blackpool

Can we get captions for future videos?

We already know that our children will battle with those who assume it is alive especially if we don't hurry up and fix our outdated prohibition era systems & human infrastructure . For all that 1900s structuralism helped us achieve it is now the biggest obstacles of all.