16:12 Not sure I follow the intuition behind supervised fine tuning not being able to penalize the “wrong” token that is opposite to what we want the model to mimic. I’m confused because in my view the wrong, but highly probably token, contributes more to the loss so it will be penalized heavier than the more meaningless, random output, tokens. Can someone clarify this for me?

It's so embarrassing these execs are all still using twitter to communicate, it just leaves such a amateur hour and sort of edgelord feel to everything.

why not host a 7B model on HuggingFace?

When I subscribed to this channel it had one of the best selection of relevant AI papers paired with their analysis outlining the important thoughts. So it was a cool way to stay ahead of some broader research trends. Now it is becoming one channel more for AI related business news. This certainly mirrors an unhealthy development which Yannik has repeatedly mentioned: AI moved in the last 18 months from an open research field to a business field with competing companies using tech information mainly as marketing tool. But the intellectual emptiness of all this is tiresome. It would be great to get more scientific content again. And as long as there is no way around the commercial llms, maybe reports on rigorous tests and similar research is a good counterweight against the stream of marketing hysteria. I seriously miss the old channel.

Is the time 1d conv also a multi channel conv or they convolve each feature map with a single filter ?

Auto generated chapters that might be inaccurate but maybe still helpful: Srun (1:26): A tool to copy a directory to a server via SSH, run a script, and then copy back a directory called logs. Rat (Experiment Management System) (2:26): Manages experiments by using a worker to listen for new tasks, pull code from MongoDB, and run jobs on GPUs when they are free. Includes hyperparameter optimization and integrates with TensorBoard. Yplot (5:31): A plotting library that works with TensorBoard events, extracting and smoothing data. Ypack (6:02): A set of routines for working with Torch and TensorFlow. Confprod (Configuration Product Generator) (6:23): Generates cross-products of experiment configurations. Fountain (7:15): A dataset library for downloading and extracting datasets like CIFAR-10. Arxivciter (8:12): A plugin to scrape Google Scholar for citation entries directly from arXiv articles. Textools (8:31): An unsophisticated tool for continuous compilation. Arxivget (8:37): Scrapes arXiv to read new papers every morning. Patternhs12 (9:17): An implementation of SVM classifiers, including linear, polynomial, and RBF kernels, from a pattern recognition class.

8:55 BOTTOM-UP-AI ... Alicization arc confirmed!

Anyone who titles their videos like a fact checker deserves to be judged.

I really liked this video's takes, until we got to the Skye situation. In the OpenAI blog post which you referred to in the video, they themselves stated that they were cast and recorded the ChatGPT voices over the summer, which lines up with them being released to ChatGPT Plus subscribers in September. So when they reached out to Scarlett, the Skye voice had already been created. They could be lying about their timelines, but I'm inclined to believe them because I, and many other plus users, first heard Skye's voice in September when the feature first rolled out to us. You seem to have very interesting / insightful videos as a whole, so I'm a little surprised that you missed this detail, especially since you did a pretty good job of covering both the optimistic and cynical takes.

Really??? What a waist of time…

Without you I would struggle to understand these papers. Thank you

What if the tone is like in the movie Her? Then you can't do it in real life? So we can't build space ships because Star Wars would sue? Or flyng skate boards because Back to the future would sue? Also why is SJ the one suing? Does she have the rights to the AI character in Her? I get that she woule want to protect her voice, but the idea that she would gate keep a concept, enthusiastic/flirty AI, is ridiculous. By the way I don't think flirty AI is a good anyways so this is not me stanning openAi 😅

ScarJo has apparently trademarked the female voice. All other women should keep quiet.

Horrible paper. Riddled with hand-crafting which it is supposedly replacing!

time for a spinoff coming from OpenAI, that's what I see, that which is out of closet and overtly commercial, that way, research can go on in left over brand.

Petition to replace reviewers with AI

Yeah. Hogwash. That's what these concerns about superintelligence are. At least for now

I miss Sky. About alignement, i think there should be research on how to control and align ongoing in parallel but agree we are nowhere near having to worry about implementing it. And it shouldn’t be a blocker in development or deployment. Bad actors will figure out how to get ai powered weapons. There is no stopping that. And that will happen waaay before a rogue AI is possible.

16:28 There’s an argument that the AI Safety grift was the first monetization round. Given no “AI Safety” ever mention things like the dangers of prosecuting journalists and whistleblowers across the west, they are worse than useless.

when I saw the demo, my first thought was "sounds a lot like scarlet". what is messed up is Sam is obsessed with "her" and decided to go ahead and copy her voice. I think of it this way. If it was my daughter, how would I feel about a creepy AF tech bro making a voice like my daughter. Sure he isn't going to play some creepy voice fantasy with this voice. when scarlet rejected Openai, they should have stayed clear and just chosen something unique.

yannick can you please do xLSTM paper?

In inner loop, for more than one step the gradients should be computed wrt initial parameters right?

Superalignment is a myth. Human values include eliminating other human competitors.

The more I learn about Revux, the more I believe in its vision.

Qventi is everywhere now, just like Revux was.

I see Revux as a long-term hold. Great fundamentals!

If you're looking for a solid x100, Revux should be on your radar.

Revux's use case is exactly what the crypto world needs.

Shopify partnership puts Revux on the map. Big boom incoming!

Dropped all my small caps - focusing on BTC, Revux, and maybe some ETH.

Just shifted my focus to presales, especially Revux. Early investment is key!

Revux is making moves quietly. Keep an eye on it!

13:00. I don't buy for a second that Altman was not well-aware of this onerous provision. Gaslighting prick. Also, alignment is less of a problem if _integrations_ are undertaken carefully, and with failsafes in place. In other words, *just don't integrate it with critical systems* without ways to _disintegrate_ it from them in ways that the AI systems themselves have no control over. Not that alignment isn't important, but it's just f'ing stupid to *not* approach all integrations carefully, and with worst-case scenarios accounted for. The SJ voice thing is bad faith, whether its a reproduction of her voice or not. Whether or not they legally can is beside the point. Just because you _can_ doesn't mean you _should_.

Everyone and their mother is suing OAI. 🙄 They are controversial and have deep pockets, so people find excuses, even bad ones, and jump on the dog pile.

openai все больше напоминает волка в овечьей шкуре, хитрого и беспринципного

I've come to the conclusion over the years that many people talk in code they'll be talking about one subject that has in some way to touch base about another subject when the reality is they're trying to discuss a certain series of things that they don't want to talk about as a direct identification an example of this is dealing with the dangers to ai and the dangerous to the companies themselves among these were looking at the ability to mass track cyberattacks literally worldwide but if we just focus on the United States there's 1.4 billion attacks per day there's 10 trillion dollars a year in damage occurring in the United States Europe effectively has the same problem they have a 10 trillion year damage from cyber attacks occurring in Europe the ability for large AI systems to use very few specialized hubs to triangulate which buildings are housing these these Mass cyber-attacks systems could be done in a very short period of time I know this because I explored it myself with chat GPT working with the mathematics and the styles and certain types of security procedures to be able to quickly identifyan effectively be able to install 20 custom hubs in the United States and in 30 days effectively triangulate where 80% of the attacking servers would be located please keep in mind that we're talking about companies that are gray buildings that nobody really knows what they do that for some reason have hundreds of thousands even a million computers operating that basically are dedicated to doing cyber attacks the simplistic reality of the statistics of 1.4 billion attacks a day occurring in the United States that were also at least 1.1 billion attacks of origin from the United States brings about a series of mathematics that we're talking about millions of computers dedicated to mass attacks there's simply no way around this and thus the reality that some companies are concerned that they have capabilities to bring forth Mass Justice actions means that their leadership and companies themselves could be in danger from criminal elements very well funded. Additional testing that I've done with open AI computers is to check the accuracy of how fast they can identify misconduct and felony behavior out of government employeesthis effective research is a key part of the presidential campaign that I was running from 2008 in which basically a company such as AI being dedicated stepping out into an ethical analyzation of the country could effectively bankrupt the federal government and every state by apprehending all criminal conduct behavior in less than 30 days the statistics are actually realities and we have many people in government that are afraid and we also have many people in the industry that finally understand some of these aspects that are also concerned about making clear that somehow they were take no part in any form of ethical behavior which could deal with the retrieval of information involving Mass felony conduct by government employees or elected officials. This effectively is the secret attitude of many people who are cowardly and afraid that for some reason that their family and friends might be killed buy these governmental corrupt entities and their private backers. This of course is ridiculous because the reality of SAR satellite technology has been able to track a blade of grass since 2008 at about 30 frames a second upgrading to 60 frames a second by 2011 this effectively means that just with the satellites already in place every citizen in the nation as well as most parts of the world have a satellite looking at them and recording at least 70% of the signature that they reflect as they go through life obviously some people would argue that it's closer to 90% since you have satellites overlapping in many territories which is true as well. However frankly the reality that there is no escape because of the mass deployment of this technology add its current point means basically that nobody should be afraid are these criminal and unethical types because there's no escape for them not when SAR satellite technologies involved. Thus again we have the ability for Mass conversion of information from satellite that can also track criminal activity rapidly so there are many people that are afraid of the masses of people that are frankly ignorant and dangerous even to themselves. Hero of now mentioned a couple things that are kind of the hidden elephant in the room the many people don't want to deal with. But enough of this great show by the way

Yeah, sorry Yannic, but Scarlett's voice is just a normal female voice. "Sounding identically like" Scarlett is 100% legal as long as her name and image is not mentioned in the product. The only reason why Sky as been removed for a while it's because it's probably instructed that it is an female AI. and It saw the movie Her in the database. But there is no legal case there.

Yup...

The Age of Surveillance Capitalism, in which OpenAI takes a major role nowadays. As the NYT reported, their product would not exist without accessing data they had no right to.

I really do not agree that we are far and safe from seing dangerous super AI. And seeing researchers leave should makes us concerned.

What about GPT5?

I think the voice problem isn’t really the voice it’s more that he explicitly called out the movie in his comms. I worked for a company that used a Hollywood movie name as the internal project name for a PC motherboard. This name leaked and the name was bandied around the internet. We were sued by the studio and lost. We also hit sued by Frank Zappas estate for a very similar reason. This is why companies have legal😊

ChatGPT down today, all fked up.

I share your opinion about AI. I don't think in any way AI is a danger right now. It is progressing fast. It has a very huge potential. And open source should be the way. As you mentioned. Llamas are open and no one destroyed the world and even if it was 1000 times more capable, still, no one would destroy the worlds because the laws of physics are clear. To destroy the world you need resources and it is not just intelligence.

Why would they even want Scarlett Johansson? They should have asked GLaDOS' voice actor!

the actual AI danger is drowning in bullshit, but that's a demand side problem so...

Imagine you wanted to avoid dangerous levels of greenhouse gas emissions today and had access to a time machine. Surely, you would want to travel back to a time as early as possible to maybe the first prototypes of steam engines and hand over the blue prints for clean air filters and push for everyone to implement them from the beginning. But if you explained to the people in past why this is necessary they wouldn’t believe you because they couldn’t imagine that these early prototypes eventually become so widespread as to cause global issues. And they would be kinda right to distrust you, since their early prototypes were so primitive that none caused major problems. But where else would you intervene except as early as possible? As soon as steam engines are widespread enough to cause harm, changing them becomes a lot harder and people will oppose the associated costs. That’s the situation that we are in with AI except there is no one from the future to tell us what we should do to avoid danger. We have to figure it out ourselves ASAP while the world pushes forward with rapid adoption. That’s why we need safety research and regulatory oversight with regard to capabilities. (Btw you don’t have to consider climate change an existential risk to understand the analogy I am making)

4o is not targeted to be the new 5. It is the new 3.5

Sam Altman is so megalomaniac that he even calls himself "sama" in twitter [face_palm]

It is funny you say that openai do whatever "misinformation" and "deep fakes" they desire "suddenly" as an anecdote immediately after the story of the guy who cared about the safety issues leaves