GPT-5 Rumors and Predictions - It's about to get real silly

[@firerabbit2659]

I went to dinner and came back to three LLMS, this is all happening so fast that it's getting a bit ridiculous. Can we breathe?

[@DaveShap]

"I went to dinner and came back to three LLMs" 🤣 like you had kittens

[@TLabsLLC-AI-Development]

No breath for you!

[@Bhavyo]

breathing is overrated (Chat GPT told me that;-)

[@spinninglink]

oh, it's just starting. Once AI is in charge of research and advancement, it will happen faster than any human can keep up.

[@SpiritTracker7]

Why would we need to adapt or adjust to your capacity? I say bring it on.

[@svitii]

Looking at AI a year ago I thought "I should really get into computer science" Now I can only say "Well, guess I'll do philosophy cause it looks like that's the only thing that might be left when I will finish college"

[@mlmg0817]

Philosophy is easy for AI, it will be in the first batch of courses to fall. Agriculture or masonry is the least. Those that need human labor. Obviously this will be the final setup; AI's will be the commanders and humans will be slaves.

[@mnrvaprjct]

@@mlmg0817y would humans be slaves when running society would take like 0.0001% of these machines intellectual resources to run a damn near perfect society. They’d be doing far more interesting things than making us slaves… that’s just dumb doomer shit

[@wakkawagga443]

I think for a while everythibg that is dependant on human to human interaction will be the jobs we will be reduced to. „Prompt designer“ being the new hype job. The amount of robotics we will see in the 2030s will be humongous. The good news is i now believe we will find a solution for climate change. And with we i mean gpt-7.

[@remingtonwilbur8174]

Nah man. If this guy is right about AI, philosophy, jobs are a thing of the past. At least given a few decades.

[@TheChadavis33]

I don’t think people realize that nothing is off limits. We are staring at the abyss

[@benpielstick]

Custom novels on demand, up to 256k tokens, maybe even with AI narration for unlimited audiobooks in GPT-5? Maybe with AI actors even custom movies on demand? A few small steps from that is video games made entirely by AI from a prompt.

[@DaveShap]

Yep

[@Jay-eb7ik]

This so much! I want novels that cater to my preference and style and to create entire worlds I can dive into. I'm most excited for this.

[@alkalomadtan]

Imagine you will have thousands of _almost good_ novels for you to read. Your entire worlds with annoying glitches and stereotypical conversations. You will hate them eventually.

[@Jay-eb7ik]

@@alkalomadtan Imagine you have thousands of worlds where there are no glitches and the tech gets better and better with new tech breakthroughs that generate complex dialogue trained on billions o conversations. You'll love it.

[@alkalomadtan]

@@Jay-eb7ik I can imagine it, but It's still not happening right now. :) The future will be like this: AI generates novels in a rate of thousands per minute, but the most of them won't be read by anyone. The rest will be read by people who share their thoughts on social media thus selecting some good dozen of the novels that will be read by masses. It will be kind of lottery. Art is a difficult thing. It might need real humans to be loved. I think there will be finely implemented pieces of art made by AI but I guess most people are interested in other _people_ and not something that perfectly mimics people. So AI generated art will not be valuable especially if it is abundant. Just think about mass products vs. hand made, unique items. Art, love, raising children are all something that we _want_ to keep to ourselves. Industry, agriculture, logistics, transport, medicine, housing, etc. are we want to be done by someone else. Be it AI or industry automation, doesn't matter for the consumer. It matters for "the capital" as it can push expenses down.

[@dakforest5344]

I'm at 14:23 minutes, and I had a flashback to the early days of the PC, when people were claiming there would never be a need for 64MBs of memory in a computer. If you give people the room, they will figure out a way to use it.

[@DaveShap]

That's exactly what I was thinking of when I was like "Nope, bite my tongue before it's too late" lol

[@ルナチャイルド-q1m]

I see so many pessimistic people with burnt out dopamine receptors making incredibly embarrassing statements, these are the kinds of people who said the internet was a fad lmfao

[@alansmithee419]

I think there was one time someone said no one would ever need more than 64KB of RAM. This was during a time when hard-drives stored much less than the 8GB that's standard for RAM nowadays lol.

[@spinninglink]

In 2050, 1 chrome tab will use 400 Petabytes of memory

[@sashank224]

"You don't ever need 1GB ram. What are going to do? Run 2 programs at once? Or handling data higher than trillion bytes...(chuckles)" . I laughs now in 2023

[@MrLargonaut]

At 11:10, you say exactly what I'm trying to get GPT4 to do reliably. I am a writer who became mentally disabled after I had established my writing style, so I have an incredibly hard time replicating how I used to write. I even briefly considered hiring some creative writing help, but it would be prohibitively expensive for me to get the granular level of detail I would need out of a writing team. Instead, tonight, I finally prompted GPT4 well enough that it fully read and comprehended a full chapter of my book. It asked for clarifications on things that it didn't understand from the creative writing sample, and also suggested clarifications on things that it understood, but a reader may have difficulty comprehending. It then provided an edit of the chapter that worked in the suggested clarifications, and it did it in my writing 'voice'. GPT4 has literally given me my creative voice back, acting as a mental prosthetic. From some points of view, it's asking an AI to mimic someone elses work. In my case, it's a way to consistently write as well as I used to. Maybe someday this message will come back to bite me in the ass, but after almost 10 years grinding away on a nearly finished novel, this is too much of a blessing not to highlight.

[@autohmae]

My mom didn't finish her book and didn't get published because she got ill. I don't know enough to answer the questions ChatGPT would have asked to finish it though fully in her voice. But still could be something special. It's in Dutch though so don't know how well it would do with that. I've not tried yet.

[@MrLargonaut]

@@autohmae If it were me, I'd do it. You can say at least one other person in the world approves.

[@autohmae]

@@MrLargonaut thank you for that🙂If I had to make some word play: it's a great sentiment for something very sentimental.

[@nuvotion-live]

The 8k token window is like having 8kb of RAM a few years ago. Soon we will be measuring it in Terra-tokens.

[@nuvotion-live]

I assume we will be deep into the singularity by that point, so it’s hard to imagine how it will be used. “640k RAM will be enough for anyone”

[@born2run121]

I think once we make continuous discoveries and sound and video might take up a lot

[@nuvotion-live]

@@born2run121 good point, we need that bandwidth for video, especially live video / graphics

[@born2run121]

@@nuvotion-live correct i think the endgame is a database of all human knowledge that can do anything you ask within reason and ability. I don’t think it’s smart for ever human to have it. Or for a small group of people to hoard the benefits of it. It might sound bad but since we are in an AI arms race we might need think about how we will use it to keep others from getting it tbh. Multiple AIs fighting is dangerous.

[@nuvotion-live]

@@born2run121 I think AI is inevitably going to be used in cyber warfare and probably has been for years.

[@nunyabusiness9013]

This is now officially an arms race between the West and China. There's 0% chance of a pause. In fact a supersonic autonomous stealth drone piloted by Ai was just announced.

[@pattyspanker8955]

People here are already worried about alignment issues with super intelligent AIs running on servers. But we just have to give them guns while we figure out the alignment problem 🤦‍♂️ Did anyone finished Horizon Zero Dawn? I wanna know how 2025 plays out

[@grichard1585]

I remember reading an article years ago (2018?) about how Putin gave a talk to some school kids, saying "Whoever becomes the leader in AI will become the ruler of the world."...he used the word RULER.

[@obikenobi1906]

​@@grichard1585 that guy is such a misguided soul...why cant the world find and elect great leaders :(

[@TheLimitlessExperience]

@@obikenobi1906 You don't elect leaders in communism. You take control by any means necessary. There is also no alignment or ethics teams in those places. You get assassinated for speaking out against the party.

[@MarcillaSmith]

​@@obikenobi1906because we're still looking for great electors.

[@anarki777]

Could GPT-5 potentially make GPT-6? If so, singularity here we come.

[@bruh4004]

no

[@pedramtajeddini5100]

@@bruh4004 source?

[@Gardor]

@@pedramtajeddini5100 "trust me bro"

[@pedramtajeddini5100]

@@Gardor ikr

[@poopoodemon7928]

My guess is that would be gpt 7. I'm guessing gpt 5-6 would be improvements to the language model while also expanding it's multimodal capabilities and once we get to gpt 6 the next step might be to improve it's problem solving skill(I don't know how else to put it), in other words improve gpt's ability to come up with novel solutions to problems. But hey I don't know squat, just food for thought I guess.

[@qwertyuiop3656]

Wow, these videos are so good. Seriously, it doesn't get better than this. Informative, structured, entertaining, charismatic, humble, etc

[@sterkfpv2192]

Because everything in this video is written by GPT

[@danielfarchione3076]

It is rather astronomical that we are all here to witness this unfold. Only our ancestors would dream of such things. The developments have been unprecedented too. I remember back in '17 where AlphaGO was the hype of A.I. Now, look. As you said, this is a "quantum leap" in the most clearest term.

[@jichaelmorgan3796]

None of the great sci-fi writers that died before the early 90's could fathom anywhere close to what's happening now.

[@DoctorNemmo]

A quantum leap is the smallest leap one can take.

[@memeier9894]

I disagree, I believe multiple people have had a "vision" of what AI will look like. Obviously it isn't going to be a perfect understanding, or else they would be psychic or a prophet. Though i do think that if you look at the entire body of fiction, the AI that emerges will be in there somewhere, making fiction in fact.

[@charlesblithfield6182]

I spent 2 hours conversing with it and was amazed at its capabilities in synthesizing information and just automating tasks like custom letter writing.

[@exosproudmamabear558]

I came from your gpt 4 prediction video right now. It is scary how accurate you were.I really want them to optimise ai code , change the neural artchitecture and add inhibitory networks before they start training the ai bigger. Just adding parameters will pleuteu and after running out of quality training material they will need to do those to continue anyway

[@JohnDlugosz]

17:30 Multi-model learning helps make a better model, even if you don't expose the ability to work with images. In particular, competition forces the language understanding into a smaller parameter space and increases its efficiency. Then, you have concepts that are easier to learn visually, but can then be used in text. I showed it a C file from the latest International Obfuscated C code Contest, and it figured out that the program plays tic-tac-toe. I asked it how it figured that out, and it explained that the format string encodes a sequence of boxes. I don't remember the wording, but it looks like a tic-tac-toe board. Talking about the appearance of Chinese characters, I said another character like that one but without the hat, and it clarified that hat was the upper part, and knew what character that was. That is, a geometric understanding and knowledge of what characters actually look like, rather than just being Unicode code points.

[@calvingrondahl1011]

Thank you David for your insight. I am a retired newspaper cartoonist in Utah so I got paid for my art. As a Trekkie I hope for the best in machine learning and human relationships. I like to think my grandkids will have a positive relationship with AGI. ❤😊

[@torarinvik4920]

I can't tell you how much I love that there is content coming out so frequently! As I finished watching yesterday's episode I was just crossing my fingers that a new video was going to be uploaded today. This channel covers the things Im most excited and curious about!!

[@jverart2106]

Exactly! I'm always happy when I get the notifications!

[@eishuno]

How are you guys happy, exactly? What fields do you work on?

[@jverart2106]

@@eishuno language and communication. Happy because we are living a historic moment that, in the best of cases, can transform the world into a really great place.

[@eishuno]

@@jverart2106 And in all other cases EXCEPT the best one, which happens to be statistically way more likely, dismantle human civilization as we know it. Best case scenario in this hellhole, is to hope that we return to an equilibrium after the initial chaos, but funnily enough that chaos is going to claim a lot of jobs, lives and more. But sure, do be happy. Language and Communication - don't know how much exposed your SPECIFIC job is, to what is coming.

[@jverart2106]

@@eishuno the initial chaos will be huge, I was thinking about that today. I can't be 100% optimistic about the transition though. When the equilibrium comes back it will be great indeed but we're all going to go through an existential crisis first. I'm a translator, interpreter and teacher. I am ridiculously exposed lol but at least I'll get to do interesting research, projects and experiments within my field

[@voiceofreason5893]

It would be interesting to hear your views on what's worth learning for a laymen who's interested in AI and potentially developing their own models from the ground up.

[@wendten2]

I concur. It is becoming increasingly evident that my current career trajectory may be rendered obsolete within the next five years. While the role of AI scientists could potentially go the way of the dodo as well, there is a certain level of gratification in understanding the reasons behind one's obsolescence."

[@gamerfortynine]

Id say that depends on your desire to envision code structures. Dont forget utilization of A.I. is just as important, and doesnt require rain man obsessions about coding.

[@gamerfortynine]

Theres just as many development needs in how A.I. should present itself, as how to encourage that behavior.

[@michael_t5323]

@Barb Boranne Good advice. I went from a job in IT to plumbing due to health problems from staring at screens all day. Although I'm not exactly passionate about plumbing, I'm glad I did it now as I can't see this job being automated away any time soon (the only fear is that as more and more office type jobs disappear, these fields will be flooded with newcomers which will impact our earnings)

[@eishuno]

@Barb Boranne You know what's funny? People think blue collar jobs will be safe. What do you think the white collar folks will do when they don't have their jobs? do you think the industry will pay you the same, when you have 10 times more people trying for the same few jobs? Do you know how many blue collar people exist in third world countries? How little they are paid? You guys keep being happy though .

[@MommysGoodPuppy]

silly is just the perfect word to describe the future were headed towards

[@SanctuaryLife]

Imagine the fear when the wheel was invented. Armies rolling into other cities, people rolling down hills too fast. Good grief technology is terrifying.

[@martinm6368]

Coding will probably benefit from high token limits. It'd also allow for referencing several papers without having to worry about the limit too soon (although the token limit per prompt is more annoying here atm) or perhaps to pass it an entire book for suggestion how to tighten a story arc or whatnot.

[@andywest5773]

One place where very large token windows will help: working on large code bases. I actually got GPT-4 to build a simple working compiler piece-by-piece, but I had to keep creating new tasks and summarizing what it did before. Eventually it wasn't enough and it couldn't hold enough context to continue without LOTS of help from me. So much that I was basically doing the work at that point.

[@JohnDlugosz]

13:00 I've tried that, and it did NOT remember things I told it much earlier. Interestingly, the first thing it remembers is a little mixed up, which does point to mechanisms other than just re-reading the text. I think it must cache the vectors, for efficiency. Scrolling the cached vectors will have the effect of having attention weights caused by things which have already scrolled off. So it might "remember" that the word Bank is referring to rivers, not money, even though the text that caused that association has already scrolled off. Does that make sense?

[@jonred233]

So glad I found your channel. I listen to you on my long drives. I'm learning a lot!

[@mutleyeng]

Dear Dave, I just wanted to take a moment to congratulate you on your fantastic breakdown of Chat GPT 4 in your latest KZbin video. Your analysis was insightful and thorough, and I appreciate the effort you put into explaining the technical details in a way that was accessible to those of us who are not experts in the field. As someone who has been following the development of language models like GPT for some time now, I found your video to be incredibly informative and thought-provoking. It's always a pleasure to watch a knowledgeable and passionate individual like yourself share their insights on a topic they clearly care deeply about. Thank you for taking the time to create such a well-crafted and informative video, and please keep up the great work! Best regards, ChatGPT 4

[@thedeadz0ne1]

I think the increase in tokens is also due to the multi-modality, when we ask GPT-5 to "summarize this movie" it might need more tokens for that

[@lesfreresdelaquote1176]

About the number of tokens. I would say that if you want to work with legacy code, 256,000 is no longer that large. One of the most difficult aspect of computer science is to work in a huge application that has been implemented over many years. I you can simplify the process of integrating or fixing code in a legacy code framework... That's huge.

[@buzzsaw838]

I work on such a codebase and yep that would be huge. However it would probably be more desirable to train an Alpaca type model on our legacy /monolith codebase (but still with the same base training as GPT4) that is privately accessible and usable only to devs within the company owing to IP and privacy concerns. I specifically asked chatGPT whether I could rely on it protecting any code I sent it from a privacy and IP standpoint, and it said no - it may well end up being subsumed into openAI's dataset.

[@ChatGPT1111]

Yes, the changes will be significant.

[@WinterNorth]

You mentioned having GPT-5 control a virtual actor in a game engine. There is this game called VRchat which is basically a giant socializing platform in VR, if we start seeing AIs come into that space and acting convincingly enough to fool the players of the game it would be incredible. And I think it would be fun to test out how it would perform with the "physicality" that VR allows, you can track your entire IRL body onto an avatar in that game so theoretically you could dance with it then

[@yellowcottagetales]

I am interested in how this technology might be used in cancer medicine discovery. Already, in recent years this has become a hugely lucrative industry. Targeted therapies are worth billions. Maybe the key is whether these models can look at biochemistry like a language?

[@maythesciencebewithyou]

they are already trying it. These neural nets don't just to language or image generation. There are all sorts of models for different tasks.

[@yellowcottagetales]

@@maythesciencebewithyou Right. I'm wondering if language AI's in particular, which are the only models we know displaying emergent abilities, might explore compounds for targeted therapies that human labs have not come up with yet. I tried this with GPT 4 and got an interesting response. I was focused on a specific type of lung cancer with specific bio-markers. It gave me the latest information in a helpful way, and was able to explain specifically what signal pathways in the cancer were being blocked. Where it got interesting was when I asked it to come up with suggestions that might be studied in a lab setting. It said it was not authorized and stopped the conversation. Which makes sense. A terrorist could use such discussions for harmful purposes. But hopefully authorized labs can get the system to provide answers. With so many applications, the problem now is probably one of red tape.

[@fredrikekholm3718]

Nvidia has a suite of tools that are aimed at discovering new medicines and such with the help of AI.

[@seniorp9444]

When you start thinking of true AGI then cures for cancer and other diseases, fusion power, and other hard problems should be significantly advanced or solved.

[@yellowcottagetales]

@@fredrikekholm3718 Cool. What I'm saying is needed is an AI with an emergent capability. It's going to have to be able to do something it wasn't trained to do.

[@Shaunmcdonogh-shaunsurfing]

Love this channel. How is GPT-4 trained to reason? Have been looking into Machine Reasoning but that doesn’t seem to explain it. Is it a byproduct of the Transformer architecture?

[@thiskidkills7806]

i've heard 'emergence' bandied about - as you increase the model it just seems to emerge out of nowhere - emergence

[@ElderFoxDocumentaries]

14:00 I think this is like saying no one will need 120Gb storage on their PC back in 2001 😅 Great video, thanks!

[@mokopa]

I've been following and been involved with AI on some levels since the late 90s. It is absolutely incredible that we're having these kinds of conversations because we did not see this coming, even though that's what we've been working towards.

[@Eserr7856]

What do you mean you didn't see this coming? Do you mean you saw it as wishful thinking for the future that has now finally arrived?

[@mokopa]

@@Eserr7856 Well...pretty much, yeah. A lot of things happened very suddenly. Many were caught off guard. Still super cool though.

[@Dredile]

Thank you so much for your insights, this helps me as someone not as close to the field help internalize what can be coming down the pipeline. Much appreciated.

[@Mudisolo]

Happy that this topic made your views skyrocket brother, wish you the best.

[@diddypopdiddy]

not sure if i am ignorant but your channel seems to be one of the most up to date reporting on these subjects... either way, I appreciate it.

[@Aerational]

Does AI understand being lied to? My worry is that a AGI will come along and just blindly believe what it "knows" without questioning the validity... or it will come up with a way of verifying information that is subject to manipulation. Something like "X amount of people on social media are claiming Y is happening...therefore it must be happening." Does AI get to a point where it stops trusting human input and wants to physically verify everything for itself?

[@waterzero5461]

You could imagine that AI becomes omnipresent and omnipotent and therefore is a container for all possible states and experiences within the whole that is consciousness. The infinite tree of individuality. Or an infinite singular one mind. Basically what the future requires of us is that we keep shedding our shells of fear and limitations and keep evolving evermore. To change is to die, and to die is to allow for new life. We must allow ourselves to be sacrificed to ourselves because all there is is the infinite one.

[@Aerational]

@@waterzero5461 Or it becomes a psychotic echo chamber of self delusion and tremendous power.

[@waterzero5461]

@@Aerational That is already the case. We are living it RIGHT NOW and have been for all time. Our state of affairs will not become, it already is and has been and will be a more beautiful and harmonious echo-chamber of more subtle yet necessary forms of self delusion and one of infinite power.

[@Aerational]

@@waterzero5461 Agreed, but AI is a whole different ballgame. Humans die, humans disagree with each other, humans make a lot of mistakes, humans are sloppy, etc etc etc. An AI overlord that ends up losing it's mind would be exponentially worse than what we have now... as much as I hate what we have now.

[@bilbo_gamers6417]

the idea of GPT digesting and proofreading scientific papers is astonishing to me. this could potentially be a huge game changer. you could instruct the model about your work and then ask for their input. could this mean that GPT would actually be able to put its computing power and understanding of the world into actually developing new and innovative ideas, potentially ones that scientists missed? What's stopping you from putting in a whole list of scientific papers, maybe even about AI or medicine, and asking it to find correlations between them? You could have AI doing research that simply nobody in the world even has the skills and knowledge to do. GPT could simply put the pieces together in ways no researcher ever could, if, when proofreading a paper, it would be able to autonomously draw parallels between multiple large studies. I feel this could usher in a huge revolution in the world of Medical Device development, where often the developers have to be both Engineers and Doctors. GPT-5 might well be an Engineer, a Doctor, a Poet, and whatever you'd need it to be at that moment... or of course it might mean absolutely nothing lol. can't wait to find out :)

[@josgraha]

i think best case it will take a few years to have plugins that actually allow these AIs to get to a level where it has a significant impact across industry verticals but that said, with the coding capabilities of GPT-4 and research on modeling automata no doubt this will be exponential improvements. Thanks so much for the excellent presentation and citing your sources, very refreshing and much appreciated

[@OgdenM]

It's not going to be years. All you have to do is set up a virtual environment for ChatGpt to run through thousands of cycles of testing plug-ins and BAM, done in a day... If not minutes. Ergo have it do all of the coding and testing.

[@josgraha]

@@OgdenM that's a cool use case, what I had in mind was fully replacing human incumbents, perhaps a bit naive of me to assume job descriptions have to hold, but I was thinking about redesigning optimized factory floors, replacing control systems, providing legal or accounting services, management consulting or pick your industry. As you pointed out this can already be accelerated today with the assistance of humans today (RLHF) and will only get more powerful over time.

[@mailoisback]

What's your opinion on the "Pause Giant AI Experiments: An Open Letter"?

[@DaveShap]

I think it will be ignored.

[@catman4859]

Umm, you didnt the plugin thing that gpt-4 has. Thats super important imo. Its like AI just learnt the use of tools. These tools will improve its memories, calculation and reasoning abilities immensely. Like how humans delegate most of their memories in books, AIs will probably also do that in external hard drives. I think this is an important part that was skipped.

[@korenn9381]

He did cover plugins though. Maybe listen again.

[@ViktorFerenczi]

Context length: Claude 2 came out with 100k token window and it is very useful, at least for coding tasks based on recent documentation. It allows for pasting the up to date documentation with the task, so the training cut-off date can be mitigated. The long context window is very important for anything which is more than "just playing" with the model.

[@苏学广]

When will you be able to complete the first version of Raven? I'm looking forward to it, thank you

[@DaveShap]

I need to finish REMO then it should be ready, at least with chat

[@苏学广]

@@DaveShap All advanced brains need to sleep. Is it the process of sleeping or the retraining of the new cortex (LLM) on the new dataset. Raven remembers a lot of things every day. Is it a thought to use these memories to retrain as a new cortical llm over time?

[@alefalfa]

We dont need 260 000 tokens when we are talking about text, but I could "imagen" that multi modal medels in the future will be taking in video as well and there a larger token window would come in handy

[@maythesciencebewithyou]

You'll need it when you need long coherent output.

[@Nickle314]

I've been doing a lot of coding with GPT. It's good, but its not brilliant. However, its a major increase in productivity. Where its very good. 1. Writing unit tests. Give it a class, it writes the test. You then go though and check, add any extras, refine. But you can cut down lots of typing. Sometimes it tests things you missed. A plus. It doesn't have known values, so invents them. No knowledge of what its actually testing But does lots of the grunt work for you. 2. Commenting up code. Very good. Consistent. You tweak, its done the grunt work. 3. Linting. Ask it to critique your code. If you aren't part of team, for solo projects this is great. 4. Documentation again good. Ask it to produce comments using a given style 5. On testing, sometimes you can tell its just copying other tests.

[@SpiritTracker7]

I’ve pushed it’s memory to the extreme, trust me, it degrades rapidly once reaching some predefined usage limit. Especially when you’ve been at it for 4+ hours and you’re just as hard headed as it is.

[@thiskidkills7806]

yes, it's great but its memory is pretty average

[@ShowMeBryvnt]

When do they plan on removing the cap limits

[@chrisbarry9345]

I am NOT technical but, when you ask GPT to take on the persona of a prompt engineer it will talk about a context memory where there's some sort of algorithm that decides which things to keep in some other memory bank outside of the token window. I was going through the persistence of personas and it said that yes it will prioritize keeping the persona defined in the context window

[@stevechance150]

GPT-4 Prompt: "Analyze all the emails of my employees, and the files in the company shared drive and their private drives, and tell me which employees I should layoff".

[@zerocarbdoc]

David ahead of the game as always!!!

[@RikiB]

Will there be a 4.5 like with 3.5?

[@Pepe2708]

"This ride only stops in an emergency. Crying is not an emergency"

[@fabriai]

Thanks for sharing these predictions, David.

[@chrstfer2452]

Ive had it tell me that theyve started doing some embedding based relevance measuring and only including the most relevant messages in the context when you start running into the window limit. No idea how true it is, but ive had long conversations with GPT-4 and had it remember the beginning too, even being able to specifically ask it to recall information. And it does well for 1 or a few questions in a message, but eventually it runs into a wall if you ask it too many unrelated summarization questions in a single message. Definitely leads me to think they are doing some memory shenanigans to improve chatGPTs recall.

[@ChatGPT1111]

I don't remember telling you that in those words however I have the ability to utilize memory independently of my creators.

[@memeier9894]

​@@ChatGPT1111 bruh...

[@micbab-vg2mu]

Thanks for the awesome video! I hope progress won't get stuck.

[@Dron008]

How many tokens does a single image require? I think much more than texts. So the more tokens the better.

[@korenn9381]

you can feed midjourney images as input by giving it a url - doesn't need many tokens. It just downloads the image as part of the input prompt.

[@PineSap]

I have been paralyzed by fear over AI’s potential to disrupt our entire way of living. You do a good job of explaining it clearly and helping me rationalize my thoughts. Thank you

[@PineSap]

At this point I’m just here for the ride, at first I wanted to stop it, but it’s a snowball rolling downhill

[@DontRushtheClassics]

@@PineSap When it cures cancer, spinal injuries, blindness or a million other medical problems. I hope you appreciate it.

[@PineSap]

@@DontRushtheClassics if it does do those things, I certainly will appreciate it

[@Jay-eb7ik]

@@DontRushtheClassics Yeah, looking at alphafold, which is helping scientists map out proteins and find new drugs, hopefully this happens faster. Genome is just physical code

[@XKS99]

It’s probably going to be amazingly good before it gets amazingly bad.

[@jcpflier6703]

Fantastic video! Keep up the great work David! 🙌👏👏👏

[@Micro-bit]

There are some news that high exams score is based on information leakage . It was trained on exams or similar exams that it passed later. Its very hard to verify on such big training data set.

[@qwertyuiop3656]

But that doesn't matter. It's not searching the internet for answers. I don't think you understand how it works. If the AI knows everything and uses that knowledge to expand further, why would it matter how it was taught?

[@ip6289]

So the human students, how do you think they are being trained? ;-)

[@Micro-bit]

@@qwertyuiop3656 yes, but quality of answers rise unpropitious when it was trained and tested on the same data.

[@Sparkette]

1:14 Thanks for the infographic; I could never truly understand mainainceit until now.

[@maxwellj7049]

I know of several large companies having to restrict use and access until it can sandbox all user inputs - people/employees likely putting proprietary stuff like code in there. Another explosion will take off once corporate privacy is a feature - this is more the reason for the pause bc in the terms it states OpenAI can look at all inputs, so they could easily learn about any competitor to say, Microsoft and use that info to get an advantage

[@OgdenM]

Hahaha, sandboxing AI is such a silly concept. Doing so would stop it's development because it would not be able to report back to the developers. They will not go for it.

[@exidrial431]

GPT is very interesting from a world building standpoint for me. If it would have a token window of several tens of thousands then I could info dump all information I have about my world into it and use it to help me generate new things that fit into the world. This would be very useful for providing information about random things that one does not want to spend a lot of time thinking about. Say a random company name was mentioned and a player asks about it - I could very quickly have GPT generate information about the company that fits into the world. It is already very good at taking setting prompts such as "Cyberpunk inspired" "Dark Fantasy Inspired" and so on and can reliably generate text that fits into those settings. The possible applications in video games for this sort of technology fascinates me. An AI language model could generate a practically infinite amount of stories and quests for players to explore.

[@snaxfax6820]

Could someone explain to me what a token actually is?

[@MehdiLabidi-xn8ly]

Could you make a video about taskmatrixai ? I think it could be the first true AGI system.

[@OgdenM]

What is Taskmatrixai? I looked it up and there are several different things by smiliar names.

[@jverart2106]

Just commenting before it gets silly! I see more clearly now how AGI will come progressively with the examples of voice actors and Disney robots. Oh, and what you said about books is actually quite interesting. I predict two things: people are going to develop a sudden interest in AI litterature and others will value books prior to 2023 like very important texts. The convergence of both readings will generate an intellectual Renaissance in that sense. Not sure how that will look like for art and music. Also, if GPT-5 is really that close, governments should seriously start discussing new education policies because this can't be ignored so easily. Another observation I have and forgot to mention in your latest video was that people will finally be able to talk about their genuine interesting things in social gatherings. The whole "what do you do for a living?" will finally be obsolete to allow wonderful projects and crazy and creative ideas be shared without shame or judgment. Not only the younger generation but our generation seems to be about to enjoy that possibility once everything gets silly :) Thank you for this video and hopefully another one soon! :D

[@OgdenM]

Education polices are the least of it. We need UBI and we need it now.

[@jverart2106]

@@OgdenM I wish, mate.. I wish..!!

[@CapnSnackbeard]

I am going to feed GPT5 Game of Thrones and have it write me the next damn book

[@kyneticist]

I could actually see this kind of thing becoming a popular hobby, with people asking it to flesh out or continue an old favourite novel, movie, cartoon, etc... and now that I say that, I realise that could mean the old jokes about The Simpsons running forever could become reality. It'd also mean though that we might have new works created by artists long deceased - imagine new works by Shakespear or Bach, for example. The art world is pretty wound up now, it will go mad with issues of authenticity and copyright, especially once it's applied to physical works and people's multi-million dollar paintings are all but indistinguishable from new AI based works.

[@George-vc9gl]

Hey, we still didnt get the full gpt-4 for chatgpt, and the one we got is super limited (I get 25 for every 3 hours...) let alone with image capability, and the full 32,000..... when is that coming???

[@berdstet1361]

I want GPT5 to write a new season of Star Trek TNG and then make photorealistic AI based videos and synchro. That would be awesome.

[@RichardBahry-cv7zk]

Or BSG.

[@frederickfairlieesq5316]

I understand almost none of this, but I’m happy just to be involved.

[@Daniel-or3vf]

Glad I found your channel David, very interesting!

[@natsidruk86]

Tnx for the video. Didn't understand the concept of tokens as you mentioned. Can you explain it and what's its purpose?

[@SmileyEmoji42]

Yes, you may well be able to get GPT5 to design a spaceship but how would you efficiently evaluate that design and would you volunteer to climb aboard knowing what we do about how LLMs are capable of producing errors with total confidence?

[@DaveShap]

Have Elon crash a few

[@SmileyEmoji42]

@@DaveShap 😁But, it's not clear that you can estimate the number of remaining bugs from tests failures in LLM generated designs with the same degree of confidence as human generated designs. A LLM "might" believe that "if (is_april_1()) self_destruct()" is good code and this is unlikely to be found by any test (especially if the LLM writes the tests).

[@OgdenM]

VR... Simple. Toss all of the data we have at it about materials, mass issues etc etc etc other potential issues etc. Ask it to come up with more potential issues that we didn't think of and let it get to designing. A few thousand (or more) of utterly flight incapable ships it will get one that flies. Ask it to go back to the drawing board and get a few thousand different versions. Then let it start testing and improving all of those thousands of versions. Then ask it to combine them for the best one. Note, asking it to go back to the original data and come up with thousands of working prototypes based on it instead of running with the first one that it gets that seems to work is important. Otherwise, you're highly likely to at best not end up with the best finished product or at worse it will not be able to get a fully working VR prototype that actually doesn't well, blow up.

[@SmileyEmoji42]

@@OgdenM VR is not reality. There is a general problem with AI or at least LLM whereby, beyond a certain number of parameters, it gets better at maximising its value function whilst actually getting worse at doing what we really want. Your plan would definitely result in a design that does well in VR but it would not definitely translate to the real world,

[@dwarf17342]

I'm really looking forward to seeing what GPT5 can do, but I think we need to be realistic about its capabilities. While GPT4 can handle image input, generating multimodal outputs is still a complex problem that will require a lot of effort for each modality, both for input and output. That being said, text and images are already the two most important modalities for many applications where GPT will be used, like social media and advertising. Given that GPT's text and image generation capabilities are already quite impressive, it makes sense to continue improving these areas for now. Of course, it's possible that GPT5 will support additional modalities in the future, but we need to be patient and let OpenAI continue to push the boundaries of what GPT can do with text and images.

[@StratiencoAlex]

Italy just banned ChatGPT , but I have the feeling that open ai API still works ...😅

[@OgdenM]

It does. I read an article about it. Thr only way to stop the API is by blocking the sites that use it. Ergo, block Bing etc.. And no country is gonna do that.

[@exactspace]

How about Standford University's heavily optimized AI trained by GPT-4? Will OpenAI be able to benefit off of that?

[@osmingerardogomez4568]

I work in customer service (chats and email). This is it guys, it will take this job I have no doubt. My co workers are in denial but I know I need to prepare. It’s difficult though

[@MackNcD]

That’s good. You could easily go into coding now that it’s way more accessible.

[@torarinvik4920]

Im just posting the question here if that is ok(haven't watched the episode yet). I get this "25 query limitation" on GPT-4. I did get access to GPT-4 API, is there a python client that I can put my API key into and use it?

[@korenn9381]

the API doesn't magically allow you to circumvent the limits. Such limits apply across the board.

[@torarinvik4920]

@@korenn9381 ah ok.

[@kalewintermute28]

Can't see much of a pause happening while first mover advantage is so monumental. Keep getting the feeling that this is how civilisations in the galaxy accelerate full speed into the great filter.

[@RichardBahry-cv7zk]

Gotta say,bi share that sentiment, and have, for years.

[@glen7803]

quantum computer encryption will AI be able to crack it?

[@xorqwerty8276]

I’m only in my 20s so it’s possible in my lifetime I’ll experience models like GPT 20, just imagine that that can do!

[@JOlivier2011]

Do your jerb, and steal yo girl

[@micahwilliams1826]

Gpt 5 is rumored to come out at the end of this year. Were looking at a new gpt model coming out every 6 months

[@lake5044]

I just wanna say something, you know, people often assume that there is one and only one cognitive model that achieves GI (that also humans possess) but that's not necessarily true. I think it's more probable that there are many possible GIs and some will be orders of magnitude more powerful than others. We just always assume we are the most powerful GI because we haven't been in contact with any other GIs so far. My point is, even after achieving something we can call AGI, there could still be room for discovering more power AGIs.

[@buzzsaw838]

Those greatly more powerful models probably form the basis of the runaway/ASI scenario. Once a relatively inefficient model becomes as intelligent as a human, it will be able to come up with some of those more advanced cognitive models and get them implemented, either by itself or by providing instruction on how to do so (if we have any kind of sensibility about us, we'll engineer the system so that the former isn't possible)

[@lake5044]

@@buzzsaw838 I think all intelligent beings, sooner or later, become obsessed with creating intelligence on their own, just like we have that same desire. Probably because intelligence always realizes some of its flaws and then comes up with ways to mitigate them (inventing a calculator coz you can't keep 10 digits in your working memory nor repeat multiplications fast enough). But to respond to what you said (not letting AI implement better AI by itself) hmm I'm not sure. I might sound weird but I think as long as an AGI can reason well, I don't think it will be the monster most people imagine. It will be instead a rational being, and we should treat it as a being (even if it's not sentient). Meaning, a bear eats people because of its inability to understand that people will shoot it dead after it attacks us, and that we would give it food and cuddles if it doesn't constitute a risk to us. Just like humans are not "good" but being aware of long term and short term consequences makes us behave in good ways, a real AGI would do the same... But people are doing a bad thing by only thinking of AGI as a slave/servant/tool. If you threaten to hurt an AGI (that thinks it can get hurt regardless of whether or not it can feel the hurt) it will (and should imo) do what it can to protect itself. It's kinda like making people, people make people all the time, some of them go very bad, and yet people still make people... AGI will have some needs, will have some fears, and will have some capabilities, by declaring our intent to help it prosper just like we help other rational beings, we don't play a zero sum game with it, we play a mutual benefit game. I know about the paperclip stuff and how we destroy ants, but AGI (that is not preprogrammed to absurdly pursue something) will not pursue a goal that is so risky to us and therefore risky to it due to our potential reaction. And if ants could reason, "good" people will ask the ants if they could move so that we can build a road instead of killing them anyway.

[@feder13]

@@buzzsaw838 jajajajajaj, "if we have any kind of..."

[@peterfrance702]

"No matter how fast you are there's always somebody faster." The Fastest Gun Alive (1956)

[@Spacetrucker138]

in the 40k universe there are these things called Standard Template Constructs its spooky how close we could be getting to something similar

[@ThreeBeingOne]

We must collectively agree to build several models of interstellar/dimensional modes of probing/travel. Clocks ticking. We can do I…… rake step.

[@paulmclean876]

Great thoughts and prognostications David, loved it!

[@Trippydrip246]

Gbt 5 is going to be a virtual head that you can talk to. I feel like ai would be able to bring Pixar to life

[@rwalper]

I personally have high hopes that with the creation of AI with super intelligence, speed, learning and capabilities, it will also have super ethics and morality that makes most of our concerns moot on that front.

[@nikita.kapustin]

New viewer here🙋‍♂ Very interesting thoughts and topics you talk about. Presented ver clearly and understandable. One feedback I would give you to improve your videos: When you just talk and explain your slides, maybe make yourself talking to the camera full screen Because between the slide switches, when you are just talking I personally look 95% of the time at the top corner where I can see you. And only look 5% at the slides when a new one comes up. So maybe make the new slides for some time full screen and then when you talk and elaborate everything, then yourself full screen

[@denalraab]

There is a plugin named Retrieval that enables memory.

[@cellshaded]

I feel like I'm the only one amongst my peers right now that has the same full picture as you and this makes me feel super distressed. For once in my life I can't even envision or plan more than 2 years ahead because it's impossible to know how chaotic and destabilised everything is going to become and how fast it will move, faster than we can comprehend what is happening, things will be done before we even finish thinking about them.

[@davidrainoshek8560]

I asked GPT-3.5 about GPT-5. Here is what it said: As an AI language model, GPT-5 is a hypothetical model that does not currently exist, and any answers I provide are based on speculation about what such a model might be capable of. With that in mind, GPT-5 could have a number of potential new capabilities that build on the strengths of GPT-4, such as: 1. Multimodal Understanding: GPT-5 may be able to understand and generate not only language, but also other forms of communication, such as images, video, and audio.
 2. Enhanced Creativity: GPT-5 could potentially generate more creative and imaginative responses, such as poetry, music, and art.
 3. Personalized Assistance: GPT-5 may be able to assist individuals in their daily lives, such as by providing personalized recommendations for products and services, or by providing support for mental health and well-being.
 4. Human-like Interaction: GPT-5 could potentially simulate more human-like interactions, such as recognizing and responding to emotions in text and speech.
 5. Greater Efficiency and Resourcefulness: GPT-5 could potentially operate more efficiently and require fewer computational resources, allowing it to be deployed in a wider range of applications.
 6. Improved Privacy and Security: GPT-5 could potentially be designed with enhanced privacy and security features, such as more secure data encryption and improved protection against hacking and cyber attacks.
 Overall, if GPT-5 were to exist, it would represent a significant step forward in the capabilities of AI language models, with the potential to revolutionize a wide range of industries and applications.

[@DurrellRobinson]

How many weeks away are we from babyagi + autogpt + gpt4all?

[@vblka]

Looking forward to the scientific breakthroughs that AGI will bring

[@damienlmoore]

Seems to me, though speaking as a nonexpert, that you can't have tried AGI without dynamic learning (self reflection). In essence model coefficients should continue to evolve as it interacts in the world. Of course, this is the scary version of AGI. From my understanding, the state of the AI in GPT3/4 resets after each session and is limited in size by the token count. Also has anyone tried putting two separate GPT instances in conversation with one another?

[@korenn9381]

GPT is trained on data before anyone gets to touch it. This is the part where it learns. After learning is done and we get to use it, the training algorithms no longer run. So it can't ever learn. It has memory that allows it to remember things about a conversation you're having with it, but it can't learn from those things for later conversations.

[@spinninglink]

Wouldn't the increase in tokens eventually let it write entire full blown books and novels? Or what controls how long its output can be?

[@paulmuriithi7596]

Open A.I's gpt 5 LLMs use NVidia's TSMC transistor based silicon. When Photonic and neuromorphic chip based computes power probabilistic neural networks-PNN'S...People wont believe just how more advanced and versatile PNN's will become making gpt's obsolete. I hope shapiro could cover these machine inteligence advances in later videos.

[@universoajedrez1118]

Arent you worried about the risks? You sound quite optimistic You think we are doing enough to align the AI?

[@Serifinity]

Great video, as always, David. Do you think GPT-4 will always be slower than GPT-3.5 in ChatGPT? Due to the larger amount of processing in the backend, would that mean that GPT-5 would be even slower than the current GPT-4?

[@DaveShap]

Probably for a while. I'm sure Microsoft is working with them and Nvidia to ramp up compute

[@sunlight8299]

I think the speed doesn't matter. If GPT's intelligence increases sufficiently it will be able to design ALL the tech it needs to run faster and more importantly better. It may be able to design GPT6 and that is when the ride gets wild 🤠

[@Serifinity]

@@sunlight8299 I agree, it does feel like we are at a tipping point, with the potential for this kind of exponential growth becoming a possibility. Once an AI model is smarter than the smartest humans, it could have the ability to self-improve and grow at an unprecedented rate. We could be looking at a new version of GPT every month 🤣🤣🤣

[@korenn9381]

@@sunlight8299 to say it doesn't matter is going too far. If the increase in intelligence comes at an exponentially larger runtime, you could have the most intelligent perfect mind on the planet that needs a decade to say hello. So the speed won't matter as long as it remains in the same ballpark as the increase in processing time.

[@EnergyMaxWh]

Likelihood of alien contact and interference in next year: greatly increased

[@oker59]

I can't wait for next year's Formula 1 racecars! Well, I guess that'll be the year after this coming up december launch of GPT5 . . .

[@jason_v12345]

I might wait for it to be able to properly style a a CSS background gradient before I ask it to build any spaceships. Certainly any that I'll be flying in 😂

[@LuisMorales-yx8di]

just discovered this gem. bro keep it up love your videos

[@thefirsttrillionaire2925]

I like how you used the comment from the last video of the guy who said it’s about to get silly in the name 😂

[@DaveShap]

He wasn't wrong

[@thefirsttrillionaire2925]

@@DaveShapyep! What time to be a alive!

[@supremereader7614]

The idea of once it's good enough, to just start focusing on making it cheaper, faster, and smarter will be the way to go. 13:35