Thinkstr

Пікірлер

@lionardo 14 күн бұрын

This video is sooo cool. I have a question isn’t the problem with hidden variables that there are an infinite amount of them. So how would this model scale?

@Thinkstr 14 күн бұрын

Hi, thanks for watching! I think a big problem with hidden states is gradient vanishing/explosion, so even though there are infinite possible hidden states, they have a problem holding information for too long. The model might need to be a lot larger to have longer memory. If you use ChatGPT a lot, you probably notice it forgets what you said a long time ago.

@GMatthew-s6q 14 күн бұрын

i lasted 5 minutes, time to do anything else.

@nikitavorotnyak9661 16 күн бұрын

Unwatchable

@jokerrfox354 17 күн бұрын

That's a great take there. It's been a year since I've seen this sort. Glad this still hit my YT algorithm

@MimiMangetsu 19 күн бұрын

Thank you for the video! You highlighted some of the key points of the book very well

@The_hepe 20 күн бұрын

Interesting

@MLDawn Ай бұрын

3 years since this video was published. I wonder where you are in your project?!

@Thinkstr Ай бұрын

I'm glad you ask :D I actually just published a paper and made a video about it! I'm trying to make the paper open-access right now, but you can also find a free version on arXiv. kzbin.info/www/bejne/oniQfnWYqtxrsKs

@abdulrazique4779 Ай бұрын

Is it not possible to increase number of classes dynamically? like at start we only know there is 5 classes only(0-4) but we don't know how many more classes will come so made the model with 5 outputs, then 3 more classes came and we add 3 more neurons to last layer (let say 5, 8 and 9) also at this stage we don't know how many more classes will come and at end 2 more classes came so we add 2 new output neurons for 6 and 7.

@Thinkstr Ай бұрын

Maybe that would work, but if we stop training with examples of the earlier classes, I think forgetting could still be a problem.

@AbdulRazique-z2j Ай бұрын

@@Thinkstr yes, but what if we update fisher matrix before every new task. will it work? means is it possible to add new weights every time we add new classes in the model?

@Thinkstr Ай бұрын

@@AbdulRazique-z2j Huh, I really don't know... You're reminding me of GANs which make progressively larger images, first learning small images and then expanding them.

@AbdulRazique-z2j Ай бұрын

@@Thinkstr na not GANS, just neural network, in which we can increase output neurons of last layer when new class introduce.

@camilavanordergonzalez8128 2 ай бұрын

great video!

@Thinkstr 2 ай бұрын

Thanks! This was a book I wanted to share with the world!

@user-qw1rx1dq6n 2 ай бұрын

I really have to ask is this compatible with a reward function?

@Thinkstr 2 ай бұрын

I actually use it as an intrinsic reward for RL in my latest vid! :D kzbin.info/www/bejne/oniQfnWYqtxrsKs

@ЕвгенийИванов-ю2б 2 ай бұрын

Hi! Really nice work! If I get that correct, in your case actor and critic use that hidden state instead of observations. At the same time the hidden state is the conceptualization of the world, provided by the forward model. The goal of the forward model is to predict the future observations and at the same time keep sustainable world view by not being overly surprised by seeing something irrelevant (on which it’ll be difficult to predict the next observation given an action). Can I ask you: do you freeze the forward model weights while training actor and critic? I guess forward model can learn patterns relevant to predict future state or patterns that are also relevant for the actor and critic. What are the ways of backpropagation? For example in GANs you freeze a discriminator while backpropagating towards generator.

@Thinkstr 2 ай бұрын

@@ЕвгенийИванов-ю2б Hi, thanks for watching! Yeah, your understanding sounds like it's about as deep as mine, but I can't guarantee that's all there is, ha ha. About backpropogation, I don't freeze the forward model, but the hidden states are detached before the actor and critic train with them. I could consider the forward model, actor, and critic to be one model making actions, predictions of value and predictions of observations, and then the curiosity term could incorporate all those parts, but I tried it, and I think those hyperparameters where too tough to fine tune.

@ЕвгенийИванов-ю2б 2 ай бұрын

@@Thinkstr Thanks for your answer! If I got that correct, it's a bit like if in your game a dog is chasing a cat, and a cat moves away from a dog. They see and must recognize each other among other animals. And for deep features you can train VAC by restoring images of them, or you can use pretrained ResNet50 conv part, which has trained on that domain specific task of classification, including among other classes cats and dogs. And It's a kind of dilemma what should be primary, what should be secondary in terms of deep features, because usually deep feature vector doesn't correspond to the original image vector in terms of size, you have to compress the information. Actor and critic may be interested in some domain-specificity and that may not be provided (hypothetically) by the conceptualization of the world, learned by forward model... I mean my internal mental state should be build upon features that are relevant to my specific task to a certain extent, not only good upon ability to predict the future state, if I get that right... Anyway great job! There is shortage of something practical and palpable in terms of FEP usage in reinforcement learning. If I write an article on FEP (maybe some time)) ), I'll definitely make a link to your article.

@Thinkstr 2 ай бұрын

@@ЕвгенийИванов-ю2б Oh, wow, thanks! Just saying its a palpable use of the FEP is an honor, because it's such a hard idea to grasp, haha. And yeah, the actor and critic might be better off seeing observations themselves, but it saves a lot of computation time to use the forward model's hidden states instead, because then you only need to process the observations once!

@aesthetic_GIRlie1347 2 ай бұрын

Thanks a whole lot for the first few lines of summarization in the video. It explained very nicely the basic concepts

@Thinkstr 2 ай бұрын

Thanks! I use a model kind of like this in my latest video: kzbin.info/www/bejne/oniQfnWYqtxrsKs

@papierspeelt 3 ай бұрын

Kurt Friston??

@Thinkstr 3 ай бұрын

This video's an old one, my latest dives deeper! kzbin.info/www/bejne/oniQfnWYqtxrsKs

@SubToGetNOTHING 3 ай бұрын

I say we call them water dwellers who’s with me

@Naker-l1e 3 ай бұрын

Wow!! I have been waiting for this video for a long time; I didn't think the results would be so impressive. I will definitely read your paper carefully! I am working on applying the free energy principle and active inference to Transformers. Do you know of any work on this? Congratulations from Spain on your work.

@Thinkstr 3 ай бұрын

Thanks for watching! The paper in the arxiv should eventually be replaced by the same paper with changes by Neural Computation like border-spacing. Also, I'll eventually upload a video which is a little deeper on the math than this one. I haven't used transformers much, but maybe I should look more into them!

@F_Sacco 3 ай бұрын

Loved this video! Also i think this is relevant to training LLMs. Often dataset curation is really important because you want the LLM to learn only from "high quality text"... But shouldn't the LLM be capable of recognizing high quality text and ignore bad text on its own? Ideally you would want the LLM to ignore text that is low quality or that he knows very well and only focus on what is "high quality" I think having an LLM capable of being curious or bored just like us humans will solve this problem

@Thinkstr 3 ай бұрын

I like that! Maybe the FEP could help with vanishing gradient by figuring out what information is good.

@eternadicha5621 4 ай бұрын

Magnifica enseñanza

@ЕвгенийИванов-ю2б 4 ай бұрын

Hi! I really like your work and recently wrote a paper about an evolutionary inspired way of neural network gradual complexification. But can't publish that because don't have arxiv endorsement. Maybe you know someone to whom I can address?

@Thinkstr 4 ай бұрын

Oh, I might be able to do that! What part of arxiv are you trying to publish in?

@ЕвгенийИванов-ю2б 4 ай бұрын

Computer science, artificial intelligence

@ЕвгенийИванов-ю2б 4 ай бұрын

@@Thinkstr I've posted that on GitHub and described the experiments. Tried to reach you through LinkedIn 😊

@полное_пузо_хинкалей 4 ай бұрын

9:40 racism jumpscare

@Thinkstr 4 ай бұрын

Ha ha ha

@microwavecoffee 4 ай бұрын

Stupid anime reference from me but in the anime psychopass they averaged out the brains of the most societally outcast people to create a superintelligence to manage society, kinda feels similar to remembering the most memorable examples 😂 Great video 👍

@Thinkstr 4 ай бұрын

I watched an episode or two of psychopass but didn't get that far! I gotta keep going... Popculture references aren't stupid, they can be great ways to communicate! One of my favorite animes is Gurren Laggan, folks have serious spiritual relationships to that one, haha

@microwavecoffee 4 ай бұрын

@@Thinkstrthat's a classic for sure!

@microwavecoffee 4 ай бұрын

Love your videos! Thanks for this Edit: I coded a very basic version of this, with a JavaScript RPG library and ChatGPT for an educational game where the villages teach the player about the blockchain (I don't like crypto but it was required for the hackathon). It was fairly barebones without any complex system in the background. Something that I think is currently really underdeveloped in Computer Science are long-term agents with memory systems. Like the idea of memory creation and memory retrieval is incredibly fascinating to me, but also Cognitive Architectures is an under-researched field. What do you think could be a good benchmark for evaluating long-term efficient memory storage in agents? Models for forgetting memories and retrieving memories efficiently are so interesting to me.

@Thinkstr 4 ай бұрын

Hi, thanks for watching! SmallVille is really impressive to me even though I complain about parts of it. Like you, I've tried ChatGPT's API myself, and it's hard to work with! For long-term memory, the first thing that comes to my mind is Catastrophic Forgetting, where the benchmark is in sorting MNIST's handwritten digits. If you train a model to tell the difference between 0s and 1s, it'll do fine... but then if you train the same model to tell the difference between 2s and 3s, it'll forget how to tell the difference between 0s and 1s! So it's important to make the model remember old tasks. I happen to have some video about it! :3 haha. I notice the audio isn't great. I've learned a lot about video-editing along the way. The problem: kzbin.info/www/bejne/roCldnhnn6tnsMk&ab_channel=Thinkstr A solution: kzbin.info/www/bejne/m4fYiGahmbWIrsk&ab_channel=Thinkstr That solution in reinforcement learning: kzbin.info/www/bejne/nKncpaeca7lomaM&ab_channel=Thinkstr

@microwavecoffee 4 ай бұрын

@@Thinkstr Thanks a bunch!

@minecraftermad 4 ай бұрын

13:30 more like it's just more believable than humans, which is a slight difference but a difference none the less. it's good at lying :) finally, automated lying! "it's got what people crave! lies it's all a person needs" 22:30 honestly i think humans are obsolete design, personally i welcome our future overlords and grave guardians, they'll be better than us. *insert scene of aliens finding earth which is just a chat gpt generated hellscape of incomprehensible acts and features* anyways, here's hoping a 1.75 trillion dimensional pachinko game wont become sentient before we figure out what sentience is.

@Thinkstr 4 ай бұрын

Sometimes I ask for help in python coding. It might say "just use this function," and when I try, python tells me "that's not a real function." 😤haha

@weirdsciencetv4999 4 ай бұрын

Generally with the free energy principle you’re minimizing the level of surprise from an internal model and the environment. This can be effected by either altering the model or taking action in the environment. Could you frame your work in these terms?

@Thinkstr 4 ай бұрын

Hi, thanks for watching! If I understand correctly, the paper in this video applies both those paths. The model is improved based on new observations to minimize surprise. The optional way to do that is by taking actions finding parts of the environment which are still surprising until they are understand. I think this is what is called Active Inference. Although, a criticism of the FEB is that taking the action of sitting in a dark room definitely limits surprise. But if someone knocks on the door, that's surprising and leads to uncertainty! So it helps to explore and meet the neighbors instead.

@weirdsciencetv4999 4 ай бұрын

@@Thinkstr yes, I get that much, great explanation there. These actions though initially increase the level of surprise, which of course the algorithm then beats back with a stick. The end result is a model which efficiently explored the space in which to model things. I think when you said “essentially sitting in a dark room would eliminate surprise” is pretty dead on. So would we say an additional mechanism, that effects this exploration is at play, that it must exist, it is essential?

@Thinkstr 4 ай бұрын

@@weirdsciencetv4999 That might be good reason for entropy and extrinsic rewards. Starting with lots of randomness in action-selection can make someone leave the house out of boredom, and getting hungry (like missing an extrinsic reward) might give motivation to find a grocery store.

@weirdsciencetv4999 4 ай бұрын

@@Thinkstr i think the FEP is onto something. I want to make evovable models that demonstrate this. Essentially use artificial evolution to chart out plausible mechanisms in biological and artificial networks. I love your channel, I subscribed!

@Thinkstr 4 ай бұрын

@@weirdsciencetv4999 Thanks! Not this month, but next month, I'll post a video about my own paper. The FEP was absolutely impenetrable when I first heard about it, but it's always making more sense and more related to biological thought as I work with it.

@Giraffeythegiraffe 4 ай бұрын

😢

@Thinkstr 4 ай бұрын

You're probably one of the good giraffes! ❤️

@Giraffeythegiraffe 4 ай бұрын

@@Thinkstr yes I am

@KaliFissure 5 ай бұрын

Dark matter is decayed Neutrons. Dark energy is the expansion caused by the decay from neutron 0.6fm³ to 1m³ of amorphous hydrogen gas. Neutron decay cosmology. A homeostatic universe maintained by the reciprocal processes of electron capture at event horizons and free neutron decay in deep voids. Neutrons in at event horizon They take EinsteinRosen bridge from highest energy pressure conditions to lowest energy density points of space Neutrons out in deep void Decay into DM creating DE Then they fall.

@bluesteno64 6 ай бұрын

I LOVE THAT BOOK!

@Thinkstr 6 ай бұрын

Me too! The TV show is good too, but I think it mostly retells the book.

@monzpush9354 6 ай бұрын

brooo iv been baked az n thought about what its like to be fish loads of times 😯

@Thinkstr 6 ай бұрын

🐟🐠🐡blub blub🐟🐠🐡

@skalibercaelesti 6 ай бұрын

Having a huge amount of text on screen whilst also talking is not so pleasant, we're incapable of reading and listening at the same time, so you're competing with yourself. Sometimes I don't need to stop the video, because you're reading the text word for word, and sometimes I should have, but didn't bother because it seemed as though you were paraphrasing the full text you'd already put on screen.

@Thinkstr 6 ай бұрын

Hi! You're totally right. I think I've improved over time, but I have a long way to go. Thanks for telling me.

@CoperliteConsumer 6 ай бұрын

Kanye West moment.

@jeil5676 6 ай бұрын

Ask yourself: "Do you like fish sticks?"

@Thinkstr 6 ай бұрын

Cannibalism ☠

@elihyland4781 6 ай бұрын

loved this!

@Thinkstr 6 ай бұрын

@stevezastrow9252 6 ай бұрын

Love those kids at the end... "It's not raining, it's sprinkling!" "My mom said it's raining" "Well my mom said it's sprinkling." "Raining" "Sprinkling" "...well, you're not real. I'M real."

@mr.k7457 6 ай бұрын

Got pretty deep there, very cool. Good luck with the algorithm!

@Thinkstr 6 ай бұрын

I don't usually get so many views so fast!

@christosgiannopoulos828 6 ай бұрын

Ok. Look. Here's the thing. Animal groups are like colours. We can't exactly tell the exact point in which one becomes another, but we can all identify which one we're looking at

@env0x 6 ай бұрын

that doesn't sound like science at all.

@leitmotif7268 6 ай бұрын

Really enjoying your content, subbed! Thanks for being here!

@leitmotif7268 6 ай бұрын

“You’re not real, I’m real” the toddler’s precipitation semantics debate is my favorite video of all time thank you so much ❤❤❤ “Ow! You poked my heawrt”

@Thinkstr 6 ай бұрын

It's so adorable, I like how the adult just filmed and let it play out.

@leitmotif7268 6 ай бұрын

@@ThinkstrI love the twins’ dynamic 😂 little aggressor in the debate and her sister the little passive healer who comes in to make sure he’s ok after the heart poke ❤🥹sweet babies

@jennaemanuel6733 6 ай бұрын

is it weird i find this video kinda hot?

@Thinkstr 6 ай бұрын

Hahaha, don't worry, these are consentacles!

@ddogg9255 6 ай бұрын

One common argument from creationists is that an intermediate species would not be optimally adapted to neither land nor sea. I think the main problem is that they lack perspective on the variety of niches and the magnitude of the timescales that evolution works on. Evolutionary biologists might understand these as an abstract concept, but still let the tradition of classifying into singular species and a limited set of environments pollute the discussion. I agree, this has more to do with semantics.

@Thinkstr 6 ай бұрын

Hi, thanks for watching! Semantics can help, we gotta communicate somehow. But we shouldn't get bogged down!

@StaminatorBlader 6 ай бұрын

creationists argue against an idea of evolution thats like a misunderstanding and simplification of the darwin era theories. thus what they argue against has nothing to do with oir current theory of evolution and if they understood what it even says they would have no arguments left.

@Thinkstr 6 ай бұрын

@@StaminatorBlader I guess our understanding of evolution has evolved too!

@giannixx 6 ай бұрын

Exactly. You can say different species don't even exist and you'd also be right. But concepts such as species, genera etc are very useful for studying and classifying life and understanding the genealogic relationships between different living beings.

@DetectiveStablerSVU 6 ай бұрын

It's the same as controller mapping on PC through pure trial and error, or even etch a sketches sort of have this experience for most people that aren't wizzes with them

@Thinkstr 6 ай бұрын

Ha ha, cool metaphor

@DetectiveStablerSVU 6 ай бұрын

@@Thinkstr I love your videos, man. Keep up the awesome work.

@ЕвгенийИванов-ю2б 6 ай бұрын

Greetings from the former Soviet union! I learn English and I'm interested in the confluence of computer science and biology.

@Thinkstr 6 ай бұрын

Hi! I know more about computer science than biology, but both are interesting when they work together.

@pickleddolphinmeatwithhors677 6 ай бұрын

Hell yeah

@КОЛИНОРЕГОНСКИЙМУЗЫКАНТ 6 ай бұрын

Привет, я из США а я учусь Русский язык))

@srijans3867 6 ай бұрын

Bro, your videos are amazing

@Thinkstr 6 ай бұрын

Thanks! They're really fun to make. I just got a new microphone, too, so my voice should be a little better.

@srijans3867 6 ай бұрын

@@Thinkstr cool man. keep them videos comingg 😄

@xg.55555 7 ай бұрын

One more interesting thing about spirals... Poliwhirl was one of the pokemon in the run to be the mascot of pokemon. Poliwhirl was the favorite Pokemon of the inventer of pokemon, too.

@Thinkstr 7 ай бұрын

One of my favorites, too!

@kshitijkapoorr 7 ай бұрын

You're soo underrated smh

@Thinkstr 7 ай бұрын

Thanks! Honestly, it might be better that way, if I got billions of views I'd be stressed out, lol

@Omnicypher001 7 ай бұрын

so ideally you have a data set of not just poor handwriting, but examples where 2s look like 1s and 1s look like 2s, so it gets good at solving the edge cases. you want there to be a fine line between concepts, and weird examples allows it to classify clearly. labeling ambiguous examples is worth more than labeling obvious examples, because they better define the boundary of the vector space holding these answers. extremely ambiguous examples are by definition, at the boundary of their classification, and if they cancel each other out, you don't really need normal examples, because those are just the average of the extreme examples. Weird examples don't just make it better at remembering, it makes it better at understanding the actual boundaries of the concepts its classifying.

@Thinkstr 7 ай бұрын

I think that's a good way to put it. If you remember the really unusual cases, maybe they're describing the most notable features.

@rogerzimmerman304 8 ай бұрын

Thanks for creating this video, this makes active inference and the free energy principle much clearer, and also explains how the brain thinks to minimize perception and reality. The examples were great, and I understood it correctly; that we prefer to assume we are full (not hungry) vs hungry, so our tendency is to eat to minimize the error (free energy) between thinking we are full and being full, also when we think we will accomplish something we put more effort to accomplish it so that we don't have an error (free energy) between thinking we accomplished it and not doing it. The active inference is the actions to minimize the free energy. I also think this is why once people make up their mind about something, even if they are wrong they are very difficult to change their mind to what is correct.

@Thinkstr 8 ай бұрын

Yeah, I think that's one way to view it! I'm working in reinforcement learning right now, so I kind of use it the other way around: the agents seek things they can't predict, so they can make better predictions in the future.

@Eta_Carinae__ 8 ай бұрын

I think that a better analog for the FEP is in RL, than a generic Neural Network, just because with RL, you have a limited number of resources to both learn and optimise your bees' behaviour. I think in his seminars, Friston points specifically to RL as something the FEP generalises (as a instance of Expected Utility Theory IIRC).

@Thinkstr 8 ай бұрын

I've gotten FEP to work pretty well in RL! I can't give much information yet until I publish a paper, but it can encourage exploration to minimize uncertainty.

@enamulhaque7135 8 ай бұрын

I have downloaded MiniImagenet data set which is already divided into train(64 class), test(20 class), val(16 class) into .jpg file. But you had .pkl data. Is your code compatible for my data set?

@Thinkstr 8 ай бұрын

I'm guessing so, but you may need to change either your data or the way my code uses it. Good luck!

@enamulhaque7135 8 ай бұрын

@@Thinkstr can i have the data you used?

@Thinkstr 8 ай бұрын

@@enamulhaque7135 I'm afraid I don't remember where I got it.

@whiteraven6260 8 ай бұрын

"If fish don't exist, the fish were inside us the whole time, right?" Iconic

@Thinkstr 8 ай бұрын

Ha ha. There's a book called Your Inner Fish, I should give it a read.

@whiteraven6260 8 ай бұрын

@@Thinkstr sounds like you're diving deep into the subject :^]

@whiteraven6260 8 ай бұрын

@Thinkstr I didn't even compliment your video, my bad 🤤 Great work, Mr Squid sir, I thoroughly enjoyed it, from start to finish ✨️

@Thinkstr 8 ай бұрын

@@whiteraven6260 Thanks! This is by far my more popular and most controversial, haha.

@DeltaLxKupa 8 ай бұрын

SENTRY DOWN !!!!!

@Thinkstr 8 ай бұрын

...Buildin' a sentry!

@microwavecoffee 9 ай бұрын

When you mentioned dropout layers I actually started laughing! That's such a positive approach to life

@Thinkstr 9 ай бұрын

Haha, thanks! My mum asked the neurologist we saw, "could this be a good case-study?" lol

@microwavecoffee 9 ай бұрын

@@ThinkstrSounds like it would be!

Ең жақсы KZbin

Пікірлер