A Curiosity Algorithm for Robots Based on the Free Energy Principle (with Dork Matter Girl!)

Go check out Alona's channel! / @dorkmattergirl
The rest of our conversation: • What is Consciousness?...
Also, no offense to JJ Abrams, haha. You can leave a story's mystery box unopened, I just think there should be SOMETHING in the box! Empty boxes feel like false advertising. Pulp Fiction gets away with it because what's in the suitcase is unrelated to the plot, it's a MacGuffin.
patreon.com/thinkstr
00:00 Unrelated
01:49 Introduction
03:44 Abstract
05:34 Experiment
07:17 Bayesian Statistics
16:05 DKL and the Free Energy Principle
18:02 Transformer/Forward model/World model
22:33 Expected Free Energy
26:37 Chart and Pseudocode
29:43 Results
32:02 Conclusion

[@weirdsciencetv4999]

Generally with the free energy principle you’re minimizing the level of surprise from an internal model and the environment. This can be effected by either altering the model or taking action in the environment. Could you frame your work in these terms?

[@Thinkstr]

Hi, thanks for watching! If I understand correctly, the paper in this video applies both those paths. The model is improved based on new observations to minimize surprise. The optional way to do that is by taking actions finding parts of the environment which are still surprising until they are understand. I think this is what is called Active Inference. Although, a criticism of the FEB is that taking the action of sitting in a dark room definitely limits surprise. But if someone knocks on the door, that's surprising and leads to uncertainty! So it helps to explore and meet the neighbors instead.

[@weirdsciencetv4999]

@@Thinkstr yes, I get that much, great explanation there. These actions though initially increase the level of surprise, which of course the algorithm then beats back with a stick. The end result is a model which efficiently explored the space in which to model things. I think when you said “essentially sitting in a dark room would eliminate surprise” is pretty dead on. So would we say an additional mechanism, that effects this exploration is at play, that it must exist, it is essential?

[@Thinkstr]

@@weirdsciencetv4999 That might be good reason for entropy and extrinsic rewards. Starting with lots of randomness in action-selection can make someone leave the house out of boredom, and getting hungry (like missing an extrinsic reward) might give motivation to find a grocery store.

[@weirdsciencetv4999]

@@Thinkstr i think the FEP is onto something. I want to make evovable models that demonstrate this. Essentially use artificial evolution to chart out plausible mechanisms in biological and artificial networks. I love your channel, I subscribed!

[@Thinkstr]

​@@weirdsciencetv4999 Thanks! Not this month, but next month, I'll post a video about my own paper. The FEP was absolutely impenetrable when I first heard about it, but it's always making more sense and more related to biological thought as I work with it.