How can we add knowledge to AI agents?

Lance Da Costa models cognitive systems. He's a PhD candidate with Greg Pavliotis and Karl Friston jointly at Imperial College London and UCL, and a student in the Mathematics of Random Systems CDT run by Imperial College London and the University of Oxford. He completed an MRes in Brain Sciences at UCL with Karl Friston and Biswa Sengupta, an MASt in Pure Mathematics at the University of Cambridge with Oscar Randal-Williams, and a BSc in Mathematics at EPFL and the University of Toronto.
Please support us / mlst
/ discord
/ mlstreettalk
Summary:
Lance did pure math originally but became interested in the brain and AI. He started working with Karl Friston on the free energy principle, which claims all intelligent agents minimize free energy for perception, action, and decision-making. Lance has worked to provide mathematical foundations and proofs for why the free energy principle is true, starting from basic assumptions about agents interacting with their environment. This aims to justify the principle from first physics principles. Dr. Scarfe and Da Costa discuss different approaches to AI - the free energy/active inference approach focused on mimicking human intelligence vs approaches focused on maximizing capability like deep reinforcement learning. Lance argues active inference provides advantages for explainability and safety compared to black box AI systems. It provides a simple, sparse description of intelligence based on a generative model and free energy minimization. They discuss the need for structured learning and acquiring core knowledge to achieve more human-like intelligence. Lance highlights work from Josh Tenenbaum's lab that shows similar learning trajectories to humans in a simple Atari-like environment.
Incorporating core knowledge constraints the space of possible generative models the agent can use to represent the world, making learning more sample efficient. Lance argues active inference agents with core knowledge can match human learning capabilities.
They discuss how to make generative models interpretable, such as through factor graphs. The goal is to be able to understand the representations and message passing in the model that leads to decisions.
In summary, Lance argues active inference provides a principled approach to AI with advantages for explainability, safety, and human-like learning. Combining it with core knowledge and structural learning aims to achieve more human-like artificial intelligence.
POD VERSION: podcasters.spotify.com/pod/sh...
www.lancelotdacosta.com/
/ lancelotdacosta
Interviewer: Dr. Tim Scarfe
TOC
00:00:00 - Start
00:09:27 - Intelligence
00:12:37 - Priors / structure learning
00:17:21 - Core knowledge
00:29:05 - Intelligence is specialised
00:33:21 - The magic of agents
00:39:30 - Intelligibility of structure learning
#artificialintelligence #activeinference

[@george5464]

You can really sense with this talk and Lancelot papers that he is going to be an integral part to neuroscience and AI for a number of decades 🤘

[@diga4696]

Such a great conversation, i just love to chill and listen to mlst. This is why I am alive.

[@gavindheilly3620]

Thank you so much for your channels work. Truly the best source of discussion on these topics that I've been able to find and with guests that continue to inspire & surprise. Truly humbled to have access to such a great resource. Thank you.

[@MachineLearningStreetTalk]

🙏 thank you!!

[@lucianamanolea348]

amazing interview again!

[@petermorsestudio]

Great interview. Thanks!

[@johntanchongmin]

Great interview! I have been researching on adaptability for a while now, and here are some of my insights to add to the conversation: 3 predictions of mine (that I exemplified in my recent paper "Learning, Fast and Slow" for fast and adaptable agents in changing environments): 1. Reward is not generalisable, we should do goal-directed learning to predict actions given a start and goal state without any reward/value function - Reward is more of an engineering hack, if you change the environment, rewards change and it takes time for Bellman updates to change every intermediate state 2. Memory is needed for fast learning - referencing past memory of episodes is much faster than training a neural net to get the right policy by backpropagation 3. Probabilities are not able to model the world - there are just too many possibilities to model - we should model it with memory retrieval and matching to nearest memory based on a query-key mechanism like that of a transformer

[@sapienspace8814]

Computational language is Fuzzy Logic, but "Fuzzy Logic" is a self incriminating terminology, yet it is well aligned with the "chaos" of the real world, where everything has an associated error. Combining Fuzzy Logic with Reinforcement Learning allows for learning of inference rules in the state space, and hence, "intelligence", a merging of language, learning, and logic.

[@vinca43]

Really enjoying this. Regarding "there has to be a dynamic that governs the world...", that's a pretty strong statement. Maybe, "it's possible that a dynamic governs the world." Tim's comment about the principle being a lens, versus actual representation of a real thing is spot on.

[@swayson5208]

cryptocracy D:

[@Robert_McGarry_Poems]

Great stuff

[@xgxfhzxfuhfjgfhgf]

Than you for the video

[@Thanos916]

From Claude: "Here are some questions that seem to still be open based on the discussion in the transcript: What is the best set of core knowledge priors to build into agents to enable efficient structural learning? How can we reverse engineer and represent human cognitive priors? How exactly can we scale up active inference agents and free energy-based models to handle complex, realistic environments? What techniques from deep learning and other AI methods can be integrated? What are the mathematical limits on interpretability and explainability, even given approaches like factor graphs? Is there a level of model complexity beyond which intelligibility breaks down? How restrictive is the free energy principle as a theory of intelligence? Could there be other equally valid mathematical formalisms of intelligence that make different assumptions? Is structural learning fundamentally brittle and opaque, relying as it does on composition of lower level building blocks? Can the overall learned models be made interpretable? How can we build free energy-based agents that display human-like flexibility and adaptation? Should lifelong learning be incorporated and if so, how? How do we evaluate and test if the behavior of free energy-based agents truly matches human intelligence in a deep way? Are existing benchmarks sufficient? To what extent do the benefits of the free energy principle come from the overall agent-based framework rather than active inference specifically? So in summary, open questions remain around issues like priors, scaling, flexibility, evaluation, and the scope of the free energy principle as a complete theory of intelligence."

[@MLDawn]

Lance is such a great academician and individual! This was a great conversation! Do we know the title of the paper that Lance was talking about at 02:56 ?

[@renanmonteirobarbosa8129]

We need those memes on a T-shirt

[@markkeeper7771]

🎯 Key Takeaways for quick navigation: 00:00 🧠 Introduction: Path to AI and Neuroscience - Lance's background: Lance, a PhD student with Karl Friston, discusses his journey from pure mathematics to AI and neuroscience. - Motivations: Motivated by a family history of psychiatric diseases, Lance chose AI for its potential profound impact on society and delved into neuroscience and the Free Energy Principle (FEP). - FEP Exploration: Lance's introduction to FEP, its exciting potential, and the need for underlying mathematical foundations to justify its claims. 02:53 🤖 Foundational Work on the Free Energy Principle - FEP's potential: Lance highlights FEP's claim that all intelligent agents minimize free energy, emphasizing the need for rigorous mathematical foundations. - Thesis Focus: Lance's work revolves around laying out mathematical foundations supporting FEP, addressing the lack of rock-solid reasons behind the principle. - Upcoming Papers: Anticipation of forthcoming papers that aim to provide convincing mathematical justification for the applicability of FEP to intelligent agents. 07:34 🕹️ Understanding Intelligence and FEP Applications - Dynamics and Intelligence: Lance explains the fundamental assumption of dynamics governing agents' interactions with the environment and the role of stochastic differential equations. - Lens of Physics: Discussion on the lens of physics and mathematics in expressing intelligence and FEP as a lens to model intelligence in a material, physical context. - Anthropomorphic Challenges: Acknowledgment of FEP's applicability to various organisms, emphasizing the ongoing need to refine it for exclusive application to brains for more powerful algorithms. 11:23 🔄 Evolution of Intelligence Theories - Intelligence Versions: Lance categorizes FEP's current stage as "Intelligence 1.0," highlighting the need for further refinement to achieve "Intelligence 2.0, 3.0, etc." - Integration of Approaches: Discussion on the convergence of FEP, active inference, and AI techniques like deep learning for a comprehensive understanding of intelligence. - Technological Leap: Identifying the challenges of structured learning and the importance of core knowledge as keys to advancing AI algorithms, with a focus on creating models that scale and share representations with humans. 16:37 🎯 Challenges in Structured Learning and Core Knowledge - Structural Learning Challenges: Identifying structural learning as a major hurdle and the difficulty in creating models that construct a scalable understanding of unknown environments. - Core Knowledge Importance: Emphasizing core knowledge as a tool to reduce the complexity of the search space for generative models and address optimization challenges in structured learning. - Optimization Dilemma: Discussion on the intricate optimization problem in creating models of the world and the potential of core knowledge to guide and simplify the search process. 19:00 🤔 Debates on Priors, Bias-Variance, and Human Knowledge - Rich Sutton's View: Reflecting on Rich Sutton's "Bitter Lesson" and the skepticism towards imbuing human knowledge into systems, advocating for computation over explicit priors. - Revisiting Approaches: Comparing current trends, including human-encoded knowledge in systems like ChatGPT, to previous approaches like expert systems and reinforcement learning. - Hybrid Approaches: Acknowledgment of the need for hybrid models that leverage insights from various AI approaches, including active inference, core knowledge, and engineering techniques, aiming for intelligent behavior beyond neural networks. 21:50 🧠 Goals of AI Researchers - AI researchers have different goals: - Richard Sutton and others aim to create intelligence without being constrained by human-like representations. - Josh Tenenbaum and like-minded researchers focus on human-like artificial intelligence and sample efficiency, emphasizing core knowledge. 23:20 🎲 Structural Space in AlphaGo's Learning - AlphaGo's skill space is structured but not exhaustive; there are holes in its knowledge. - The skill space is more pointillistic, revealing limitations in certain situations. - Benchmarking AI systems should consider these limitations and avoid overgeneralization from specific tasks. 25:17 📊 Benchmarks and Challenges in AI Evaluation - Benchmarks can be misleading, especially when comparing human-like and non-human-like AI. - Evaluating AI performance requires a nuanced understanding of the benchmarks and their limitations. - The current benchmarks may not effectively capture the true capabilities and limitations of AI systems. 28:37 🧩 Human Specialization and Core Knowledge - Human intelligence excels in specialized tasks enabled by core knowledge. - Core knowledge, developed through evolution, contributes to tasks like emotion and face recognition. - The challenge in AI is to reverse-engineer and incorporate core knowledge to achieve human-like abilities. 30:54 🔄 Adaptability and Age-related Constraints - Human adaptability decreases with age due to the challenge of unlearning and relearning. - Machines, with Bayesian model reduction, can unlearn efficiently, making them more adaptable. - The free energy principle highlights the challenge of human adaptability in the face of rapid technological progress. 35:43 🤖 Active Inference and Explainability in AI - Active inference provides a theoretical bridge from first principles to practical applications. - Safety and explainability are enhanced in AI systems using active inference. - The generative models in active inference are evolving towards greater interpretability, addressing challenges in understanding complex decisions. 39:07 🌐 Generative Models and Neural Networks in AI - The integration of neural networks into generative models sacrifices explainability. - Future developments aim to create generative models without neural networks for better interpretability. - Balancing machine learning components with interpretability is crucial for applications like self-driving cars and assistive surgery. 41:30 🧠 Probabilistic Programs and Factor Graphs - Discussion on the challenges of probabilistic programs. - Introduction to the factor graph approach as an alternative. - Benefits of factor graphs in understanding agent representations and information flow. 42:55 🤔 Structuring Learning Process for Explainability - Adding pressures to structural learning for better understanding. - Considerations on priors, such as parsimony, to simplify models. - Reflecting on challenges faced in specific AI challenges like the Arc challenge. 44:24 🔍 Free Energy Principle and Model Complexity - Explanation of the free energy principle for explaining the world. - Decomposing free energy into accuracy minus complexity. - Optimization process to achieve sparser and less complex models. 46:17 🔄 Different Approaches in AI - Comparison of top-down (free energy principle) and bottom-up (Tenenbaum's approach) AI perspectives. - Excitement about the convergence of models from different approaches. - Reference to an Atari-like environment experiment validating active inference learning trajectories. 48:13 🕹️ Active Inference and Learning Trajectories - Discussion on a paper demonstrating active inference reproducing human-like learning trajectories. - Insights into the efficiency and pattern similarities between agents and human participants. - Impact of core knowledge on agent performance in complex environments. 49:43 🌐 Core Knowledge in Complex Environments - Exploration of the role of core knowledge in complex environments. - Reflection on the importance of core knowledge for human-like learning efficiency. - Consideration of goals and objectives in determining the relevance of core knowledge.

[@luisluiscunha]

The tendency to undervalue or dismiss deep learning strikes a familiar chord with me. It echoes the way Marvin Minsky once disparaged the work of Frank Rosenblatt, stunting the growth of early neural network research. This dismissive attitude persisted to an extreme, reflected in the second edition of Russell and Norvig's renowned textbook on "Modern AI", which I own. Despite the book's expansive 1100 pages, a mere 17 are dedicated to neural networks. That is a world I would not like to return.

[@Wardoon]

This guy seems to be deep into this shit and will definitely contribute much to ai development

[@alertbri]

🙋

[@manuellayburr382]

With respect to explainability, I'm surprised there was no mention of introspection - this is how humans explain their actions. Surely an automatic car that could use human language to tell us why it did something would be a way to go. Not only that but we could potentially interrogate and correct its logic. Is no-one looking at this?

[@luisluiscunha]

Watson, the Pioneer psychologist, would strongly disagree with you 😊

[@manuellayburr382]

@@luisluiscunha I think that the work of Liz Spelke and others (as mentioned in the video) has shown that the *tabula rasa* ideas of Watson's brand of behaviourism do not provide an adequate model for human behaviour except under very limited conditions. There is a big difference between introspecting on a question such as "Why do you love your mother?" which Watson was objecting to, and "What mathematical equations did you use to calculate the trajectory?" The former requires a guess as to why certain feelings arose whereas the latter requires a logical argument.

[@haldanesghost]

Holy reductionism, Batman! The **physicists** are gonna save us coconut-head biologists using *math*.

[@mfpears]

Put glasses with only horizontal slits on kittens and they will not be able to perceive vertical edges as they grow up. What part of optical information is core knowledge? Object permanence isn't totally core knowledge either, or peekaboo wouldn't be interesting to babies.

[@mfpears]

Imo compute is all you need, and the order of knowledge gained is critical.

[@dunebuggy1292]

If intelligence is a physical thing, then 1s and 0s certainly will not confer intelligence. Issue is we say profound things but gloss over the contingent nature for why this profundity is either damning or permissive. Non-biologic constructs can reflect back state differences, but they're not participating in the phenomenon that produces those states. This should be pretty clear, I think.

[@davidwitt6198]

What if the non-biologic was able to control external factors ? (Controlling bionic extensions)

[@dunebuggy1292]

@@davidwitt6198 That interaction is just a state exchange predicated on some model. The lack of physicality of data means that such data has no "understanding" or "memory" to be "reflexive" towards. These words all implicate some synthetic process that can't be another series of states, requiring storage. Biological systems do not rely on storage space. The bionic example can be seen similarly with the idea that computer vision will somehow give real-world understanding. Computer vision is not giving a computer 3D vision or actual vision. Computer vision is us deciding to take 3D information to be represented by some combination of 2D data. Thus, everything, again, is nonphysical. It's just more of the same thing; what makes computer vision neat is it being a different way of combining 1s and 0s.

[@didack1419]

If that is true will just have to start using neuromorphic chips instead of binary chips. Regardless, it’s probably not true because there’s nothing fundamentally limiting in using binary. The fuzzy values of neural spikes can be represented in bits, biological neurons also compute with finite information and delete information.

[@dunebuggy1292]

@@didack1419 Refer to Michael Levin's research: biological organs have a fundamental intelligence; a notional space of experience. Just because neurons function speciously like computers, doesn't mean that they are (in fact, neurons fire at higher rates and would produce higher entropy than a regular computer could regulate, commensurately). This is all to say that the information existing in biological organs do not supersede the organs. They are simply messages to a thing that has intelligence, fundamentally. A computer does not have fundamental intelligence. It has a series of processes that act out the end-to-end representation of information. Sure, you can broaden the routes from A to Z, but all you're doing is filling a circle with squares. It's never actually homeostatically intelligent.