Are artificial "neurons" like biological neurons?

[@aminacodes]

the blender-human comparison was the highlight of my LIFE

[@mikexcohen1]

It's also a great ice-breaker at socially awkward parties :P

[@austincarter2177]

You have to write a Deep Learning book. Your Linear Algebra: Theory, Intuition, Code is the best thing I’ve ever laid hands on.

[@mikexcohen1]

Thank you for your enthusiasm, Austin :D I don't have plans to write a DL book, but I hope the full version of this course is useful. I'm currently working on a calculus book, which hopefully will be finished in the first half of 2025.

[@austincarter2177]

@@mikexcohen1 Even better then, your stuff is a game-changer!

[@jake66664]

The main justification for using copyrighted data in training commercial generative AI models is hinged on this very notion that ANNs are doing the same thing the human brain does when it learns skills like drawing or writing. Professional creatives have unknowingly had their own work used to automate them out of their own careers. This has happened, is currently happening, and will continue to happen if something doesn't change. The terminology may not bother you much but it certainly stings for those who's lives have been changed for the worse by something that probably shouldn't have happened if we were a bit wiser. That being said I appreciate you taking the time to make this educational video.

[@mikexcohen1]

Excellent point, Jake. And I agree completely that there are serious legal and IP issues with generative AI. I think we're only a few years away from someone being able to create a full-length course on ancient Egyptian history using my voice, slide template, writing style, educational approach, and humor -- and I won't get a cent of royalties or even acknowledgement. (Also: I know nothing about Egyptian history, so it would be my style combined with historical documents.) But that's a separate issue from the biological relevance of DL models, and this video is only about that relationship.

[@cortical1]

This is very similar to the ubiquitous use of the term functional connectivity in cognitive neuroscience. It is totally not connectivity. It is merely correlated super low frequency fluctuations in the hemodynamic response; it's not even neural activity. Its use is a marketing scheme and it's a genuine shame that it has now been reified with widespread use. Interregional correlations in the BOLD signal are not connectivity.

[@mikexcohen1]

Excellent point, C. Confusing, conflicting, or imprecise terminology is a real pain in many fields. In many cases, poor terminology is just a minor annoyance to adapt to, but there are other cases where terminological confusion leads to conceptual misunderstandings.

[@cortical1]

@@mikexcohen1 Yes, it sure does. We now have a whole generation of newly trained grad students who have a fundamental misunderstanding about the neurophysiology of human brain function because of "functional connectivity" taking over imaging. It has also replaced actual imaging experiments with merely passive measurements during rest, another big loss for scientific inference making. SMH.

[@ValaAssistant]

The issue here is that, you think that they are using the underlying vector space being normal numbers the modern systems are usually are using a many-dimensional underlying vector space. tensors inherently change every problem into one of geometry. They can do what real neurons can, just gotta program them to do that. No need to change terminology.

[@tauiin]

My god thank you for this, I've been trying to make sense of all this stuff surrounding Generative AI and it's honestly been very confusing, because often people-mostly reddit and hackernews types-will make silly judgements ("we're just like GPT bro! its neural networks all the way down!", "AGI is just around the corner!!!", "AI Arts real art dude!!!", "how do you know *we're* not like them?") And then people will counter with (what are imo) philosophically weak retorts ("Its just a neural network, we aren't", "it's just a stochastic parrot", "it can't think") so its nice to get some clarification on what exactly are the differences and similarities between us and these transformer models, especially from the people who I've been wanting to hear from the most around this new tech, neuroscientists! at the very least I hate Generative AI for reasons unrelated to whatever mathematics goes into their making (and that they should've never been created, although that's just me being a bit of a luddite) :P

[@mikexcohen1]

Thanks for your comment, tauiin. Artificial neural networks are amazing and fascinating computational systems in their own right, with really nothing to do with biological brains. The naming and hype is indeed annoying at times :P

[@manavsingh9958]

Hi mike i am beginner from india interested in taking your Udemy course just want some clarity . s the course updated as Ai is evolving every day is the course relevant for the scenarios today and are there any BIG PROJECTS in the course which will help me to boost my confidence by implementing the knowledge and also help in my CV .

[@mikexcohen1]

This course focuses on foundations of deep learning and AI. It is true that AI is actively developing, but the developments are mostly in scale, architectural tweaks, and applications. In terms of the foundational math and operations, not much has changed in 25 years.

[@manavsingh9958]

What about any big project's?

[@marcohemm]

Hi Mike, are the slides downloadable if I purchase the course on udemy?

[@mikexcohen1]

Hi Marco. I don't make my pdf slides available, but you get lifetime access to the entire course.

[@marcohemm]

@@mikexcohen1 I trust your expertise so I bought it

[@vctrspike]

Why a such quality content has so few views?! Only 900 that’s a real insult

[@mikexcohen1]

Thank you for the compliment :) I just uploaded this video a few days ago.

[@vctrspike]

@@mikexcohen1 also what books would you recommend to read about how biological neurons process information and generally about bio neural networks? I have already read Neuroscience for dummies and Fundamentals of computational neuroscience by Thomas Trappenberg.

[@aminsinichi]

What a fantastic video this is!

[@mikexcohen1]

Thank you, kind internet stranger.

[@Gin-toki]

Well, we do take into consideration how much horsepower a car can deliver when choosing a new one, so it's not completely detached from a horsie :P Also I really dislike buzz words within the scientific community, especially when they spread out to the public and become mainstream synonymous with certain types of technology, it's really frustrating! *sighs* Sadly it is often necessary for scientist and researchers to use those buzzwords when applying for grants for their research, because the people with money, are often easier to persuade whith these words. A long time ago, it was nuclear or atomic that was the buzzword needed to get research grants, even if what you worked on wasn't remotely related to that field. Another one was Cryogenics. And in recent years there have been Nano, quantum and now AI :P

[@mikexcohen1]

Great comment, thanks! And yeah, a lot of this issue is just people throwing around buzzwords because that's what other people expect.

[@GoodTechConf]

thanks for this

[@TheLucifer1992]

? a few neurons with ReLU activations can easily model the non linearity of biological neurons. Why does the difference in non linearity between artificial and biological neurons matter?

[@mikexcohen1]

Great point, TL: As a universal function approximator, a sufficiently complex DL model can approximate any input-output transformation that could be measured from a neuron. But that doesn't mean the DL model has the same mechanisms as a neuron, or that the two have anything to do with each other. Why does it matter? It doesn't, except when people claim that one is a model of the other. By analogy: If I showed you a paper airplane, you might think it's cool, but if I told you that my paper airplane is an accurate model of a bird because both can sustain flight for a period of time, then you'd call me crazy...

[@TylerMatthewHarris]

Love this video thank you

[@Dr.Z.Moravcik-inventor-of-AGI]

1. I've said that already some 8 years ago (about neurons) 2. It's not marketing but propaganda 3. Neurophysiology (aka this coming from us) didn't give me substantially more than I've learned already some 30 years ago (I was student then) 4. I've made video years ago that brain *MUST have some sort of address bus* for which many neurons must be responsible (kzbin.info/www/bejne/eJivgnmmabaclcU) 5. I don't care what americans say about their ai. It won't change the course of this world.

[@cortical1]

The claims are not specific to Americans. And AI has already changed the way many businesses in the world operate.

[@thomasgarman6353]

These computers are achieving something like real intelligence by implementing an abstract representation of the computation that biological nervous systems do, I think that fact alone makes neural networks a great name. And though it is true that the abstract representation is widely different, there is a fundamental process really being replicated. Intelligence isn't only reserved for neuroscience, sodium potassium pumps are not the fundamental elements driving problem solving behavior. I honestly think it would be dishonest to NOT have it named neural networks, because you would hiding the fact that YES, something like whats happening in the brain is happening in those neural networks

[@mikexcohen1]

Thanks for the comment, Thomas! But what is the basis of your claim that ANNs have the same computation as real neurons? The computations are actually quite distinct (in fact, we don't even know the basis for the brain's intelligence -- there are fundamental problems with the action potential as the elementary computational unit, and the focus on APs is based on historical precedent and convenience, not because it's established to be a computational foundation. In this sense, ANNs are not actually replicating anything biological). AI has already shown (imho) that neurons are not the only basis for intelligence; it's possible that intelligence can arise from many distinct physical and computational mediums. To me, that is a much more fascinating and thought-provoking idea than a forced equivalence of AI-to-biology that breaks down except at the most superficial glance. Anyway, thanks again for the comment :)

[@thomasgarman6353]

⁠@@mikexcohen1 well, how about back-propagation ? It updates the weights of each node in the model according. to stimulus. Isn’t that much the same as when the neurons change the distribution of receptors at the dendrites in response to a specific pattern of stimulus ? I mean a sea-slug nervous system is different than that to humans, but isn’t there a basic principe are happen in both ?The fundamental computation I am pointing is that change in weighting. And YES I am wholly in agreement with you that there are different mediums for intelligence, and that’s exactly the point. There are different mediums but there’s something fundamentally the same. It just happens that the nervous system discovered it first., I highly respect you and your work. Really, I have a copy of your book that was recommended by people at my old EEG lab, they carried with reverence. I just felt a deep offense to this video lol

[@jake66664]

@@thomasgarman6353 The brain is not doing backpropagation.

[@thomasgarman6353]

@@jake66664 I didn't say that the brain was doing back-propagation. That is a particular mechanism by which you can update the weight of a node. In a real brain the analog is changing the distribution of receptors on the dendrites so that the neuron is more or less likely to fire for a given stimulus. The two mechanisms are clearly different but the essence is the same: that you have a node whose "weight:" is being changed in response to data/stimulus

[@louiscyphre7620]

@@thomasgarman6353well , two things being similar in certain,limited aspects, doesn't make one a good model for the other and viceversa, like it was already stated by the analogy of the blender and the human in the video . just because both can take solid/liquid food and output processed food, doesn't make a blender a good model for a human. Also , computers are not "achievers" that's just a human projection.

[@AZ-hj8ym]

We need people like you to tell the truth

[@mikexcohen1]

Thanks :D although this is a point that many neuroscientists have made for many years, perhaps just not on YT.

[@BlueBirdgg]

twitter? would be nice to have you on twitter.

[@mikexcohen1]

Thank you for your confidence in me :P But I'm not on Twitter or any other social media.

[@BlueBirdgg]

@@mikexcohen1 More anti-social than me? Can't! I need you on twitter.

[@zijiexing5307]

谢谢❤！

[@mikexcohen1]

不客气 :)

[@CasperBHansen]

It’s no marketing scam. I think most people in AI would agree with you that it isn’t nearly the same as a neuron. However, changing.a name of a concept in an entire field of study is not exactly easy :P I think you’re spending way too much energy on saying that you have a better model of a neuron and being quite insulting in the process I think. Stick to “this isn’t a good model of a neuron” and move on.

[@mikexcohen1]

I politely disagree, but I appreciate you taking the time to express your view! (Also, for some context: this is one optional video from a 60-hour video course on deep learning, and I don't think a claim like "not a good model" should be made without rigorous justification.)

[@marianodiazrivera9496]