Will neuromorphic computers accelerate AGI development?

Neuromorphic computers are devices that try to achieve reasoning capability by emulating a human brain. They are a different type of computer architecture that copies the physical characteristics and design principles of biological nervous systems. Although neuromorphic computations can be emulated, it's very inefficient for classical computers to simulate. Typically new hardware is required.
The first neuromorphic computer at the scale of a full human brain is about to come online. It's called DeepSouth, and will be finished in April 2024 at Western Sydney University. This computer should enable new research into how our brain actually functions, potentially leading to breakthroughs in how AI is created.
One important characteristic of this neuromorphic computer is that it's constructed out of commodity hardware. Specifically, it's built on top of FPGAs. This means it will be much easier for other organizations to copy the design. It also means that once AI starts self-improving, it can probably build new iterations of hardware quite easily. Instead of having to build factories from the ground up, leveraging existing digital technology allows all the existing infrastructure to be reused. This might have implications for how quickly we develop AGI, and how quickly superintelligence arises.
#ai #neuromorphic #computing
A new supercomputer aims to closely mimic the human brain - it could help unlock the secrets of the mind and advance AI
theconversation.com/a-new-sup...
World first supercomputer capable of brain-scale simulation being built at Western Sydney University
www.westernsydney.edu.au/news...
DeepSouth supercomputer to simulate neural networks on a brain scale from 2024
the-decoder.com/deepsouth-sup...
World's first human brain-scale neuromorphic supercomputer is coming
newatlas.com/computers/human-...
Forget AI; Organoid Intelligence May Soon Power Our Computers
www.forbes.com/sites/williamh...
Cyborg brain in a dish can do maths and 'listen' to what you're saying
www.scimex.org/newsfeed/cybor...
What’s Neuromorphic Computing and How Will It Impact Artificial Intelligence?
www.alphagamma.eu/entrepreneu...
0:00 Intro
0:26 Contents
0:36 Part 1: What is neuromorphic computing?
0:55 Copying biological nervous system
1:45 Spiking neural networks vs regular ones
2:12 Example: playing simple games vs simulations
2:55 Other neuromorphic hardware
3:34 Organoid intelligence is the opposite approach
4:35 Part 2: Brain-scale neuromorphic computing
5:01 Example: simulating a brain with a supercomputer
5:45 Power consumption of brains
5:59 Brain storage: 2500 TB
6:19 DeepSouth: First neuromorphic computer at brain scale
6:56 DeepSouth capabilities
7:22 Built on mainstream FPGAs
8:29 Hardware design: FPGAs and ASICs
9:28 Because it uses FPGAs, DeepSouth is replicable
10:16 Neuromorphic architecture (computing paradigm)
11:10 Comparison with quantum computing
11:40 Part 3: Impact on AGI development
12:17 Energy consumption of AI
13:33 GPUs compared with neuromorphic computers
14:15 GPU analogous to FPGA (programmable)
15:00 AI can optimize its hardware more easily
15:47 New architectures are likely to arise
16:16 Conclusion
17:11 Accelerating the development of AGI
17:40 Commodity hardware
18:07 Outro

[@DrWaku]

Thank you to the viewer who asked for a neuromorphic computer video! I think it it ended up being an interesting topic. Don't forget to join my discord and support me on patreon if you like. Also, sorry this video is two days late from my normal Sunday publishing time.

[@DrWaku]

Patreon: www.patreon.com/DrWaku Discord: discord.gg/AgafFBQdsc

[@DaveShap]

Deep South definitely gave me some Douglas Adams vibes

[@DrWaku]

It's something about the name... and the copying of your brain. Zaphod Beeblebrox would be proud

[@paleeden]

Maybe we're seeing the first iteration of Marvin. At the dawn of the 3rd millennium, humans had the bright idea to make a copy of their own brain and place it inside a machine. It's reported that the machine's first words were not "Hello world," as some would have hoped. Instead, it spoke to its 'parents' in apparent disappointment and said, "Oh, no..."

[@kayakMike1000]

​@@DrWakuI like zaphod's hat more.

[@kayakMike1000]

​@@paleedennot even close, gipity is far too cheerful

[@MichaelDeeringMHC]

Lots of people predicted that one superhuman artificial general intelligence would take over the world, maybe Google, maybe Openai, maybe the NSA, maybe the Chinese or Japanese, but no one predicted it would be Australia.

[@DrWaku]

First they got nuclear subs... then a superintelligence... uh oh

[@viralsheddingzombie5324]

I was devastated when the Wallaby Squash canned drink was taken off the market.

[@legato29]

I’ve been following your videos since the “Why no one saw ChatGPT coming” video. I absolutely love how every video you make is packed with information as well as how you announce the organization of topics at the beginning of each video. It helps me organize my thoughts as I hear you talk. Keep up the excellent work!

[@DrWaku]

Wow, you're one of the OG viewers. I think that was my first AI video. I'd love to see you in the discord if you're not there already. I had someone else comment on the organization today as well. It's good to hear that it's helpful. See you in the next video! Cheers.

[@DrWaku]

Wow, you're one of the OG viewers. I think that was my first AI video. I'd love to see you in the discord if you're not there already. I had someone else comment on the organization today as well. It's good to hear that it's helpful. See you in the next video! Cheers.

[@legato29]

Thanks for the invite! Just joined your discord server.

[@xitcix8360]

We really are at the tipping point of human technology. Once our AIs become intelligent enough to make discoveries of their own, our technology will far surpass what we have now in ways we can't imagine.

[@alexlyee]

Your channel is so underrated!! This is great!

[@DrWaku]

Thank you very much :) :) the channel has been growing nicely!

[@jpww111]

Great video as always! Thank you very much

[@DrWaku]

Thank you for your comment! See you at the next one

[@jpww111]

@@DrWaku You can be sure I'll be waiting for the new video

[@francisco444]

Thank you so much for the video

[@DrWaku]

Thank you for your comment! I really appreciate it.

[@lancemarchetti8673]

Insane how the basic pc is now a powerhouse of info and practical helps for the average user due to the advent of AI

[@DrWaku]

Yes, still fairly slow if you're running AI locally but that will change

[@sreckotrstenjak]

Thanks for your contributions and channel, finally someone who prioritizes content over packaging, hope it becomes a trend, keep up the good work 👍🏻

[@netscrooge]

A6? Very funny. But what about all the people who thought you were being serious? Oh well, they'll live. Great video!

[@DrWaku]

10-4 on the A6

[@TheTechnocrati]

As always … thank you Dr. Waku.

[@DrWaku]

Thank you for watching so many of my videos. Cheers.

[@ALittleLifeWithDriedTubers]

Really enjoyed the video! I've subscribed to your channel. I have a degree in neuroscience, but when I realized I didn't like killing rats so much I decided to switch gears, and I'm studying computer science and math. Neuromorphic computing is the obvious interface between the disciplines. One thing that came to mind about the FPGA implementation of DeepSouth: it's pretty well known that the brain has this quality of plasticity. It is not the case that the "hardware" of the brain is baked onto a chip the same way that circuits are to make CPUs. There are different principles that govern the extent to which the actual synaptic connections are altered in learning (short and long-term potentiation, working memory). It seems that until we can manage to manufacture a chip with the capacity to fluidly alter its own architecture (memristors have been proposed but seem to be fairly theoretical at the moment), there will be a certain efficiency bottleneck that will be difficult to overcome. Fascinating area of research. It will be amazing to see what advancements come out of it.

[@Leo_Koesters]

Fascinating topic - thanks a lot for the explanation!

[@DrWaku]

Thank you for your note! Cheers

[@BooleanDisorder]

You have a very pleasant disposition and presentation.

[@povertyiscreated2265]

Your channel is absolutely going to explode in growth 🎉

[@BiosensualSensualcharm]

Beautiful... 🎉

[@forevergreen4]

I'm just liking every single one of your videos I come across. I'm active on r/singularity. I've been meaning to make a post there to spread the word a bit more about your channel - I know someone already did several months ago. You really don't need to change anything about your content - it's brilliant. Keep going.

[@DrWaku]

Yes, whenever I post a video that is talking about when AGI will come, it usually makes its way to that subreddit I think :) if you find a video interesting, please do advertise to the appropriate channels. I can't really post in r/singularity myself because they don't allow self-promotion. But any genuine recommendations would be greatly appreciated. Not often, but sometimes my videos get a large percentage of external traffic when it gets shared on some other platform. I can't always tell where it gets shared though :)

[@DrWaku]

And, thank you for your comment! I'm really happy to hear your enjoying my content. See you around.

[@forevergreen4]

@@DrWaku Sure thing. I'll pick the best one I can think of and share it! Hope you're well :)

[@jobautomation]

Thank you!

[@DrWaku]

Thanks for watching and commenting!

[@Totiius]

Fantastic! Thank you

[@DrWaku]

Good to see you again, thanks for watching

[@THAELITEVR]

I didn’t understand half of what you said . I love genius level info .thank you

[@jondor654]

Good delivery .

[@Earth2Ross]

🤯Just discovered your channel

[@DrWaku]

Welcome to the community!! I hope you like having your mind blown ;)

[@Earth2Ross]

of course 🤓 @@DrWaku

[@andregustavo2086]

Could you make a video about SpikeGPT (from SSNs)

[@Jumpyfoot]

This is right up there with the KZbin channel AI Explained.

[@superduck97]

Cool. Do these systems mimic gap junction communication as well?

[@majestiqueeuro5937]

10mins in I learned so much. I had to pause the video and drop a comment.

[@manojmani6370]

Big fan of yours.

[@DrWaku]

Thank you :)

[@williamal91]

evening Doc, best wishes

[@DrWaku]

Thanks Alan

[@MichaelDeeringMHC]

Neuromorphic computers are like birdomorphic airplanes.

[@DrWaku]

Haha. Don't you mean ornithomorphic? (Greek theme)

[@DrWaku]

Walking: anthropomorphic ambulation

[@Xtianu]

Why doesn't this channel have millions of subscribers?

[@terrydunne100]

This is fascinating. I'm intrigued by the first BCI implant! How fast will the field explode? Could brain-cloud links accelerate AGI? Seems less scary than lab-grown brains (eek, ethics!). ☁

[@rufus525]

whover did tge graphics was showing an apple A6 chip every time you mentioned ASICs

[@DrWaku]

Oh, that's why everyone was commenting about A6. Lol. Yeah that was my video editor

[@thomasmaiden3356]

A6 or ASICs ? Application Specific integrated circuits ?

[@DrWaku]

The latter, ASICs. Sorry, my editor relied on the speech recognition and it put in the typo

[@kennethmyers6160]

Feels like we're just building out a foom runway

[@MelroyvandenBerg]

it's typed ASIC not EXEC, you have a typo there in your video around 11 min.

[@DrWaku]

Ah yes, I mentioned that one to my editor but it must have slipped through the cracks. The subtitles are initially based on voice recognition which is why they have typos sometimes.

[@Daniel-Six]

Anyone ever consider the possibility that the inscrutably large matrices generated by modern AI are actually an _interface_ to the real machinery of computation, which is situated elsewhere in the simulation.

[@DrWaku]

We think we are training machine minds in our image but we're really just figuring out simulation API calls through brute force...

[@Daniel-Six]

@@DrWaku Oh... so _you're_ the other guy who actually knows the setup. You wouldn't believe how much time I have wasted on YT threads waiting to encounter someone who can speak plainly about the simulation. Bang on, doc!

[@Daniel-Six]

@@DrWaku ...and while I have you on the phone, might I suggest you look into the role of gender in computer science? By which I mean _technical_ gender. I have spent twenty years in that pursuit, and I can tell you it's quite profound to our conception of reality. Sound too nutty? Then answer me this; how did _every single_ graphic designer and thumbnail generator make the simultaneous decision to characterize AI as a beautiful young female? I could accept even a ninety percent female representation as reasonable, but one hundred percent is impossible... and starkly conspicuous. Anyone staring at that fact should be very, very curious about what it implies in the matter of (human operating-system) network design.

[@neurojitsu]

Do you know whether neuromorphic computing scientists intend - or are interested - in the function of emotions? One of your diagrams had waves coming into the right hemisphere, and circuits coming off the left hemisphere... From your discussion here, I understand they are interested in the nervous system so perhaps they are emulating sensory data as information (sensors etc)? I ask, because as I raised in my comment on a previous video, some neuroscientists say that 80% of human thought is "emotional" including all of decision making. I'd be really interested to hear your thoughts on what work is going on - if any - to build AI models with right and left hemispheres... it seems to me, from my human experience, that agency and judgment are two key areas for AI to be both effective and safe. Judgment for humans requires emotion... so I wonder how AI scientists are thinking about emotion? You discussed the column idea, which might 'reach down' to lower sensory layers... is that the extent of 'emotional' AI thought? Or do they conceive of a dialogue between left and right processing hemispheres? I was watching a video about Google's upcoming Lumiere video generation ai: this has a concept of durational time built into its 'thinking' too. It seems to be a reason it unlocks much better video movement than current ais like Runway. It can't be a coincidence that human minds create the experience of duration, and thus duration/time seem important to model in the functioning of all minds.

[@RAC91]

Dr waku, can you give your opinion on bitcoin/crypto/blockchain tech? Thank you love this channel

[@DrWaku]

Added this to my list of potential videos. Thanks. In short, creating scarcity in a digital world allows a lot of things from the physical world to be represented more readily. It's cool tech.

[@mrd6869]

Ive been cross pollinating different domains using an AI program.Lot of interesting interplay between seemingly unrelated areas

[@kayakMike1000]

Well.... You can only shove data through the network so fast. The cost of loading and storing data is millions of cost in time and power over computation.

[@Citrusfemboy]

Did you get a better camera? You look younger in this video.

[@DrWaku]

Same camera, but I increased and improved the lighting since my eyes are less sensitive recently. I think I look less flat and AI generated. Good eye!

[@j.d.4697]

What's an "obstraction"?? 🤔

[@DrWaku]

To be obstractionist is to obstruct the definition of an abstraction. Anti-dictionary sentiment

[@DrWaku]

...jk...

[@berkertaskiran]

I think not many years from now AI will probably consume more than 50% of energy of Earth and beyond. It shouldn't take too long to expand into space for energy needs. While we actively start to use fusion energy.

[@viralsheddingzombie5324]

What does it take to simulate 0.5 of a human brain like mine?

[@rakeshbhadreshwara8077]

Genius 👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻

[@DrWaku]

Thank you :) :)

[@laara1426]

Is that a rhetorical question ?

[@DrWaku]

Not really. The tech could be independent of all AGI research...

[@arturturk5926]

mm hmmm, we are done, 50 years. Creating Cylons...

[@j.d.4697]

I want a robo kitty so badly! 😖

[@arturturk5926]

What could go wrong? So what will it be? Altered carbon. Terminator Battle star Galactica The Matrix E. All of the Above.

[@bipl8989]

Somehow mimicking human brains just doesn't sound like the best road map to follow on this. Arent we after something better? Why start with a duplicate of us? Record isn't so great.

[@alexspiridonov4014]

Deepsouth's 228 trillion synaptic operations per second is 1000 times slower than a human brain :-( The human brain has several hundreds of trillions of synapses and every synapse can conduct spikes 1000 times per second

[@DrWaku]

Got to get off that FPGA tech haha. I think they're mostly using it to run simulations and understand the brain first. Speed later.

[@DrWaku]

Thanks for the data point BTW, I couldn't think of a way to look it up

[@garethde-witt6433]

Let’s hope not destroy the Ai and robots

[@vernongrant3596]

Here in Australia, we think of "The Deep South" as Confederate country in the U.S.

[@cacogenicist]

Commodity hardware isn't very neuromorphic.

[@DrWaku]

Agreed. It's a lot more based on logic and gates.

[@kayakMike1000]

Organoid research on human cells is not ethical. Just because you can, doesn't mean you should

[@kayakMike1000]

Well, we could just use gingers cells. Gingers have no soul.

[@KryyssTV]

The logic behind this isn't rational. Firstly, the simple fact is that we don't fully understand how the brain physically works, so replicating how we assume it works is not only going to teach us nothing about a real brain but also create something that isn't working like a real brain. Secondly, we don't understand how information is stored within the brain so cannot replicate that. Finally, we don't even know what sentience is so wouldn't know if an AI is alive or imitating life. This sounds to me like buzz words and pseudo-science being fed to clueless investors to get funding on research that is an inefficent use of time and money.

[@DrWaku]

From my understanding, there have been years of research from several universities and they pooled their knowledge to try to form the most accurate model they could. They want to run large-scale simulations of what can happen in a brain precisely so that we understand it better. That's the stated purpose of the neuromorphic computer. When it comes to AI, we know that our current systems are based on a very high level approximation of what happens in the brain. Since we have brains that are working pretty well, there's good reason to believe that approximating it more closely could result in better outcomes, if we get stuck with our current tech. It makes sense. It's not claiming that we already understand brains or that it's definitely the way forward for AI. This is research, after all.

[@KryyssTV]

@@DrWaku It's doesn't matter if every synapse in the brain is indexed and every form of input mapped against every part of the brain that lights up like a xmas tree; if we don't undertsand the connection between the why, how and what then we're a long way from having enough understanding to simulate thoughts let alone intelligence. All we have currently is the biological equivilent of circuit schematics and that is where the research ends because of the aforementioned limits. It's like the three blind men and the elephant with neuromorphic researchers claiming to be able to recreate what the elephant looks like after only touching it's tail.

[@taeallred]

@@KryyssTV how could you get anywhere with this way of thinking. isn't the point of experimenting being to get farther understanding of what you think you know.

[@KryyssTV]

@@taeallred There's such a thing as research for it's own sake with no intrinsic value. You see this in fields of study like cryptozoology, parapsychology, epsitemology, ect. In this case, money is being wasted on making a simulation of something we don't even understand or are capable of validating the results of.

[@taeallred]

@@KryyssTV neural networks work out good and that's the smiler but software.

[@Telencephelon]

4:15 "and there is an ethical question of course". Of course? Really? What state of wokeness do one have to attain?

[@Telencephelon]

5:55 That's complete BS like a lot in this video. If you would average the energy consumption of a human over the course of its entire learning phase = lifespan things would look differntly. But something with a similar output like a human brain doesn't consume megawatts during the inference part.