The Hard Tradeoffs of Edge AI Hardware

[@philippepanayotov9632]

Yet another interesting video, which condenses a lot of information into a manageable chunk. Keep it up!

[@petermuller608]

Dude your content is as outstanding as it is random. From soviet oil export over chip manufacturing to edge AI models xX

[@fennewald5230]

I write inference pipelines on jetsons for my work. The latest generation have some very attractive performance characteristics. They do pull 60W, which definitely isn’t nothing, but for our use case it’s manageable. Something that wasn't stressed is that those devices are heavily optimized around fixed point 8 bit throughput. A high-spec Orin can put out 275 TOPS, which is more than triple a 4090s 83 TFLOPS. Even if the models are much weaker, the increase in throughput opens up a lot of flexibility with system design.

[@azertyQ]

Qualcomm AI100's can do 400TOPS @ 75W

[@Smytjf11]

@@azertyQ yeah, but an Orin costs $2K

[@tempacc9589]

As someone who actually deploys edge AI I heavily disagree with how "easy" you make FPGA's and ASICS seem. For the vast majority of "smaller" projects the Nvidia Jetson series is a far better choice since they support newer algorithms and functions. Especially with the speed the field of AI is progressing at. Furthermore fp16 GPU tensor cores are basically optimized for ML inference and provide good performance if you want to spend a little extra time on converting the model, though often even that has compiling issues with newer ML models.

[@g.l.5072]

Agreed.

[@adissentingopinion848]

As a verification engineer for FPGAs, let me tell you that the Hardware Description Languages you need to use are ancient relics based off of PASCAL. They have so many pitfalls and inexplicable limitations that cause more problems than the design itself. And they need to be so complex because the design space for this stuff is extreme. When the compiler(synthesis+PAR) has to conjure up a complete mapping of billions of custom registers and their connections, you're facing the limitations of hardware, physics, software, development tools, and whatever hellish design process your company is in. If you need an FPGA for a problem, you need to be damn sure there isn't a specific ASIC or a microprocessor that can do it faster or cheaper for you. They are good for super fast timing, fast turnaround for design iterations, a pathway to ASIC dev, and as much parallelization as you can stuff through an interface (the main bottleneck!). And if you want these benefits, be prepared to build it all from the ground up. Generalized cores are liable to need tuning for extra speed, and if you're using an FPGA you're gonna need to milk it for every last drop or downgrade to a cheaper chip because cost will be king.

[@parkerbond9400]

Asianometry makes EUV lithography understandable, when we all know it's magic. Making very hard problems seem easy is just par for the course.

[@ChrisSudlik]

You're missing the latest development in edge AI: simplified models running on a physical quirk flash memory. Mythic AI and Syntiant are two companies taking advantage of this to do simple, this tech is in the earliest days and has a lot of future potential.

[@bakedbeings]

I've googled quirk flash memory to no avail.

[@jagadishk4513]

Veratasium made a video on mythic

[@ChrisSudlik]

@@bakedbeings a quirk in flash memory, I missed a word in there. Veratasium has a somewhat ok primer on the general principle.

[@aatkarelse8218]

Must agree, training models is tough, using them much less so. a bit of a none issue vid.

[@nailsonlandimprogramming]

I saw that one, and it promises!

[@StephenGillie]

Post-training pruning is very much how human neural networks learn - massive connections anywhere and everywhere over the first few years - during the initial training phase, then massive pruning to get rid of the unnecessary connections.

[@jadeaffenjaeger6361]

This is true and underlines the vast differences between our brains and machine learning hardware: Unstructured Pruning for Neural Networks tends to struggle with performance because it usually maps poorly to the existing highly-parallel and highly structured hardware. In the industry, methods like low-bitwidth quantization and custom architectures with low parameter counts (e.g. MobileNet as mentioned in the video) tend to see more use because their regular structure can exploit parallel hardware a lot better.

[@TheDiner50]

It is not even years. As soon as one got enough training for the basic task the pruning beings quickly until your ok at it. You are at a baseline training level. It is first months or years of small improvements stacking up that starts to really be noticeable. Much harder but a sign of real taut skills from doing it for real. Usually people that are really good at something look at you funny with a long stair into the distance when you comment on there 'skill' and effortless execution. Since the improvements are many times just work to them and really rather not do it in the first place. The only way a AI can be smart is if it learns to do stuff on it's own faster or better in some way without the help of the creator going in messing with it constantly.

[@MyPhone-qg2eh]

Only the relevant solutions need to be deployed. And can be accessed by a simple sorting algorithm most of the time.

[@MarcdeVinck]

That solder bridging in the stock video at 2:56 and 3:00. Yikes! Great video BTW!

[@fh80a]

Valeu!

[@fra4897]

Amazing video (as always), I am a ML engineer, it's not (always) true that the less weight the model has the less memory/faster it is. I know it's a little counter intuitive. It all comes down to how many GFLOPs the model need on some hardware

[@davidcarlsson1396]

I made a thesis at university for graduation, basically image processing for a specific, conservative industry. Scored like 90% accuracy (above human for the segment, less if you would take the entire object into account). I encouraged them to keep contact with other students and the university, because I soon realized this would take atleast 4-10 3-month long thesis works that would need to build on eachother before getting really the market leading role they easily could achieve. I had basically no contact with the company, I just did it because I suggested it. Damn now I realize I marketed it to the wrong part of the company. Will need to reach out to the actual CTO of the mother-company. Damn, that is 3-4 lost years for them XD

[@dragonsaige]

It is generally true

[@fra4897]

@@dragonsaige nope

[@refikbilgic9599]

I would like to write some personal sentences since this is something I am recently working on. There are already existing ASICs around, and it has a huge benefit compared to FPGAs, that is the "area". It's nice that we can reprogram our FPGAs anytime, but considering the edge-AI commercial devices (E.g., VR headsets), we have a very tight form factor for the chip. Then, it is feasible to say ASICs will dominate the edge-AI products. Even more, we will find the architecture of edge-AI, next to the processing units with more caches (everything on a single package = inside of our mobile devices in the future(?) ). However, we need to consider some issues with AI accelerators.. They have to work quite busy, which makes their thermal profile a bit annoying, and we need too many buffers next to cache memories (too much memory on-chip = too much area... ). We have nice cooling solutions already, but we definitely need more, or we need new methods to reduce the sparse computation of neural networks. Maybe you heard about "Spiking Neural Networks (SNN)". They provide a nice "event-based" structure to the network, which allows you to create "idle" states for your computation... That is already a nice idea to have a nice edge-AI chip with low power option !! Next, what if we make this chip in 3D? Considering the memory domination in the AI chips, what about stacking the memory die, vertically on top of the logic die? We try to answer this question in imec.

[@severussin]

You work at Imec?

[@refikbilgic9599]

@@severussin Yes… Specifically on this subject!

[@diamondlion47]

Looks like AMD is already doing that with the mi300 line of chips. Wonder what your take would be on how much of a performance benefit this could give over an H100? Thanks.

[@refikbilgic9599]

@@diamondlion47 AMD also work with imec, and indeed, not only AMD but other vendors are founding the R&D in this domain, such as Huawei, Google, Qualcomm and Intel. But these vendors mainly concentrate on the "development" process instead of "research". We can measure performance with total floating point operations per second (FLOPS), and both processors provide extremely high computation. But, as discussed, this is not enough, and we need to think about the power of consumption. In this case, one can measure the energy efficiency per operation cycle (FLOPS/Watt). But overall, performance measurement is not straight-forward and we can think about many other parameters like, logic core configurability (multi/many-core with multi-thread ops), memory subsystem (L1/L2 cache sizes, etc.), technology processing node etc... Finally, as the end user, we only see the marketing prices and advertised performance comparison for last products...

[@diamondlion47]

@@refikbilgic9599 ​ @Refik Bilgiç Thank you very much for the response. I notice you don't mention Nvidia in the list of companies, do they partner with other research institutions or are they not doing as much research into stacking or advanced packaging etc or just an oversight?

[@TheTyno37]

My fave video of yours recently! Thanks for making it!

[@nekomakhea9440]

for a second, I thought the cow photo at the beginning said "dont eat ass"

[@zyzzfanzyzzfan]

I’ve been studying neural architecture search for my PhD but ive not heard much about joint architecture and hardware search. Currently in an internship in industry and that sounds like quite an ambitious goal based on whats going on meta.

[@dominic.h.3363]

Natural language processing is what will end me. There is literally nothing else I can do as a paraplegic with dyscalculia on the ass-end of Europe besides translation, and Deepl is already near perfect between every language pair I can speak. I've got every recommendation under the sun from becoming a musician (been playing the guitar for 6 years daily, can't memorize the fretboard because of dyscalculia), to programming (can't memorize the syntax because of dyscalculia), to 3D modeling... (you get the gist, nothing even remotely related to manipulating quantities or memorizing arithmetic relations is viable), to becoming a novelist (sure, because we all know how great those people earn). Aanyway, that was my comment for the algorithm.

[@brainletmong6302]

Damn, god does curse twice indeed. Hopefully you'll figure something out soon. I'd suggest language teaching to students but I'm sure you've already thought about that one for a long time.

[@AschKris]

People who can't work shouldn't have to work, why are we advancing tech and automating jobs if we're not also reducing the necessary work to make society function?

[@dominic.h.3363]

@@brainletmong6302 I can't teach because I don't know any grammar I don't "get" grammar. Guess why. I acquired every single language I am speaking by exposure to the language, much like a toddler learns to speak by just simply absorbing the language in practice from people who are speaking around them. I've learned my first foreign language with a starting vocabulary of ~300 words within 2 and a half months at eight years of age watching TV 18 hours a day, because I was bored with nothing else to do on my summer break. At the start of my summer I watched cartoons. At the end I watched nature documentaries. English followed as my second foreign language soon after... I'm not stupid, I have a very specific impediment that makes me unable to do a large category of things, and another impediment that makes me unable to do most of the things I have left after the first impediment. The tiny sliver that is left is being made superfluous as a field of employ by AI. At the same time I have lexical knowledge about several things I would be unable to perform. For instance, you could have me in any aquarist store and I would be perfectly capable of disseminating expert knowledge to prospective aquarists about both aquatic flora and fauna, but I could neither stock shelves, or get fish from tanks that are outside my reach.

[@dominic.h.3363]

@@AschKris That's a commendable notion but I don't like to feel useless... which I do, increasingly so. I want to be able to succeed at something that gives me a sense of fulfilment. And it's not giving me the satisfaction if I am doing it for myself, I want to contribute to "making society function" as you put it.

[@AschKris]

@@dominic.h.3363 Yeah, but that's because you want to do it, it shouldn't be a requirement to stay out of the streets.

[@PaulGreeve]

I thought you would have included the mobile processors from Apple, Samsung, Qualcomm, etcetera. They all include a Neural Processor these days but I never hear much discussed in relation to them, how powerful they are, what they’re actually able to do and so on. Given the many millions of people own phones that are powered by these processors, surely the potential to bring Edge AI to everyone is now here if they are used effectively.

[@eafindme]

Chips tradeoffs generality and specificity. General computing like CPU is Turing complete but it is bad for neural network, while neural processor or tensor core is specialized in accelerating neural network computing but not Turing complete. It is not a big deal usually, just that how far they could push the efficiency envelope matters for the IoT edge computing.

[@pyromen321]

@@eafindme this is not true. All modern tensor processing hardware on mobile devices (Qualcomm, Apple, Google) is literally just a super light arm core with heavy modifications. The neural processors in all modern phones are all just as Turing complete as the main CPUs, but they are directly in control of silicon that is highly specialized for doing common neural network operations efficiently. Plus, any hardware that can do common neural network operations is guaranteed to be Turing complete unless it’s some of that analog trash that won’t be viable for 50-400 years.

[@eafindme]

@@pyromen321 thanks for the rectification. Because I'm currently working on neuromorphic computing on FPGA, it is hard to say if it could not be viable. Who knows computing paradigm will change in the future.

[@franks_9254]

My concern is, that those chips are nor easily available, if you are designing an industrial edge device. Right? While the type of chips shown in the video are.

[@StephenGillie]

Game-changing MAC (multiply-accumulate) chips which can each replace whole racks of traditional CPUs. QMAC (quantum multiply accumulate) qubits are analog bits, giving sub-bit precision, and analog outputs cut training time by 10x or more. There are several innovations in this space going past arm64/x86-64, and many will be ideal for mobile devices, but none are mentioned in this video.

[@jjhw2941]

Some of the Nvidia Jetson boards like the AGX Xavier and Orin have separate Nvidia Deep Learning Accelerators built in as well as the GPUS. There is the Hailo-8 M.2 accelerator too.

[@jamessnook8449]

Almost 20 years ago I led a project to port the neural models we ran on a Beowulf cluster to a more mobile platform. Our goal wasn't to create a processor to run solved networks like the current crop of AI processors - we built it with full plasticity so that the learning and processing could be performed on the same piece of hardware. I am disappointed that what is available today is a shadow of what we did in 2004. None of the current processors are constructed to model the neural structures required for real intelligence. They just keep modeling the same vacuous networks I used as an undergrad in the cog sci program at UCSD in the late 80's. Most of the people using the technology don't understand intelligence and sadly don't care what is required. One example: Lately I've seen numerous job postings for AI engineers who can do prediction - what they don't understand is that it isn't prediction, the missing component of these networks is expectation - facilitated by prospective memory.

[@abowden556]

There are some people who actually care about this, Numenta for example! They partnered with AMD/Xilynx, so it's not like their approach has no support. Sparsity i a huge win, but their plans go way, way beyond that. They actually want to model the brain, or a small fraction of it, and their research has already borne fruit in the area of sparsity. They are definitely wanting make hardware that more closely mirrors the brain, when they get the chance though. It's very clearly their primary area of interest.

[@VenkataRahulSatuluri0]

Are you speaking of probablistic learning sir?

[@jamessnook8449]

@@abowden556 Functionally appropriate neural processors aren't easy to design/build and won't be as efficient as current neural processors are at running solved networks, but the current methodology won't give rise to real intelligence either. Sparsity? I believe you're referring to what we called sparse connectivity - is one aspect of what is missing from the newest versions of what is really just improved back propagation networks. Another missing element is learning - even my professors thirty years ago admitted that there was no biological basis for back prop, but they had no other mechanism for modifying connection strengths. Few people are incorporating real biology in their network designs because it is a pain in the butt, and truthfully even fewer care about it. I am glad someone (Numenta you mentioned) still does.

[@jamessnook8449]

@@VenkataRahulSatuluri0 In regard to which portion of my comment are you referring?

[@VenkataRahulSatuluri0]

Sir, the last part, expectation facilitated by prospective memory

[@rngQ]

2:35 The AI-generated PCB image is a nice touch

[@LogioTek]

Qualcomm AI100 DM.2e form-factor edge AI accelerators scale 70-200 TOPS at 15-25W, also various Qualcomm Snapdragon mobile SoCs fit the role of less powerful 30 TOPS and under edge AI accelerators. Qualcomm is pushing pretty hard into that direction with SoCs for edge AI, vision AI, drones/robotics, ADAS, and level 4-5 autonomous driving including software stacks. They even have AI research division dedicated to solving problems with edge AI model optimizations and other things.

[@tobuslieven]

Maybe ASICs for standard image processing layers like edge and depth detection, where a lot of the hard work can be done efficiently and in a standard way, then FPGAs or GPUs for more specific neural nets that take the final layer from the standard ASICs and do something interesting with them.

[@CrazyEngineer314]

You should take a look at Perceive's ERGO chip it seems to be a gamechanger in this field

[@maus3454]

I was wondering whether it is possible to un-learn or de-learn (undo a learning) of a learned neural network? So it will forget things or patterns

[@dragonsaige]

This is definitely possible, for example by simply further training the model on something else, but why would you want that?

[@maus3454]

@@dragonsaige In case it learned something you don't want it to

[@cavemaneca]

Using edge processors for neural networks at work, it's amazing to see how quickly the field is growing and how quickly their capabilities grow.

[@jairo8746]

5:35 It is really silly to think you can try pruning something that nobody has an idea of how or even why it works, to make it work more efficiently. I imagine it would be similar to cutting cables in an electrical panel and checking if the machine powered by it is working faster or slower, without knowing which room or component had had the power cut out.

[@alefratat4018]

Yes, pruning is one of the most overrated approach when it comes to optimizing the accuracy-performance trade-off of neural networks. But because this idea came from renowned gurus of the DL field, lots of people think it's the way to go. In practice, it very often leads to disappointing results.

[@dragonsaige]

On many popular models pruning can give gpu class performance on cpus with only a few percent accuracy drop on modern inference engines.

[@Bareego]

The main issue I have with AI is that we let silicon/software learn under some evolutionary pressure, and we pick the one that does the job best. But we don't actually understand what made that particular resulting structure do the job best. We harness the gains from complexity without learning about the part of the structure that makes this so efficient and why. It's like someone finding a wand and wielding it without knowing how it does what it does. Part of this is that more complex problems require an AI network that is so complex, that we have no hope of understanding what makes the best working model tick. I don't think we're in any danger of being taken over by AI, but that the information we could learn from AI for making our own designs are rarely or never learned. As edge AI is concerned, I suspect we'll get finished AI solutions preprogrammed. This leaves a small requirement of processing for the edge device, which should keep power requirements low. Most devices are VERY specialized in their purpose. Much easier to run an expensive AI centre and churn out cheap pre-programmed devices.

[@droid16beta97]

Pre-programmed as opposed to what? They don't train the models on the go...

[@vishalgiraddi5357]

And for that there's a topic called explainable AI which aims to predict not only th result, but also explain why the particular result was obtained, presently explainable AI models are not as powerful as conventional black box models, but they exist

[@bakedbeings]

Look into anaesthesia. We've been putting millions of surgery patients a year under deep chemical anaesthesia for decades without knowing the mechanism.

[@tt10tt]

​@@bakedbeings excellent analogy.

[@glock7061]

The thing with fpga is that you have to manually program entire network in vhdl or something similar. Which is cumbersome and not very easy. Belive me I tried.

[@ristekostadinov2820]

don't worry, he mentioned that in the video 😄

[@gorak9000]

Sounds like you need a script to take your model and generate the VHDL automatically. I've done it in the past - VHDL doesn't have the automation you need sometimes to do what you need, so write an external script (c/c++/python, whatever) that generates the VHDL you need that's too tedious to do by hand.

[@eafindme]

I made a CNN to FPGA microarchitecture hardware transcription automtion software, so I only have to focus on CNN model and design. Of course, it is on exotic computing domain so not a typical binary computing.

[@glock7061]

@@eafindme What about something more complex than CNN? Self attention for example

[@illeatmyhat]

It seems more sensible to program the network in something like Amaranth (previously nMigen) in Python.

[@Hyo9000]

Fwiw, CUDA doesn't really use C or C++. It has a very restricted API from which you can call upon its features, and yes it is accessed from "C", but its semantics are very different. Branches are expensive, and recursion is not allowed. And malloc doesn't exist.

[@alexandrep4913]

Video summed up: There is no shortcuts, this is going to require technology we just dont have yet to really get what we want from AI/Deep Learning

[@Stroporez]

What about in-memory compute and neuromorphic chips?

[@murdercom998]

ty for your hard work =)

[@hypercube33]

Love the potato PCB ai generated image at 2:44

[@hushedupmakiki]

The way you close your video sounded exactly like my highschool coaching tutor. He would give really in depth and well explained lessons, but the hour ended, closed shop and got out in his car before the class' bags were even packed 🤣

[@kennethtan6403]

Thank you and much Love from the Philippines.

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@Maleko48]

thanks for highlighting the NPU on a stick... might have to check that out

[@Andrew-pc8zn]

Wake up babe! Asianometry posted!

[@monstercameron]

you didnt mention in-memory computing or analog computing as possible solutions. Take a look at Mythic AI, looks like the will get around the perf and cost by using an analog and in memory computing methodology to get around the von nueman bottleneck and use a flash memory manu process which is relatively cheap and dense.

[@PhazerTech]

Good point. Also the memristor is a breakthrough that could play an important role here too.

[@slicer95]

Mythic AI has shut down.

[@jannegrey]

This is gonna be interesting.

[@NathanaelNewton]

Heh I agree

[@jannegrey]

@@sub-vibes agreed

[@mbabcock111]

Brainchip AkidaTM is the world’s first commercial neuromorphic processor. It mimics the brain to analyze only essential sensor inputs at the point of acquisition-rather than through transmission via the cloud.

[@eelcohoogendoorn8044]

'FPGAs have a lot of potential'; if I got a dollar for every decade thats been said... id be able to buy a fancy cup of coffee, at least. Not sure if FPGAs have some kind of massive marketing campaign, or if the general idea just appeals to nerds; its like FPGAs are to EEs as what fusion is to physicists. To present FPGAs as having even a fraction of a precent of the relevance to edge AI as jetson nano or GPU type architectures does, makes presenting ITER as a practical solution to power generation look intellectually honest and well informed by comparison.

[@isaidromerogavino8902]

that's pretty harsh, but not gonna fight with you on the fpga issue, never used them. On the other hand, I do have my reservations about the feasibility of the ITER project, as well. However, I'll admit the sentiment is, in part, simple intuition derived from a bare understanding of physical phenomena/concepts. The other part of the sentiment coming from a (comparatively) more rigorous mathematical formation at school. Would you be so kind to guide me towards some source or reference to complement my knowledge on the theoretical/technical difficulties which inform your opinions on ITER?

[@eelcohoogendoorn8044]

@@isaidromerogavino8902 This is a fairly recent decent video on the issues; though I think she is still being too kind: kzbin.info/www/bejne/gnuXiGSdYpt9n7s&ab_channel=SabineHossenfelder

[@timng9104]

not sponsored but arduino nano 33 BLE with ARM M4F chip is so good, I have pretty good accuracies with some custom applications (shadow gesture sensing). There are also concepts on reservoir computing, recurrent networks, echo state networks, would love to hear your take

[@napalmholocaust9093]

I have "The Edge" open in a back tab. Yt free movies has it up now. Or did, they only stay free for a few weeks sometimes and when I get to it its gone and private. I had to sit through "Rob Roy" dozens of times myself in earlier days.

[@FobosLee]

wait. Isn’t it an AI generated image at 2:40 ?

[@jadeaffenjaeger6361]

The edge NPU with arguably the best chances of widespread adoption are ARM's Ethos-U and Ethos-N NPUs which are made to complement their Cortex-M and Cortex-A processors respectively. To my knowledge they do not exist in (commercial) silicon yet but will like have a head start over the offerings of small startups due to their seamless integration with an already established ecosystem.

[@cjmitz]

I’m looking at using multiple edge ai devices ($3) in parallel. Each running a real time algo and then combining outputs. NNs will get progressively more efficient and will easily run on modern micros. Micros are the future over fpgas but an fpga can form part of the parallel to serialising etc.

[@yelectric1893]

What’s a micro?

[@smooooth_]

@@yelectric1893 I'm assuming short for microcontrollers

[@yelectric1893]

@@smooooth_ ahh. Wow, well having a dozen picos and communicating the results could be pretty robust

[@Shaunmcdonogh-shaunsurfing]

Fantastic video

[@cryptoinside8814]

So AI computing is nothing more than a super-fast computer either sitting at the data center or edge. It's like a super IBM PC vs IBM mainframe in the old days 🙂 People just love to put lipstick on a pig and call it something different, but it's actually the same sh*t 🙂

[@severussin]

Basically

[@NathanaelNewton]

4:50 *realizes that I have been using premier pro the hard way*

[@alecdvor]

Is there an rss feed for the news letter

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@arpinfidel]

a topic i'm really interested in is the use of analog computers for NNs. most NNs are not entirely precise by nature so I think it would make sense to run it with analog chips

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@bobdobs23]

Am I mistaken is not Tesla with their dojo configuration among the leaders?

[@temistoclesvaldes5312]

O man i got this 18 minutes fresh? nice

[@J_X999]

Video on US chip restrictions?

[@sodasoup8370]

Honestly i cant see how this problem could be "solved". on an Edge device like for traffic recognition you need fast AND precise Inference. Like the Video said, most ways of cutting corners just dont apply. I fear that this will result in overspecialized hardware which will flood the market after some cheap product with good software support gets released...

[@alefratat4018]

There is a common misconception (encouraged by the big tech companies for good reasons) that says that using bigger and bigger neural networks is always better. That's true only if you want to solver very generic problems (such as the ones tackled by those big tech companies). However, a lot of real-live scenarios and use-cases are much narrower and could be addressed by a set of specialized lightweight, cleverly designed, neural networks.

[@sodasoup8370]

@@alefratat4018 i am just starting to learn ML, but isnt it in the case of Computer Vision just a bunch of Convoluted layers which take 90% of the computing Power? The fully connected layers after that are important for the model but you cant save much resources with them...

[@geoffegger]

Any thoughts on hailo?

[@droid16beta97]

I don't feel like any of those are AI-specific issues but apply to all kinds of computation.

[@ChrisHarmon1]

My father LOVES the same movie. Must be a dad thing.

[@vishalgiraddi5357]

Interesting topic indeed, My own final year college project is based on post training quantization of models Currently working on performing 2 bit quantization

[@tsunamio7750]

thanks

[@FerranCasarramona]

Another edge AI techonolgy is spiking neural networks, which can reduce consumption a lot. Brainchip is a company offering this technology

[@artscience9981]

As an electrical engineer, a video like this is music to my ears.

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@first-thoughtgiver-of-will2456]

Constrained (edge) models will perform well once we have pretrained ANN based online reinforcement learning algorithms that solve discretization (episode length and SAR map resolution) and search strategy (not epsilon decay, a better bellman equation). I also don't think parameter quantization is reliably stable enough for chaotic systems (deep nets) and stochastic dropout mine as well be Monte Carlo architecture search. Quantization is a useful techniques but I don't think its the future of edge learning and is a nascent attempts at porting big data to the edge instead of finding an edge first solution or integration to deep net models. Inference at the edge is underutilizing the edge. We need fine tuning at the edge and continual learning. Tesla's closed loop data feedback and update pipeline is an interesting approach that utilizes the constantly shifting domain seen by the model at the edge.

[@VenkataRahulSatuluri0]

And then there came software programmable ASICs to run inferencing for generic AI models

[@matneu27]

Learned a lot more thanks to your video and (I assume) the ai generated voice. Anyway ai could maybe one day optimize its power consumption and manage a power grid to optimize the distribution.

[@sunnohh]

I work with NLU and it is almost all snake oil

[@odopaisen2998]

> Silicon Compiler : kzbin.info/www/bejne/fX6cgX53m7mfo7Msi=b7se3OuI42jfNYkM > Parallella : kzbin.info/www/bejne/rIecl5anirqYZ7ssi=2dtIp--sL6L4iiKP&t=830

[@pyromen321]

I’m amazed you didn’t mention Qualcomm, Apple or Google. All of them have very good edge AI solutions compared to everything you mentioned in this video. Whenever you open up your phone’s camera app, data is being fed to multiple neural networks for autofocus, lighting, super resolution, noise reduction… etc. Qualcomm even has their AI hardware exposed to app developers. Not sure about apple and Google, though.

[@EyesOfByes]

0:49 It's the dude from Ireland. Duh ;)

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@larryslobster7881]

more ai vids love this

[@bluegecko6770]

Thanks for your enjoyable content

[@samgeorge4798]

Love your use of AI generated images throughout the video.

[@jonathanpeace3167]

Its all about uJ/inference and low Iq; checkout Syntiant and Greenwaves for example...

[@goodfortune6399]

That was a good movie

[@ryanreedgibson]

You need to start charging for this. By no means stop posting.

[@johnshaff]

Power not processing is the problem

[@RonJohn63]

0:06 Eat mor chikin

[@Smitty_Werbenjagermanjensen1]

I don’t think I’ve ever heard of the terminology ‘edge’ devices, I believe they’re called IOT (internet of things) devices in Cisco and maybe elsewhere

[@amandahugankiss4110]

Okay I love you goodbye!

[@deusexaethera]

Today I learned Github Copilot exists.

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@computerconcepts3352]

interesting 🤔

[@Erik-gg2vb]

Will be interesting to see how Musk's, not sure if it is apples, DOJO works out.

[@arubaga]

We should demand open source firmware, or edge AI could be insecure at brain stem level. 😮

[@nomadhgnis9425]

Optical cpu🤔

[@styx1272]

What about the Akida 1000 by Brainchip Corp ? You seem to have missed a winning entry. And deleted my post ?

[@RonJohn63]

The real problem is that computer neural networks don't work like biological neural networks.

[@jasonwaterfalls2935]

Why not offload computing to an edge cloud? This combined with 5G would solve the latency constraint.

[@alefratat4018]

Not possible for scenarios when you have to cope with hard real-time constraints. Which are quite common actually.

[@again5162]

I hope we eventually destroy disease with better computers especially cancer

[@MaxxPa1]

Hello you have been selected among my lucky winners DM via the above name on telegram to claim your prize 🌲 🎁.

[@GavinM161]

Good lord, what tosser bought a license plate with "TSLA S1"!?!

[@irvingchies1626]

Deep learning ≠ A.I. Actual A.I. should be able to learn just by watching once or twice, then a little practice just like a human

[@JeffreyCC]

Why do I always get you in my recommendations, also when I blocked you in every way possible?

[@alex15095]

Why did you block him?

[@JeffreyCC]

@@alex15095 Because his video's were occupying a third of my daily feed without watching any of them

[@alex15095]

@@JeffreyCC KZbin algorithm 👍

[@daviddevlogger]

Second

[@TheMoeShun]

tinygrad

[@jmatx]

Edge, edge, the edge, the Edge -- what is the Edge, edge?

[@reddixiecrat]

Sounds like you’ve never heard of Palantir

[@brettyoung6045]

The reason there aren’t that many good edge chips is because the market is small.

[@coraltown1]

I am really not all that impatient for ultra powerful AI to appear on The Edge, as so much of this tech gets misused. Generally, it's just not making the world a better place.

[@V3racious3]

If I hear about the Asianometry Newsletter one more time today I'm actively spending resources against it.

[@nailsonlandimprogramming]

Great video @asiaonometry! Iḿ suspect but I tend to favor NVidia basically because is easier to deliver the models and inference code, basically what changes is the Docker container and a few libraries. Besides that flexibility comes at a cost of power consumption. For instance if I want them running 24x7 on solar independent power I need a large solar panel AND lot's of batteries

[@AscensionNow1]

thanks