INSANE Machine Learning on Neural Engine | M2 Pro/Max

Taking machine learning out for a spin on the new M2 Max and M2 Pro MacBook Pros, and comparing them to the M1 Max, M1 Ultra, and RTX3070.
get TG Pro for yourself: www.tunabellysoftware.com/tgp...
👀 More gear I use: (including course creation and youtube stuff): www.amazon.com/shop/alexziskind
Thanks to Peter Lin for his examples (his repos down below).
▶️ Apple Silicon and Developers Playlist - • Apple Silicon and Deve...
TensorFlow Metal Backend on Apple Silicon Experiments - github.com/tlkh/tf-metal-expe...
Tensorflow training scripts - github.com/woolfel/mac-tensor...
ML MacOS Performance - github.com/woolfel/ml-macos-p...
#m2pro #m2max #machinelearning
💻NativeScript training courses - nativescripting.com
(Take 15% off any premium NativeScript course by using the coupon code YT2020)
👕👚iScriptNative Gear - nuvio.us/isn
- - - - - - - - -
❤️ SUBSCRIBE TO MY KZbin CHANNEL 📺
Click here to subscribe: / alexanderziskind
- - - - - - - - -
🏫 FREE COURSES
NativeScript Core Getting Started Guide (Free Course) - nativescripting.com/course/na...
NativeScript with Angular Getting Started Guide (Free Course) - nativescripting.com/course/na...
Upgrading Cordova Applications to NativeScript (Free Course) - nativescripting.com/course/up...
- - - - - - - - -
📱LET'S CONNECT ON SOCIAL MEDIA
ALEX ON TWITTER: / digitalix
NATIVESCRIPTING ON TWITTER: / nativescripting

[@NilavraBhattacharya]

Thank you from the bottom of our hearts for doing these tests, and not joining the video editing and Final Cut Pro bandwagon.

[@AZisk]

Happy to help!

[@somebrains5431]

Video editing doesn’t seem to hammer ram and gpu cores. Maybe that will change with M3.

[@coder-wolf]

Truly! Alex is one of very few who actually focuses on software devs when reviewing the Macs. Hugely appreciated!

[@CAA84788]

Yes, this is really helpful. I always had to try to use those video editing/photoshop results as a proxy for what I really wanted to know. Great resource!

[@Khari99]

As an amateur data scientist, I can't tell you how happy I am seeing you do these tests because of how niche our field is. Now I just need to figure out how to use CoreML with all the models I was working with in Tensorflow lol

[@mr.anirbangoswami]

How hard is it going to be

[@ef7496]

@@joytimmermans wow 😮 man how much experience you have ? Why don’t you make a video on that ? Please I am looking to start learning all that can you help me with a roadmap?

[@remigoldbach9608]

Great variety of test in your videos ! Amazing !

[@SonuPrasad-xt4yr]

Great video! Never thought ANE would be that powerful, Thank you for sharing your expertise and providing such valuable content. Keep up the good work!

[@AZisk]

Glad you enjoyed it!

[@gerryakbar]

Hi Alex, really thanks to your lots of ML benchmark in Apple Silicon. However, I think the benchmark configs can be different that they should maxed out for each arch. You can use bigger batch size for Apple Silicon since they’re using unified memory

[@rhard007]

Your content is the best for developers on youtube. You should have a Million Subs. Thank you for all you do.

[@avocado9227]

Excellent video. Keep it up!

[@AZisk]

Thank you very much!

[@user-er3pz8ev6d]

Thanks, machine learning is the thing I was looking for and only you making such tests

[@AZisk]

Glad to hear that!

[@daveh6356]

Cheers Alex, great to see something actually using the ANE. Something's seriously wrong with the M1 Ultra, it has double to M1 Max resources including a second CPU & ANE and more GPU cores even if they're a little weaker. Any chance you could check the CoreML config to see if the extra resources can be used?

[@drweb1210]

The ANE is something different. As I understand it, it’s designed for matrix (tensor) calculations in contrast to the CPU. I’ve trained NN using python and tf, and i know you can format the trained model in a way so it can utilize the ANE on iPhones using swift, the performance is amazing IMO. However now I started to go a bit deeper into swift, I want to try and train models on the ANE 😅. Awesome video btw. Love this kind of content, glad i found this channel.

[@ZhuJo99]

well the way how it's done is not that important as a final results. We are not using computers focusing on tool itself, but on getting job done. And seems like Apple did a pretty good job with their processors :)

[@riteshdhobale8210]

Hey I'm currently in 1st yr of cs n specialising in aiml confused which laptop I should buy windows with GPU or m1 or m2 pls help n if mac are the thing for an aiml engineering

[@drweb1210]

@@riteshdhobale8210 Consider this. If you are just starting, you will most probably work with more basic models, math and general coding (python, R, JS, MOJO...). For most task the M1, M2, M3 macs will do just fine, they are much more than you'll need for learning. When you start with DL (deep learning), the macs are still good but start to get very expensive (M1, M2, M3 MAX). This does not mean that you HAVE to buy a MAX model, things will just go faster on them, the PRO models will do ok as well. Most of DL is done on GPU clusters anyway (AWS, AZURE...). All that being said, if your main focus is to go for DL directly (which i don't recommend, but most people do), or you want to play with already existing models than go for a laptop with a good GPU. Last thing to take into account is the VRAM. The one thing that separates the macs and PCs (for now) is the underlying memory architecture. Because macs GPUs and CPUs share (in a sense) the RAM, you can load much larger models into memory to do inference. Simple example an RTX 4060 has 8GB of VRAM, and the macs can get access much more RAM (up to 128 GB) but this costs a lot of money. If you go for a mac go for the 16 inch. Hope this helps.

[@marcosarti9029]

I was waiting this video with all myself! finally!

[@SiaTheWizard]

Amazing examples and tests Alex. I was actually looking for YOLO test for Mac and this was the best video I've seen. Keep it up!

[@rubenhanjrahing7324]

really needs a lot of content just like this

[@user-ul4lj6cu9b]

This is the video I need. Thank you!!

[@cipritom]

Very informative! Thank you!

[@sivovivanov]

Great video! I'm looking at an M2 Pro for PyTorch related work - any chance we can get a video on some of that as well? Thanks

[@igordemetriusalencar5861]

Now I really want you to get your hands on a 4090 new notebook to test it to the limits.

[@AZisk]

believe it or not, I was considering this, but you are literally the first one that asked.

[@lightytan5404]

@@AZisk yes, please. So far nvidia new gen seems promising. But how it stands up against M2 Pro/Max?!

[@GlobalWave1]

@@lightytan5404the new laptops or portable laptops with the intel new hx laptop processors with the 4090 sound like helicopters and forget battery life but they are monsters. 😂

[@drreg2513]

​@@GlobalWave1 better have helicopter than 105 degrees cpu/gpu

[@LeicaM11]

Should not squeeze a 4090 into a small laptop!

[@MarkoChuValcarcel]

Thank you Alex for this great video. I'm very impressed with the results comparing the CIFAR10 with 1024 batch size, because it shows the difference in RAM speed between Apple M processors and the RTX-3070, the RTX is faster doing the calculations. By the way, my Desktop RT-3070 took 301.787s , 270.702 and 234.107, with 64, ,128 and 1024 batch sizes. During the test My RTX used 178 watts !!! we have to add the power usage of the CPU and other components. Another interesting thing I've notice is that if you consider that every GPU Core in the M1 Max, M2 Max and M1 Ultra processor, has the same processing power, you can almost find the images / second starting with the M1 Max times, just doing a simple aritmetich operation, this is very interesting because in Deep Learning training the performance of the M1 Ultra scales equals or proportional to the number of GPU cores it has, something that is not valid in other tasks. Finally, it would be very interestin to compare the inference speed with a Jetson Xaiver NX o similar board, because this NVIDIA boards cost more than US$1000, I think that a Mac mini M2-Pro could be faster than the Jetsons in inferences, and could replase the Jetsons in some tasks, the Jetsons have many advantages over a Mac Mini of course, for example they have many encoding/decoding engines to process many streams of video in parallel. But who knows, maybe someday we will see a robot with a Mac Mini with an M processor in it.

[@MeinDeutschkurs]

Great! Thank you very much. I'm interested in AUTOMATIC1111-API-SD-image-creation-differences between these devices. And there is also a question: Is there any way to bring stable diffusion into the neural engines?

[@theoldknowledge6778]

Amazing video! Can you do a comparison video running YOLO? I’m very curious to know how many fps these machines can pull up and it’s a more visual test. Thank you!

[@yongjinhong5533]

Hey Alex, have you tried increasing the number of CPU workers? As most of the computation overhead is in transferring data from CPU to GPU in the macs.

[@as-qh1qq]

Subscribed! Was looking for some academic benches and got them. Any chance of benching on simulation workloads like fluid or EM sims?

[@zyzzyva303]

Nice video Alex.

[@aady392]

Great video Alex. Thanks much, did you tried m2 with 24 gb in similar test?

[@yacinezahidi7206]

Thank you Peter and Alex for the video. Could you do this with pytorch as well ?

[@acasualviewer5861]

What I wonder is if the high RAM M2 Maxes (like 64GB or 96GB) can train significantly more complex models or use significantly bigger batches simply because they have more ram than most discrete GPUs.

[@sharathkumar8422]

Data Loading: The dataset used to train a model is first loaded into the system's RAM before it can be utilized. If the dataset is large and the RAM is insufficient, it can't load the entire dataset at once, which can slow down the training process as data needs to be constantly loaded and unloaded. Batch Processing: Deep learning models are typically trained in batches due to computational limitations. The batch size (the number of data points that the model sees at once) directly affects how much RAM is used. Larger batches require more memory but can lead to faster and sometimes more stable training. However, if the batch size is too large for the available RAM, it will cause an out-of-memory error. Model Size: Larger, more complex models (more layers, more nodes) require more parameters and thus more memory to store those parameters. Additionally, during the training process, the system also needs to store other information such as gradients and intermediate layer outputs for backpropagation, which further increases the RAM usage. Parallelism: If you're using a framework that supports it, and you have sufficient RAM, you can train multiple models or multiple parts of a model simultaneously, which can significantly speed up the training process. Speed: RAM is much faster than disk storage. So, the more data your RAM can hold, the quicker the access time, and thus, the faster your model can train. Swapping: If your system runs out of RAM, it will start swapping data to disk, which is a much slower process and can drastically slow down the training process.

[@acasualviewer5861]

@@sharathkumar8422 I understand the theory. But I want to see benchmarks. In theory the M2 Max with maxed out RAM is great for ML. But I'd like to see some benchmarks in practice. Performance is also based on bottlenecks, and if the GPU is insufficient, it doesn't matter how much RAM you have, you can use a GPU with inferior RAM and it will still blow you out of the water.

[@lyncheeowo]

thanks for you making the amazing video! one question: what's that chart? how do i visualize the cpu & gpu use on my macbook pro 14'' 2023?

[@alphazutn1274]

More on the ANE please. If you can find a test made for PyTorch/TensorFlow and that also has a version for CoreML and compare Windows vs Mac.

[@kingcoherent]

Thanks for this! I ended up plugging for a 24GB M2 Air because I thought it would be a while before Apple silicon would be at all useful for (real world) DL. perhaps I'll be needing another machine sooner than expected! Doesn't the Nvidia have almost 6K cores? I would absolutely expect it to trounce the Apples, even though it's old - I was most surprised how close some of these benchmarks were. But ultimately, until more of the DL frameworks add proper support for Apple silicon it's a bit of a moot point and I imagine most (Mac based) developers will continue to use cloud resources for their work. Of course once there is proper CoreML support in Tensor Flow / PyTorch then you may train/refine on a laptop.

[@MiladMokhtari1995]

these new macbooks are so cool i wish I could afford one :( great video!

[@_xkim00]

Great comparisons, I am planning to get 96gb RAM for my dL models as well.

[@_Einar_]

Lovely video as always! I've got a question, which Macbook would you consider 'worth the money' for a Data scientist and/or a ML/AI engineer ? Obviously it depends on the work one does, but it seems some tasks require terabytes of Ram, not GB's and so upgrading to 96 won't cut it anyway. On the other hand, going to low will enforce one to always use cloud services. At this point, I've tried a M2 macbook pro 16" base model (16 Gb ram), and I've run out of ram computing scattering transforms on a relatively small (2Gb) dataset. So the choice for me must be in the range of 32-96 I suppose.

[@lucasalvarezlacasa2098]

I'd probably say 64GB is the sweet spot.

[@arijr71]

Thanks - great video and comparison between the different MBP models! Planning to ditch my Intel MBP and (maybe) my Linux RTX-PC. Apple Neural Engine has an insane potential for computation purposes on Apple Silicon macs. Is it really so that Apple Neural Engine does not expose any proper API to be used for generic (python) OSS tech-stack DL training purposes? Only Apple CreateML in training and CoreML inference?

[@francoispro3799]

Hi Alex, thanks for this video ! (and your interesting channel :) ) I vote for a DALL·E alternative to run on your M1 Ultra/MacBooks arsenal to generate funny images for the next video ! :)

[@user-ou4go8lq8o]

Hi Alex, Do you think the mac studio m2 max with 30 core gpu would be enough for machine learning?

[@trustmub1]

Accoustic Windscreen (Whats ver that is) ... lol that killed me 🤣

[@Techning]

As a PyTorch user these type of comparisons with benchmarks written in PyTorch running on the M1/M2 GPU would be awesome :) I believe the results will probably be similar though.

[@MosWaki7]

they're actually very different, tensorflow has been optimizer for apple soc, but pytorch is nowhere close in performance on apple soc

[@Part-Time-Larry]

@@MosWaki7 was going to say just this.

[@cosmincadar3655]

Statistics look cool, but can someone explain which end user use cases can benefit from ANEs? e.g. if I own one of these MacBooks, in which scenario would I benefit the most of ANEs? Thanks.

[@AdamTal]

Can you compare base stock m3 max to top stock m3 max (I don’t mean customization, just stock options) any ML benchmarks would be great. Thank you

[@arhanahmed8123]

Hey Alex, It's nice to see that Tensorflow is working well in M2 chip, anyway,where do you live Alex?

[@haralc]

Can you also compare the ANE with Intel's Compute Stick 2?

[@samz905]

Very helpful info, thank you! With Nvidia 40 series laptop coming out soon, it would be very interesting to see how M2 GPUs perform against the likes of 4090, 4080 in deep learning tasks

[@woolfel]

in terms of raw compute power, the 40 series has more. The main limitation is memory and how much you need. if 16 or 24G is enough, paying the NVidia tax is better than apple tax. If you need more than 24G, M2Max might be the cheaper tax. Only way to know is take your workload, run some benchmarks and then figure which fits your needs.

[@mahdiamrollahi8456]

What we need in pytorch is to specify the device like cpu or cuda. What we have to do to use gpu or ane in Apple Silicon series?

[@pierrew1532]

Sorry so regarding the comparaison with PC laptop, is the extra $$$ of the M2 (max) worth it for data science work ?

[@watch2learnmore]

It would be great if you could revisit the Neural Engine's impact now that you're benchmarking with local LLMs.

[@blacamit]

Hello Alex Can you tell me which Linux distributions would be ideal for starting a career in programming? I'm a newbie java programmer.

[@dorinai]

3:17 - one way I prevented the swap on my M2 Pro 14 inch MacBook Pro with 16GB ram was to disable swap. and I have ran into any problems, the performance will degrade if you do this on an 8gb 13 inch macbook with m2 or m1 because it relies on swap for increasing performance but on the 16gb with swap off, not only does it help ssd longevity but it also never runs into any performance hiccups

[@mrcoffeelaplace8322]

I'm still deciding between the 14" with 10cores and 32Gb over 12cores 16Gb. Is the SSD such a problem or could i get away with 512GB SSD?

[@niharjani9611]

Which IDE did you used in m2 macbook pro ? Hoping for an answet 😅

[@lula5425]

Sir, please test engineering softwares on parallel on MAC like CAD and CAE , solidworks or ansys

[@giovannimazzocco499]

Did you consider repeating the benchmark for M3s?

[@jackyhuang6034]

i want to learn machine learning. Should i get m1 pro which is way cheaper or the latest m2 pro？

[@fezroldan9545]

At Alex I saw your prior video, I'm on the verge of purchasing a new laptop. I'm not a gamer, but rather an engineer who is focused on learning analyst tools along with ML tools. Currently have MacBook Pro 15 in from 2018, seeking to trade in and buy a new MacBook Pro or build my own PC. Been with Mac for a while, still love apple and MacBooks. Your videos have a been great help on determining which laptop to choose and understand fundamental applications based on specifications of laptop along with end application. Keep it up Alex!

[@zakpro4007]

ultimately which mac did you buy?

[@JBoy340a]

Thanks for doing this test. It was quite eye opening. I am getting a M2 14" pro and was wondering about the requirements and how much upgrading the base models memory would help. Looks like I would have to upgrade the memory and go to Max to get a big performance increase. Since I have access to cloud based systems with GPUs and TPUs I think I will just go with the base system.

[@AZisk]

glad it was helpful. Thanks for your courses 😊

[@JBoy340a]

@@AZisk thanks for the kind words.

[@SGLogic-O-Maniac]

Was the ANE test a model training or model inferencing? Can we expect using ANE for training PyTorch/Tensorflow anytime in future? I am blown away by the efficiency of the M1/M2 lineup. I never throught I would say this, but I kinda want to trade-in my Ryzen 5800H/RTX 3060M Legion Laptop and a kidney or two for those shiny M2 Max 16-inch.

[@eksjoker]

All info I've found so far on ANE for training has been a dead end. So I'd love to know what happened here as well.

[@RitwikMukhopadhyayBlueThroated]

Hi @AZisk, could you please do a similar comparison with Intel Core Ultra 9 laptop, not to mention to check it's GPU and NPU.

[@ADHDintothewild]

great dude!

[@AZisk]

Thanks!

[@08.edenaristotingkir86]

Should I get 14 inch m2 max with 38 core gpu and 32 gb RAM or 30 core GPU and 64 gb of RAM. Does RAM really plays a big role in training?

[@xavhow]

In the future, if these ML can tap into M1/M2’s neural engine and co-op with the GPU, could be even faster?

[@r12bzh18]

What program are you using to monitor the GPU? I have installed something called iStats - but yours looks a bit different. Great video! I installed tensorflow-macos and tensorflow-metal in a venv virtual environment but sometime I get some errors and it stops. Tricky install!

[@AZisk]

iStatistica

[@aceflamez00]

istatistica

[@kitgary]

I am interested to see how the 14" M2 Max performs, are there any significant difference between the 14" and 16"?

[@aaryanchadha]

thermal throttling, to prevent fan noise, overheating, apple reduces cpu and gpu performance on the 14 inch, also there's no high performance mode on the 14 inch which a lot of devs use as battery life is great

[@haralc]

How can the Apple chips neural engine performance can be different in the same generation when they have the same number of neural engine core count?

[@ritammukherjee2385]

hey Alex can you make a video on running Gpt4all falcon on m2 mac air

[@MarsRobotDeveloper]

¡Gracias!

[@AZisk]

glad you enjoyed

[@Lancelotxxx]

would be cool to revisit this, with LORA stable diffusion training comparison and other real life application of AI for average devs

[@MarkMenardTNY]

I think the total wattage usage of all of the Macs is lower than what my 3090 would draw. His saying the M1 Ultra drew like 59 watts almost made me laugh.

[@AIBeautyAlchemist]

i am play with stable diffusion recently, i first install it on my 1m pro Macbook pro, it works but slow. Now i am just trying to training some LoRAs, does it it working on M1 chips? and is anyone has experience on training stable diffusion models or LoRA on Mac, how it compare to RTX GPS???

[@carloslemos6919]

Any idea on how to run openai whisper on apple neural engine?

[@revan6614]

Thanks a lot for this! Does the M2 Pro Mac Mini (16-core neural engine) perform similarly? I'm trying to decide between the M1 Max Mac Studio and the M2 Pro Mac Mini for machine learning. They both have the same specs aside from the Studio having a 24-core GPU compared to the 16-core GPU of the Mac Mini. Would the difference of the M2 Pro over the M1 Max be more worth it than the 24-core GPU vs. 16-core?

[@ZhuJo99]

depends un usage. M2 Pro has more high performance cores than M1 Max. M1 Max has more GPU cores.

[@riccardrosen2073]

One thing YT has taught me is that productivity is only about creating videos. Thanks.

[@cgmiguel]

Nice benchmarks! I’m wondering how many kWh on your bill energy 😮

[@AZisk]

only shows up as a spike when i bring a desktop with an RTX card in here :)

[@urluelhurl]

What are the advantages of using a M2 max pro that does not have a dedicated GPU when for a similar price I could buy a P15 with a RTX 5000 that comes already equipped with Ubuntu and Nvida data science packages?

[@BrickWilbur2369]

any way I can get Photoanalysis in apple photos to work faster.??

[@Colquhoun7]

Just got an M1 Max 24c 64gb in Sept 2023 for video, feel like it’s still a good enough machine today.

[@gdotone1]

interesting the processors never get to 90%+ usage... is that OS, micro-coding, hardware ?

[@TheNameOfJesus]

This video is interesting. But the reason I'm hesitating to get the M2 is that I don't know how much it will speed up my neural-engine limited application. Basically, my app is the MacOS Shortcuts app which is running in an infinite loop doing an OCR of the entire 4K screen. It takes 4.1 seconds to perform OCR on the entire screen. If the M2 can do the same work in less than that, I'll probably buy it. I realize that my situation may also be dependent upon SSD speed, because the screen has to be put into a file to be sent to the Shortcut which does the OCR. Is there a way to do an OCR of the screen without putting the screen into a file first?

[@usptact]

Alex, how did you run tf-metal-experiments? I followed all the steps and got errors during runtime. Some google search showed this due to tensorflow for macos being too high - 2.11 installed in my case.

[@AZisk]

try downgrading tf

[@usptact]

@@AZisk Thanks, that did the trick. The version 2.11 was no good, downgrading to 2.9 worked!

[@AZisk]

@@usptact nice!

[@henryjiang9990]

SO which one should I get?

[@PerpetualPreponderer]

Schwarzenegger at the end of this video: "I'LL BE BACK..."

[@geog8964]

Thanks.

[@gufransabri9012]

I have an HP Victus laptop with RTX 3050 Ti Laptop GPU (4GB RAM). I use it for deep learning and the 4GB RAM is less than sufficient. I always run into OutOfMemoryError I'm considering buying the Macbook Pro 14inch M2 Pro with 16GB RAM, the one that you tested in this video. Should I buy it? Will it be sufficient for me? Can someone give me an in depth answer. My use case is not that of a beginnner AI student. But im also NOT training LLMs. For example, I'm doing a project where I'm training an Efficient Net b5 model and I cant use batch size more than 8. otherwise it gives me OutOfMemory Error Anyways, can somoeone please help me. Should I buy the M2 Pro Macbook 14inch with 16GB ram?

[@roberthuff3122]

🎯 Key Takeaways for quick navigation: 00:00 🖥️ *Introducing the Machines and Models for Machine Learning Testing* - Brief Introduction of M2 Pro and M2 Max and models to be tested - Introduction of the machines to be used, inclusive of a PC laptop 01:23 🔍 *Deeper Insight into the Models* - Explanation of the varying models such as resnet, mobilenet, distal bird, and bird large 02:48 💻 *Detailed Documentation of Training Deep Learning Models * - Description of setting up a conducive environment for dependencies, - Highlights on the heavy dependency on memory for the training 04:12 ⏳ *Experimentation on Multiple Devices* - Descriptions of how tests are kicked off on different machines - Provides real-time feedback on how different machines are responding to the test 06:05 📈 *Wrapping Up and Analyzing the First Set of Results* - A look at the results obtained from the various machines - Commentary on the impact of GPU core count on the results 07:55 🔄 *Another Round of TensorFlow Experiment Focusing on the PC* - Explanation of another TensorFlow experiment that includes the PC - Description of the different batch sizes and the dataset used in the experiment 10:12 🔮 *Insights from Third TensorFlow Experiment with Different Batch Sizes* - Analyses on the outcome of the second batch of experiments - Insights on how power usage directly impacts the speed and results 11:50 🧠 *Focusing on the Usage of Apple's Neural Engine* - Introduction of a test that demonstrates the Apple Neural Engine - Commentary on the impressive results driven by the Neural Engine 13:54 📈 *Results from Experiments with the Neural Engine* - Description of the results obtained from the experiments involving the Neural Engine - Observations and insights drawn from the results 15:28 🔚 *Conclusion and Closing Remarks* - Final thoughts on all conducted experiments - Encouragement to subscribe for access to more tests and software developer content. Made with HARPA AI

[@AZisk]

wow thanks!

[@krosser2123]

Nice reference for Star Trek fans. 👏

[@cheettaah]

Wondering why Apple didn't put the Ultra into a MacBook. 60W is still much less than 175W graphic cards in Windows laptops.

[@onclimber5067]

Would be amazing to see this test done with the new M3 models, since they are supposed to have a much better GPU.

[@axedemon2274]

hey, i am student who is studying machine learning, can i buy m2 pro for coding and stuff?

[@peterwan小P]

I would love to know what kind of models can the neural engine handle and how fast can it handle it in comparison to cpu and you.😂😂😊

[@rupeshvinaykya4202]

Which sw used for screen recording, plz share.

[@AZisk]

On mac i use this : prf.hn/click/camref:1100libNI. more specifically the toolbox has different screen cap features

[@furoriangurinu6601]

The point about llm's not being able to be run is false. A friend of mine ran a huge vision transformer and a huge BERT model on her M1 Air with 16gb base config. The swap is insane on these machines.

[@davidchodur7773]

I think you can utilise also tensorflow with ML cores. I tried it on M1 air and did some machine learning where CPU and GPU were basically idle.

[@choc3732]

How was the performance? Also how did you do this? lol

[@DeepThinkingGPU]

3 months later, my mac mini with the Apple M2 Pro with 12‑core CPU, 19-core GPU, 16‑core Neural Engine 32GB unified memory, is coming this week, they say its good for AI stuff, anyone know what I can do with AI with it? in June/July of 2023, thanks

[@MarkoChuValcarcel]

I can't wait to see the performance of the next M2 Ultra processor, it will reduce the gap between Nvidia and M processors in some training tasks !!!

[@ZhuJo99]

well the gap is reduced in favor of Apple Silicon, when taking power consumption to performance ratio. You can do stuff on laptop while running on battery, with Nvidia you need 10x more power (CPU excluded) and huge loud PC connected to outlet - even running it on laptop.

[@hikkiartemis4087]

What is better: one or two rtx 4090 or some m2 for ML/AI? Can you make such video?

[@AlexanderHyll]

The 4090 has more processing power ~FLOPs, but if you actually need that much processing it can be cheaper to run on a server than buying cards right out. If this actually matters, seeing as the cards are already so fast, you might want to prototype locally and train remote from a cost perspective. Also remember other factors play a big role too, for example thermal throttling on most 4090 laptops under prolonged loads of that magnitude, not to mention the wear and tear on the hardware, available RAM, unified memory vs random access/dedicated graphics memory etc.

[@the_cluster]

The ANE benchmark results for the M1 Ultra are astonishing. Especially where the M2 Pro was faster. Indeed, according to the specification, the M1 Ultra chip contains twice as many Neural Engine cores - 32, while the rest have only 16. The M1 Ultra was supposed to be faster than any M1 / M2 Max or Pro; in this case, it does not matter that the M2 has a slightly higher clock speed or more GPU cores. However, 32 ANE cores do not always give a performance boost. Very strange.

[@ef7496]

You made it so technical, so for beginners it’s difficult to understand which one was the best overall. Can someone tell us the final result of which one is has the best results overall? Thanks

[@antor44]

Very interesting video, but the data is explained too quickly, I have to set the KZbin player speed to at least 75%.

[@vivienseguy]

I wish we could use the Apple Neural Engine for training!

[@aggiechase37]

Can we now?

[@vivienseguy]

@@aggiechase37 I think not. ML training library such as PyTorch / tensorflow use the MPS kernels which seems different from Apple ANE. I haven't tried CreateML though. Maybe ANE is just not optimised for backpropagation.

[@aggiechase37]

@@vivienseguy I gotta wonder if that's in the works

[@ggsoloplayer]

what is this pc laptop

[@balubalaji9956]

asitop not working anymore.\! any body had any luck?

[@GHOSTYYY90]

For For getting into/studying ML, would you recommend the 16", M2 Pro 12-Core, 32GB, 512GB, 19 core GPU Or 16", M2 Max 12-Core, 32GB, 512GB, 30 core GPU price difference is 180 bucks Do I need 1TB ?

[@Someperson273]

Definitely 1 TB since 513 GB has reduced SSD speed. I would recommend M2 Max 1 TB

[@MarkPharaoh]

If this is just a curiosity/hobby I would just look at refurbished M1 stuff.