Raspberry Pi Supercomputer Cluster

  Рет қаралды 793,908

Gary Explains

Gary Explains

Күн бұрын

Пікірлер: 616
@GaryExplains
@GaryExplains 5 жыл бұрын
For those asking about the rack, I don't remember exactly where I bought it. But here are a few from Amazon that you might like: geni.us/3AAUtfx and geni.us/DbWsT and for the UK this one: geni.us/rGjT6
@DDBAA24
@DDBAA24 5 жыл бұрын
Does this method your demonstrating using the MPI library allow these RPi's to combine system resources ? I know you said that it sees 4 Pi's = 16 Cores , does it also pool the RAM ? Assuming the answer to both those questions are yes , can you still enable ZRAM within the cluster ? When compiling programs on the Pi it tends to hit swap after a point , ZRAM allows the Pi to swap to the RAM instead of the MicroSD (im sure you familiar). I would want to use a setup like this for compiling , does this cluster configuration help me in that regard ? Thanks Gary .
@kevindeng1889
@kevindeng1889 5 жыл бұрын
I suggest another rack with fan kit: www.amazon.com/dp/B07MW24S61 and for the UK: www.amazon.co.uk/dp/B07J9VMNBL
@jamesanderson478
@jamesanderson478 5 жыл бұрын
Hello Gary, I would like to thank you for making this video and maintaining such an informative channel. you are very easy to listen to and your explanations are to the point. Keep up the good work. Today was the first day i saw your channel link and decided to give it a go. the raspberry pi is a great pltform. However the raspberry PI Zero is a very compact development board as well. The Pi Zero does not have the compute power of the big pi. however the integration of the wifi on the board makes it one of my favorites. I built a super computer using the PI Zero as the main board. I loaded each PI with aversion of Windows 2008 data Center Server. I used RUFUS to flash 64 GB microSD cards with the OS. Once i had the OS working I cloned it using tis program... clonezilla.org/clonezilla-SE/ I have a multi-slot microSD card reader. It holds 24 microSD cards and connects via usb 3.1. I installed teh Windows 2008 Data Center Server on the microSD cards in about 15 minutes. Windows Server 2008 Data Center can cluster up to 32 machines processors at once. The Windows platform is very stable yet it is a little large. About 3 GB on each microSD. I used the data center server services to aggregate all the Raspberry PI processors and resources. i used the Wifi on each chip to bridge them all together. I only spaced them about 1/2 inch apart. The network and teh data processing is so fast that it returns almost instantly. I mainly use the cluster just to browse the web and play games. It is absolutely over kill on any kind of gaming or graphics program. I set the paging file on all the drives to 512 initial max 4096. I also connected an external 3.0 6 TB hard disk to the USB mini on one of the PI. Number 32 in the cluster. Then all teh data i download goes thru the other 31 and passes down stream directly to the external like a funnel. I can download a 4K movie in about 55 seconds with my AT&T fiber. I thought you might be interested in looking at this flavor of PI super computer. Of course i have Python, PHP 7.2 and AMPPS installed on the cluster. Automatic load balancing and wifi VLAN tagging. The windows 2008 server data center can open up a whole venue of new and powerful applications you may be interested in
@eg3730
@eg3730 5 жыл бұрын
@@kevindeng1889 hi
@kevindeng1889
@kevindeng1889 5 жыл бұрын
@@eg3730 hi
@fincrazydragon
@fincrazydragon 3 жыл бұрын
A long time ago, a man named Bill Gates had a vision: "A computer on every desktop" Now, thanks to Raspberry Pi, a new vision has emerged: "A supercomputer on every desktop"
@SS-ARYAN
@SS-ARYAN 3 жыл бұрын
But if every desktop computer is a supercomputer…
@belaolson8172
@belaolson8172 3 жыл бұрын
@@SS-ARYAN then we can only dream bigger, my guy 😎
@gustavojhonson7876
@gustavojhonson7876 2 жыл бұрын
considering the physical limitations of transistors, the only way to turn a single device into a supercomputer is through the cloud.
@emanyatta
@emanyatta 3 жыл бұрын
Wow! 12 minutes super computing lecture gives you more than a 4 year bachelor degree
@PersonALANty
@PersonALANty 3 жыл бұрын
Guessing you have that 4 year bachelor's degree and you're referring to it aren't you?
@gerboog
@gerboog 3 жыл бұрын
No. Just no.
@jarrettg7937
@jarrettg7937 3 жыл бұрын
I know you're trying to compliment the video (maybe inflate the audience's ego?) but I think your university ripped you off...
@infinity5288
@infinity5288 3 жыл бұрын
the university teaches you in more detail and less effort. this video is less in detail (in a nutshell)
@PersonALANty
@PersonALANty 3 жыл бұрын
@@infinity5288 Less in detail with more effort put into explaining the detail. I would say that teaches more, because the less effort put in, the less that you get your point understood, therefore you teach LESS because it is not taught, just stated.
@achill3sAp0
@achill3sAp0 5 жыл бұрын
Thank you Gary!! I learn more watching one video than spending hours on so called Tech Channels.
@jamesanderson478
@jamesanderson478 5 жыл бұрын
I agree... He is very good at the explanation and easy to listen to.
@falcondarkshadow
@falcondarkshadow 4 жыл бұрын
Watch tech quickie on yt
@philh98
@philh98 3 жыл бұрын
@@falcondarkshadow agreed linus and the gang really do good job there
@falcondarkshadow
@falcondarkshadow 3 жыл бұрын
@@philh98 definitely
@joakimjocka8022
@joakimjocka8022 5 жыл бұрын
This is by far the best example i have seen on this topic, excellent vid
@joseph6750
@joseph6750 5 жыл бұрын
What I find interesting is that that program you ran is the equivalent of what was run on the EDSAC computer in the 1950s when it was doing nothing else and you are generating more prime numbers in 30 seconds than it could in just under 10 years.
@1MarkKeller
@1MarkKeller 5 жыл бұрын
WOW!
@AbhinavSubramanian
@AbhinavSubramanian 4 жыл бұрын
Yeah, even the chips you find in those musical birthday cards have more computing power than all the Allied Forces put together did in WW2. It's crazy.
@dashboy007
@dashboy007 4 жыл бұрын
@@AbhinavSubramanian but we went to the moon on that power?
@bnbnism
@bnbnism 4 жыл бұрын
@@dashboy007 even your current computer/laptop/phone no matter the model is still many times more powerful than the greatest computers of the ones used for the first few moon landings
@dashboy007
@dashboy007 4 жыл бұрын
@@bnbnism I was trying to be sarcastic. There is no way my phone today could power anything else but itself, let alone a rocket ship.
@emd1999
@emd1999 5 жыл бұрын
The example you used with primes is one of concurrency rather than parallelism it seems. This is a very good primer on the basics of high performance computing though. Good video.
@ridingnerdy6406
@ridingnerdy6406 5 жыл бұрын
What people forget about the old microwulf clusters is they use 2 gigabit connections to per board to share data. The Pi3 has 1 ethernet connection capped at 300mbps, which made clusters actually slower than a single Pi. Now that the Pi4 is here with true gigabit and USB 3 support to add a second one, a Pi cluster might actually be a viable project.
@hammercanttouchthis
@hammercanttouchthis 5 жыл бұрын
So this video is misleading? 🤔
@bigmacbeta
@bigmacbeta 5 жыл бұрын
@@hammercanttouchthis it was a nice simple example.
@johndunlap9139
@johndunlap9139 4 жыл бұрын
It depends on the problem you're trying to solve. If the problem you're trying to solve requires minimal network bandwidth(small inputs and outputs) but requires a large amount of CPU processing time... Then the older pi's will work just fine.
@gregjalepeno6769
@gregjalepeno6769 4 жыл бұрын
@@hammercanttouchthis It was obviously a demo of a theory of clustering put to practise, not a video about optimisating data bandwidth and latency.
@jagardina
@jagardina 5 жыл бұрын
I do enjoy your videos, even though I already know pretty much everything you discuss usually. And I have recommended it to people who do need to learn about a topic. Great format, production quality and content. Thanks for making this.
@bobcat_the_Lion
@bobcat_the_Lion 5 жыл бұрын
Thank you Gary. There are a lot of videos on how to build a cluster with multiple rapberrys, but this is the first time I actually see it running as a cluster. All other videos stopped after the build, or ran them as individual computers.
@leledumbo
@leledumbo 5 жыл бұрын
Reminds me of concurrent & parallel computing class back in college, specifically the grid computing chapter. The classic example we used back then was matrix multiplication, while for the project we choose to parallelize inefficient sequential sorting algorithm with final goal to beat quicksort up to certain data size (because eventually quicksort still wins, it's just a much more efficient algorithm after all).
@1MarkKeller
@1MarkKeller 5 жыл бұрын
*GARY!* *Good Evening Professor!* *Good Evening Fellow Classmates!*
@GaryExplains
@GaryExplains 5 жыл бұрын
MARK!!
@PEGuyMadison
@PEGuyMadison 5 жыл бұрын
I looked a few of these Raspberry Pi clusters and for less than $800 I bought a used quad processor 32 core Xeon Dell R820 with 96 GB of memory..... and it just works. Sure when it's running it consumes more power but it's a unified memory across the 4 processors which makes HPC easier.
@HShango
@HShango 5 жыл бұрын
Very informative, i've always considered making a mini supercomputer (raspberry pi 3 +b)
@Standbackforscience
@Standbackforscience 5 жыл бұрын
Man I love this channel, always something interesting to learn
@antonnym214
@antonnym214 5 жыл бұрын
excellent explanation. Thank you! Back in the early days of the IBM PC, I wrote a game with virtual robots that did combat in a virtual arena, and each "Warbot" ran its own program, which was an interpreted language I wrote just for that game. The language was called R-Code. In this case, The R-code interpreter was running 5 programs at once, and each program had it's own simultaneous i/o. That was pretty cool in the old DOS days before windows and multitasking.
@hammercanttouchthis
@hammercanttouchthis 5 жыл бұрын
What version of DOS did it run on? And did you mean it ran on the IBM PC or XT? :)
@antonnym214
@antonnym214 5 жыл бұрын
@@hammercanttouchthis By that time, 1991? I was running it mostly at work on IBM XTs running MSDOS 3.3 or later, as i recall because we had 3.5 inch floppies. The entire programming environment and game fit on a single diskette. It was only 10,000 lines of QuickBASIC code. I never released it to the public, but I had one other friend who was interested in programming who liked writing R-code for the warbots.
@nyanates
@nyanates 2 жыл бұрын
Reminds me of Robot Wars. My friend and I had a blast programming our robots to pummel each other in the ring.
@mxcollin95
@mxcollin95 4 жыл бұрын
Really interesting! I’ve always wondered how that worked.
@DavideOrlando1969
@DavideOrlando1969 5 жыл бұрын
Yeah, i did it months ago with 4 raspberry pi 3 and MPI4py and it work very well! I also used psh (parallel secure shell), very useful tool.
@AbhinavPandit1
@AbhinavPandit1 4 жыл бұрын
Tht why i like to watch ur show... So many things to learn
@mav29
@mav29 3 жыл бұрын
perfect explanation 2:30 - 3:00 planning to have one built soon aside from having rpi4 , i was also thinking about orange pi to have as another alternative then mix them if possible just need more research on this thx sir Gary
@prashanthb6521
@prashanthb6521 4 жыл бұрын
Wonderful way to explain Gary, thanks a ton.
@certs743
@certs743 5 жыл бұрын
Forgetting the cost of power for a moment I am curious how the performance compares to a PowerPC based cluster which was probably the first "out of the box" consumer level hardware solution available that could be configured as a supercomputer cluster.
@GaryExplains
@GaryExplains 5 жыл бұрын
Yeah, I agree that would be interesting. In fact building different clusters from various bits of historical and new hardware and then benchmarking them would be quite interesting, but alas very time consuming!
@johndoe1909
@johndoe1909 4 жыл бұрын
@@GaryExplains my master thesis where done in the early 90:es, and it was about creating dynamic computing clusters using heterogeneous computers (various hardware architectures at the time). Given the overall limitations we identified types of problems which could scale using the available technique. Great fun and on the cutting edge for it's time. The main benefit was that the computer clients connected where largely unaware that they committed computing cycles, the jobs was running int the background. The base was done in pvm, in many aspects the successor to mpi.
@nilloviz
@nilloviz 5 жыл бұрын
You should make a video about quantum computers. A lot of youtubers have failed to present that topic in a neat way...
@Masterr59
@Masterr59 4 жыл бұрын
This was such an interesting video. One of the best I've seen in a long while!!
@GaryExplains
@GaryExplains 4 жыл бұрын
Wow, thank you!
@antonfernando8409
@antonfernando8409 3 жыл бұрын
awesome, never knew anything about super computing, and now i know, thanks.
@ykhatat
@ykhatat 5 жыл бұрын
Thanks I learned something new today!
@mohamedshuaau632
@mohamedshuaau632 5 жыл бұрын
4:14 On serious note. Love the channel. Love the video. Very informative. Thank you!
@positivevibrations5103
@positivevibrations5103 10 ай бұрын
Wonderful explanation!! Great channel.
@GaryExplains
@GaryExplains 10 ай бұрын
Glad you think so!
@ThomasGodart
@ThomasGodart 3 жыл бұрын
Nice work, Gary! And if you want to remove the overhead and speed up calculations greatly, you can switch from Python to Golang, for example, and have microservices do the work
@lorensims4846
@lorensims4846 5 жыл бұрын
Dragonfly BSD is an OS designed specifically to handle a cluster like this.
@gregorykusiak5424
@gregorykusiak5424 4 жыл бұрын
Loren Sims does each node have to match the rest, or can any machine get added to the array?
@Continus
@Continus 4 жыл бұрын
This just shows the purpose of Raspberry Pi, a learning tool. I never thought they would bring it to Server/Clusters. It's a great teaching tool from basic programming to now supercomputers. Raspberry Pi may not be a world record PC nor a Supercomputer with Tera flops in processing power. But it has proven to be a super teaching device that's caught a lot of interest world wide for those who want to jump in and learn. And a great gaming emulator! d^_^b
@coows
@coows 5 жыл бұрын
Overclock them and add heatsink and fans to the cluster for extra power.
@strenuousbobbykushner
@strenuousbobbykushner 4 жыл бұрын
Gary, this video was informative :) Thank you
@semco72057
@semco72057 5 жыл бұрын
That is a neat idea of making a Super Computer from a group of small computers like the ones you mentioned. I wonder if IBM has thought of that since they are into the Super Computer business.
@He_isI
@He_isI 5 жыл бұрын
That was done with the PS3.
@anthonya.jumelles7103
@anthonya.jumelles7103 5 жыл бұрын
Yup, that was a project that the US Military did because the PS3s had a lot of cores in a relatively compact form. It made being able to source machines from around the world really eaay.
@mixcocam
@mixcocam 4 жыл бұрын
Super cool video - would be great to see more examples!
@mihir206
@mihir206 5 жыл бұрын
Hey Gary could you please upload a step by step video to achieve node cluster????
@mrxmry3264
@mrxmry3264 3 жыл бұрын
About checking if a number is prime: square it, subtract one and divide by 24. If that is a whole number (no digits after the decimal point) it MIGHT be a prime number, otherwise don't waste time checking further.
@GaryExplains
@GaryExplains 3 жыл бұрын
Yes, there are plenty of different ways to check for primes are likely primes. But that isn't the main point of the video.
@maycodes
@maycodes 4 жыл бұрын
Thanx a lot Gary. merry christmas.
@midcardheelhd8522
@midcardheelhd8522 2 жыл бұрын
Wow.. I'm an idiot and you explained this so well that i think i could try to do this.
@gristlevonraben
@gristlevonraben 5 жыл бұрын
I just desire to have one pi for audio, one for video, and one to run them, to turn three pi's into a great desktop computer.
@kerph
@kerph 5 жыл бұрын
That would be pointless, because unless your doing instense audio processing, there would be no point in having a dedicated audio pi, and just one pi is not enough to do much video editing on, and at that point what would your third be used for if not everything else?
@calvint3419
@calvint3419 3 жыл бұрын
Thanks Gary. I also tried Apache Spark on Jetson Nano and it works. So I expect Apache Spark can work with Raspberry Pi too. The concept is the same.
@blazing-angels
@blazing-angels 3 жыл бұрын
Imagine if you had hundreds of Raspberry Pi's and then used this method to create the ultimate super Raspberry Pi
@prototype9000
@prototype9000 5 жыл бұрын
Linux has a kernel module that handles the nodes so you dont have to make complicated programs
@GaryExplains
@GaryExplains 5 жыл бұрын
Which module are you referring to?
@JohnUnsub
@JohnUnsub 5 жыл бұрын
I'm also interested
@AlejandroRodolfoMendez
@AlejandroRodolfoMendez 4 жыл бұрын
For what I knew it is in arch Linux, maybe
@pmccraken
@pmccraken 3 жыл бұрын
Nice. Very clearly explained and demo'd
@fuzzylilpeach6591
@fuzzylilpeach6591 4 жыл бұрын
That is super cool! (Pun not intended.) It's one of those things that I'd want to do just out of the fun of making it, regardless of how actually useful it is. Of course it is useful for many things.
@AungusMacgyver
@AungusMacgyver 5 жыл бұрын
Great explanation!
@familyaccount4753
@familyaccount4753 4 жыл бұрын
wow great video dude!
@wandiletembe
@wandiletembe 5 жыл бұрын
😎 Real Tech Channel. 👍🏾🙏🏾🇿🇦
@GonzaloOviedoLambert
@GonzaloOviedoLambert 5 жыл бұрын
amazing explanations. Great work, thanks
@aakashpatil3228
@aakashpatil3228 2 жыл бұрын
MPI - messaging service used in supercomputer cluster Scatter n gather (lesser data, high latency) (more data, less latency as compared to single core) Public n private key- security
@9a3eedi
@9a3eedi 5 жыл бұрын
I would much recommend using the raspberry pi compute module 3 instead of the regular boards. It has the same specs as the regular pi but are much more suited for this sort of thing, they're smaller so you can pack them more densely and need less wires, and can be ordered in bulk and cheaper. Unfortunately though you'd need to make your own custom board to connect them all together.
@GaryExplains
@GaryExplains 5 жыл бұрын
Just to be clear, you would recommend the modules to people who have the skills, knowledge and money to make their "own custom board to connect them all together."
@9a3eedi
@9a3eedi 5 жыл бұрын
@@GaryExplains that's correct. The assumption is that someone willing to build a cluster of raspberry pis will also be willing to make a custom board for his custom application
@GaryExplains
@GaryExplains 5 жыл бұрын
@@9a3eedi Well that is a bit of a bad assumption. I just built a cluster and I am NOT willing/able to make custom boards.
@9a3eedi
@9a3eedi 5 жыл бұрын
@@GaryExplains hmm... Makes me wonder if there's something available off the shelf..
@9a3eedi
@9a3eedi 5 жыл бұрын
@@GaryExplains just googled a bit and found something sold for 259 dollars that supports 5 cm3s, connected with switched GbE. Very cool :D
5 жыл бұрын
Could you use a lower latency/higher throughput interconnect like direct PCIe connection to increase the performance? For a few computers it could be possible (certainly not on large scale supercomputers since PCIe 3 has max. cable length of 8 inches).
@kopai555
@kopai555 4 жыл бұрын
Closet we can get now a day in my Datacenter that i take cared is FC (Fiber Channel). Every single server in DC and inside cluster are connect together in network via FC for lowest latency and maximum transfer rate.
@DataHotep
@DataHotep 4 жыл бұрын
We use infiniband in an HPC setting. Its connected through the PCIE bus.
@slowerpicker
@slowerpicker 5 жыл бұрын
Nicely done. Thanks!
@s.j.3247
@s.j.3247 5 жыл бұрын
Thanks for the nice explaination 😁
@markphillips8019
@markphillips8019 5 жыл бұрын
Why did you not use the Lite version of Raspbian? You lost no end of MIPS to running the GUI.
@0lAlex0
@0lAlex0 5 жыл бұрын
And would disabling the Desktop interface from raspi-config increase the performance?
@markphillips8019
@markphillips8019 5 жыл бұрын
@@0lAlex0 heck yes. You'd have gained about a 20% performance increase. The GUI is a beast.
@clintgossett1879
@clintgossett1879 5 жыл бұрын
But the demo video would suck
@YouArentValid
@YouArentValid 5 жыл бұрын
Because he doesn't actually want to calculate prime numbers as efficiently as possible, he just wants to make a video about cluster nodes lol.
@markphillips8019
@markphillips8019 5 жыл бұрын
@@YouArentValid Yeah, this struck me as one of those "I need content for my channel" videos. I'll bet he took it from a howto found on the Interwebs too? Plus, it's been done to death already.
@PepsisFormosa
@PepsisFormosa 5 жыл бұрын
You should try this with the rockpro64 and connect each board together through the pcie to get really low overhead. Might even get to write your own kernel and mpi layer!
@NoorquackerInd
@NoorquackerInd 5 жыл бұрын
PCIe doesn't act super friendly all the time. It's probably better using InfiniBand cards
@PepsisFormosa
@PepsisFormosa 5 жыл бұрын
@@NoorquackerInd Well if you're going to use a comm card, it would be a lot easier to put a ten Gb network card in the pcie slot. I was just thinking to keep costs down, you could try to run the messages over just the pcie lanes.
@DAVIDGREGORYKERR
@DAVIDGREGORYKERR 5 жыл бұрын
There are new SBC micro computers which have a RYZEN 8 core processor called the UDOO BOLT V8 get eight and you could have 64 cores.
@BubbafromSapperton
@BubbafromSapperton 5 жыл бұрын
Looks nice but would I be able to play Pong on it?
@twistednickster2653
@twistednickster2653 3 жыл бұрын
Nah we won't prob get that for another 5-7 years :(
@mjs2016
@mjs2016 3 жыл бұрын
@@twistednickster2653 not sure if ur joking or not but pong can be played on a web browser if your lazy, or if you have some time, you could get a retro console emulator and set up a pong ROM on it. Or you could program one with python.
@twistednickster2653
@twistednickster2653 3 жыл бұрын
@@mjs2016 i am joking lol dont worry i wont woosh you
@surjagain
@surjagain 4 жыл бұрын
Really loved this video :)
@Pauluz_The_Web_Gnome
@Pauluz_The_Web_Gnome 4 жыл бұрын
Hi, I have created a cluster programm, that runs before I even press
@Mangalover345
@Mangalover345 4 жыл бұрын
now the real question is, Can you play games on it? can it play the benchmark games?
@joehaines748
@joehaines748 4 жыл бұрын
Great job. Thanks for the video.
@ianwalker6546
@ianwalker6546 5 жыл бұрын
Nice video, really well explained!
@dryoldcrabman6890
@dryoldcrabman6890 3 жыл бұрын
This was fantastic!
@neelarjuna2536
@neelarjuna2536 5 жыл бұрын
This is one of the best tech channel in the KZbin.
@leenshelly
@leenshelly 4 жыл бұрын
enjoyed this video well presented
@pppluronwrj
@pppluronwrj 5 жыл бұрын
something new! thanks prof Gary
@alexanderwingeskog758
@alexanderwingeskog758 5 жыл бұрын
On my old Amiga I did a lot of fractal scenery animation and it took ages, this would be a really good demo of connecting more computers thru a slow bandwidth link. And raytracing also, Lightwave was really great as renderfarms go, one master/server (with GUI) then just send the resources to a bare minimum program that actually calculates the different images and sends it to the master/slave and it is pretty good on resource management as it just hands the nodes the images that is not done (hard to explain) but it really meant that you could connect pretty much anything... a slow computer, a fast computer and so on... it used everything at 100% all the time. Do Povray exist on the Pi? If it does that might be a good start for a great demo of connecting PI's :-)
@kestergascoyne6924
@kestergascoyne6924 4 жыл бұрын
Thank you Gary.
@mqtrx
@mqtrx 3 жыл бұрын
i would love to see people make raspberry pi supported or just games for the os and then see people make a pi super computer to get like 500 fps or even a video editor supported for the os
@HerrHafiz
@HerrHafiz 4 жыл бұрын
thanks for the info.need to study more about the clustering pi .For example begin to cluster 2,then 4, then 8, ....until recently Oracle already built the 1060 Pi s
@DonaldAnderson
@DonaldAnderson 5 жыл бұрын
Gary, I was using two vastly different computers. Computer 1 System 76 Meerkat with an i5 7260u processor. That computer alone could do the prime number program in about 4 seconds. 2nd computer, celeron based ChromeBox but in developer mode and also running Linux. Both systems using PopOS! 19.04 from system 76. When I add the celeron computer into the mix it takes MUCH longer. Like 1 min 30. I thought that even though the second computer was much less powerful, it could at least add some help to the calculations. I was somewhat surprised by the outcome.
@GaryExplains
@GaryExplains 5 жыл бұрын
I think with MPI Scatter and Gather the controlling node waits for ALL the computers to finishes their calculations, so slower computers in the cluster will slow everything down. You can write the program differently so it doesn't have that problem.
@DonaldAnderson
@DonaldAnderson 5 жыл бұрын
@@GaryExplains Thanks! I'll look into that!
@usquanigo
@usquanigo 5 жыл бұрын
Making a Pi cluster is cool, but common. Even with desktop clusters. The problem is - once a home user makes one.... then what? What can actually be done with it, that doesn't require writing code (other than playing with light blinking sequences). (Pi calculation, prime number generation, and benchmarks are basically the same as making the lights dance in sequence. Just short term novelties, not actually a use.)
@BobBeatski71
@BobBeatski71 5 жыл бұрын
I have three HP DL380 G6 machines, 32GB each, each connected by four Gb Ethernet to a DLink switch. They run OpenFOAM cfd software. Each machine puts out 300W of heat when running flat out. So a simulation costs the same as a 1kwh electric fire and I have to make sure I have decent air flow in the house, else it becomes unbearably hot !
@usquanigo
@usquanigo 5 жыл бұрын
That sounds really cool (though you just said it was hot). When you say "simulation", what are you actually doing with it?
@BobBeatski71
@BobBeatski71 5 жыл бұрын
@@usquanigo I run air flow studies to see what effects changing the internals of electronics units has on where the air flows and whether the flow is faster / slower. For example, a friend suggested, out of curiosity, moving a series of ventilation holes which separate two electronics compartments, to see what effect that would have. Turns out this was a much better solution than that which went into production. Now his idea is causing a revision 2 of the final metal work.
@usquanigo
@usquanigo 5 жыл бұрын
@@BobBeatski71 That is seriously cool. But also sounds like work (business). Not home-gamer (as AvE would phrase it).
@Amam-xu3xr
@Amam-xu3xr 5 жыл бұрын
Good now i can create my own server
@Smarkalbert
@Smarkalbert 5 жыл бұрын
First of all, Thank you Sir for doing this video, Is any bloggs of people specifically doing this project and maybe expanded the project? Please advice, I want to join.
@mianunya7607
@mianunya7607 4 жыл бұрын
now we need a raspberry pi 4 supercomputer
@GaryExplains
@GaryExplains 4 жыл бұрын
Yeah, I keep thinking about making a video about that, but I want to run a different workload on it so that's not just a repeat of this. However I can't think of an appropriate workload that would be easy to set up and demonstrate.
@bigmacbeta
@bigmacbeta 5 жыл бұрын
Great job again Gary.
@JUSTaCringeChannel
@JUSTaCringeChannel 5 жыл бұрын
Fantastic channel love it
@theoneyoudontsee8315
@theoneyoudontsee8315 5 жыл бұрын
i know a guy that used 40 raspberry pie 3b 2gb sbcs using a ethernet switch. recently upgraded it to have a better 40 sbc rack using 20 1ft cables and 10 2ft cables and 10 3ft cables all cat6 ethernet 24 gauge matching 100% copper to replace a mess of all 3ft cat5e ethernet cable 20 28 gauge and 20 26 gauge. i did not think how there linked together would matter but after upgrading just the rack and the ethernet cable in arguably a funny pointless expensive way ended up getting a more steady workflow that was vary noticeable. over a 24 hour period even got 1.2% more work dune witch is not margin of error as of this time have more than a year of data after the change and a year before and the thing with raspbery pi clusters is after you buy the cable you dont need to buy more as cat6 ethernet 24 gauge 100% copper cable should be more than enough for some time. i was told the different length of the better matching cable matters with some meta of load priority and how the sbcs link does matter total news to me and the data backs it up.
@maj429
@maj429 5 жыл бұрын
There's this thing called punctuation... JK good analysis.
@athul7227
@athul7227 5 жыл бұрын
Thanks for your knowledge
@IBITZEE
@IBITZEE 4 жыл бұрын
PS... if dare to suggest a 'petit' correction to the video... in the demo show the htop from the 4 Pi(s) and dont forget to select the process sort of each htop to CPU load...
@DavidOwensuk
@DavidOwensuk 4 жыл бұрын
Great video Gary:-)
@GaryExplains
@GaryExplains 4 жыл бұрын
Glad you liked it!
@Gdolwell
@Gdolwell 5 жыл бұрын
the r pi4 changes everything
@shokama
@shokama 5 жыл бұрын
Cool video! One question, though: can the Raspberry Pis be connected via GPIOs and make them behave like the cluster in the video? Will it be more effective that way?
@CodeXND
@CodeXND 5 жыл бұрын
there should be a way to do this on old android phones
@IBITZEE
@IBITZEE 4 жыл бұрын
As always... great info... great hob!!! you're the man,,, ;-)
@wychowanynawinie
@wychowanynawinie 5 жыл бұрын
Great stuff but there is a much more simple solution called k8s 😊 and of course there is smaller version of k8s called k3s check it out ! In company where i work we use k3s in IoT infrastructure and its work perfectly for example in edge devices.
@GaryExplains
@GaryExplains 5 жыл бұрын
Kubernetes is an container management system. A very different thing.
@RobertBartlettBaron
@RobertBartlettBaron 5 жыл бұрын
@@GaryExplains you are correct about kubernetes being a container management system (sort of, I like to think of it as an orchestration layer) but incorrect in your assertion that it is a very different thing. I am running a couple of kubernetes clusters, I have about 20 or so nodes each having multiple cores (just like your set of rasberry pi having 4 cores). Each node runs its own operating system, has certificates to ensure secure communication, and fast networking between them. All nodes have docker and the kubernetes software running on them, a couple of designated masters - running the scheduling software on them. So you have written a small program that you then load in to each of the nodes. Similarly, I can write a program that is encapsulated in a docker container. This container can then tell kubernetes to start 15 or so containers to run and passing in the values that they need to compute over (the scatter part of your example), then they pass back the result via sockets or a shared network drive. What kubernetes is doing for you is keeping track of all of the nodes and scheduling things for you, so that you as the user of the cluster don't necessarily know what the exact topology of the cluster is - you just say give me a node like ... . From an end user perspective, kubernetes is much simpler to use than your rasbery pi. In actual setup, it is more complex.
@GaryExplains
@GaryExplains 5 жыл бұрын
@@RobertBartlettBaron Exactly. And if you can't see that what you have written is very different to MPI and HPC then I can't help you.
@autohmae
@autohmae 5 жыл бұрын
@@RobertBartlettBaron I would say setup of k8s has gotten a lot easier over time, rke is probably easiest.
@gamearmour3431
@gamearmour3431 2 жыл бұрын
Question! Can a raspberry pie supercomputer be used for blender software for faster renderings! And can the raspberry pies be configured with gpus.
@TheB1nary
@TheB1nary 5 жыл бұрын
Fantastic video! Subscribed :)
@consig1iere294
@consig1iere294 4 жыл бұрын
It would be great for us regular people to see real world examples, how about a blender render in action?
@bentleyplays125
@bentleyplays125 5 жыл бұрын
I would use Udp to send packets to one of the hosts, then that host sends to the next host, etc etc and then the final host with all that information processes it and sends it to the main computer
@garyharris8082
@garyharris8082 5 жыл бұрын
thank you...subbed really intersting video.
@8bitbrainz
@8bitbrainz 5 жыл бұрын
But... can it run crysis?
@simonralph7720
@simonralph7720 5 жыл бұрын
Only in 480p
@--Valek--
@--Valek-- 5 жыл бұрын
@@simonralph7720 medium settings?
@woodie07
@woodie07 5 жыл бұрын
Maybe, but then you will have a burnt pi.
@Miraikon
@Miraikon 5 жыл бұрын
Nothing can run crysis!
@EvolvedApe
@EvolvedApe 5 жыл бұрын
i HAVE A $3000 DESKTOP AND IT STILL WONT RUN SMOOTH ON HIGHEST SETTINGS. I'm beginning to believe nothing will run crysis on highest settings. lol
@carloslemare6060
@carloslemare6060 4 жыл бұрын
I love your channel!
@GaryExplains
@GaryExplains 4 жыл бұрын
You are so kind
@theseamusexperience
@theseamusexperience 5 жыл бұрын
Cool video! We have a few raspberry pis laying around, this would be a fun project. Are you going to the SciPy 2019 convention in Austin?
@GaryExplains
@GaryExplains 5 жыл бұрын
No, unfortunately I am not going to SciPy.
@philh98
@philh98 4 жыл бұрын
What are some use cases for a super computer of this small of a caliber? (I am a bit new to the tech scene so if this is a dumb question, that is why)
@SwapnilLonkar
@SwapnilLonkar 5 жыл бұрын
Thanks for complete sherlock holmes on your github..
@artgressick
@artgressick 5 жыл бұрын
Great video!
@XenonG
@XenonG 5 жыл бұрын
Lemme drag up an old joke, BEOWULF CLUSTER!
@ratmadness4858
@ratmadness4858 5 жыл бұрын
SlashDot refugee?
@vapourmile
@vapourmile 5 жыл бұрын
Very good.
Full Windows 10 Desktop on Raspberry Pi!!!
8:16
Gary Explains
Рет қаралды 426 М.
The Petabyte Pi Project
22:27
Jeff Geerling
Рет қаралды 2,2 МЛН
Bike Vs Tricycle Fast Challenge
00:43
Russo
Рет қаралды 107 МЛН
Как подписать? 😂 #shorts
00:10
Денис Кукояка
Рет қаралды 8 МЛН
My Daughter's Dumplings Are Filled With Coins #funny #cute #comedy
00:18
Funny daughter's daily life
Рет қаралды 12 МЛН
YOU Get a Supercomputer! - Supermicro AI Workstation @ CES
15:14
Linus Tech Tips
Рет қаралды 1,5 МЛН
Raspberry Pi 4 Review - Dual displays and upto 4GB of RAM
17:19
Gary Explains
Рет қаралды 325 М.
Begun, the CM4 Clone war has
15:47
Jeff Geerling
Рет қаралды 280 М.
Every Developer Needs a Raspberry Pi
27:27
Sam Meech-Ward
Рет қаралды 650 М.
When Did Raspberry Pi become the villain?
21:54
Jeff Geerling
Рет қаралды 1,6 МЛН
Top 10 Uses for a Raspberry Pi (400th EC video!)
11:24
ExplainingComputers
Рет қаралды 242 М.
When you Accidentally Compromise every CPU on Earth
15:59
Daniel Boctor
Рет қаралды 834 М.
Homelab upgrade: New Raspberry Pi Cluster rackmount
14:44
Jeff Geerling
Рет қаралды 557 М.
My thoughts on framework after daily driving it for 2 years
16:34
Louis Rossmann
Рет қаралды 710 М.
NVIDIA REFUSED To Send Us This - NVIDIA A100
23:46
Linus Tech Tips
Рет қаралды 10 МЛН