10Gb Home Network (P2) - Peer to Peer

[@Pietro_CALTABIANO_CH]

Incroyablement bien expliqué! Et même après 4ans, cette série de présentations reste une véritable référence. Merci beaucoup iTechStorm, je regrette seulement qu'il n'y ait pas plus de vidéos sur cette chaine.

[@Itechstorm]

Thanks for being patient for the release of part 2. I wanted to be sure things were accurate, tested, detailed, and something I could put my name on. I really hope everyone enjoys this video as much as I enjoyed making it! If you learned something, consider subscribing and liking the video. I even have a "tip jar" available on my channel if you feel compelled. Thanks for your support!

[@jolejo7]

+iTechStorm so you use the Mellanox one? the ConnectX-2 Family:? I thinking to buy some but there not much information about compatibility with EXSI. Did you test one in a ESXI? Tks

[@Itechstorm]

+jolejo7 I didn't test with ESXi but I see matches on the VMware HCL: www.vmware.com/resources/compatibility/vcl/result.php?search=Mellanox+connectx-2&searchCategory=all

[@Maisonier]

Can I install 2 or 3 Intel x540t2 on a motherboard Msi Z170-a Pro with a celeron, use the onboard for the wan, install pfsense and use that device as a router and switch with 10gbe ethernet for 6 computers? or there isn't enough lines in that motherboard? or the lines doesn't matter when is working as a switch?

[@TNW1337]

yes @@@Maisonier

[@char00lesg]

dude.. Came here to say that your videos are awesome. Fun to watch and learn. Now I'm hooked. Keep it up.

[@Itechstorm]

+char00les Glad you are enjoying the content! Thanks for the post.

[@michaelreyes9997]

I really enjoy the information and the quality of the production, keep up the great work!

[@Itechstorm]

+Michael Reyes Awesome! I am glad you found it useful. Thanks for the feedback.

[@SteveOwensRoswell]

Thanks for these 10Gb videos Have 10G up and running now between my Plex server and Storage server.... very very fast transfer speeds now!

[@Itechstorm]

+Steve Owens awesome! That's what I love to hear! Thanks for the update.

[@ilyai746]

for DAS should be ok....old pc like nas :)

[@ipullstuffapart]

Great quality video on 10g, just wanted to note that at around 5:45 you mention GigaThreads/second, where in this case it's actually GigaTransfers/second

[@Itechstorm]

+ipullstuffapart very possible. English is hard!

[@vasilispapas482]

Thank you for all these knowlege!!!! Grate job!!!! For the Gays how ask for distance :You can use SFP to FIBRE adaptors for the conections and then you can use fibres with limits of distance 400meters for the chip ones and 11km for the expencives! Vasilis GR

[@BloomerzUK]

Really enjoyed the first part, thanks for posting the 2nd. Going to make a coffee and enjoy watching now. Cheers from England.

[@BloomerzUK]

+JBloomfield1990 Very good video. You should share this to reddit, specifically /r/homelabs ? They will certainly appreciate it. We have a Mellanox InfiniScale IV in the office, 40GBp/s - mental speeds.. Not had a chance to play with it yet.

[@Itechstorm]

+JBloomfield1990 Good afternoon from the other side of the pond. Coffee sounds like a brilliant idea! Bonus thumbs up for the suggestion.

[@BloomerzUK]

+iTechStorm Correction: /r/homelab is what you want.. remove the 's' at the end. Those guys would appreciate the videos for sure. Maybe even reference it in their FAQs.

[@Itechstorm]

+JBloomfield1990 Thanks for the feedback. You are right on target with your suggestion. This content is truly aimed at the home-labbers of the world. I'll post in a bit if someone hasn't already beaten me to it. Just curious, how do you plan to use the Mellanox InfiniScale? Top of rack switch? Enjoy the setup! That's truly the fun part, isn't it?

[@BloomerzUK]

+iTechStorm No problem - happy to help. I absolutely love videos like yours; knowledgeable, instructional and most importantly informal! Don't think anyone's has shared it yet on reddit. The Mellanox InfiniScale is going to be strictly used between three Dell R720's for CFD (Computational Fluid Dynamics) operations. It's really for low latency for clustering - it's improved production work by some margin.. Some jobs used to take a week, now can be done in under a day!

[@mety9325]

Very useful Information about the possible bottleneck PCIe Slot. Thank you

[@joshua.jebadurai]

Dude. You're a God among mere mortal geeks.

[@bobcauthen]

Great videos! Please post the third one. I am very interested also in the software you will use in the third video. Keep up the good work....

[@Itechstorm]

+Bob Cauthen thanks!

[@ejemm]

Well shot and explained! Great job!

[@Itechstorm]

+ejemm thanks!

[@theodoro89]

Awesome video! Can't wait for the build video of the switch! :D

[@Itechstorm]

+theodoro89 thanks for the feedback!

[@liketruth8053]

Yes I cant wait ether. Its making me a little crazy that this is the second video I have watched about connecting a nas to one computer, why do this when you could just plug the SSD right in the computer I am using. ? Is their a reason I would want to use a Nas server that only serves one computer ?

[@TheInternalNet]

I remember this when it was first released. Can we have a 40gb update??

[@techdocstr]

Thankyou for sharing.

[@jolejo7]

What 10 GB switch :O never thought it was possible to make a custom switch :p Great video, look really professional

[@geogmz8277]

I want to see that.. as well..

[@Itechstorm]

+jolejo7 Awesome. Glad you enjoyed it. And yes, I'm going to build a switch in the next video. XD

[@jolejo7]

+iTechStorm I'm really curious about this because, i thought it would not compete with dedicated hardware for switching (ASICs)

[@Itechstorm]

+jolejo7 The switch build is not intended to compete with an enterprise layer 2 switch. But for now I can say it will yield some interesting results and open the door of possibilities where people feel very limited to begin with.

[@jolejo7]

+iTechStorm Can wait to have to news about it :) Are you into the networking field? I'm still at university but my first test of the cisco ccna is scheduled in July

[@kfl611]

This kind of made my brain hurt. Need to watch it more than once.

[@sabotpfl]

What is the best method to ensure that my PC is using the 10Gb connection vs the 1G onboard? My Window 10 client continues to use the slower NIC. I am scratching my head at this point. Sorry, I am not a techie but love to experiment. Thank you for your awesome videos!!!

[@seans9168]

Mr. Sabot you would need to set the IP address to be on a different network for your 10Gb interface. And configure the remote end to be in the same network. For example your 1Gb adapter might be in network 192.168.1.0/24. So you would put your 10Gb interface in network 10.10.1.0/24. When you want to transfer using the 10Gb link connect using the 10.10.1.x IP. That way your PC knows to route the traffic over the right pathway, in turn getting the 10Gb speeds. Next time you do a file transfer in File Explorer access your remote system using \\10.10.1.x\sharename and your traffic will route over the 10Gb link. Hope this helps!

[@cvazquez743]

I had the same issue it tuns out it was the sharing configuration, the adapter by default goes to public sharing, you must Turn on network discovery and Turn on file and printer sharing. This is for Windows 10 Pro.

[@sergeantseven4240]

Prices have dropped on switches and NICs since this video came out. It might be better now to just get a switch with 10Gb ports on it. Also the NICs are cheaper now.

[@bigchew3149]

Id Like To See a Updated 10Gb Video,As I Have Ben Thinking of Going /Experiment With 100Gb..LOL. What Do You Think about 100Gb Networking here in Late 2019 I would like to see if it would be something worth Experimenting With or not! Mainly For FreeNas/Maybe UnRaid ! Thanks

[@tobyhines7587]

Wow, No way I'm trying this. But fun to dream about.

[@Itechstorm]

+Toby Hines I did all the hard work so you could have all the fun! Living life on the edge with 10Gb. That could be you!

[@hefwilliams5400]

Great series, at the end of this video you mentioned you built your own switch? Can you share anything on that yet?

[@Itechstorm]

+hefyn williams I don't want to share any details just yet. But is there something specific you would like to see in the video?

[@hefwilliams5400]

Hi, thanks for reading... really keen to get a setup going with 3 or 4 ESXi 6 hosts running on 10GB for VSAN / Vmotion, without the expense (and noise) of a 10GB switch in my homelab :)

[@Itechstorm]

+hefyn williams thanks for your patience. I am back to working on it full time starting tomorrow. Thanks for an example. I will use your example in the video.

[@raheelhameed1]

Hey iTechStorm, Thanks for all the effort you put towards creating these very useful tutorials. My desktop is about 55 ft away from NAS, if I use MNPA19-XTR Mellanox ConnectX-2 cards, can a fiber chord be run between them at that distance (I don't see DAC cable can accommodate that big of a distance)? If yes, what transceiver modules I should be using and what fiber cable? Can you please point me to links for the components that will help in linking the two cards at 55 ft distance? Thank you so much!

[@Itechstorm]

+raheelhameed1 yes, transceivers and fibre will work. Make sure the transceivers are compatible with the cards you have. Finisar seems to be popular. Look for multimode 850nm transceivers/fibre with LC to LC connectors. Fibre may be designated as OM3.

[@josephhh1745]

THANKS FOR VIDEO.

[@legominimovieproductions]

Wow... the first consumer boards with pcie g4 are just about to get distributed and it was already n the list back in 2016?

[@diegopineda8840]

Bad ass. Thank you sir I am going to do this at home and work!

[@Itechstorm]

+Diego Pineda good luck!

[@pfunk897]

Hey man. Thanks for the videos. I've been waiting to see the diy switch video. Do you think that's coming soon? I'm assuming it's something along the lines of bridging multiple 10g base-t nics using FreeBSDs network stack(?), or is it something more complex or altogether different in architecture? Anyway, I'm excited to watch. Thanks again!

[@Itechstorm]

+pfunk897 glad you are enjoying the series. I really want to get this next video knocked out too. I've been keeping everyone waiting long enough. I'm optimistic that it will be finished in a couple weeks. As far as how I'm going to do it, you are on the right track. But I'm going to take a direction that a lot of people are not expecting. But that's a good thing, and I'll explain why. I plan on keeping things as simple as possible considering the more advanced nature of what I'm trying to accomplish with the 10Gb home Network Series.

[@tcc5750]

How would one connect a 10G SFP+ switch to a 10G nic if each use different transceivers? Ubiquiti XG16 seems to use it's own proprietary Ubiquiti type transceiver, but my Mellanox nics have their own transceiver types correct?

[@Itechstorm]

+TCC ya, sometimes that can get tricky. Not everything is going to talk to everything else. A lot of the time you have to plan carefully as to minimize the differences between your gear. Network hardware vendors would prefer you stick with their brand for all you components of course. This is not always realistic or practical. In your case look for a third party transceiver that is compatible with the switch and NIC. Check Finisar. They are supported on a lot of platforms I believe.

[@tcc5750]

iTechStorm For some reason I didn't get an email notification you replied! Thank you for replying, and your reply helped me. I googled around more, and I decided to order from amazon so if the DACs don't work I can return and order a different set.

[@Itechstorm]

+TCC excellent! Glad I could point you in the right direction.

[@TNW1337]

A get switches that all the cables are compatible B have a Company make you a custom DAC cable. C Use SPF+'S With Fiber.

[@taoli5497]

very cool. Thanks.

[@atomsbudgetreviews8713]

Awesome info. Thank you. I'm currently looking at changing my network setup. I run two Hp Pro curve 1800-24g switches. I've built a FreeNAS dual Xeon system which has dual onboard 100/1000 ports. Can I use these ports to connect to my existing switch and then add 10Gb PCIe cards to connect to my main pc?

[@Itechstorm]

+Jesse A ya, you should be able to add a 10Gb adapter to your FreeNAS box. Check for compatibility before you buy anything. FreeNAS can be picky about drivers sometimes. Direct connect your two systems, and configure a different subnet for your 10Gb traffic. Don't forget to reference your new subnet to get the better speeds. Good luck!

[@Univers314]

Nice and what is your hardware configuration for the two workstations. type hdd, Memory clock speed, Mainbord, cpu, ........Is this also possible for notebooks?And what with many small files? Is this also 1 gb/sec Thanks in advance Gino

[@Itechstorm]

+Van Gino I just going of memory here... System1: cpu 3.5 i7, 32GB RAM, SSD, Asus desktop mainboard; System2: cpu 2.5 xeon quad, 16GB RAM, SSD, HP mainboard. For it to work with notebooks it would need to be embedded 10Gb port. I remember seeing thunderbolt to 10Gb Ethernet adapters too for macs.

[@decafpancakes]

I want to add a network share but i want it to use the 10g card, how would i do this.

[@capturingperfectionphotogr1144]

I wish someone answered this question as the details on how to do this were assumed in this video and left out and skimmed over.

[@newart88]

Thanks for the great video.

[@Itechstorm]

+Saleh Al glad you enjoyed it!

[@newart88]

Waiting for your router video :)

[@Maisonier]

Can I install 2 or 3 Intel x540t2 on a motherboard Msi Z170-a Pro with a celeron, use the onboard for the wan, install pfsense and use that device as a router and switch with 10gbe ethernet for 6 computers? or there isn't enough lines in that motherboard? or the lines doesn't matter when is working as a switch?

[@WizardNumberNext]

Chelsio is actually quite a surprise cards with uploadable firmware will update its firmware by themselves on firmware upload-neat

[@WizardNumberNext]

DACs are coming in any lefthand between 0.5m and 7m I have seen 0.5m, 1m, 1.5m, 2m, 3m, 5m, 7m and 10m the last one is outside of spec and need DAC only card like Intel AF DA spec allow for up to 7m lenghts provided standard fully desired SFP+ port 10m is possible only on DAC only SFP+ port

[@cesarcasas]

Excellent

[@neuro112]

Is it possible to use fibre cables instead of SFP+, the reason is my computer to my NAS is over 10 meters (SFP+ cable). The distance is about 40 ft. Can you recommend any cables? Thanks

[@Itechstorm]

+neuro112 Absolutely! That's another reason I recommend the SFP+ form factor. You will need a fiber SFP+ transceiver that is compatible with your 10Gb card. So in the case of Mellanox you would look for Mellanox compatible 10Gb fiber module 850nm LC. 850nm is the wave length, and LC is the connector type. For the cable get multi-mode (mm) LC-LC 850nm. Or search for OM3 fiber. This will take you 300m or 984ft :)

[@endlessmindset8808]

Anything you would update/recommend since I found your video in 2021? I need to transfer large 100gb+ files constantly throughout the day from computers A and B to computer C

[@giuseppefalco5106]

HI, BEST VIDEO i have two workstation pc at 25 meter distant, can i use fiber cable and transceiver with mellanox card to increase distance of connection....instead of dac (direct attach copper.. 1, 3, 5 is too short ) ?

[@Itechstorm]

+Giuseppe Falco yes that should work, just make sure you get compatible transceivers and the correct fiber to match.

[@giuseppefalco5106]

Thank you so much for the reply Obviously the modules get them compatible, do you have some sfp + 10gbe to suggest it? The important thing and that low costs and that works fine ... for fiber cable I had thought of the OC3 multimode since the network does not exceed 500 meters I would like to buy a switch to put between the 2 pc workstations, because i have 2 more workstations within my network to connect to fiber, one to 33 meters and another to 50 meters, i was thinking of buying a tplink T1700G-28TQ because it has low cost (300euro) and have 4 SFP+ 10gbe ports to connect 4 pcs for data exchange, This router in the description says it serves as an uplink and does not explain if it is suitable for data exchange, according to you I can use it as a normal data exchange router, but instead of using ports rj45 (ethernet) to use sfp+ ports? This tutorial is great, I accept tips from experts It would take a tutorial that describes how to make a fiber LAN 10gbe not ethernet but REAL fiber Thanks to everyone for the answers

[@kingneutron1]

Just a thought, but you should probably have at least a quad-core i5 or equivalent processor to do fast(er) networking... I wouldn't attempt it with say a Core i3 unless you're doing nothing else on the box but data transfer (no GUI or user-interactive apps) and are prepared to deal with less than top speeds

[@didnthurt4077]

Can i add a 10G single Networkcard for p2p connection to my NAS and at the same time use the standard onboard LAN connection to the 1G Home Network? Basically, can i have 2 network adapters running at the same time (1G+10G)?

[@Itechstorm]

Sure, why not? As long as you have an open slot compatible with your card.

[@codychoe]

Are you saying if there is 2 ports on NAS and you have only 2 computers to connect to, you don't need switch? Just connect cable directly from computer to NAS?

[@K_shabby]

If I wanted to connect 3 PC’s would using a dual port card in each of them and then connecting each pc to the other two work? Alternatively - my main work station is the only one which actually needs to talk to both other PC’s So could I connect my main workstation to my render server AND my storage server using a dual port card?

[@89hunt89]

YOU CAN DICTATE WHICH CONNECTION IS USED BY ADJUSTING YOUR METRICS PER CONNECTION.

[@MichealG]

Thanks a lot for your video. You said in the video that you\ll connect your zfs. Did you manage to use your card with freenas? If yes, what's the model number of your card?

[@Itechstorm]

I decided to hold off on the FreeNAS build until they release 10 stable.

[@jcnash02]

I’m using Chelsio on FreeNAS and XCP-NG

[@chazfaz3595]

Very Helpful & informative iTechStorm! Thanks! Being a mere intermediate level computer geek, my understanding is often spotty. I do understand how the hosts file works, establishing associations between names or aliases and IP addresses. And I see that in your setup, "HSPC1" & "HSPC2" point to the respective IP addresses you assigned to the two 10GB adapters; but I couldn't catch how those aliases can then used in Windows to establish a direct connection to the desired host. Anybody catch that? It went by so fast. Right now my system occasionally chooses the 1GB network to connect. Using the alias would, as you suggested, prevent this apostacy.

[@stevewinner]

That's the part that I didn't catch either. I get putting the aliases in the hosts file, but didn't see where he connects to the other pc using the hosts file alias

[@garycrumrine9538]

So, I am wanting to cable up my house, which has PC's on multiple floors, and all will exceed the 15 ft limit. What do you do for longer distances?

[@LeighHart]

If you want 10Gb then your best option is fibre optic links - you can get SFP+ modules for the vendors mentioned in this video from sites like fiberstore for very cheap. Switch wise, there's a few out there - unifi xg16 is affordable all things considered. my opinions are my own.

[@Phil-D83]

Any switch with more than 2x gbe ports are stupid expensive. Not sure why in 2016-2017....

[@Itechstorm]

And the point you make is what fundamentally drove me to build this solution, and roll this video series. The whole "fun" factor might have had something to do with it too.

[@attilavidacs24]

Can I make my own cable using LC-LC fiber patch leads as I have spare SFP connectors?

[@TrilleX421]

A little confused, should 10Gbit network have the same netmask as the 1gbit network?

[@chrisumali9841]

Where is the 10gb switch build video? Thanks for part 1&2

[@Itechstorm]

+Chris Umali I post all my video status updates to my Facebook site: facebook.com/itechstorm

[@chrisumali9841]

Thank you

[@fabslyrics]

great video !!! thanks how does that differs from Mellanox Infiniband 20GB/s ? same setup and functionnalities ? and just differents connectors or far more?

[@smakarl0]

And on Linux?

[@erlingpj]

If I use a 10GbE dual port card (Intel Ethernet Converged Network Adapter X520-DA2) and Windows Server 2016/2019, will I be able to use NIC teaming to get 20 Gbps between the two peers?

[@Itechstorm]

Yes, BUT you will only get 10GB per session, even though it shows as a teamed pipe. You would be better off linking that second port to a third computer and bridging the two local ports to switch traffic through the host in the middle.

[@hanse81]

I understand the installation but what I didn't understand from this video is the following: When doing a normal file transfer from one computer to another, how would you be able to make the computer use the 10Gb connection? Is Windows just that smart so it selects the fastest option? I want to connect my gaming computer to my fileserver in the same room with 10Gb.

[@Itechstorm]

You are designating the 10Gb link to be a different network (10.x.x.x). So when your client pc requests something from the 10Gb network it will use the 10Gb link.

[@lrrrruleroftheplanetomicro6881]

whats the reason for editing the windows host file? You've been entering the ip addresses and the respective computer names?

[@DesertCookie]

Windows won't use the connection by default.

[@lrrrruleroftheplanetomicro6881]

@@DesertCookie Thanks! But 3 years later I stll didn't get round to doing 10 or 40Gb ;-)

[@DesertCookie]

I'm just leaving the information here for passers-by :).

[@lrrrruleroftheplanetomicro6881]

@@DesertCookie Does that mean u don't care about me at all ? :-(

[@DesertCookie]

Arent't you too a passer-by. Not having been that human you were so many moons ago? Now older, wiser, filled with wisdom, and potentially regret about never having tested 10Gb LAN.

[@robertagans8083]

great tutorial, curious what is the top file transfer speed I should be able to get? I have 2 like workstations Sun Ultra 27 W3540 XEON, I seem to top out about 300MB, using the Mellanox MNPA19-XTR adaptor. thx

[@gjergjibullari]

Are these Chinese version any good? I see that Mellanox MCX311A-XCAT has a Chinese version sold on eBay as well as a Made in Israel version which I trust is higher quality.

you built your own 10 gb switch? wheres the video link please?

[@Itechstorm]

+Jay Mee Yes, I built a 10Gb switch and had a blast doing it! Now I just need to build the video that shows how I did it. Unfortunately the video is more difficult and time consuming than the switch. Lol. Oh well.

[@CineTechGeek]

+iTechStorm Keen to see this. I was looking at this too.. Use 4 PCIe slots with 3 dual port cards then bridge them all together.. I did a quick test with three cards to three systems and it seemed it would work well.. But do you think there is much of a performance hit over a real 10GBe switch??

[@Itechstorm]

+CineTechGeek of course it's going to depend on the gear you use and how compatible it is together. Makes all the difference in the world. That being said, a dedicated purpose-built switch will perform better to some degree. That may only be 10ns or 100Mb better. But I look at it like this... Why do I need the kind of switch purchased by a data center? As a home user you will never notice the difference in the end.

[@CineTechGeek]

+iTechStorm I look forward to your part 3.

[@alopextos8629]

+iTechStorm Before you finish the video, would you tell in advance what os you are using? If it is linux or bsd, which distro? Thanks.

[@TNW1337]

Just buy a mikrotik 317 16 ports of 10Gb love!

[@39zack]

I see the oldest cards (the one that was made for PCI-e 1.0) are the cheapest, but will they work on windows 10?

[@Itechstorm]

Subscribe and Like my Facebook channel for frequent updates on the release of my latest videos! Check it out: facebook.com/itechstorm

[@Chaz042TFC]

Soooo, part 3?

[@BryantBeers]

I've been wondering about part 3 myself. Really interested.

[@Itechstorm]

Is this what you are looking for? :) kzbin.info/www/bejne/pmScnnmwbLSFmdE

[@Chaz042TFC]

+iTechStorm Yes, Yes it is :D

[@Itechstorm]

+Bryant Beers posted!

[@sukhoi01]

Hi, I used the same 10gbe Mellanox to connect from my computer to NAS but the NAS gets disconnets and crashes i think if i use 10 Gbe connection for a prolonged duration. if i use 1Gbe it works without issues. Can you suggest what i may do which can resolve this issue?

[@yonashabtamu7724]

HI do you Support for DELL OptiPlex 790 or 9020? thank you

[@tylertc1]

Love this stuff - decided to follow along step by step - purchased same cards, got stuck at the mlxup portion - but was able to continue forward without updating the firmware..but annoyed because I want to do that part and I know it's not working because I'm missing something (not seeing it something etc) Man, having the damn'dest time with getting mlxup to run. I know I'm missing something basic. But am getting C:\Users|TYLER~1\AppData\Local\Temp\sfxter_Tyler' is not recognized -as internal / external command or batch. Which the external / internal command not recognized has plenty of fixes on Google..which I've completed (although I know that's nearly impossible for you to tell..as with this stuff it can be one little setting somewhere that you'd have to really see to be able to assist with. But I'm curious. When you put mlxup in the sys32 file path. I note that you're in a folder named Mellanox. Did you create a folder within Sys32 called Mellanox and this is where you put mlxup. From the video (which I've re-watched countless times) - although quick -it just looks like (you said) you dropped it right in System 32. But for the life of me I can not get it to recognize mlxup of any version. My system is 64bit naturally made sure I downloaded the 64bit version for Windows (running W10). I can go on - and I'd be happy to make a quick video of what I've done. But wanted to say things for putting this information up, it's been a lot of fun to follow the build thus far and try to replicate it! helpdeskgeek.com/how-to/fix-not-recognized-as-an-internal-or-external-command/ - is an example of things I've tried and most other point to that exact same fix - but for some reason still not working. For reference I'm not telling you this as in asking for help. Just where I'm at but most importantly that I appreciated you posting the content and providing a series that someone that likes to dabble with this stuff can follow.

[@tylertc1]

I should note that I continued forward - got them on a separate network and can ping them from one machine to the other and vice versa....but I can still tell that I've got the original firmware of 1000 vs the fw-ConnectX2-rel-2_9_1200-MNPA19_A1-A3-FlexBoot-3.3.400.bin

[@Itechstorm]

Hey Tyler, sorry to hear you've had so much difficulty with the firmware update. I wish some vendors wouldn't make it so difficult sometimes. First I'd say if you got things working and you are seeing good speeds you may want to leave the cards at their current version. It's isn't always necessary to update firmware/drivers. I put that in the video really to point people in the right direction in case they needed to for whatever reason. I did mention throwing the executable into the c:\windows\system32 directory. But the binary can sit anywhere. You just need to make sure the mlxup utility is in your PATH variable somewhere. Good luck!

[@tylertc1]

Thank you for the response. Yea I've got it in the path - followed that part but I know I'm missing something simple lol. But it's still good fun, and yea with the speeds where they should be when testing, I've left the firmware versions alone for now. Just like solving problems or figuring it out since I'm not as familiar - fun challenge. And I definitely understand about noting that in the video. A lot of people don't follow that step of checking / updating firmware versions (granted sometimes you'll have to roll it back if you're working with some specific xyz software or piece...but generally always update). Thanks man, look forward to more tinkering if you've got stuff in the pipeline. If not - at least thanks for these!

[@fabricetoledano4040]

i really tried hard and it s not a walk in the park ... on windows 10 2 identical dual port connectx-2 but "opensm" service shows port is down , plus always shows as unplugged with qsfp dac well plugged. ... OFED VPI installed and card recognized ... any idea why ?

[@geogmz8277]

But can you use these dual ports to be configure in lacp or lagg setup? I don't see why not.

[@Itechstorm]

+Geo Gmz Of course you can! :D

[@geogmz8277]

+iTechStorm cool, like your videos.. by the way check "extreme" switches.. they're very cheap and capable of L2/L3 10gbps.. They were no very reliable for "enterprise" but good for home/small office or lab..

[@Itechstorm]

+Geo Gmz Good to know. Thanks for the update.

[@chris-rh8bc]

cute cat whats his/her name can you get cables longer then 16ft ?

[@Itechstorm]

She is known on the channel as Cat5. She's the co-host and a bit of a diva sometimes.

[@Itechstorm]

+chris I believe you can, and I think they are known as "active" direct attach copper. Length would be around 32 - 50ft I think. They are going to be hard to find potentially and probably expensive. It's a better bet to get a couple of fiber transceivers (10Gb) and pickup a compatible fiber cable on eBay. No more distance limitations when you go that route.

[@TNW1337]

Yes you can there are active DACs But it prob cheaper to just buy 2 SFP+'S and Custom made OM4 Armored fiber cables. Look up on amazon or Fiber store dot com

[@dartfrogdk]

Hello Very good video, i bought a meleanox connectx-2 cards with cable, to use with windows 10, but windows 10 goes into BSOD with error on file mlx4eth63.sys, you hear of that problem

[@digitalmetal7714]

I've ordered a couple of PCIe X4 Mellanox ConnectX-3 EN PCIe 10GbE MCX311A-XCAT CX311A Controller cards have i made a mistake not getting PCIe x8 cards? or is x4 fast enough for single port cards in a PCIe Gen 2 x4 slot? I think by your diagram it should be ok at 16Gb/s Bandwidth? Thanks!

[@cleverusernames]

Awesome man thanks!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[@Itechstorm]

+cleverusernames glad you enjoyed it so much!

[@cleverusernames]

Was wondering if i this was possible then found it was but couldnt find the non arm and a leg costing parts, thanks man got it ordered and waiting!

[@Itechstorm]

+cleverusernames awesome!

[@angeloskouratzinos8796]

I'd like to know what you use your vmware server for

[@Itechstorm]

+Angel TheZombieKitteN I use it for a bunch of different things like staying up to date with latest virtualization goodies, lab environment, testing different tech for my videos, etc. I don't use it for production stuff because I'm constantly making changes. I do have a few friends that use theirs to host services for customers, like email and web.

[@windows7ge357]

I plan to do this with my Windows workstation to my Linux server except I plan to use SFP+ to Fiber-optic modules . I won't get full 10Gbit speed but 1Gbit is bottlenecking my servers RAID. Both my workstation RAID and Server are capable of 250MB-300MB/s so 10Gbit will triple by network speed and high capacity SSD's or a larger HDD RAID will give me true 10Gbit in the future.

[@Itechstorm]

+Jonathan Uhos perfect! That's a great use for 10Gb! That is essentially the original problem that gave me the idea to create this series.

[@windows7ge357]

iTechStorm I got a pair of HP 10GbE Mellanox ConnectX-2 671798-001 for $48, A pair of 10GBASE-SR SFP+ 850nm 300m DOM Transceivers for $32 & a 15M (50 foot) LC UPC to LC UPC 850/1310nm fiber-optic cable for $11, total cost to almost triple my point to point connection speed: $124 provided I figure out how to configure the cards, plus they're refurbished 10Gbit cards so life expectancy will be questionable. This is my first time trying 10Gbit & fiber-optic I was careful to pick things that appeared compatible.

[@Itechstorm]

+Jonathan Uhos I've seen where certain transceivers are not compatible with certain cards and so on. Best thing you can do is check the HCL. Otherwise just cross your fingers and hope for the best! Magic ferry dust helps sometimes too.

[@windows7ge357]

Some quick google searches show the card is being sold with transceivers at the same spec as what I ordered so I have high hopes it will work. I don't know if the transceivers have their own firmware or something that has to be compatible with the card. With any luck they'll work, honestly my biggest concern is driver installation. I'm kind of hoping FreeNAS has a built in driver compatible with the adapter.

[@Itechstorm]

+Jonathan Uhos true. Hardware is only as good as the driver.

[@christopherbartleson8918]

Don't they have PCI-E x4 10GBe network cards?

[@giuseppefalco5106]

@iTechStorm ... mellanox connect-x 2 (MNPA19-XTR) work on windows 10 ?

[@Itechstorm]

+Giuseppe Falco yes it does. I'm using the Connect-X2 myself on a Windows 10 box.

[@tim3172]

@5:47: 5 gigathreads per second? PCI-e is measured in gigatransfers/second (GT/s).

[@dominikdobrowolski9681]

Why mask /24 for PtP connections. Use mask /30 or /31 if between non windows hosts.

[@Itechstorm]

I used a /24 for a very simple reason. The majority of people do not fully understand subnet masks so I used a mask everyone recognizes to simplify for them. I know people that do understand it will slice the subnet however they please.

[@ColderthenIceCold]

Why did you use "hcsp" in the host file?

[@LazyTurtle587]

Hello, purchased installed HP NC523SFP 593717-b21 593742-001 593715-001 10Gb network card driver & added to HP z840 workstation slot 5, but i am getting error always when system boot, Windows 11 pro installed...How can i resolved it...???

[@MrPhendrill]

Thanks for this video. In french "merci pour cette vidéo". Iwill probably follow your advices because 1 Gb shows clearly it's limits. "Blue inch" and subscribe of course. Hope i will be understood (my english is poor)

[@Ryan-zc1tz]

So what would be the purpose of a 10gbit switch if a direct connection is possible?

[@74862]

+iTechStorm With the MELLANOX MNPA19-XTR do you know if I can use a different type of cable to increase the distance between the two systems? I want to install one card in my windows 10 PC and the other in my FreeNAS Storage server. The only problem is that they are about 16 meters away from each other. Will these work with any type of optical cable? Awesome video btw! It's incredibly helpful

[@Itechstorm]

Hey Jonathan, you can use fibre but you need to make sure you get matching transceivers and compatible cable with the transceivers. Others have tried Finisar transceivers and typically OM3 fibre. Could get something like this: www.finisar.com/optical-transceivers/ftlx8573d3btl OM3 fibre will allow for 300m. That should cover your distance just fine.

[@74862]

iTechStorm thanks! I just bought a pair of these from ebay and they should be arriving later this week. I will have to look into getting these since my FreeNAS computer is about 16 meters away from my PC. While you're on here I dont suppose you know where to get the drivers and firmware for these cards(MNPA19-XTR)? Mellanox's website is incredibly confusing when it comes to locating them for their end of life connectx2 hardware. I spent over an hour on their website and had no luck. Current drivers and firmware only seem to support connectx3 and up.

[@74862]

Wow, Thanks a lot! I have been struggling with this for a while so I really appreciate your help. If I'm reading this correctly, ConnectX2 needs WinOF 4.80 and firmware version 2.9.1200 It is a little confusing though because it says firmware version 2.9.1200 and above. So I'm wondering if I can use firmware version higher than this. Eitherway I'm willing to bet the card will work so long as I download this driver and update to 2.9.1200 firmware at the very least. The first link in your post appears to be broken but the second link seems to show which drivers/firmware are compatible with their products.

[@74862]

+iTechStorm Hi TechStorm, I'm sorry to bother you again but I was hoping you might be able to help me out again. I've installed the card and confirmed that is shows up in 'System Information' I then use 'get-netadapterhardwareinfo' and confirmed that it shows up at the correct bandwidth Name Segment Bus Device Function Slot NumaNode PcieLinkSpeed PcieLinkWidth Version ---- ------- --- ------ -------- ---- -------- ------------- ------------- ------- Slot06 x16 0 2 0 0 6 0 5.0 GT/s 8 1.1 This was more challenging than I though because I had to try the card in several PCI-E slots since it kept showing up with a PcieLinkWidth = 1 instead of 8. I installed the driver but my problem is that when I run 'mlxup' I get this: PS C:\Users\jon> mlxup -E- No devices found or specified For some reason the firmware tool is unable to find the card. This happened in every PCI-E slot I inserted the card into. Even the ones that gave me a PcieLinkWidth of 1. I bought 2 of these cards from ebay and I get the same problem with both cards. I dont suppose you have any words of wisdom that could help me out. I tried running mlxup using sever versions. I tried running the latest version V4.5 and then tried the verison you are using in your video V4.3

[@74862]

+iTechStorm After doing some more investigation it appears there is a problem possibly with the driver or the registry? When I opened 'device manager' the device status was: "This device cannot start. (Code 10)" I assumed it was a problem cause by the fact that the card was installed in a PCI-E Slot that wasnt giving me full bandwidth when I installed the driver so I uninstalled the driver and reinstalled it. but now I get a new error: "Windows cannot start this hardware device because its configuration information (in the registry) is incomplete or damaged. (Code 19)" I'm at a complete loss

[@JoseJimenez-mp1iy]

Hi there. Do you think that could be possible to boot diskless PC from 10GB card from a pc base san nvme server? let say my song pc and my; at full speed, fast virus recovery, cheap storage, flexible configuration, (win, ubuntu, hackintosh...) fast system... etc

[@liketruth8053]

OK now we have two computers talking in 10Ge, Comp A has the 4T ssd, Comp B is like a workstation Suppose I had a third C and forth D computer and simply unplugged computer A and plugged in computer D or C when needed. That seams simple enough. Why is a switch so expensive ? Even if I set up a remote B C D switch that was controlled manually but remotely - that would be possible. So again Why is a switch so expensive ? If 10Gbe can run on a Rj45 Cat 7 ... why wouldn't a switch with RJ45 ports work with this 10GbE pipe. ?

[@Styler_144]

Chelsio S310E-CR getting Bluescreens on Windows 10 :( but working fine on WIndows Server 2012 R2

[@vinatron8075]

You have a lot of wasted addresses on your peer to peer network using a triple 255 subnet mask when you could have used a /30 subset mask or 255.255.255.252 network saving wasted addresses because with a net mask of /30 you have 1 network address 2 host addresses and 1 broadcasts addresses I’m working on getting my CCNA routing and switch certification and that is under chapter 8 of introduction to networks and also covered a little in chapter 7 just thought I’d point that out

[@Itechstorm]

Yes, I completely agree with you. I only used a /24 because most people will not understand what you just explained and I wanted to keep things simple to a degree. I figured if someone was familiar enough with networking they would carve up the address space to suite their own needs anyway.

[@mety9325]

@@Itechstorm can you two elaborate on why it is better to not "waste" addresses ? does it have to do with broadcast traffic? you wouldn´t need the addresses anywhere else, so that cannot be the reason?!

[@Itechstorm]

This is only a problem for enterprise configurations where they might have hundreds or thousands of network devices. Home users don’t need to be concerned about this issue.

[@mincreng1]

its good information, can i setting for iscsi?

[@Itechstorm]

+AHMAD RAMADAN yes, this setup is perfect for iSCSI. Most of the 10Gb cards have offload technologies built in, like TSO (TCP Segmentation Offload). In addition, Direct Attach Copper (DAC) has the lowest possible latency which is very important for use with storage.

[@derrickmehaffy]

This is what I'm looking forward to seeing as well. I actually work in the Data Center field so of course I get to use some of our spare hardware for my own "home lab". (Yay free power & cooling!) Currently running: 3x Dell R610's with XenServer 7.0 (4 LACP Bonded 1Gbps 'VM Network' and 1 Management 1 Gbps) 2x Dell R410's with XenServer 7.0 (3 LACP Bonded 1Gbps 'VM Network' and 1 Management 1 Gbps) 1x Dell PowerConnect 6248 for the VM Network 2x Dell 1950's for PFSense (Way overkill I know, had them since High School) Few other pieces of equipment for management/VPN/Etc Only thing I want now is a storage server and the ability to do HA (aka vMotion) through iSCSI, only problem is our lack of 10G stuff, so I wanted to add to my lab as cheap as possible. Looking forward to seeing your build, hopefully its something I can create using some of our spare stuff + a few green bits of paper. Also NICE job on your video quality, you don't have many but your easy to understand, to the point, and great visuals on your videos is refreshing. Certainly earned another sub within the first 30 seconds of your video.

[@Itechstorm]

+Derrick Mehaffy awesome awesome feedback! Looks like you have a great lab in place. Good work!

[@yousafkhar]

Is it possible to setup a wireless 10Gb Home Network? As i have more laptops than desktops at home. @iTechStorm

[@Itechstorm]

+yousafkhar someday maybe. But that day is not today.

[@yousafkhar]

Any solution for Laptop users :(

[@Itechstorm]

+yousafkhar if you have Thunderbolt on your laptop you could get one of these amzn.to/2c6WLkM

[@rocketman221projects]

You can use 802.11ad. It can go up to 7gbps and runs on the 60GHz band. It requires line of sight and has a range of a couple meters.

[@lauwengyew8218]

got it, thanks

[@lauwengyew8218]

Another question, I have edited the Host file. How do i start it to make sure that I am connected to the 10GB instead of the 1 Gb during file transfer. thxs in advance

[@Itechstorm]

+Lau Weng Yew File Explorer \\10.10.10.x

[@chrismoore9997]

Looking for the third video... ?

[@JeremyWorcester1]

Seems it's not up yet.

[@WizardNumberNext]

why reduced bandwidth? 16GT is much bigger then 10Gbps actual data transfer speed on PCIe 1.x is 250MB/s or 2Gb/s, this gives 16Gb/s which is more then 10Gb/s PCIe is full duplex unidirectional bus (this means it have full speed uplink and full speed downlink and both can and do operate at the same time) by the way of w are talking windows, then do not expect 1GB/s at all, as this takes more then just 10GbE adapter even on Linux getting over 931MB/s is a challenge and that already takes in account 9600B MTU to get serious transfers you should get infiniband or RoCE (which is almost like infiniband over EtherNet)

[@mountainspirit3552]

I followed all the step from your video. But two PCs cannot even ping each other.

[@timchambers5242]

Correction: GT/s is Giga-Transfers per second = 10^9 Transfers, nothing to do with threads in this context. Multiply this by the PCIe Width (x1, x2, x4, x8, x16) to calculate Gb/s (Gbps). Divide Gb/s by 8 to calculate maximum GB/s (GBps).

[@WizardNumberNext]

it is NOT 5 Giga Threads but 5 Giga Transfers this unit was chosen, as not all transfers are data, and not all data is bits which you want to transfer - therefore transfers, as this is 5'000'000'000 transfers a second, but not 5'000'000'000 bits a second

[@guillermogalvanb]

This is fucking amazing :D

[@Itechstorm]

+Guillermo Galvan lol. Hell ya!

[@tonycarballo8660]

+iTechStorm Can you post a good link to download iperf and Jperf ?

[@tonycarballo8660]

+Tony Carballo BTW Excellent job on the video!

[@Itechstorm]

+Tony Carballo My bad, I was supposed to include that in the video description. Video description updated. And here is the link too: SoftPerfect RAM Disk | www.softperfect.com/products/ramdisk/

[@ManofCulture]

Nice :3

[@lauwengyew8218]

can you give me the link to download the mellanox connectX2 driver, It is not at their website,. thxs in advance

[@Itechstorm]

+Lau Weng Yew see tabs at bottom of this link: www.mellanox.com/page/products_dyn?product_family=63

[@74862]

Did you ever find the driver? I've looked on the mellanox website for the past hour and cant find the damn thing anywhere

[@redwanhasan1721]

Probably should have asked this in your previous video but does anyone know what kind of difference between Realtek, Intel and killer? I mean is there really any difference at all? Assuming all of them are at gigabit speed, CPU utilization is not a big deal, I'm thinking about throughput and latency.

[@Itechstorm]

+Redwan Hasan there are differences in the 1Gb NICs too. Particularly in the enterprise versions where they offer offload functionality, teaming, jumbo frames, etc...

[@redwanhasan1721]

iTechStorm yes I have seen older one had like 4k jumbo frame and newer ones have 9k I believe but for a given time say now is it worth buying intel gigabit network cards? I'm using realtek gigabit right now and wondering is it worthy to switch to intel?

[@Itechstorm]

+Redwan Hasan If your Realtek cards are working as you expect them to, I don't see any reason to switch.

[@redwanhasan1721]

iTechStorm that's what I told myself and my NAS and seedbox runs perfectly fine and i can play games with these onboard realtek chip too but was curious about the whole advertisements saying Intel is far better than realtek or killer. Anyway thanks a lot!

[@Itechstorm]

+Redwan Hasan glad I could help