40 Gigabit Server Rack Upgrade FAIL

Thanks to Lexar for sponsoring today's video! Don't like our puns? Frankly my dear, I don't give a NAND. Get your RAM on here: amzn.to/3ycLgoJ
In the quest for a 40 gigabit switch in my server rack, sometimes you come across a deal you just can't pass up. For $175, I was able to get my hands on a brand new Brocade ICX 6610 40 gigabit switch upgrade. And it works... for only $3,000 extra....
BUT FIRST.... What am I drinking?
You all know I love Sticky Hands Double IPA from Block 15. Well this is "Brewer's Cut" Sticky Hands with a unique blend of 6 DIFFERENT hop varieties. It did not disappoint. But after this video.... I might need another. 8.1% ABV
Links to items below may be affiliate links for which I may be compensated
Check out today's parts -
Brocade 6610 Switch: ebay.us/ZpI6pp
HP Infiniband Network Cards: ebay.us/vIk9aE
And check out all my network gear in my rack -
- Network -
UniFi UDM-Pro: amzn.to/2RV7gPg
UniFi G3 FLEX Camera: amzn.to/2EsGFpr
UniFi G4 PRO Camera: amzn.to/32W9RhU
UniFi USW-24-POE Gen2: amzn.to/3iZcCEK
Mikrotik CRS317-1G-16S+ on Amazon: amzn.to/3p5k5Gp
Mikrotik Desktop 5-port 10Gb Switch: amzn.to/2TvVezP
Mikrotik CRS328-24P-4S+ 24-Port PoE + 10Gb Switch: amzn.to/3fHpGjf
Ubiquiti UniFi UDM-Pro Router/Firewall: amzn.to/3fA3Spm
10Gb DAC cables: amzn.to/3wLrWeJ
HP NC523SFP Dual 10-Gigabit PCIe adapter: amzn.to/3fCjEjB
Asus XG-C100F 10Gbps PCIe 3.0 Network Card: amzn.to/3g1Ulqj
Grab yourself a Pint Glass or Hoodie at craftcomputing...
Follow me on Twitter @CraftComputing
Support me on Patreon or Floatplane and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
www.floatplane...
Bossa Antigua by Kevin MacLeod
Link: incompetech.fi...
License: filmmusic.io/s...

[@evilone4fun]

I hate to say it but after I saw the title and then the box my instant thought was licensing fail. When I shared the link with my boss I said without watching the video just the thumbnail and the model number "what was the failure" he came back with license or unsupported SFPs. Brocade is notorious for their licensing. I have had many times when C level management or the procurement people have said I can get this for 10-100K cheaper with this other vendor and we have to then explain to them why the quote they got was for an unlicensed or base license unit. Sorry you got burned on this.

[@JasonBoth1]

I thought the same thing

[@mndlessdrwer]

Absolutely. Brocade is atrocious about their licensing. Like, Cisco does licensing for certain feature sets, but I don't think I've encountered switches from Cisco that literally turned off half the ports if you didn't buy the license for them.

[@fohdeesha]

The licenses for this model are freely available with a 2 second Google search in his case, and don't affect the rear qsfp ports

[@SirHackaL0t.]

@@fohdeesha Do you have to pay extra for the licences?

[@fohdeesha]

@@SirHackaL0t. no, hence the word "freely"

[@W1ldTangent]

Linus in the background kills me everytime 😂

[@gimmegaming5345]

Being your already using Mikrotik switches would you not look to replace it with CRS326-24S+2Q+RM. I purchased this switch which has 24 10g and 2 40g ports. If you only need 2 systems at 40g it might fit the bill. I think its about double the price but it would be new and you get a warranty and no license issues.

[@mukit2339]

I saw it coming but it still hurt to see the typical license road block.

[@LesKingBNE]

What's happening with the audio in this video? Buzzing at the start and sounding underwater while going through the rack.

[@ronaldnaeyaert3653]

Opportunity for do another video on who to hack the license from STH :)

[@LrdLnitari]

Everyone's human. Have a good drink.

[@bonemealmc]

There is a very popular STH forums post about Brocade switches and licensing that might be of help to you. It’s called: Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)

[@benw3768]

Replying to try and bump this to the top. You can get yourself a license for much less than $3200.

[@TheTuxprogrammer]

Fohdeesha ftw

[@endersgame485]

Agree'd i have the 24 port version of this currently working in my rack, only one set of 40gb ports work as 40gb though, the other set are for breakout cables only!

[@bgeek23]

Thanks for posting.

[@mrtontovr2608]

@@TheTuxprogrammer Fohdeesha indeed. There are so many ICX switches going for pennies on eBay because they're not licensed. I've been buying them up and licensing them myself for my lab. They're great bits of kit when you don't need to worry about the licenses anymore :)

[@gangmeow]

I'd check with fohdeesha. He helped me configure my brocade icx6610.

[@chetmanley1755]

my icx7250 wouldn't be functioning without fohdeesha

[@gangmeow]

@@chetmanley1755 ya hes amazing. I have full 40 gig and all my Cisco sfps work great. And I have stacking enabled. Runs my entire network. Absolutely amazing.

[@scottbutler5]

It's been fifteen years since I first got bit by hardware licenses and they're still the dumbest thing I've ever encountered in computing. You already own the hardware! It literally costs them zero more dollars to just let you use the hardware you already bought! I will continue shaking my fist at the sky over this for the foreseeable future.

[@jondelaire]

Had that happen to me with a juniper switch, it’s EOL and the web interface won’t open without a license, I am now stuck to CLI instructions.

[@c.m.7037]

It's horrible. You can't even operate most hardware now without license, it's dead paper weight, such a waste of materials.

[@billyhatcher643]

brocade are pieces of shit this hurts small businesses with this type of nonsense

[@billyhatcher643]

@@c.m.7037 yep we own nothing in todays time this is how they milk people dry we cant do this at all unless were a big company otherwise they dont give a damn about us alot of this will become ewaste eventually

[@MaxusR]

Some modern vehicles even won't heat your butt if you haven't bought a license. The heater is there and the money for the hardware was spent but it just won't work.

[@JuliePGUK]

Licensing is getting out of hand, a license for the speed of the connection has to be up there with heated seat subscriptions

[@btraker]

This is normal for Brocade, they've been doing port licensing since the 90s.

[@zyeborm]

@@btraker Most vendors have tbf, they all suck. Microtik are unusual in them not doing it. Good guy microtik

[@metalwolf112002]

You are limited to 55mph... please tap your NFC enabled debt or credit card against the screen if you wish to unlock higher speeds for the duration of this travel. *stop for fuel after driving on interstate for a few hours* You are limited to 55mph... please tap your NFC enabled debt or credit card against the screen if you wish to unlock higher speeds for the duration of this travel.

[@AndyScott]

I swear the Aries LTT Store sight gag is my absolute favorite and I'm going to be sad when that sponsorship is over 😂...well played guys...well played

[@bastian433]

There is a servethehome thread called "Brocade ICX Series (cheap & powerful 10gbE/40gbE switching)". It contains some very useful information

[@pixel_vengeur391]

OH, finally somewhere I can chime in! A gentleman by the name of Fohdeesha has flooded the ServeTheHome forums with their extensive knowledge of Brocade switches, and, by Googling their name, you can easily find a Setup and Licensing guide for Brocade ICX series switches :D

[@TheTuxprogrammer]

Check out Serve the Home's forum post on the ICX series, there is a way to license these now that they are all EOL. A few people I know run this switch with the config found there and it seems to be working alright from what I hear.

[@parsons151185]

I got the PoE model last year. Bought it due to the STH forum page. I've left a comment but it might not be visible due to it containing a web link. I've tweeted it too. I think he should checkout the Celestica DX010 next ;) I'm hoping to use my Dell S5248F as part of my home lab... though I need to register it to sort out the licensing situation with that. And some solar panels as the angry pixies sure are becoming expensive in the UK

[@rdmclark]

I went with a Brocade 6610 poe switch based off what I learned from STH for cheap. I love having 40g for my Truenas and 10g to a few clients. Using Connectx-3 3 40g cards

[@davidchan1733]

Spot on Tux..... I did that exact thing for the Brocade/Ruckus ICX7150 switches. It works fine. Check out ServetheHome's forum post. It covers how to open up the licenses step by step.

[@morosis82]

Just a heads up, I hear there's a group that frequent the STH forums that can obtain the requisite licenses for you, at least for the 10G ports. I've been looking at switching to this switch also to get a flew more 10G ports from my FlexFabric 5800 HP.

[@Tekgnome.]

They released everything now since they are EOL. I bought a 6610-48p for $49 and everything is unlocked now.

[@aaronchamberlain4698]

Also: I’m strongly considering starting an open source project to reprogram Fiber Transceivers. It’s silly that it’s come to that. FS has a box that does it, but it’s tied to an online portal, making it a weird proposition. I want the hardware and software to be fully open source.

[@pbrigham]

Thanks for sharing, another company to add to my list to the ones that I will NEVER buy any kind of hardware.

[@CatalystReaction]

share the list pls

[@attilavidacs24]

@@CatalystReaction Cisco charge a ton for unlocking features too on their newer routers and switches.

[@jamescollins6085]

Has anybody made any custom firmware for these switches?

[@Doesntcompute2k]

@@jamescollins6085 Nope. Really hard because vendors like using various processors (CPU) as the core of a group of switch-chips in each different switch. They of course just recompile the source code, and presto, but you and I?---NOPE (big nope). Sad, I know.

[@jamescollins6085]

@@Doesntcompute2k Ah, I see. Thank you for explaining.

[@playhelper1]

anybody else hear a buzzing in the main video but not the intro? Clears up after 4(ish) minutes

[@blazingmatty123]

Yeah I'm getting that too

[@playhelper1]

comes back in the outro as well. Good content as always though!

[@bobhope1160]

Yep same

[@rett.isawesome]

Unfortunately that's the NSA surveillance 5g waves. Not much one can do.

[@CraftComputing]

No idea what happened there. We didn't notice it in editing, but it's definitely there. That's my bad....

[@nudelwolves616]

I have had the same situation very recently with Cisco. I bought two Cisco Aironet 3602i AP from ebay. I thought buying used enterprise level hardware to basicly reuse was not only a good thing but also cheap as hell and a good challenge for me. Oh how wrong I was. After I got the APs I realised I need new Firmware for them as I don't have a controller. Cisco though only gives this firmware out to company's with a service license. So after back and forth with Cisco I found myself with two perfectly fine APs which I cannot use, because Cisco wouldn't give me the firmware. Not to mention these APs were already out of service but still have their autonomous ios software locked behind a giant paywall. If these APs were still being sold I could understand. But Cisco didn't sell these anymore for several months and it doesn't make sense for any companys to use this. As private Consumers cannot reuse them as well Cisco effectively creates tons of unnecessary ewaste

[@billdberger7407]

You're starting to figure out that these companies couldn't care less about environmental concerns beyond the ESG scam. Reduce, reuse, recycle is real environmentally sound policy/business practice yet how many of the F500 truly practice any of it?

[@I4getTings]

If you try again, be on the lookout for Aruba APs that support "unified" the AP-345 and newer I think. The new ones do require tracking down their proprietary serial cable though.

[@adam.gibson]

But it does make *cents* for them to not allow it to be used... other customers would need to buy new hardware (with a chance it could be their products) instead of making use of old stuff. All we can do is when there are alternatives, make sure to purchase from different companies that don't restrict products like this. Radio Control (RC) hobby is a good example. A lot of RC Radio/Transmitter companies limit software features based on what product you buy or require license upgrades to enable them. FrSky is really getting popular because they don't do that. The hardware they sell have all software features enable for all models of product. Pricing is less than alternatives to for the same feature and hardware.

[@DJ29Joesph]

They sell them online because most sysadmins know how to get the firmware free online.

[@zyeborm]

I have seen sites with cisco bin files kicking around btw

[@Jelly420]

The good ol, "ye got a license for that m8?" pitfall

[@seancondon5572]

loicense*

[@VelcorHF]

I always tell people the hardest part of IT is failing the saving throws. Sometimes you just hit a wall due to licensing, hardware configs, no longer supported firmware and just greedy vendor issues.

[@NickyNiclas]

Wonder if there is a crack for the license like there is for the Supermicro IPMI stuff.

[@marcogenovesi8570]

yes there is, there is a site called "fohdesha" where a dude has written some info to do that

[@Darkk6969]

There shouldn't be a crack. There should be universal license keys for EOL equipment with a disclaimer that the manufacture no longer supports it so use it at your own risk and is for non-business use. Most of us would have been fine with it.

[@wzcx]

Jeff, take a look at Fohdeesha's Brocade guide - it covers license unlocking for this exact switch.

[@fohdeesha]

Congratulations on being the only person in the comments spelling my nick right

[@wzcx]

@@fohdeesha ha, thanks. I have a sneaking suspicion that this was an intentional failure to make it a multi video series, but we shall see.

[@vascovalente3929]

It’s going from ridiculous to criminal that most companies now brick their hardware unless we pay them for the privilege of using the products we buy them .so, I buy a used car from some guy and after that in order to it to work, I need to go to The automaker company and pay them also for it to start moving , and even 10 times more that i payed the original owner .

[@davidmcken]

This is the primary thing killing the used market for kit such as this which in turn drives more sales so don't expect it to change any time soon.

[@petegilbride8230]

@@davidmcken You spelled EXTORTION wrong, bruh....

[@saxophonescott94]

There's a guide on how to do it! Sent it via twitter dm! I've been about to pull the trigger on the switch myself and found the guide recently.

[@joshuawaterhousify]

Just a heads up, there was an odd buzz under the audio while you're in the office. Not something I've heard on previous videos, so I figured I'd mention it in case it was missed. Love the content itself, and definitely good to know just how predatory licenses can be on this kind of thing.

[@deusqain]

I heard it when he was in front of the Cabinet. I presume It's the "noise filter" to remove all the server noise.

[@codec862]

It sounds almost like a broken cable or a ground loop

[@sp00n]

@@deusqain No, the hum was actually gone when he was in front of the server rack. It's at the beginning and at the end of the video.

[@Blackkatana]

Yeah there is a definite hum when he is in his office. There is a static white noise in front of the rack as others have said but it is different than the hum as mentioned.

[@Chris-yc3mm]

That buzz is very annoying

[@niklasxl]

this is why easy return laws need to be a thing, you can always miss something about a product and only notice it once you have it

[@JoakimKanon]

Quick tip with heavy rack stuff: Temporary install something in the unit below, so you won’t have to carry all the weight whilst installing the new gear.

[@kanarie93]

yeah, those old HP G4 DL360's empty cases are great weight holders :+

[@pearcomputers2542]

the septup.exe from patchbox seems like a great tool for that type of scenario. Haven't used it myself though.

[@ewenchan1239]

Sometimes, I would actually test stuff out on a bench first to make sure that the system/unit powers up and is able to do what it is supposed to do in order to look for potentially damage from shipping, especially when ordering from eBay.

[@nopenope9416]

setup.exe is AWESOME. I use it everyday.

[@youtubegaveawaymychannelname]

I love when you show the fails. It truly helps the rest of us avoid the pitfalls that would be expensive to find out for ourselves.

[@neggles]

fodeesha on the STH forums has a full guide for unlocking every single feature on these...

[@HueMongus101]

Mikrotik CRS354-48P-4S+2Q+RM could replace all your switches. A great addition would be either the MikroTik CRS354 or CRS326 to add QSFP+ to your SFP+ setup

[@Darkk6969]

I love MikroTik gear. I have several at home and been flawless for me. I too eventually want to upgrade to 40 gig but for now 10 gig networking is fine. I agree the hardware licensing sucks. Especially in the used market when the gear could have had a second life only to be e-waste'd. It's really a waste.

[@SAerror1]

I have two of these and they are awesome! (Well, I have the non-PoeE version) …moral of the story is to keep buying MikroTik :)

[@Ufphen]

Love Microtik, it's basically enterprise gear without the licensing

[@prla5400]

I had a microtik airfiber thing in a rural town

[@MM-vl8ic]

check serve the home forum There's a guy there that has done a bunch of work with the Brocades He's stuff got me straighten out... I'm using a Mellanox sx6036G (Ethernet) dual linked ConnectX-3 SMB/RDMA between my PCs and the icx 6610 SFP+ 40gb into the sx6036G and a 6450 10gb fiber(pair) into the mellanox, (outside cameras starlink etc so if anything takes a lightening hit it only "deletes" the 6450) I'm trying to learn networking......

[@DavesGarage]

The opening line is awesome!

[@jongmassey]

Have a read of the STH forum epic thread on these, there's solutions for your DAC and licensing issues. Probably should have had a good read before starting!

[@nathankooistra7788]

Hopefully he sees this. That thread helped me a ton

[@MrMaster2k]

As soon as I heard the words Brocade - I understood immediately why this video was labelled as a fail. Brocade even licenses each port on a Fibre switch individually and you have to buy THEIR SFP's for each port.

[@thomasbonse]

Considering you already use Mikrotik hardware, you might consider their CRS354-48G-4S+2Q+RM switch. Alternatively, I personally like the EX4300 series switches from Juniper.

[@leesturgill5798]

Fohdeesha ..from STH forums

[@PinkyTech]

Welcome to enterprise networking. Cisco is no better. We bought 30k enterprise switches then had to spend another 10k to enable routing protocols. The industry is just going that way unfortunately.

[@junker15]

WHAT I'd expect this from NX-OS, but the code I have doesn't require licenses for routing (that I know of; by the time I get them, a license file is pre-installed and all teh routing Just Works) (I'm old-school IOS spoiled so have never had to deal with additional licenses just to enable something like BGP or OSPF (or EIGRP, which I expect on a Cisco router for FREE; they made it))

[@PinkyTech]

@@junker15 It was NX-OS. You can do some routing, but the amounts you can do are limited and no supported routing protocols without having a license...or none you would want to use in 2022.

[@zuexs690]

40Gb?!? Cmon Jeff, just jump to PCIe fabric switches already ;)

[@marcogenovesi8570]

40gb is for losers

[@MactelecomNetworks]

“Get in losers” best opener ever 😂

[@danielbarnes3406]

I think the clip at 13:50 would have made a great opener :)

[@jerrycrumety1571]

Just bought one of these now, I've done my research on it before purchase and found that there's work arounds to the issue you had, mine was pretty cheap so I'll see soon if changing the file and ID codes work. Thanks for the video

[@ShinyTechThings]

I do that with brackets all the time or you can unbend the top and *"retrofit"* the card. 🤓😎

[@Arachnoid_of_the_underverse]

Yep, even synology charges extra licenses on its home NAS systems for the surveilance station app above the "free" two cameras.

[@HueMongus101]

Unlike brocade, the Synology licenses are transferable

[@leithacullen]

It is just me or can you hear a high pitched hum in the video?

[@charlesp7535]

On the 10 gig ICX switches, you can just type a cli command to 'install' the 10 gig license and it enables the ports. It might also work for the 40 gig.

[@Oyashiro_Chama]

the 40gb don't require a license, but it has to be in switch mode not router, requiring the sw firmware.

[@Oyashiro_Chama]

To add only 2 ports are 40gb the other 2 are 10gb x 4 breakout type.

[@fohdeesha]

@@Oyashiro_Chama the 40gb ports work just fine with the full router firmware

[@Oyashiro_Chama]

@@fohdeesha Not sure if the 6610 is full 40gbps routing capable though, should work great for 40gbps switching though, I use one as my main switch and it works great and I have dozens are spare parts due to my job decommisioning all our Brocade equipment.

[@fohdeesha]

@@Oyashiro_Chama the 40gbE ports will route at full line speed regardless of features enabled like the rest of the ports

[@olpqay]

10:40 EE here. Just wanna point out: 9600 is your baud rate. Meaning symbols/second. You are encoding 8 databits into one symbol meaning your effective bitrate is at 9600*8 = 76.8 kBit/s

[@LucienneGainsborough]

I believe each symbol only represents one bit in RS232. And given 8b/10b encoding it only yields 960Byte/s or 7.68kbit/s

[@olpqay]

@@LucienneGainsborough You are right... It's a simple NRZ code and one might say OOK/2-ASK. So yes, each symbol represents one bit. 8 info bits, 10 code bits (consisting of start bit, data bits and one stop bit). Your calculation checks out!

[@Jetta4TDIR]

If you like the Brocade licensing issues, wait till you're met with their firmware upgrade issues... They're great fun too 😉

[@GYTCommnts]

This is infuriating... The trend of having to "license" hardware like sort of asking for "permission" to use it is just BS. It's supposed that the trend was intended for manufacturing products that last longer or because of an OPTIONAL service of support or something like that. But, products last the same or less AND you are more and more obligated to pay a fee for use the thing. Jeez, wonderful future ahead if we don't fight this.

[@wiziek]

It isn't, this is enterprise gear, with additional support. You can't slice silicon/cpu/asic, this is done to offer one variant of switch and client can decide if he need 40g/10g/ whatever combination. Did you see how windows server is licensed for virtual machines and 2/4 cpu motherboard? Some thing...

[@GYTCommnts]

@@wiziek I understand, and agree with you in some aspects. I was talking particularly about the trend in general and don't like it. But that's me.

[@brycedavey1252]

Well, welcome to the world of QSFP :D I had a similar issue with mellanox IS5025

[@djdrastic1]

Time to get into Mikrotik my man. Also assume anything from Brocade is going to be licensing hell.Some older gear of theirs we had per port sfp+ licensing on their 48 port sfp+ switches.

[@dracotrapnet]

I knew the fail after reading the title and hearing Brocade. "I bet it's licensing!" I have yet to light a brocade up before. We had one in house for a review that was sent to trial along with a 3par 3200 SAN. The fun part was the Brocade was sent all 10g copper and we had 10g MMF on the 3par. We never got around to even lighting up the Brocade. We just shipped it back. We just got a pair of sfp+ adapter plates for our Summit Extremes and ran with 10g MMF sfp+.

[@franklincerpico7702]

I would be looking for any sort of hack to unlock that ability, screw those guys. Paying for support is one thing but paying to unlock something you already have is bullshit.

[@TevisC]

Time to go Office Space on that switch.

[@paulbrooks4395]

Had a project at work with dual 20 Gbit to Cisco Fabric Interconnect. The best part of the build was finishing the DRS cluster in VMware with dual 20 Gbit VMotion ports. VMotions were insanely fast, as VMware uses both paths without any need for LAGs. It opportunistically uses paths similar to SMB multi channel. The funny part was how the customer’s SANs couldn’t even come close to saturating a single 10 Gbit iSCSI path. Moral of the story is that spinning rust isn’t useful above 10 Gbit unless you have dozens of hard drives.

[@0tzeF]

Have a look at the Mikrotik CRS354-48G-4S+2Q+RM or CRS354-48P-4S+2Q+RM

[@MysticFists]

Whatever mic was used for the sitting at table portions of this video has some feedback noise/hum that's quite grating to my ears :(

[@rush2489]

After firmware release 8.0.60 you can install a license using command: license install perpetual xxxxxxxxxxxxx But it is honor system that you have purchased said license, and if you ever need support and or are found in non-compliance.... a world of hurt you will find.

[@MrHellbear1]

Could you revisit this project with the Fohdeesha guides for licensing?

[@aboron_]

I went "40G" on the backbone of my home lab a while ago using a Mellanox sx6036 from ebay for about $200. It has 36x 40/56G ports and can work for ethernet and or infiniband - i set mine to straight up ethernet only 56G mode. 56G only works with compatible Mellanox cables/cards, but these are also pretty available on ebay (56G Compatible ConnectX3's can be had for about $50-75). It also supports a breakout mode for some ports that splits a single 40G qsfp into 4x 10G sfp+ connections - this is good for systems that only have 10G. There is a ton of great support for this model on the STH forums.

[@joshhartwig1266]

Those sx6036's went through the roof price wise when the chip shortage hit. Not sure if they have came back down to a reasonable level. IIRC that switch is nice as the power consumption is not insane like some of the others.

[@BobHannent]

I have the SX6012, which wasn't working, I think I need to use the FAE command to enable off-brand DACs

[@SirHackaL0t.]

Wow, an excellent price.

[@ajhieb]

Yeah, I snagged one of those a month or so ago on ebay for $135 or somewhere thereabouts. It didn't include the ethernet license and after failing to find any "free" solutions I pulled the trigger on another ebay auction for the license ($50) so I still got a working switch for less than $200. So far it's working like a champ as my homelab backbone network.

[@petegilbride8230]

@@ajhieb There are TONS of links to help unlock all those features. Shame you spent the money already. :(

[@halldorrunarhafliason2771]

Enterprise licenses are absolutely the worst thing - as someone that has been in the industry for 16 years it has only gotten worse somehow :(

[@smithincanton]

Thank you for this warning tail of Brocade. I'll never touch their hardware.

[@Prophes0r]

I made a different mistake with my 40/56Gbit setup. In my case, I could get the licenses from Mellanox for free as a home user/student. The difference with my setup is that it is IMPOSSIBLY loud. Neighbors coming over to see what the noise is loud. 'I can hear it while wearing headphones, it's on another floor of the house, and my doors are closed' loud. Unusably loud. I was in contact with someone at Mellanox to see if I could bypass the fan-protections so I could watercool it. But that project is stagnating. Oh! and it also draws like...120W while IDLING?!??!! 300ish under full load? I shelved the switch and just direct-attached my workstation to the VM Host. The plus side is it motivated me to build a short rack, rackmount my workstation, and move my VM host next to my desk. Why? Because DAC cables are the "cheap" option. But they aren't THAT cheap. A single 0.5m QSFP+ DAC cost me more than TWO used ConnectX3 cards...

[@danmerillat]

If he ever gets the 6610 running it's got some pretty beefy noisemakers in it too. There's a whole modding community just to quiet the fans on it.

[@JimtheITguy]

When you said new in box the first thing that came to mind was Licensing, its why UI and Mikrotik are making great inroads for this sort of environment

[@XenArena]

You can flash it. There is even a yt on it. And you will have what you desire. :)

[@yuuki-yuuki]

"Doing research is very important" Sure, when there's a post about how to get them licensed right there within just one click of google search button, you know how important it is.

[@fohdeesha]

lmao gottem

[@OzSigns]

Thanks for the video mate. No wonder they are so cheap used. They basically just a 1gbe switch.

[@BlackICE1973]

just get Mikrotik Switch with 40Gbit QSFP Slots.

[@xtantaudio1]

you can upgrade/unlock the 40G ports using the instructions on the STH Forums for FREE. NONE of the DAC Cables will work with the Brocade Switch. It takes an MPO Fiber cable. I already dealt with this. send me a message, I can help !

[@mrmotofy]

Or just post the info publicly and link it or share it with everyone

[@OldMan_PJ]

Am I correct, licenses for just 2 ports is $3,500?

[@CraftComputing]

That is correct for the two 40Gb ports.

[@808jpm]

@@CraftComputing if you want the ICX6610-48P let me know...I decided to go Juniper after hearing how loud it was and not getting links up. I knew about the STH post mentioned above but I thought I tried the first couple ports which shouldn't need a license. There's another comment here that mentions this, I'm pretty sure two of the 40gig ports on the rear are only used as channelized ports, so max config is 2x 40gig, 16x 10gig, 48x 1gig.

[@808jpm]

@@CraftComputing You should have tried your cables back to back with the network cards too. If you have the VPI (Infiniband + Ethernet) version instead of EN (Ethernet only) you might find the interfaces running in Infiniband mode. I believe you can setup the card to default to Ethernet, or you may need to configure your driver...that's easy on Windows, not sure about the specifics on FreeBSD or Linux. For an added twist, if connected to a switch when first powered up/initialized the VPI card should come up as Ethernet. Later when you want to use the second port or reboot a server while the switch is rebooting or off you will wonder why your Ethernet interfaces are gone.

[@CoryMT]

This is unfortunate, but I appreciate you making and releasing the video. I wasn't aware of this even being a thing, and I'm grateful for the warning.

[@KidKool31yo]

For $175, I would make a video of myself throwing it straight into the trash and move on to a different solution

[@mrmotofy]

And be sure to plaster Brocade all over and mention it in multiple videos to remind and inform people to avoid them.

[@Warkwarkwark]

Arista switches are hard to get firmware updates for, but licensing isn't a problem, and it's easy to find a key to unlock optics from other vendors. 7050QX would scratch that 40gig itch.

[@wombatdk]

Just having gone through licensing nightmares myself (and just resorting to hacking the f'n thing), I can sympathize. It seems the entire industry is stuck on "let's bilk customers with nightmarishly complex and expensive licensing schemes". I think Cisco started it in the 90s, the rest just saw the scam worked and followed.

[@DmnkRocks]

wait until you discover dell switch licensing.... especially if they have a bug in the firmware that expires your unlimited license and send out notifications two days beforehand - when you're on vacation

[@jgonzo1995]

I say get skip 40 gig and get the Mikrotik CRS504-4XQ-IN - you can run 100 gig to the storage and editing rig and then break out the other 2 ports to 4 x 25 gig and get the rest of your servers up to 25. Win-win.

[@funcrusher213]

Juniper EX4300 is a great 40g option.

[@the-electric-eye]

And here I was happy I just went to 10gbe lol

[@voodooutt]

Dude. Overkill with the box knife lol

[@PCTechHustle]

Licenses... always the licenses... Network engineer as my day job here... I know your pain... you pay to play!

[@tryp1cac340]

Jeff have a look at the Mikrotik CRS326-24S+2Q+RM it has 24 10Gbps SFP+ and 2 40 Gbps QSFP+

[@Prophes0r]

So. a 10Gbit switch? That's not 40Gbit. And no, having 1-2 uplink ports at a different speed doesn't count. At that point you are better off just using point-to-point links.

[@tryp1cac340]

@@Prophes0r its a 10GB switch with 2 40GB ports, considering the switch he brought was a gig switch with 10gb and 40gb i dont see your point, also he wants 40Gbps to his NAS and backup server which is why i suggested this switch as it has enough 40Gbps ports for this and everything else can be 10Gbps

[@tryp1cac340]

@@Prophes0r and yes he can ptp his backup and file server but then he still has to have a 10Gbps connection for regular LAN connection this way he still would only need one switch for all his 10Gbps and 40Gbps needs and one switch for his 1Gbps client needs. also means if many 10Gbps clients are hitting the file server at the same time the connection to that switch is bigger without having to mess around with link aggragation which can be a pain to set up corretly

[@jdl3408]

As soon as you said brocade and eBay I thought “no way all those ports are licensed” tough break.

[@fohdeesha]

Licenses are freely available with a 2 second Google search

[@RootSwitch]

If you take another stab at 40Gb, I'd recommend the FS S5860-20SQ. Its an FS-rebranded Broadcom switch with 20x 10Gb, 4x 25Gb, and 2x 40Gb. And 2 or more can be stacked which is nice so you can do MLAG/VPC, for that completely necessary redundancy in a home lab. I havent had any transceiver compatibility issues with a wide variety of models. Or theres that new Mikrotik CRS504-4XQ-IN that has 4x 100Gb QSFP that supports breakout cables. Kind of wish I had waited for that when I moved my lab to 25/40Gb because its much cheaper.

[@jpdube1]

I owned this S5860 and it's an amazing switch for the price. And best of all you don't need any licences to use all the speed available

[@HowToITLLC]

Anyone else hear a humming of fans?

[@skaal_]

Brocade … It’s a Trap! Nice PSA for anyone who has yet to run into some of the administrative headaches with various networking manufacturers and their interactions.

[@CraftComputing]

I'm mad at myself because I should have known better.

[@KyleYoung101]

@@CraftComputing Definitely Check out the Serve the Home thread on these. There is methods to enable all ports on these EOL switches. There is a reason these are so valued. I run a 7150 and 7250 at home, and ran a 6610-24 prior. They are great switches, and its not hard to get all ports functional.

[@joaopedroalbernaz]

There's a horrible hum on the audio

[@ejbully]

MELLANOX MELLANOX MELLANOX MELLANOX

[@computersales]

Ugh brocade switches are the worst. Only switch brand I have personally ran into like that. I recently had my hands on a Dell Z9100-ON 32 port 100Gb switch. I didn't get much of a chance to play with it because I screwed up the boot order. It was basically a backplane controlled by an atom based PC. I believe the switch OS was somewhat open source with different options available. High end switches are beyond my skill set beyond reselling them. You might look at the Dell S6100-ON though. It would potentially give you access to 10Gb, 40Gb, and 100Gb.

[@RepaireroftheBreach]

Did you figure out how to get 40G working? I’m in need of a NAS for editing and VFX work. Have lots of chunky multilayer EXR files and need lots of speed and space. Trying to find a NAS that will accommodate and it’s not easy. If you found a solution please let me know. Thanks!

[@Paulnt04]

Look at the ServeTheHome forums about the ICX 6XXX series licensing. Also, the 2/2-5 and 2/2/6-9 ports are 4x 10GbE links each, so they support breakout cables, but I’m not sure if they support 40G but as long as the 40G card on the other side can support 4x 10G links and can be bonded, then you should be fine. I run the ICX 6610-48 at home as my core switch with a MikroTik router

[@oliverlind6596]

im sorry to hear about that, but thank you soo much for pointing this out, i had a bid in for a Brocade unit, glad ive been outbid, will definitely never look for another one again

[@McCuneWindandSolar]

My 10 gig is all used equipment so is my storage server, and my video surveillance server. I use Blue Iris and I want to get even more out of it and been looking to get a video card just for rendering because the Intel processors with Quick sink, is only so good when everything is dumbed down. were I want a very power full system that can handle 32 4k or better cameras. now worried about power. I just want better AI and higher quality video. You have any recommendations one Video cards. been looking at the Telsa's as I can use a different video card for out put since will be viewed over the network. Been thinking about getting another big server just for video surveillance.

[@TheGrowOp]

I got an R710 for free from a friend and man it's been a hell of a ride. Knew nothing about servers a month ago and now know just a tiny bit more than that. Really neat stuff to get into

[@coreygriffith2467]

The icx 6610 is a mint bit of kit. But yes there is a license component. Hopefully someone before me has sent you to the serve the home forum on how to get the 8x10g ports going. As well as for some good info on what ports are real 40g and which are breakout only. If not I suggest hurrying on over to their forums as there is a nice lengthy thread about the icx switches

[@pikeyMcBarkin]

Ah A teaching moment for all you home-labbers. Great video.

[@jeffnew1213]

Late last year I purchased the new Ubiquiti Pro Aggregate switch to replace my HEAVY, LOUD, POWER HUNGRY enterprise Arista Networks 10G switch. The Ubiquiti, for $850 plus the cost of GBICs, is 28 SFP+ ports and four SPF28 ports capable of 25GbE. Using 10Gtek modules for 1Gb, 10Gb and 25Gb fibre, I rewired my network with slim Cat 6A cables from Monoprice, popped four Mellanox 25G cards in two big Synology NASes and two Dell PowerEdge servers and... never looked back. Silent, cool running, decently fast, and not ridiculously expensive.

[@93vxhybridhatchback]

Because if this video, STH and fo who should not be named, i ordered one of these switches today(my aruba s2500 died last night). 3 times as many 10g ports and two because why not ports for about the same price as my s2500 was?? Yes please.

[@lezion]

Hey Man! Return that metal pile of junk and consider crs504_4xq_in or Celestica Seastone DX010. They are quite cheap for 100gbe and you can still use them for 40gbe as they are backwards compatible. If you want cheap qsfp28 cards, look for HP Ethernet 4x25Gb Single-Port 620QSFP28 or QL45611HLCU as they allow for 4x25 operation.

[@servalous]

It's not like its known by the -> experienced IT Sysadmin FULL PORT LICENSED

[@thegreyfuzz]

HP expected me to pay $1600 for a service contract to cover one failed 1yr old 48port 1G Aruba POE+ switch, THAT IS STILL UNDER WARRANTY (half of the POE ports just stopped being POE)! Warranty is useless with a service contract (that is basically the cost of a switch). Threw it in the junk pile and went back to Cisco, after 25 years of using nothing but Cisco in my datacenters and my homelab (WAY before it was cool) without a single problem, honoring warranties/repair (even for my labs), why I even gave HP a shot I will never know.