Understanding Intel's Quick Assist Technology

[@markm0000]

These videos are so much better than they used to be. It feels so chill and close like you’re a good senior admin that cares to help the new guys. Thanks Wendell ❤

[@pyramid_5]

I'd love to hear more about the actual architecture and perf details, like: is the core-to-qat bandwidth dedicated, or does it consume pcie or memory bandwidth? what's the latency hit on sending a block to qat, and thus what's the break-even block size vs. on-core processing? are there NUMA node considerations with which cores talk to which qats?

[@RetroPianoGuy]

the only core-to-qat traffic is to give qat memory pointers and kick it off. it does not consume pcie bandwidth. it only consumes memory bw because it has to read the encrypted data from memory and then write decrypted data. in other words, you do not "send a block to qat". you send some pointers, and it fetches the encrypted data and writes the unencrypted data and then sends an interrupt to inform the driver that the job is done. the assignment of jobs to qat engines is up to the driver. as Windell said, there are often multiple qat, but he did not mention that there are multiple engines within each qat. the driver will assign to qat/engines based on availability.

[@josephdtarango]

Integrating Intel Quick Assist Technology (QAT) into our systems would significantly boost performance in client VPN, low-level network intrusion detection, and homomorphic computing tasks. For instance, it could bolster encrypted malware, virus, and exploit detection on Linux servers, enabling efficient sinkholing or reflection of malicious traffic. Features like multi-layer encryption, zero-copy processing, content-addressable memory, and in-place detection packet aggregation could address performance and latency hurdles in secure traffic zones.

[@HyenaEmpyema]

In some alternate universe, there's a Wendell talk show host with a live studio audience, and at the end of the show he gives away Xeons and other server gear to everyone.

[@spiralout112]

So this means Microsoft will be licensing accelerators soon?!? Love how the overview was explained, seems like pretty cool tech, have to keep it in mind for the next firewall upgrade!

[@666Tomato666]

already is, and QAT PCIe cards are also available

[@LackofFaithify]

The real question is what hardware actually has qat..... It's like Carmen Sandiego without a catchy theme song or computer game or tv show: who knows? Problem with the atom having it was that it was usually, if not only, on appliance based hardware. The newest netgate appliance (4200) has a new atom, but it is an atom without qat, so not like it's something anyone can just have to tinker with. Well, not anymore. Intel really doesn't learn lessons. Keep everything nice and out of reach.

[@totojejedinecnynick]

Couple years ago Intel started losing to EPYCs in IPC and perf/watt metrics. Also core count. When ARM started to eat their cake too, Intel responded by introducing QAT saying you don't need them pesky CPU cores, just use our QATbandaid instead! (and in few select benchmarks they were suddenly able to beat their competition).

[@timramich]

​@@totojejedinecnynickYour comment tells us you didn't watch the video

[@totojejedinecnynick]

@@timramich how so? Wendell even mentions how it was designed to help weak Atoms in the first place. Nowadays they are using it to get around power limitations their cores have. Offloading to defacto ASICs is cheaper in terms of power. Sure I won't mind using it if it is already in silicon I paid for, hell, I use proto-QAT in way of Intel Quicksync every day for encoding.

[@timramich]

@@totojejedinecnynick You literally said it was because Xeons were losing to Epycs, which is not why QAT was made. It was to compete with dedicated networking silicon. Why do you use QuickSync to encode? For actual production end result files? If so, gross.

[@totojejedinecnynick]

@@timramich If YOU have watched the video then perhaps you have noticed it is about offloading more than just networking (even if initially it was a crutch to help weak atoms). I was pointing out how Intel positioned it and marketed it - as a direct response to being ounumber and outclassed in raw performance. And when was that? Just look at previous videos that Wendell made, it was exactly that - an answer to getting their ass kicked by EPYCs. It was made because they saw it coming and wanted to have some selling point (which arguably they have) It is up to customers to judge that, I am sure some will find it useful if their workload can leverage QAT (if the price is right). But I doubt it is a key selling feature in majority of cases - when it is so much simpler to just throw more powerful cores (and more of them) without having to worry about aditional sw setup. Bigger number better (sadly). and fyi - I use quicksync for security camera encoding. Works like a charm, in production, reliably, while sipping power and not needing dedicated GPU for encoding or bogging down CPU.

[@TeflonBilly426]

When I wanted to upgrade from my Atom C2758 based pfSense router to build a 10GB ethernet home network, I making sure the replacement had QAT as well, I was able to score a great deal on a Supermicro D-2146NT based system. I'm golden for years to come (with horsepower to spare.)

[@stefannilsson2406]

Hey, that's a lot of nice information. Could you make a video explaining DSA? (Data Streaming Accelerator) Seems like all the 4th and 5th gen Xeon scalable CPU's have one, but I can't figure out what it does.

[@jp-ny2pd]

X86 is slowly morphing into the Cell phone HW methodology. Everything is becoming a HW accelerator.

[@tuqe]

Opened genuinely wondering if I could get enhanced remote support for elderly people via some intel low level magic, I guess accelerators are cool

[@lucimon97]

My server cpu is an i7-1065g7 in my old Surface Pro 7 sitting behind my tv. Setting up Ubuntu on it was a bit of a pain, but it does all I need it to for now.

[@magfal]

15:58 I hope AMD leverages some of their Xilinx IP for on chip acceleration too.

[@kingkrrrraaaaaaaaaaaaaaaaa4527]

Well the A.I engine is exactly that. I'm hoping they add some IP from Pensando to their APUs and CPUs. Also adding it to their GPUs.

[@digitalizeddeath]

Hey Wendell what about the PCIe Adapters like the 8950 I was wanting to setup a small overkill nas with 10 gb Quickassist and a video card for transcoding Like a skylake hp z series box probably have to drop to 2.5 gb unfortunately with pcie limitations Was also looking at a dell power edge dual Xeon system and tossing in my Xeon phi Take it to the absolute limit for accelerators and co processors

[@indignasmr7379]

Could you test the powerdraw? I have an old server and it draws 250W no matter what I'm doing, if anything, so I don't know if that's feasible. In my case, I have an obsession with hyper-convergance in the mini homelab space. A very low power computer with accelerators and a core or two that has all the performance for when you need it (say, a heavily modded game server) in Proxmox is the dream. Preferably cheap, so I can litter my friends' houses with them.

[@KyuVulpes]

I am not using any as I am finally getting a homelab set up. I do have 1 intel machine so I should check to see if it has QAT.

[@janus-h6c]

your phone is unplugged

[@michaelrichardson8467]

IT WORKS WITH ZFS???? Amazing

[@789654123654789]

Alright, I'm sold. Where can I get this juice in one of those Atom C5000 chips in an ITX form factor? My New Year's resolution was to give my NAS/server/homelab/everything a swift kick in the ass and get up off the tower of Raspberry Pis with UASP SATA adapters. (Apologies to Jeff Geerling) Double pronto if they're going to IPO.

[@Galileocrafter]

Would QAT do anything for a high speed (10 GbE or more) WAN connection on OPNSense with some firewall rules and Suricata? I'd figure for DPI, QAT could be interesting. And would it do anything for VLAN routing? Edit: Some C5000 and P5000 boards from Supermicro seem to have the QAT variants and SFP28, might be even more interesting than the Minisforum MS-01.

[@vprwave]

For openwrt there is work on openssl offloading with qat, as a pointer (it's in the forums). edit: he gives a pfsense example too, towards minute 16.

[@samiraperi467]

Cherry Trail may have been the first Atom with QAT, but it was far from the first *Atom*.

[@sloqlap]

I want a guide for taking advantage of AES-NI on a homelab or servarrr setup since 13th gen Intel stuff has it.

[@eDoc2020]

AES-NI is an instruction set extension. Most compatible applications will make use of it automatically.

[@LiquidExodious]

Wendell is an angel 💜💜

[@HeneryH]

Just being a regular Joe, I am just trying to get my Chinese minicomputer using a C3000 Atom processor with QAT for ProxMox and OPNsense to take advantage of the C3808 Denverton QAT offloading. Here in Q3 2024 it is hard to know what needs to get done. At the moment I ma trying to rig up some sort of benchmark test setup.

[@daniellundin8543]

Thx!

[@KarlMeyer]

Very cool technology. I wonder how long it'll take until we get a multitude of different ASICs built in to one CPU that are designated for assisting all kinds of different use cases. One for transparent data encryption. Another for facial recognition. Another for visual/audio upscaling ect. Kinda defeats the purpose of a CPU though...

[@GameCyborgCh]

maybe but the closer we get to the end of moore's law we will need to find different ways of processing data faster

[@totojejedinecnynick]

@@GameCyborgCh or you know, abandon x86 behemoth for good. There is a reason why ARM rules the cloud... give it couple of years are RISC-V might join in too

[@claysweetser4106]

I'm more curious about the possibility of FPGA hardware that can be utilized by software to "create" hardware accelerators as needed. You would just buy a number of generic FPGA "cards", and have them automatically be specialized as needed.

[@totojejedinecnynick]

@@claysweetser4106 sadly there is a reason why ASIC designs win over generic FPGAs in mass production. That is why we have specialized networking chips, sata controllers, etc instead of just general purpose FPGAs attached to PCI lanes. Not only you would need a ridiculously massive FPGA but it would be probably slower yet more power hungry compared to something well designed and fine tuned for one specific job. Not even getting into security side of things...

[@zodwraith5745]

If GPUs have taught us anything features are going to become more important than raw grunt as we move forward. The downside to this is much like Nvidia Intel has more pull in what they can get developers to implement, leaving AMD playing catchup when these features get used.

[@axelriet]

I have a Gen2 PCIe 16x QAT 8970 adapter and trying to use it in Windows 11 (not Server) to accelerate compression for an in-memory cache I’m developing. There are firmware files for the card in the software distribution but no tool to flash it, and I’m still tinkering. Apparently, Gen2 adapters only support Deflate and not LZ4 as the Gen3/4 embedded in processor packages appear to support. There seems to be a huge amount of latency (in the order of 1ms) which precludes its use to compress small buffers. I’m not sure yet I’m using it as it’s meant to be used. You have to be careful about the type of memory used for the buffers as this thing operates independently of the CPU and the memory must be pinned. There is much to learn. The jury is still out to decide if it’s useful for my application.

[@amateurwizard]

I've got a Tesla P40 that I don't use because the blower fan addition prodtrudes too much from the side. I've been thinking of taking a blower cooler from a device of the same generation that could mayyyybe cool it... IDK it's been bothering me. I'd like to play with LLMs with it.

[@hamesparde9888]

Do you mean it's part of the un-core? I Guess it could be on a different tile, but it's on the CPU package right?

[@totojejedinecnynick]

except it has PCI-E address... and it presents itself as another /dev/ item... I don't even want to know what a mess it must be with cache coherency and interrupts. Seems fast tho.

[@gcs8]

Saddly, I am running E5-2687W v4 or E5-2673 v4 everywhere, so, no fun QAT for me.

[@johnsmith1966]

What I don't understand about qat is how do I know which CPUs have it? Is it only server CPUs? How do I know which CPUs have it "for free"? Why would I buy an Intel CPU if it either (1) doesn't have the capabilities I need or (2) requires what feels like a ransomware payment to "unlock"? On top of all of that, qat isn't even plug and play. It's sounds simpler to just buy a modern Zen-based CPU and use software-based tooling instead of buying Intel for features that might or might not exist.

[@ABaumstumpf]

"how do I know which CPUs have it?" by doing what Wendell said - look at the specs.

[@exciting-burp]

If z-swap were to support this that would be a massive deal, z-swap is already a no-brainer and this would just push it further.

[@maximum988]

Ive got a qotom box with a C3758 that has QAT.

[@seeibe]

Yeah using dedicated hardware for encryption/decryption is a very good idea that could never be exploited by state actors

[@opshlds]

Engagement challenge accepted. It doesn't appear my Intel(R) Xeon(R) CPU E5-2680 v4 supports QAT, at least not that I was able to find.

[@DavidSchmitt]

Is there any way these qat devices are available at AWS?

[@DaleEarnhardtsSeatbelt]

Im quick assisting this videos watch time.

[@GIANNHSPEIRAIAS]

i think the dpu approach will be the norm in he future given that amd probably will add to it most of xillinx tech (i assume?)

[@totojejedinecnynick]

Xilinx FPGA... if I was in AMD C-suite I would be proposing a subscription based, software defined on-demand accelerator upgrade! Want more this or that? We can program it into your FPGA ! (actually I could see that to be a better choice than maze of QAT variants in each SKU)

[@sfalpha]

DPU can also have do control-plane thing by itself independent of Host CPU because it run it's own operating system. It may can still send remaining data for slow connection until it finished while host crash or rebooting.

[@terrycook2733]

Is there ray tracing on a Avatar movie?

[@BVN-TEXAS]

What type of keyboard is that you are using ?

[@TheMsdos25]

Looks like the Model F Labs beam spring keyboard he reviewed.

[@user-el4su7tl6f]

Where is that level1techs computer from?

[@nasanasax3757]

I got access to : Intel Xeon X65xx Intel Xeon E5-26xx Intel Xeon E5-26xx V2 Intel Xeon E5-26xx V3 Intel Xeon E5-26xx V4 Did some testing using ProxMox VM On most of them.

[@ABaumstumpf]

Why are there now people asking if their specific CPU has QAT? Look at the specs - it will tell you.

[@dgSolidarity]

You might want to tweak your gate/compressor. Volume rapidly all over the place sometimes. Keep on rambling.

[@simon3121]

NIC offload is becoming pretty stupid. In a K8s cluster, there are a lot of layers, and inter container traffic also wants to be protected. So only CPU instructions that can be fed through the different virtualization layers makes sense.

[@markm0000]

Lmao Intel can’t learn their own mistakes even in 2024. Once this tech was made it should have been in every single processor locked behind a license key. Sigh. Now this is an extra level of annoyance.

[@labor4]

bad luck with the pcie adapters. no support anymore, and blacklisted by vfio for sr-iov.

[@hamesparde9888]

Too bad I'm cheap and use a RockPro64 for my NAS. LOL. 😅

[@Ensue85A]

Always install the Haiku OS....lol.

[@Battery3]

Bang🎉

[@porklaser]

If anyone ever put this tech in to a top mount addon module for a Raspberry Pi would be called "QAT in a hat?"

[@deimosian]

CPUs I have? 2650Lv4s. Do they have QAT? LOL

[@daikucoffee5316]

Intel acceleRacers 🔥😂

[@chromerims]

Thank you, Level 1 Techs 👍 My imperfect spidey-sense tells me QAT (Quick Assist Technology) ---being as old as it is and therefore also unoriginal--- will not be among the requisite product differentiators propelling Intel back to the forefront . . . even though having dedicated compute like QAT or AICs (dpus) for compr-decompression and encryp-decryption activities is plainly sensible. Separately, does Intel have any defensible moat around QAT commercially / from IP standpoint? Kindest regards, friends and neighbours.

[@QuintonCraig]

Now do Microsoft's quick assist :D

[@sativagirl1885]

Can QAT help bitcoin mining farms owned by #billgates?

[@CandyGramForMongo_]

I thought they wanted to rent that crap.

[@TheJkilla11]

And no one is talking about this bs.... Next-Generation Secure Computing Base (NGSCB; codenamed Palladium[1] and also known as Trusted Windows'[2]) is

[@woswasdenni1914]

so tldr co processor all over again

[@magneticshrimp7429]

youtube is eatin my comments, cant talk about the c-word anymore I guess :D

[@johnbeer4963]

I've got an accelerator in the car. Sometimes I stomp on it. Sometimes I take My foot off it.

[@ronnyspanneveld8110]

Used a BAD word "Intel" so disliked.

[@timramich]

Yeah, but GPU video encoding, and any hardware-based encoding, is garbage.