NVIDIA’s AI: Virtual Worlds, Now 10,000x Faster!

[@JoshSmithEE]

I work in factory automation and this is absolutely a hard problem. I look forward to this solution making it into standard toolsets.

[@SIRICKO]

You'er not gonna be working at all my man onces this get here.

[@markmuller7962]

But then I can't spend 8 hours a day tightening bolts 😞

[@0AThijs]

@@SIRICKO I'm pretty sure that op doesn't just tighten bolts all day...

[@jtjames79]

I'm a miniature war game player. I hate painting. I can follow instructions on how to make a robot arm. What a time to be alive!

[@monad_tcp]

I want something more realistic than Factorio or Satisfactory, this seems like a cool game to make. Those H100 are going to be very cheap after the second AI winter starts very soon when the LLM bubble pops. Time for cool and useful things to be done with them. What a time to be alive.

[@pandoraeeris7860]

Finally getting down to the real nuts and bolts of AI research!

[@peanutnutter1]

You didn't screw this analogy, you nailed it!!

[@CamAlert2]

What a tool.

[@MilkGlue-xg5vj]

​@@peanutnutter1This is but a bolt in building autonomous AI agents. The future is gonna be nuts!

[@Speedrunner.007]

@@MilkGlue-xg5vj the future is gonna be nuts and bolts!

[@mycul_]

You're telling me this thing can plug in a USB correctly on the first try? Uh huh ok pal, lets easy down with all of the sci-fi

[@businessmanager7670]

this is evidence based. stop your ignorant dreams

[@geniferteal4178]

Yes, actually someone taught me to look for the EDIT: "seam" on the connector and face that down. It usually means the logo faces up. Regardless once i figured that out, I got them right more often than not

[@Ou8y2k2]

I can do that too... with usb-c

[@SantinoDeluxe]

i heard usb-a are 4 dimensional which is why you gotta flip em over twice

[@geniferteal4178]

@@Ou8y2k2 edit so it makes more sense. Sorry.

[@TheAkdzyn]

Dr Zsolnai-Fahér, I don't know whether I've mentioned this before or if it's relevant to your interests in anyway but I am Malawian (southern Africa). I hope you know the impact of your work. These quick updates with clear explanations of topics offer first timers a chance to learn something new without being overwhelmed. thank you for creating this channel and uploading so frequently. Personally, when I started watching your content I never thought I'd be watching entire Nvidia tech exhibitions and product launches but now I find my self sitting through hours of the launches and enjoy it thoroughly. Thank you and "What a time to be alive!"

[@Patralgan]

What a Time to be alive!

[@TwoMinutePapers]

That is absolutely amazing and really made my day. Thank you so much!

[@TheAkdzyn]

@@TwoMinutePapers You're welcome sir! 😄

[@RedOneM]

1994: "Playstation released a week ago it is so dope! Can you imagine what we'll have in 2024?" 2024: *SIMULATING* *A* *GAZILLION* *NUTS* *IN* *PARALLEL*

[@spazneria]

We really went and grew up

[@iceseic]

we managed to surpass their expectation

[@FusionC6]

so many NUTS

[@AA-iq6ev]

@@FusionC6 Thts what she said

[@gamefun2525]

@@FusionC6 deez nuts

[@MrYerak5]

No WD-40 simulation?

[@Sekir80]

What does Water Displacement recipe 40 has to do with bolts? Nah, I get, don't worry.

[@wobbers99]

Now that would actually be a massive paper. Love it!

[@undefined6512]

I tell ya hwat

[@palleppalsson]

10 papers along the line we get the "soaked in saltwater for years bolt that you have to heat with the welder and whack with a hammer -simulation"

[@MikevomMars]

THIS is the level of physics I want to use in VR games and apps! 🤩

[@michaelleue7594]

...Why in those? Like, I understand AI training for robots in the real world or for simulating designs for factories, but under what circumstances would you want to hyperaccurately screw on a bolt in a video game?

[@marcoshalberstadt7646]

@@michaelleue7594 Literally any game that requires physics simulations, special in VR. Of course you have to calculate the trade off between realism, performance and fun, but there is always some niche communities that want to just maximize realism at all costs.

[@SirNightmareFuel]

At the moment games would typically solve this situation with a "that's good enough"-style trigger box, and scrubbing through a pre-baked animation. The excitement here is we could skip this jarring and unimmersive experience, with a much more uniform and intuitive solution. Also it's getting closer to simulating a more granular and 'realistic' world, which is inherently more immersive. ​@@michaelleue7594

[@HiddenExp]

@@michaelleue7594 massive melee combat sims!

[@Luizfernando-dm2rf]

@@marcoshalberstadt7646 So basically it all boils to down to porn, as always

[@JorgetePanete]

This is nuts!

[@user-cg7gd5pw5b]

You think you're funny, huh? Well, guess what! You are, I laughed.

[@Mertiven]

and bolts!

[@bilbobaggins5938]

It is fairly screwy.

[@Simplicity4711]

They don't screw up a lot. Mostly down.

[@jlrutube1312]

@@Simplicity4711 I'm going to bolt over to my computer right now and try this.

[@Kuchiriel]

This will become a metric unit someday. Nuts and bolts per second.

[@iusearch]

Don't think so, easy to cheat on this particular case

[@SkultétyBendegúz]

Americans will literaly use anything but the metric system smh

[@john_blues]

You should see his video where they had a cats per second measure.

[@Ryan_Thompson]

And rural areas will still be getting 10 Nbps while rich urban centers will be seeing 1000 Nbps. "I just crash my car a block from home every time, because I can assemble a new one faster than I can find a parking spot." What a time to be alive!

[@cherubin7th]

We already have a metric unit for such things: Hertz.

[@3d-illusions]

So the breakthrough was using sdf collision detection?

[@Felipehez]

i think that houdini has some options to use sdf for collisions, but I don't remember it being particularly fast

[@voinea12]

Game engine developers: 👁️👄👁️

[@locinolacolino1302]

Game Engine developers make use of SDF based physics all the time, it's just this level of fidelity is rarely required in games so those clock cycles are used for other things.

[@monad_tcp]

@@locinolacolino1302 "is rarely required". As someone who plays Space Engineers I disagree, clang is my enemy, I wish I could stack 8 GPUs in parallel and increase the simulation to 800hz. I always thought it was janky because the physics runs at 60fps, but that's too slow. Increasing the mesh detail decrease the performance exponentially, not linearly, that's why SDF based physics don't do that. But this paper proposes a solution that doesn't need more detail, so you can actually increase the simulation to 800hz and thus decrease janky. I think I'm going to make a factory game that's actually physics based instead of just being a logistic simulator. Imagine real items in the conveyor belts, that's going to be fun.

[@voinea12]

@@monad_tcp in Godot, a simple chair falling lags the whole game, though maybe I'm doing something wrong idk

[@dreadthedrums]

Do game engines use SDF for collision? I have seen things like Niagara in UE5 use it, but generally collision is done with a collision mesh, or using original geometry (very expensive)

[@theneonbop]

@@voinea12 you can get the godot jolt physics addon which should work much better for most things

[@pranjal9830]

What a time to be alive.

[@kronux3831]

This right here is a prime indicator of what I think is going to be humanity’s biggest innovation over the next decade: an increase in our ability to simulate complex things like this in a super efficient way. Can’t wait to see how this tech scales in the next few years

[@mumiemonstret]

The best simulation has to be the simulation of football simulants at 0:47 - so accurate!

[@SteelBlueVision]

Accurate if someone dropped some crystal m-e-t-h in their gatorade, maybe...

[@fgmenth]

Factorio 2 is going to be AWESOME

[@ShikariHybryd]

"You can even plug in that USB-A adapter in a virtual world accurately; a feat almost impossible to mere mortals." 💀I laughed so hard

[@RC-1290]

I had to simulate bolt interactions for VR. It makes much more sense to approximate the bolt. But if you use this example as a way to describe other complex collisions, then sure, valuable.

[@vadimfedenko]

as a guy who works on this stuff, i can say that complex geometry simulation of the bolt threads is not necessary. as soon as the bolt enters the nut, it is enough to turn them into a single physical object, where there are only 2 interconnected degrees of freedom between the nut and the bolt (rotation with movement along the axis of the bolt). physical stimulation is needed only at the very edge of the bolt and nut to determine whether the bolt has entered in the right way

[@akissot1402]

And how i am supposed to study the series of events that causing a jamming ?

[@MrTehkaiser]

It's impressively embarrassing that you don't understand why the physical sim is an absolute necessity in this context. Especially seeing as you work in the field, that fact makes it significantly more embarrassing.

[@6TheBACH]

Wait, I don't understand. Are they still using triangles with the bolt or they are using other stuff?

[@lucas_sapa2985]

Instead of triangles they are using SDF (signed distance fields) a form of mathematical calculation of the volume of the object. Its what Unreal Engine 5 uses for good and fast collition detection and other effects

[@badumtsy]

So, soooo nice to see new research around SDFs!!! It almost makes me believe that my recently finished thesis involving SDFs wasn't for naught xd

[@SteelBlueVision]

2:21 - Bolt SDF (SDF stands for SIgned Distance Field, btw) visualized as point cloud, Bolt SDF visualized as mesh. Both of these are a mistake for a mechanical part being made to a mesh approximation of a solid, both the source and the cause of all of the collision detection problems that follow. The moment you triangulate and _then_ do collision detection on the triangulated meshes, you are effectively doing collision detection wrong. It doesn't matter whether the resultant mesh is made of quads or nGons or how "perfect" it is topologically speaking, no matter the mesh, it's all triangles in the end and this plethora of triangles is what will fed into the collision detection algorithm(s). Compare and contrast this method with how any CAD application or mechanical application worth its salt, represents mechanical solids like literally nuts and bolts. They are represented as non-destructive solids and surfaces backed by exact mathematical formulas that describe them with a good heap tablespoon of Constructive Solid Geometry (i.e., bool operations like union, intersection, and difference, all performed mathematically and perfectly - try that with a mesh). The moment you map the solid to a triangle based mesh, even for something relatively simple in the grand scheme of things like a bolt, you get a proverbial crap-ton of polygons to consider for collision detection or you cheat and come up with a bolt that looks like it came out of some sort of FPS shooter with hexagons for curves. This is the case, at least, if the object of interest has curves and then requires a considerable amount of mesh geometry to approximate it. It can never be exactly mapped to a mesh, only approximated with greater and greater accuracy at the cost of an explosion in polygons (read, triangles) that are used to form those mesh based approximations. Then you have to deal with that vast sea of triangles when you do your collision detection, making it a very expensive process, even if you take shortcuts and break up triangle groups into volumes, along with other similar techniques used to cut down on the test of "which triangles are colliding, or worse yet, which surfaces are intersecting that cannot intersect, because they are rigid bodies.

[@DustinRodriguez1_0]

When you said "everything is simulated down to the last millimeter"... you were just using an expression, right? It has to have higher fidelity than millimeter-level to be useful in any real life context... especially dealing with things like machined parts, 10,000ths of an inch are the common unit. A millimeter is gigantic in comparison.

[@AviramYageena]

What a time to be alive! Thanks for the content!

[@someonetwofake]

In this video 1:58, I see significant artifacts with convex decomposition. Are there really no good non-approximate convex decomposition algorithms, and is the convex decomposition shown in the video the best researchers currently have? I recently created a convex decomposition algorithm for colliders for a video game destruction system that quickly generates a small number of convex pieces, does not create new triangles and produces no artifacts - the convex pieces perfectly represent the initial mesh's outer shell. Could it be that my algorithm could be useful also in this kind of simulations?

[@fruitcake4910]

Well, publish it

[@crawkn]

Humans learn to correct the alignment of the nut with the bolt by feel, with trial and error on every attempt. A robot could very easily learn to derive the proper thread alignment from a visual analysis of the bolt threads and the nut threads, to start the nut from the ideal position every time.

[@richcolour]

The local blacksmith dropped his dog off to me to look after. But as soon as he left it made a bolt for the door.

[@sstolee5978]

Wow! Extremely cool paper. I was also stoked when you showed The Witness footage, one of my favorite games of all time... do you enjoy it much as well? Seems kinda out of place here, but I'm not complaining.

[@Darkstring]

Could this be used in juntion with physics simulation? IE having a model create an organic structural model, then testing its validity via simulating its ease of assembly? Could be used to make things for machines to put together, rather than human hands.

[@weegee9209]

can be the potential game physics engine revolution as we've been stuck at the lack of a effective collision detection method for games for a while now, these days real time path tracing is pretty much viable, we are only waiting for the same revolution happen to the physics aspect of the game.

[@wobbers99]

Thank You for your video's. They really help me understand Light transport and Physics. I cannot imagine how un-educated on this subject i would be without your efforts. This video made me think of a zillion possibilities. :) Be nice to see that technique for Collison detection in Unreal Engine.

[@monad_tcp]

Finally useful things that are not related to LLM ! I want to make a factory game that's super realistic in the way things are built, now I can. What a time to be alive

[@UrbanArT69]

Accurate and fast collision detection is especially important in VR Games, as you get really close to the object you are holding in VR. So I really do hope to see an implementation of this in a game engine like Godot

[@mho...]

a mass collision system, that doesnt explodes, shakes & runs in real time?! thats just cool, another piece needed for a Holodeck!

[@thelatestartosrs]

I would use this for simulating agriculture equipment. I want to make a platform where we can train over robotic configurations in order to develop efficient robots for harvesting berries in fields. This needs fine detail for the plants but it also has to be a simulation to support weather events and robot damage over accidents, excessive use, normal wear and tear, etc.

[@errorhostnotfound1165]

The source in the bottom left says 2022, so is this very new or 2 years old? I am confused

[@alexeykulikov5661]

Can this technique translate to video game collision/physics simulations? Is it general enough? Those gains, if combined with automated enough approach to setup the properties for in-game objects, could result in actually interactive sandboxes and immersive sim games, at last.

[@antoniobortoni]

To create the best simple robot that can talk and perform tasks, focus on enhancing a good small multimodal models and training them in 3D virtual worlds. This approach improves task efficiency and reduces training risks. Combining these models with verbal interaction capabilities will make robots highly practical and accessible for various settings. Let’s revolutionize human-robot interaction!

[@LyricsQuest]

Accurate factory simulations to cost-effectively optimize factory layout and performance before ordering all the factory equipment and parts, leading to more productive factories and increased production across the economy. The second industrial revolution, cool.

[@lockaltube]

4:19 - Dumbledore plugged in USB cable calmly

[@billr3053]

3:40-3:46 I’m dismayed that the bowls show a distinctive polygon shape at the rim. I wound have thought that by now graphics boards’ algorithms would do circles as primitives. Fundamental building blocks.

[@d3crypt3d]

HOOOOOOOOOLY, this will be insane...looking forward to implementing this on my own

[@ryanpiotr1929]

I'd just like to use this to study the relationship between particle size distribution and maximum packing density for various shapes.

[@nutzeeer]

i have recetnly thought about SDFs being used for complex geometry! to now see it in action is really cool. there must be some sort of algorithm in place to create SDF functions from any given geometry. cool! SDFs can do much more than what triangles can offer. the geometry is mathematical, it can be infinitesimaly detailed!

[@travian821]

I never thought that a realistic simulation for robot training was particularly hard. If techniques like this go forward robots may actually become useful in day to day tasks. Kinda neat/terryfying.

[@akasht5]

This is really cool! Is it just kinematically consistent or are they simulating contact forces with friction accurately or are they fudged…

[@afterthesmash]

When you get a 10,000x speedup, the actual story is that no one had seriously looked at the problem before. That's going to happen a lot in the near future. There's so much we can do, and so much we have yet to tackle in a serious way with our new superpowers.

[@מבוססעלסיפוראמיתי-ק8ו]

maybe an even more elegant solution to problems like this is a combination of the visual rendering of the mechanics for the optical sensors of the machine learning algorithm together with pressure sensors for the algorithm to learn in combination - when i plug in a USB port or tighten a bolt my eyes are rarely what i rely on to know it's on tight, and i'm not blind. for the calculations of the nut and bolt - why calculate the whole bolt and not the main line of the thread and from there calculate the surface area that will come in friction? asking wanting to genuinely know

[@N0RZC]

Incredible! Cant wait to see what the future brings.

[@mm-rj3vo]

This will soon be implemented in videogames, and it will make simulating mechanical things, as well as destruction physics a bajillian times more realistic

[@MyLittleMagneton]

This is going to be huge for optimising manufacturing processes.

[@MrEricGuerin]

is this method (physic engine) is something general enough? is it something that could be also used by game industry ? (as you know in video games, physics is very approximative - in general complex visible shapes are approximate with simple convex shapes.

[@code4chaosmobile]

wow, I immediately thought of the Bobiverse series of books and being able to design everything in virtual. Until now this was still a huge roadblock. In a couple papers we could get Open Source factory solution competitions.. build widget x with fewest steps.. or the most :D and not to far along from deploying the original machine to make all the rest ;)

[@NOTORIOUS404]

I work in aircraft maintenance and since 2015 I knew my job was among the last to lose to automation. While I still think that is true I no longer feel so confident about the time frame.

[@SEB1991SEB]

Once they programmed into the robotic arm "Lefty loosey, righty tighty", they were halfway there really.

[@Estrallio]

the virtual robotics revolution, the digital answer to the industrial revolution, is here!

[@samstarlight160]

What's the success rate of a custom-made machine? Is this a close enough match for general purpose robot arms to be implemented in a real-world setting? Or is an 85% success rate for the initial attempt still leagues away from being acceptable?

[@RedOneM]

A practical machine for mass production purposes would need 99.99% accuracy and a reliable way to report the the 0.01%

[@DanielAusMV-op9mi]

Oh wow, factories with machines that produce our stuff for us? What a time to be alive ! ❤❤

[@V0IDWARE]

thank you Ren for letting me know about whatever this is

[@orbatos]

The actual optimisations are what we want to know, I'll be pouring over this paper.

[@kazitox]

it's always interesting to see the kind of things AI struggles with that you wouldn't even think of, and how well AI can do it once you've figured it out.

[@gibsongk]

Favorite new KZbin channel

[@marcelbricman]

i‘d use it for building houses. i honestly think this is going to be a huge application in the future.

[@kwullums]

Hey I haven't seen the video but have you seen any papers about these social media string parsers that people are claiming can read minds?

[@ezebentivegna1676]

Is it posible to use spline lines following the peaks and valleys of the geometry to simulate physics?

[@TheForbiddenLOL]

Wow I can't wait for your next video called "NVIDIA's new AI: 50,000x faster Virtual Worlds with Supercharged Ray Tracing!"

[@yahm0n]

That seems really silly. The obvious way to handle nut/bolt interaction is to take over with custom code so that the nut and bolt effectively become one object that is managed algorithmically, eliminating the need to even have a geometry interface between them.

[@Overt_Erre]

I'm annoyed that through all of the "how amazing this is, just look at it" stuff, you didn't explain how they actually reduced the contact point calculation. very annoying

[@TheSepetto]

This is nuts! Pun aside 2 more papers down the line and we are face to face with a revolution in our way of living?

[@CielMC]

SIGNED DISTANCE FIELDS? Valve actually changed the world

[@matthewferguson8369]

Wait, why can't they just get a high 3d resolution from lasers? They've done that to map cathedrals and towns, right? I'm missing something on why the number of triangles is important, it's a lot, but that's still easy for computers to compute the physics of right?

[@unreactive]

this is potentially game-changing for robotics

[@Jacobk-g7r]

So we use this in conjunction with omniverse. Or we could use it with cortical labs research.

[@Yggdrasilrune]

So, when will this hit game engines like Unreal, and how close are we to Matrix-like virtual realities? I would think, if we can have precision simulations of physics at scales that small, this should make things much more realistic in a virtual setting, and maybe we can use those precise 3D models as a basis for accurate tactile information so touching things would feel real, though we would need a proper user interface that would allow us to feel it, or a brain interface that would send the right signals. I just really want the matrix.

[@bakedbeings]

Signed Distance Fields are either turning up in more applications lately, or I'm just now noticing them. We've come a ways from metaballs.

[@Jakestar29]

The USB joke was gold

[@MM-ts9jy]

When I was a kid I always daydreamed about videogames, and in these fantasies I always pictured a game that simulated all things to perfection (specially water), even before I knew what a game engine even was. I'm so excited to see a technology that will eventually bring my dream to reality, and I will be able to live it. If AI doesn't take my job, that is. And I'm a software engineer, so you can guess how fucked I am.

[@Khofax]

I see this first being used in repair tasks that are too dangerous or inconvenient for humans to access with a human commanding the robot to perform certain tasks. Although I’m not sure how superior this would be to good ol direct controller based input.

[@nixes1636]

This is crazy amazing. At the start of the video I was like this is surely impossible.

[@tablab165]

Whoa! That's nuts! ...and bolts.

[@jk35260]

I think tactile feedback data is still crucial. Without tactile feedback, how does the AI knows when the screws don't match or there are defects.

[@jeffg4686]

craziness - I liked the bowls!

[@unrealdevop]

Oh man, this would be great to have in Unreal Engine.

[@ThaMentalGod2003]

u asked what we would like to see this be used in. id like to see this in videogames! 😎

[@freyna]

"What a time to be alive!" ❤

[@Soar-hw2rw]

Neuro link, LLM, cheaper and faster simulations, VR. I know what I am waiting for.

[@test-uy4vc]

What a nut time to be hard alive! 🎉

[@Sentientforce]

Shoot a spinning nut right over the top of bolt. Then only 2 outcomes, if it’s aligned it will fasten or it will fall off.

[@scarletevans4474]

Can't wait for these machines to start putting together some microelements that human simply wouldn't be able to do, because of big precision and tiny forces needed!

[@markosz22]

I want a new Garry's Mod kind of game with this level of physics simulation.

[@Axel_Andersen]

I think " robot tailored to a specific" task is as oxymoronic as "general purpose robot" is tautological.

[@Gameboygenius]

I was hoping The Witness would get a mention beyond simply being an example of a video game!

[@BunkerSquirrel]

30,000 b.c.e: me think. You think. What if rock think? Today: rocks thinking

[@lirothen]

Curious how it handles cross-threading

[@sleepyinterface8819]

I was somewhat expecting that USB joke

[@Thedeepseanomad]

If I get the money, I will Buy Nvidia and turn it into an open sourced international research foundation for humanity (and Earthlings in general).

[@medooazmi]

him: what would you fellow scholars use this for my dirty mind ;)

[@brightblackhole2442]

fine, but you need to remember that the nut must not get jammed over time

[@danielv6906]

Why simulate nuts and bolts at the highest level whilst not being assembled? Once a bolt is in place it can return to being a static part. And when referring to "simulated" here, we obviously leave out actual engineering simulations using FEM, correct material, stick-slip, tribology etc etc. Granted, this will be done in the engineering stage, not the assembly stage. Anyway, I'm sure we'll reach the point where we can do both in the same environment and have AI assist us with everything from ideation, specification, design, validation and production planning/factory organization. It's just a crunching game 😁👍

[@mdsmatheus]

Does that mean we're finally getting real-time liquid simulation for video games?

[@havisvh]

You missed the chance to say... "thats Nuts!"

[@shahbazfawbush]

This is nuts!