NVIDIA’s AI: Virtual Worlds, Now 10,000x Faster!

[@JoshSmithEE]

I work in factory automation and this is absolutely a hard problem. I look forward to this solution making it into standard toolsets.

[@SIRICKO]

You'er not gonna be working at all my man onces this get here.

[@markmuller7962]

But then I can't spend 8 hours a day tightening bolts 😞

[@0AThijs]

@@SIRICKO I'm pretty sure that op doesn't just tighten bolts all day...

[@jtjames79]

I'm a miniature war game player. I hate painting. I can follow instructions on how to make a robot arm. What a time to be alive!

[@monad_tcp]

I want something more realistic than Factorio or Satisfactory, this seems like a cool game to make. Those H100 are going to be very cheap after the second AI winter starts very soon when the LLM bubble pops. Time for cool and useful things to be done with them. What a time to be alive.

[@TheAkdzyn]

Dr Zsolnai-Fahér, I don't know whether I've mentioned this before or if it's relevant to your interests in anyway but I am Malawian (southern Africa). I hope you know the impact of your work. These quick updates with clear explanations of topics offer first timers a chance to learn something new without being overwhelmed. thank you for creating this channel and uploading so frequently. Personally, when I started watching your content I never thought I'd be watching entire Nvidia tech exhibitions and product launches but now I find my self sitting through hours of the launches and enjoy it thoroughly. Thank you and "What a time to be alive!"

[@Patralgan]

What a Time to be alive!

[@TwoMinutePapers]

That is absolutely amazing and really made my day. Thank you so much!

[@TheAkdzyn]

@@TwoMinutePapers You're welcome sir! 😄

[@pandoraeeris7860]

Finally getting down to the real nuts and bolts of AI research!

[@peanutnutter1]

You didn't screw this analogy, you nailed it!!

[@CamAlert2]

What a tool.

[@MilkGlue-xg5vj]

​@@peanutnutter1This is but a bolt in building autonomous AI agents. The future is gonna be nuts!

[@Speedrunner.007]

@@MilkGlue-xg5vj the future is gonna be nuts and bolts!

[@mycul_]

You're telling me this thing can plug in a USB correctly on the first try? Uh huh ok pal, lets easy down with all of the sci-fi

[@businessmanager7670]

this is evidence based. stop your ignorant dreams

[@geniferteal4178]

Yes, actually someone taught me to look for the EDIT: "seam" on the connector and face that down. It usually means the logo faces up. Regardless once i figured that out, I got them right more often than not

[@Ou8y2k2]

I can do that too... with usb-c

[@SantinoDeluxe]

i heard usb-a are 4 dimensional which is why you gotta flip em over twice

[@geniferteal4178]

@@Ou8y2k2 edit so it makes more sense. Sorry.

[@RedOneM]

1994: "Playstation released a week ago it is so dope! Can you imagine what we'll have in 2024?" 2024: *SIMULATING* *A* *GAZILLION* *NUTS* *IN* *PARALLEL*

[@spazneria]

We really went and grew up

[@iceseic]

we managed to surpass their expectation

[@FusionC6]

so many NUTS

[@AA-iq6ev]

@@FusionC6 Thts what she said

[@gamefun2525]

@@FusionC6 deez nuts

[@Kuchiriel]

This will become a metric unit someday. Nuts and bolts per second.

[@iusearch]

Don't think so, easy to cheat on this particular case

[@SkultétyBendegúz]

Americans will literaly use anything but the metric system smh

[@john_blues]

You should see his video where they had a cats per second measure.

[@Ryan_Thompson]

And rural areas will still be getting 10 Nbps while rich urban centers will be seeing 1000 Nbps. "I just crash my car a block from home every time, because I can assemble a new one faster than I can find a parking spot." What a time to be alive!

[@cherubin7th]

We already have a metric unit for such things: Hertz.

[@MikevomMars]

THIS is the level of physics I want to use in VR games and apps! 🤩

[@michaelleue7594]

...Why in those? Like, I understand AI training for robots in the real world or for simulating designs for factories, but under what circumstances would you want to hyperaccurately screw on a bolt in a video game?

[@marcoshalberstadt7646]

@@michaelleue7594 Literally any game that requires physics simulations, special in VR. Of course you have to calculate the trade off between realism, performance and fun, but there is always some niche communities that want to just maximize realism at all costs.

[@SirNightmareFuel]

At the moment games would typically solve this situation with a "that's good enough"-style trigger box, and scrubbing through a pre-baked animation. The excitement here is we could skip this jarring and unimmersive experience, with a much more uniform and intuitive solution. Also it's getting closer to simulating a more granular and 'realistic' world, which is inherently more immersive. ​@@michaelleue7594

[@HiddenExp]

@@michaelleue7594 massive melee combat sims!

[@Luizfernando-dm2rf]

@@marcoshalberstadt7646 So basically it all boils to down to porn, as always

[@JorgetePanete]

This is nuts!

[@user-cg7gd5pw5b]

You think you're funny, huh? Well, guess what! You are, I laughed.

[@Mertiven]

and bolts!

[@bilbobaggins5938]

It is fairly screwy.

[@Simplicity4711]

They don't screw up a lot. Mostly down.

[@jlrutube1312]

@@Simplicity4711 I'm going to bolt over to my computer right now and try this.

[@MrYerak5]

No WD-40 simulation?

[@Sekir80]

What does Water Displacement recipe 40 has to do with bolts? Nah, I get, don't worry.

[@wobbers99]

Now that would actually be a massive paper. Love it!

[@undefined6512]

I tell ya hwat

[@palleppalsson]

10 papers along the line we get the "soaked in saltwater for years bolt that you have to heat with the welder and whack with a hammer -simulation"

[@voinea12]

Game engine developers: 👁️👄👁️

[@locinolacolino1302]

Game Engine developers make use of SDF based physics all the time, it's just this level of fidelity is rarely required in games so those clock cycles are used for other things.

[@monad_tcp]

@@locinolacolino1302 "is rarely required". As someone who plays Space Engineers I disagree, clang is my enemy, I wish I could stack 8 GPUs in parallel and increase the simulation to 800hz. I always thought it was janky because the physics runs at 60fps, but that's too slow. Increasing the mesh detail decrease the performance exponentially, not linearly, that's why SDF based physics don't do that. But this paper proposes a solution that doesn't need more detail, so you can actually increase the simulation to 800hz and thus decrease janky. I think I'm going to make a factory game that's actually physics based instead of just being a logistic simulator. Imagine real items in the conveyor belts, that's going to be fun.

[@voinea12]

@@monad_tcp in Godot, a simple chair falling lags the whole game, though maybe I'm doing something wrong idk

[@dreadthedrums]

Do game engines use SDF for collision? I have seen things like Niagara in UE5 use it, but generally collision is done with a collision mesh, or using original geometry (very expensive)

[@theneonbop]

@@voinea12 you can get the godot jolt physics addon which should work much better for most things

[@kronux3831]

This right here is a prime indicator of what I think is going to be humanity’s biggest innovation over the next decade: an increase in our ability to simulate complex things like this in a super efficient way. Can’t wait to see how this tech scales in the next few years

[@fgmenth]

Factorio 2 is going to be AWESOME

[@pranjal9830]

What a time to be alive.

[@3d-illusions]

So the breakthrough was using sdf collision detection?

[@Felipehez]

i think that houdini has some options to use sdf for collisions, but I don't remember it being particularly fast

[@mumiemonstret]

The best simulation has to be the simulation of football simulants at 0:47 - so accurate!

[@SteelBlueVision]

Accurate if someone dropped some crystal m-e-t-h in their gatorade, maybe...

[@vadimfedenko]

as a guy who works on this stuff, i can say that complex geometry simulation of the bolt threads is not necessary. as soon as the bolt enters the nut, it is enough to turn them into a single physical object, where there are only 2 interconnected degrees of freedom between the nut and the bolt (rotation with movement along the axis of the bolt). physical stimulation is needed only at the very edge of the bolt and nut to determine whether the bolt has entered in the right way

[@akissot1402]

And how i am supposed to study the series of events that causing a jamming ?

[@MrTehkaiser]

It's impressively embarrassing that you don't understand why the physical sim is an absolute necessity in this context. Especially seeing as you work in the field, that fact makes it significantly more embarrassing.

[@ShikariHybryd]

"You can even plug in that USB-A adapter in a virtual world accurately; a feat almost impossible to mere mortals." 💀I laughed so hard

[@RC-1290]

I had to simulate bolt interactions for VR. It makes much more sense to approximate the bolt. But if you use this example as a way to describe other complex collisions, then sure, valuable.

[@badumtsy]

So, soooo nice to see new research around SDFs!!! It almost makes me believe that my recently finished thesis involving SDFs wasn't for naught xd

[@6TheBACH]

Wait, I don't understand. Are they still using triangles with the bolt or they are using other stuff?

[@lucas_sapa2985]

Instead of triangles they are using SDF (signed distance fields) a form of mathematical calculation of the volume of the object. Its what Unreal Engine 5 uses for good and fast collition detection and other effects

[@Darkstring]

Could this be used in juntion with physics simulation? IE having a model create an organic structural model, then testing its validity via simulating its ease of assembly? Could be used to make things for machines to put together, rather than human hands.

[@richcolour]

The local blacksmith dropped his dog off to me to look after. But as soon as he left it made a bolt for the door.

[@SEB1991SEB]

Once they programmed into the robotic arm "Lefty loosey, righty tighty", they were halfway there really.

[@monad_tcp]

Finally useful things that are not related to LLM ! I want to make a factory game that's super realistic in the way things are built, now I can. What a time to be alive

[@AviramYageena]

What a time to be alive! Thanks for the content!

[@sstolee5978]

Wow! Extremely cool paper. I was also stoked when you showed The Witness footage, one of my favorite games of all time... do you enjoy it much as well? Seems kinda out of place here, but I'm not complaining.

[@someonetwofake]

In this video 1:58, I see significant artifacts with convex decomposition. Are there really no good non-approximate convex decomposition algorithms, and is the convex decomposition shown in the video the best researchers currently have? I recently created a convex decomposition algorithm for colliders for a video game destruction system that quickly generates a small number of convex pieces, does not create new triangles and produces no artifacts - the convex pieces perfectly represent the initial mesh's outer shell. Could it be that my algorithm could be useful also in this kind of simulations?

[@fruitcake4910]

Well, publish it

[@SteelBlueVision]

2:21 - Bolt SDF (SDF stands for SIgned Distance Field, btw) visualized as point cloud, Bolt SDF visualized as mesh. Both of these are a mistake for a mechanical part being made to a mesh approximation of a solid, both the source and the cause of all of the collision detection problems that follow. The moment you triangulate and _then_ do collision detection on the triangulated meshes, you are effectively doing collision detection wrong. It doesn't matter whether the resultant mesh is made of quads or nGons or how "perfect" it is topologically speaking, no matter the mesh, it's all triangles in the end and this plethora of triangles is what will fed into the collision detection algorithm(s). Compare and contrast this method with how any CAD application or mechanical application worth its salt, represents mechanical solids like literally nuts and bolts. They are represented as non-destructive solids and surfaces backed by exact mathematical formulas that describe them with a good heap tablespoon of Constructive Solid Geometry (i.e., bool operations like union, intersection, and difference, all performed mathematically and perfectly - try that with a mesh). The moment you map the solid to a triangle based mesh, even for something relatively simple in the grand scheme of things like a bolt, you get a proverbial crap-ton of polygons to consider for collision detection or you cheat and come up with a bolt that looks like it came out of some sort of FPS shooter with hexagons for curves. This is the case, at least, if the object of interest has curves and then requires a considerable amount of mesh geometry to approximate it. It can never be exactly mapped to a mesh, only approximated with greater and greater accuracy at the cost of an explosion in polygons (read, triangles) that are used to form those mesh based approximations. Then you have to deal with that vast sea of triangles when you do your collision detection, making it a very expensive process, even if you take shortcuts and break up triangle groups into volumes, along with other similar techniques used to cut down on the test of "which triangles are colliding, or worse yet, which surfaces are intersecting that cannot intersect, because they are rigid bodies.

[@afterthesmash]

When you get a 10,000x speedup, the actual story is that no one had seriously looked at the problem before. That's going to happen a lot in the near future. There's so much we can do, and so much we have yet to tackle in a serious way with our new superpowers.

[@lockaltube]

4:19 - Dumbledore plugged in USB cable calmly

[@mho...]

a mass collision system, that doesnt explodes, shakes & runs in real time?! thats just cool, another piece needed for a Holodeck!

[@crawkn]

Humans learn to correct the alignment of the nut with the bolt by feel, with trial and error on every attempt. A robot could very easily learn to derive the proper thread alignment from a visual analysis of the bolt threads and the nut threads, to start the nut from the ideal position every time.

[@DustinRodriguez1_0]

When you said "everything is simulated down to the last millimeter"... you were just using an expression, right? It has to have higher fidelity than millimeter-level to be useful in any real life context... especially dealing with things like machined parts, 10,000ths of an inch are the common unit. A millimeter is gigantic in comparison.

[@thelatestartosrs]

I would use this for simulating agriculture equipment. I want to make a platform where we can train over robotic configurations in order to develop efficient robots for harvesting berries in fields. This needs fine detail for the plants but it also has to be a simulation to support weather events and robot damage over accidents, excessive use, normal wear and tear, etc.

[@alexeykulikov5661]

Can this technique translate to video game collision/physics simulations? Is it general enough? Those gains, if combined with automated enough approach to setup the properties for in-game objects, could result in actually interactive sandboxes and immersive sim games, at last.

[@errorhostnotfound1165]

The source in the bottom left says 2022, so is this very new or 2 years old? I am confused

[@akasht5]

This is really cool! Is it just kinematically consistent or are they simulating contact forces with friction accurately or are they fudged…

[@UrbanArT69]

Accurate and fast collision detection is especially important in VR Games, as you get really close to the object you are holding in VR. So I really do hope to see an implementation of this in a game engine like Godot

[@wobbers99]

Thank You for your video's. They really help me understand Light transport and Physics. I cannot imagine how un-educated on this subject i would be without your efforts. This video made me think of a zillion possibilities. :) Be nice to see that technique for Collison detection in Unreal Engine.

[@LyricsQuest]

Accurate factory simulations to cost-effectively optimize factory layout and performance before ordering all the factory equipment and parts, leading to more productive factories and increased production across the economy. The second industrial revolution, cool.

[@N0RZC]

Incredible! Cant wait to see what the future brings.

[@weegee9209]

can be the potential game physics engine revolution as we've been stuck at the lack of a effective collision detection method for games for a while now, these days real time path tracing is pretty much viable, we are only waiting for the same revolution happen to the physics aspect of the game.

[@travian821]

I never thought that a realistic simulation for robot training was particularly hard. If techniques like this go forward robots may actually become useful in day to day tasks. Kinda neat/terryfying.

[@TheForbiddenLOL]

Wow I can't wait for your next video called "NVIDIA's new AI: 50,000x faster Virtual Worlds with Supercharged Ray Tracing!"

[@orbatos]

The actual optimisations are what we want to know, I'll be pouring over this paper.

[@d3crypt3d]

HOOOOOOOOOLY, this will be insane...looking forward to implementing this on my own

[@kazitox]

it's always interesting to see the kind of things AI struggles with that you wouldn't even think of, and how well AI can do it once you've figured it out.

[@MyLittleMagneton]

This is going to be huge for optimising manufacturing processes.

[@matthewferguson8369]

Wait, why can't they just get a high 3d resolution from lasers? They've done that to map cathedrals and towns, right? I'm missing something on why the number of triangles is important, it's a lot, but that's still easy for computers to compute the physics of right?

[@billr3053]

3:40-3:46 I’m dismayed that the bowls show a distinctive polygon shape at the rim. I wound have thought that by now graphics boards’ algorithms would do circles as primitives. Fundamental building blocks.

[@DanielAusMV-op9mi]

Oh wow, factories with machines that produce our stuff for us? What a time to be alive ! ❤❤

[@V0IDWARE]

thank you Ren for letting me know about whatever this is

[@MrEricGuerin]

is this method (physic engine) is something general enough? is it something that could be also used by game industry ? (as you know in video games, physics is very approximative - in general complex visible shapes are approximate with simple convex shapes.

[@mm-rj3vo]

This will soon be implemented in videogames, and it will make simulating mechanical things, as well as destruction physics a bajillian times more realistic

[@מבוססעלסיפוראמיתי-ק8ו]

maybe an even more elegant solution to problems like this is a combination of the visual rendering of the mechanics for the optical sensors of the machine learning algorithm together with pressure sensors for the algorithm to learn in combination - when i plug in a USB port or tighten a bolt my eyes are rarely what i rely on to know it's on tight, and i'm not blind. for the calculations of the nut and bolt - why calculate the whole bolt and not the main line of the thread and from there calculate the surface area that will come in friction? asking wanting to genuinely know

[@ryanpiotr1929]

I'd just like to use this to study the relationship between particle size distribution and maximum packing density for various shapes.

[@kwullums]

Hey I haven't seen the video but have you seen any papers about these social media string parsers that people are claiming can read minds?

[@samstarlight160]

What's the success rate of a custom-made machine? Is this a close enough match for general purpose robot arms to be implemented in a real-world setting? Or is an 85% success rate for the initial attempt still leagues away from being acceptable?

[@RedOneM]

A practical machine for mass production purposes would need 99.99% accuracy and a reliable way to report the the 0.01%

[@Jakestar29]

The USB joke was gold

[@unreactive]

this is potentially game-changing for robotics

[@yahm0n]

That seems really silly. The obvious way to handle nut/bolt interaction is to take over with custom code so that the nut and bolt effectively become one object that is managed algorithmically, eliminating the need to even have a geometry interface between them.

[@tablab165]

Whoa! That's nuts! ...and bolts.

[@Jacobk-g7r]

So we use this in conjunction with omniverse. Or we could use it with cortical labs research.

[@NOTORIOUS404]

I work in aircraft maintenance and since 2015 I knew my job was among the last to lose to automation. While I still think that is true I no longer feel so confident about the time frame.

[@jk35260]

I think tactile feedback data is still crucial. Without tactile feedback, how does the AI knows when the screws don't match or there are defects.

[@sleepyinterface8819]

I was somewhat expecting that USB joke

[@ezebentivegna1676]

Is it posible to use spline lines following the peaks and valleys of the geometry to simulate physics?

[@TheSepetto]

This is nuts! Pun aside 2 more papers down the line and we are face to face with a revolution in our way of living?

[@freyna]

"What a time to be alive!" ❤

[@mdsmatheus]

Does that mean we're finally getting real-time liquid simulation for video games?

[@CielMC]

SIGNED DISTANCE FIELDS? Valve actually changed the world

[@jeffg4686]

craziness - I liked the bowls!

[@Estrallio]

the virtual robotics revolution, the digital answer to the industrial revolution, is here!

[@marcelbricman]

i‘d use it for building houses. i honestly think this is going to be a huge application in the future.

[@Gameboygenius]

I was hoping The Witness would get a mention beyond simply being an example of a video game!

[@markosz22]

I want a new Garry's Mod kind of game with this level of physics simulation.

[@nixes1636]

This is crazy amazing. At the start of the video I was like this is surely impossible.

[@havisvh]

You missed the chance to say... "thats Nuts!"

[@stevemeisternomic]

This can be done even easier with an algorithm that teaches borders and padding

[@unrealdevop]

Oh man, this would be great to have in Unreal Engine.

[@gibsongk]

Favorite new KZbin channel

[@DavePetrillo]

This paper is two years old and it says this has been in isaac sim since preview 4??

[@Yggdrasilrune]

So, when will this hit game engines like Unreal, and how close are we to Matrix-like virtual realities? I would think, if we can have precision simulations of physics at scales that small, this should make things much more realistic in a virtual setting, and maybe we can use those precise 3D models as a basis for accurate tactile information so touching things would feel real, though we would need a proper user interface that would allow us to feel it, or a brain interface that would send the right signals. I just really want the matrix.

[@bakedbeings]

Signed Distance Fields are either turning up in more applications lately, or I'm just now noticing them. We've come a ways from metaballs.

[@code4chaosmobile]

wow, I immediately thought of the Bobiverse series of books and being able to design everything in virtual. Until now this was still a huge roadblock. In a couple papers we could get Open Source factory solution competitions.. build widget x with fewest steps.. or the most :D and not to far along from deploying the original machine to make all the rest ;)

[@antoniobortoni]

To create the best simple robot that can talk and perform tasks, focus on enhancing a good small multimodal models and training them in 3D virtual worlds. This approach improves task efficiency and reduces training risks. Combining these models with verbal interaction capabilities will make robots highly practical and accessible for various settings. Let’s revolutionize human-robot interaction!

[@machinegod3000]

When robots are able to knit a sock, manually - then we are screwed!

[@vaendryl]

You really went into the nuts and bolts of it lol

[@nutzeeer]

i have recetnly thought about SDFs being used for complex geometry! to now see it in action is really cool. there must be some sort of algorithm in place to create SDF functions from any given geometry. cool! SDFs can do much more than what triangles can offer. the geometry is mathematical, it can be infinitesimaly detailed!

[@Axel_Andersen]

I think " robot tailored to a specific" task is as oxymoronic as "general purpose robot" is tautological.

[@scarletevans4474]

Can't wait for these machines to start putting together some microelements that human simply wouldn't be able to do, because of big precision and tiny forces needed!

[@johnyharris]

At last, a solution to my flat pack post traumatic stress.

[@MichaelHolstine]

That's just nuts!

[@test-uy4vc]

What a nut time to be hard alive! 🎉

[@koktszfung]

Let’s wait until this is implemented in real life

[@kleinebre]

This is two minute papers with Dr. Károly Zsolnai-Fehér. But the troubles don't stop there :D

[@Michael_H_Nielsen]

"If you fear that you have a addiction. Please feed it" ;)

[@MyNames_55]

Can't wait for AI to do at least part of the hard optimizing proccess

[@Sentientforce]

Shoot a spinning nut right over the top of bolt. Then only 2 outcomes, if it’s aligned it will fasten or it will fall off.

[@MM-ts9jy]

When I was a kid I always daydreamed about videogames, and in these fantasies I always pictured a game that simulated all things to perfection (specially water), even before I knew what a game engine even was. I'm so excited to see a technology that will eventually bring my dream to reality, and I will be able to live it. If AI doesn't take my job, that is. And I'm a software engineer, so you can guess how fucked I am.