How to make a Point Cloud Renderer using Unity VFX Graph

[@SamSchiffer]

Subscribe for more tutorials and other gamedev content! Follow me on Twitter! twitter.com/DevOrenge Tutorial playlist: kzbin.info/www/bejne/rXmzoGasotSBn5o

[@warden4688]

just for any future followers of this tutorial, you need to enable Experimental Operators/Blocks in Edit>Preferences>Visual Effects, had me stumped for a bit longer than I would like to admit.

[@SamSchiffer]

That’s good to know, because I didn’t have to do that. Probably has to do with using a different Unity version.

[@김동현-f2i8v]

thank you~~~~

[@glitch0001]

Thanks so much for this, this is awesome. Warning to anyone using the pastebin code, the line that sets the height of the textures can be one pixel too low since length/resolution are ints and can get rounded down. To make sure you are rendering all voxels adjust this to something like (int)Mathf.Min(1 + voxels.Length / resolution, resolution), and break out of the for loop if index exceeds voxel array length. you will have some unused pixels in your texture but this is preferable to not rendering voxels in my case at least.

[@luciid9786]

Holy moly this is such a good video Orenge

[@alessiograncini9564]

so great and finally a good tutorial about this !

[@badgraphix9659]

Really nice, thanks for the tutorial :) Btw, are you participating in the GMTK game Jam?

[@SamSchiffer]

No, I wouldn't know what to make of the theme, and i heard about it a little too late. Great that you like the video though!

[@WireWhiz]

Hey this is awesome, I've been trying to write compute shaders to do this but failing. Now to try to recreate Teardown.... Edit: I might use this for some VR stuff and then make a tutorial about it, with full credit given to you for the rendering of course. Edit 2: Tip for those who want the particle cubes to receive shadows: Us an Output Particle Lit Cube instead of an Output Particle Cube node.

[@SamSchiffer]

Glad you found some use out of this video!

[@arsenbabaev1022]

Did shadows work well? Is it worth trying this rendering technique? Im working with voxel art.

[@jacobhansen3871]

I have tried following this tutorial 3 times and still doesn't show any points, does this still work in 2022?

[@mitkoaleksandrov5674]

Thank you for the great video! I'm interested in voxel-based simulations. I'm wondering if I pass voxels' position and color (or just update color) through c#, can I expect good performance or the update will be slow?

[@SamSchiffer]

I haven't tried this, so I'm not sure what kind of performance you'd get. Best is to just perform a quick benchmark before committing to it. You could also try changing the values (color and position) in the update section of the vfx graph, that way you don't have to reinitialize the graph every time you update the particles.

[@mitkoaleksandrov5674]

@@SamSchiffer Thank you for the reply! I know that I can use the update section, but I still need to pass some values out of the vfx graph for other purposes. I may check how quickly I can update voxel positions and color.

[@mitkoaleksandrov5674]

@DevOrenge, I have tested the dynamic update and it seems a bit slow. I have noticed that with the new version of HDP there is no "Output Particle Cube". Do you know what would be the way to create cubes? Also, do you if we can achieve the same thing with UDP?

[@SamSchiffer]

It should be the same for URP as for HDRP. Not sure about the output particle cube, maybe they just changed the name?

[@DanHearn-s8h]

in the new version of unity VFX you can no longer output particle cube

[@yaneelox]

Hi! Can I ask you how to make a spawning 1 particle at given coords (raycastHit coords)? What should I change in the code? Thank you for guide!

[@jddennison]

hey guys, anyone have any idea what this error means :[New VFX] [System] CameraSort.compute: Kernel at index (0) is invalid UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&)

[@arsenbabaev1022]

Can shadows work with this? Can it be used to optimize rendering for Large voxel art models? If i create them as particles.

[@SamSchiffer]

I believe they can receive shadows, I'm not sure about casting shadows though. You would absolutely be able to render voxel models with this, as long as you have individual point data and you don't have more than a few million voxels. It's not the most optimal way to render voxels though. The best way is by generating a mesh from the voxel data.

[@arsenbabaev1022]

@@SamSchiffer so if creating regular models is better anyway whats the point of this type of rendering? Just cool looking tech with a lot of cubes / particles? I thought this method is somehow faster then normal, ill test it out anyway.

[@SamSchiffer]

For one it’s a lot quicker and simpler to do it this way, but that’s just a side note. This method is mostly useful for rendering point clouds, since they aren’t easily depicted otherwise. But it also works for voxels as long as they are kept in the lower million range. If you have the time to code a voxel mesh generator, then I feel that is the best way to go about it. But that’s much more complex than what is presented here.

[@Vandercox]

Any idea if it would be possible to optimize that (either the settings or the system itself) if say i wanted to have 150-200 million voxels active ?!

[@SamSchiffer]

Not sure how feasible it would be to display that many particles, it might just explode your gpu. What you could try (I haven't tested this and it might not work): - use 8k textures - use multiple textures when you run out of space in a single texture - try compressing your data as much as possible to conserve memory (does your positional data fit into a single float, or can you somehow store xyz rgb and scale in a single vector4) - if every particle has the same scale, consider omitting the scale from the texture and store it in the graph as a simple float that way you may be able to fit xyz rgb in a vector4. - another thing you can do is to have a list of colors (or textures) and use the w component of your vector4 to index one of those instead of storing a color for each individual particle. also you should be aware that VFX graph has some issues with insanely large particle counts. It may start lagging even if your system can theoretically handle it.

[@Vandercox]

@@SamSchiffer Thank you for the input I was very familiar with Unity, but not so much with the HDRP & the VFX graph. This solution seems very handy for what i'm trying to accomplish and your suggestions are helpful. I will try to optimize as much as possible and see how far i can get it :) Have a nice one!

[@arsenbabaev1022]

@@Vandercox did you succeed? Any updates on optimizing whats shown in a video? Maybe you even have a git repo if im lucky.

[@Vandercox]

@@arsenbabaev1022 Unfortunately my GPU bottlenecked a lot quicker than i thought it would. I believe i got to about 2 million particles which is not bad at all and was running around 30 fps. That was using a GTX 1060. We stayed with our partnership as it is and the option was not investigated further (my test took me about 1h30 or so). I believe with a decent GPU and more optimization you can get a few millions but definitely not as much as i was initially looking for. Good luck! =)

[@SamSchiffer]

That’s interesting. I remember the most I managed was around 13 million, but that was using billboards. This all was running on a 980ti. I’m not sure but I think 5 million should be possible at a decent frame rate without using billboards

[@omsa9433]

Thank you for the great tutorial I have a question I have a collection of data points and they are 30000 in total I am struggling to render them using your code. I am really new to this

[@SamSchiffer]

That’s a tough one to help you with, really all you should need is to call the method SetPoints() with your points array and colors array and that should be it (I’m on mobile right now and wasn’t able to look up the exact method name but I hope it’s close enough).

[@omsa9433]

@@SamSchiffer sorry for the disturbance I have tried the example you did on the start method but it tells me index out of range even that I did the same thing as you

[@omsa9433]

@@SamSchiffer I was able to do it Thank you so much🙏🙏

[@warden4688]

@@omsa9433 how did you fix the index out of range exception?

[@harshitkaushik4584]

@@omsa9433 How did you fix this error?

[@bjornsyse]

If my coordinates are signed, that is some are negative values - this will be lost when converting to colors since they can't be negative, right?

[@SamSchiffer]

You can apply a transformation to normalise your coordinates before you pass them through to the shader

[@bjornsyse]

@@SamSchiffer good idea!

[@DanielSantallaa]

How would you show the original colors of the mesh?

[@SamSchiffer]

If you’re using a simple mesh that has colors for each individual vertex, you can pass the mesh.colors array into the SetParticles method just like that. If your mesh has a texture, you’d have to somehow get the color of the texture at each vertex position and store it into an array which you’d then pass into the SetParticles method. I wouldn’t know how to do that though.

[@vaiox8773]

Thanks! this is great! I've followed the tutorial and I have everything done, now I need to apply it to my point cloud but I don't know-how. I come from graphics this is just my first week with Unity, If somebody can point me where to read o seek. Thanks again!

[@SamSchiffer]

Hey, how is your point cloud currently loaded? The script I wrote has a SetParticles() method that takes an array of particles along with an array of colors. If you can manage to get your particles into that form factor, then all you need to do is pass them into that method and you're set.

[@vaiox8773]

​@@SamSchiffer Thanks for the answer! I still have lots to learn, I've Just used a custom package called Pcx to import Point clouds into Unity that I get from another youtube tutorial

[@SamSchiffer]

@@vaiox8773 I haven't heard of that package, what is the data loaded as?

[@SamSchiffer]

@@vaiox8773 It looks like it's stored as a mesh, in that case you can make a public field in the script "public Mesh mesh;" then drag the imported pointcloud game object onto it from the editor to load it into the script. In the script's start method you can get the vertices for the pointcloud by typing "mesh.vertices" and the colors should be stored in "mesh.colors"

[@vaiox8773]

​@@SamSchiffer After you import the Pcx package to Unity, it allows you to import any .Ply in binary hope not to be spamming this is the Package github.com/keijiro/Pcx/releases and this is the video kzbin.info/www/bejne/l3zJgYh5j750ntk

[@jinchaohan2485]

can we load ply or pc file instead of mesh?

[@SamSchiffer]

you can load whatever file type you like.

[@panickal]

Can this be done without HDRP?

[@Torrochan]

I am sorry, but i dont understand how to attach this now to my objects. ...

[@SamSchiffer]

If you have your pointcloud ready in unity, you simply need to feed it into the public SetParticles method provided

[@Torrochan]

@@SamSchifferThx for the fast answer but Sorry i am stupid :/. So normally i would feed it with every mesh i want to render? Can i just simply render all meshes at once?

[@SamSchiffer]

You would normally have an array/list of point positions and point colors if you’re using a point cloud, this method doesn’t rely on you passing in meshes. So you pass in all the points you want to render at once into the SetPoints method and that should do it.

[@sjoerdev]

Is there any way for these voxels to cast shadows?

[@SamSchiffer]

I'm not sure exactly, might be something to do with the HDRP settings or you can try switching to an output particle lit cube in place of output particle cube in the VFX Graph.

[@sjoerdev]

@@SamSchiffer do you also know if it is possible for these particles to have a collider?

[@SamSchiffer]

@@sjoerdev I don't think it's possible, you'd have to do collision detection separately.

[@corriedotdev]

i cant figure out why youre assinging a colour texture or even a pos scale here.. why are you assigning to textures when you could just pass the vector3 position of the vertex to the system to render? cheers

[@SamSchiffer]

I'm not 100% sure anymore, since I made this a while back, but I'm pretty sure it was because you can't select vertices individually, and kind of spawn particles randomly at the vertex positions.

[@urbanmasque]

Can you paste this script somewhere? Im running into console errors following this :(

[@SamSchiffer]

Sure thing my man. Give me a minute to get the script and upload it to github. I'll post it in the video description. Check back in a bit.

[@urbanmasque]

@@SamSchiffer Thanks I was able to slow walk through the errors in the console and resolve xD. - Can you make a video setting up the environment or hierarchy for your plugin? - Another issue I've run into is that my point cloud points are super small and do not have the thickness of the ones in your example. Did you add a particle effect on top of your example? How did you achieve that. - Also I had to animate my blend effect but yours looks automated. Im not sure if its a different version, but I would love to walk that through step by step.

[@SamSchiffer]

@@urbanmasque I added a link to the script now. In the script I have a value that sets the size of the particle cube, it's just a simple scalar that I pass into the vfx graph. Also be sure to have "output particle cube" as your particle output for your vfx graph. I'm not sure what you mean with blend effect, I didn't do anything with that in my project. By the way, you don't need to import the hdrp package, you can just import vfx graph and it'll work the same. Might be less complex to set up.

[@user-gy8xn4kt7s]

Arigato

[@SamSchiffer]

do itashimashite

[@underbelly69]

will work for URP? or please make a URP version - thanks