Remaking the Portal Effect from Deaths Door! | Using the Stencil Buffer in Unity

In this gamedev breakdown, I'll show you how the portal effect from Death's Door can be remade in Unity with some stencil rendering tricks!
The project is free to download so feel free to check it out to see how it works here (Unity 2021.3.6f1):
drive.google.com/file/d/1FC5j...
Chapters:
00:00 Intro
00:48 Stencil Buffer Explanation
03:03 Basic Implementation
04:37 Render Order Mayhem
06:03 Polish
06:44 Outro

[@krooqs]

Finally tutorials that explain not only the how, but also the why! Funny too.

[@Raccoon5]

What is funny is that the original game is using the most simple method you presented with single stencil buffer which makes the door clip. Then they switch to normal rendering when the door is close to closing. You went even further than the original! Good job Check at 0:06

[@nikefootbag]

Haha good pickup! I think the bright white bloom helps to hide the clipping too

[@RandomProduct]

This goes into collisions, but I'm wondering if you could tackle the Haunty Halls Galaxy from Super Mario Galaxy - specifically the part where ghosts are eating part of the floor. They're hiding part of the surface and turning it into a hole, changing the collision of the floor. Might be interesting!

[@QuontynTechArt]

Thanks for the suggestion! Yeah just from the videos I see of it I have a couple ideas. I imagine there is a particle emitter attached to those ghosts which emits a particle that isn't rendered in the main camera, but is rendered in a top down camera to use as an alpha mask for the ground. This is at least a common set up in a lot of games, they use similar systems for leaving trails in grass / snow, etc. Not a huge expert on collision optimization but since particles are accessible on the cpu you can do a bunch of quick sphere intersection tests and basically just turn off the collision between the player and the ground if they intersect. Intersection tests with a sphere shouldn't be too expensive. If you tried using the stencil buffer for something like that it would be tough for the intersection test just because you don't really have any info on the CPU. Would have to play around with it but that would be my first guess!

[@pixtrick]

At 6:35, the bug is probably occuring because of the second inverted box. The door frame is not rendered anymore when you look at its back but the second box is still rendered, so it's quite normal i guess. A simple solution would have been to ztest the second inverted box to render it only when it's in front of the frame but the mask shader does not write into the depth buffer so ... _Welcome to render order hell_ Also, the fact that the door frame casts shadows is a bit weird in my opinion, but hey i'm not an astrophysicist either ;)

[@QuontynTechArt]

For sure! There was already so many order-dependent draws that I didn't really want to try and throw another one in and complicate things further haha. But that's a solid idea if you were using this in a project that needed it. Also, yes, probably does not make any sense that a portal casts a shadow lol. But I kinda just liked the look of it with it so I took a creative liberty and left it in : )

[@nianyiwang]

You, sir, have my toast

[@pixtrick]

@@nianyiwang thanks mate, I appreciate it ;)

[@SharkJets]

Who are you?? 4 subs and this video quality is pro-grade! Well done!

[@QuontynTechArt]

Awesome to hear : ) Just been trying out something new

[@SharkJets]

@@QuontynTechArt not that you'll need my help at all, but once you get a couple more videos out, I'll give a shoutout on my channel. You know your stuff!

[@nianyiwang]

I didn't realize this before scrolling down to the comment secion... Liked, subbed, and why the hell does he only have 1 video?

[@marceli-wac]

Who would have thought youtube needed another game-dev-related breakdown channel... Not me, but I'm glad it's here!

[@breadleymcthicc5444]

This is incredibly well polished! This aspect of game design is one I've yet to focus on, and it's honestly really interesting. As a suggestion, I've always wondered how the effect is done where Mario transitions from 2D to 3D in Super Paper Mario. Have fun, dude!

[@hogwrangler3283]

i had a moment of epiphany thanks to your explanation of the stencil buffer, ty!

[@DivinusYT]

Great video. Would love to see more from you!

[@wat5931]

This is a really nice first video! Keep it up!

[@user-ff6mr5sc4n]

Good job,It so clear to tell me what's stencil buff

[@FudanshIKun]

Awesome explanation+ Awesome tutorial, really thanks for big help on my homework :)

[@seve8138]

Great! Hats off for the effort! 😄

[@janelu701]

This is so cool!

[@user-ff6mr5sc4n]

so great

[@tianqigao869]

awesome tutorial!

[@buu_mm6784]

VERY COOL ... 😳

[@MikeCore]

niceeee

[@user-xo8nq7dr3o]

What a great video❤. Congratulations. Is the character's movement in the project isometric like in the game?

[@QuontynTechArt]

Thanks! The movement is similar to how it works in Death's Door, although I just used Unity's character controller with a few modifications.

[@masterjcoz]

love it, I thought the portal video with the car is all I need, I have tried to make the TOWNSCRAPER export look better for weeks. The windows stick out when they are desined to be boolean tooled out of the buildings mesh. I have tried using stencil buffers but couldnt get the effect I needed. I currently have 3 cameras to generate masks to complete the effect. Which seems less efficient. As you are better than me, it would be a massive deal for the comunity if someone figured out the windows properly. For my situation, the logic is (example with rays): If it hits the outside of the window then the buildings mesh, render the inside of the winow ontop. (so not out side of window -> inside of window-> building mesh , this would be the overlapping issue). I have yet to learn how to do this with shaders. Your video has made me want to have another go

[@QuontynTechArt]

Hey! I have never touched Townscaper so I can't really say exactly how it works, although I looked up a video and I see what you mean about the windows sticking out. You're right though, rendering three cameras seems excessive and I'm sure the actual game does it a different way. I'm not totally convinced it's using the stencil buffer for that effect though... My guess is that it's doing a cheap raycast in the shader from the world space of the pixel in the direction of the view vector, and looking for intersection between that ray and the box that would be "cutting into" the building. You would then shade it based on the intersection position to determine if it was the window, or the interior wall or whatever. If it fails the intersection test entirely (ie, in the areas its sticking out from the building), you can discard the pixel or use alpha blending and just set the alpha to 0. Another option, although I think it's less likely how this is done, is to do a parallax shader effect on the window. If you look at how the windows in Spiderman for the PS4 are rendered you can try out something similar. It's basically just rendering a flat plane and then faking depth in the shader. Finally, it might also be helpful to install some graphics debugger like RenderDoc and launch it on Townscaper to see exactly what draw calls they are issuing. It can be a little troublesome to set up (especially for shipped games). But if you get it working its super helpful for reverse engineering effects cause you can see exactly what meshes / resources they are using, and if you dig deep enough I think you should be able to see what stencil values they are using (if any). Hope that helps at least a little bit! Often times I find myself trying a bunch of techniques before I figure out one that actually works haha

[@eliasayberk2609]

Great break down I wonder how valorant breach ultimate that looks like a crack texture projected on the environment

[@GrandHighGamer]

Seems like masking the outside of the 'hallway' would have worked better. So instead of only drawing stuff inside of the doorway, just stop rendering any part of it covered by the outside of the hallway.