The Math behind (most) 3D games - Perspective Projection

Perspective matrices have been used behind the scenes since the inception of 3D gaming, and the majority of vector libraries will have built-in helper functions to construct them for you. But what if you wanted to know the reasoning behind how these matrices are constructed.
I start off with a brief introduction of computer graphics and the key ideas behind 3D rendering. I differentiate between Image Order Rasterization typically used by Ray tracers, and object order rasterization as used in most video games. I then present an animated walkthrough showing the full derivation of both the orthographic and perspective projection matrices.
My derivation here is focused on the Vulkan API, but the reasoning applies just as well to other API's such as OpenGL or direct X, with the only notable differences being the size of the canonical viewing volumes, and the handedness and conventions of the xyz coordinate systems.
0:00 How does 3D graphics work?
2:05 Image versus object order rendering
2:51 The Orthographic Projection matrix
5:17 The perspective transformation
7:08 Homogeneous Coordinate division
8:27 Constructing the perspective matrix
10:29 Non-linear z depths and z fighting
11:30 The perspective projection transformation
** Resources **
Fundamentals of Computer Graphics by Peter Shirley & Steve Marschner
www.songho.ca/opengl/gl_projec...
matthewwellings.com/blog/the-...
www.insider.com/pixars-animat...
* Attributions*
Ray tracing graphics cards - Photo by Nana Dua from Pexels
Rubiks cube - Photo by Mathias P.R. Reding from Pexels
Ray tracing in video games - www.digitaltrends.com/gaming/...
en.wikipedia.org/wiki/Line%E2...

[@BrendanGalea]

For those of you who have been following the vulkan tutorial series, this video is a slight departure from the normal format. The coding portion for the tutorial will be released separately as soon as I’m finished. Future tutorials will return to the more typical structure, with code and theory interleaved. Enjoy!

[@yaircambordamorocho4506]

i loved the Homogeneous Coordinates Hero 😂

[@BrendanGalea]

@@yaircambordamorocho4506 hahaha thank you :)

[@y.m.o6171]

I really like the way you explain the theory behind these computer graphics topics without mudding with the programming part.

[@olegartamonov3439]

Amazing design of video, very accessible and interesting explanation, thanks a lot.

[@dexterman6361]

Crystal clear as usual! Now I know what z-fighting really means! Take that wikipedia!

[@func8211]

Cleanest description of the math behind view projection I've seen! Nice.

[@Enaku]

Agreed! I've been reading and watching almost everything I can find on the topic and this is by far the best explanation I've seen to date.

[@NoahDove-ei8in]

WHAT THE FUCK IS HAPPENING GOD HELP THE MATHS IS HELL, KILL ME NOW.

[@ammonwolfert]

I’m writing a rendering engine from scratch in C and this was exactly the resource I was looking for. Wonderful visualization and explanation!

[@Stratelier]

I remember, when first learning programming I tried to get (any level of) 3D rendering to work, and discovering the math of perspective transformation (e.g. x1/z1 = x2/z2, thus x(screen) = x(world) * (z1 / z2) ) was a critical breakthrough. Even if the only thing I could actually code with it was an imitation of the "starfield" Windows screen saver.

[@kmsskyquake7330]

question : are you using SDL ?

[@mackaygrange3072]

I’m working on the same project with SDL and quaternions right now and it has been a whole rabbit hole to dive into!

[@alexanderheim9690]

After watching 3blue1browns "Essence of Linear Algebra" Playlist, taking the linear algebra module in my university AND watching this video I was finally able to implement these concepts all while deriving the used methods by myself too. Thanks for this awesome video

[@BrendanGalea]

Thank you!! glad you found it helpful!

[@guitarvoicing]

Another master piece of 3d explanation. I really never understood well the black magic of the perspective matrix. It was in somehow just diving by z. Yet, your explanation is so clean and so perfect. It makes it really easy to understand. Please post more tutorials like these. You should definitely write a book about it too.

[@AlexGomes09]

Beautiful, right to the point. Not a lot of extra theory. Easy to follow and understand. Keep them coming 🙏

[@lavz24]

Found this, Great explanation this topic was killing me, I was not able to understand it before where they get the values and a visual representation of it. You have everything. Great job!

[@bellanthea]

Thank you for this video!! I don’t code much, but I love art and math and was so curious on the math behind game engines / 3d graphics and wanted to see how close or far off my guesses were. this makes me want to code!

[@BrendanGalea]

Thank you so much. Comments like this are what keeps me motivated to keep making videos.

[@ssopkc_venom]

Probably the best in depth tutorial i have ever seen on KZbin for graphics rendering. Even better than Cherno IMO

[@PBrrtrn]

Absolutely loved this explanation and would recommend it to anyone trying to understand the nuts and bolts of graphics programming.

[@kipchickensout]

This simultaneously looks easily and neatly explained and not understandable at all without an already pretty advanced level

[@zarblitz]

This video on its own is incredible, educational, and just all around superb. I learned so much and tied together so many loose ends I had dangling in my brain because of this video. Thank you.

[@rootKLM]

Two videos in a week! So glad you're making this series, I was struggling a lot with the vulkan documentation before I found your videos!

[@krumss]

Very well explained!! Good work!! Look forward to your game engine and computer graphics series!!

[@pedro_soares_bhz]

Wow, these drawings are so cool, they have a 2000's vibe. Nice.

[@aaaakko139]

Even though I thought I had already understood this before I watched, I still learn something from it. Thanks for making this video, I think it's the clearest one about how to get the Perspective Projection

[@Josh-kh1zp]

This is amazing!! I always wondered what was behind it, great Job!

[@tomatrix7525]

This is very very very good. I’ve seen too many people code in opengl etc without really understanding what is going on. Sure it works, but this intuition as to what these matrices are doing is good

[@BrendanGalea]

Thank you!

[@sardaou3944]

The best mathematical explanation. In most cases, the equations come out of the blue like from the magician's hat. Thank you very much

[@isshintheguy4172]

I have exactly no background in computer graphics, only linear algebra. Your video is masterful in its explanation. I find the tempo of your speech helps alot!

[@abdelmajidkansoussi6904]

The best explanation of perspective projection. Thank you

[@RogerValor]

never knew there is even so much more math beyond the math that my simulation itself needs, but it finally explains a lot

[@r.d.6536]

Wow, thank you so much for explaining these topics so well!

[@Skeffles]

Brilliant video. I did a computer graphics course a few years ago and we never actually went into what makes the matrix. Thanks for explaining it!

[@zilch.scienta]

THIS WAS SO AWESOME! Keep up the great work!

[@jemmrich]

Just wow. Fantastic explanation!

[@allesfresser0]

This channel is just awesome! Thank you so much.

[@philipbutler]

homogeneous coordinates swooping in lmao. Great video! I've been meaning to understand this better for about a year now and finally searched it up and I'm thrilled to have found this, perfect for my level.

[@Karmush21]

Thank you so much for this video! Finally made me understand the perspective projection matrix

[@mrshodz]

great explanation. i love the visuals; it makes it so much easier to understand.

[@BrendanGalea]

Thank you!! this was definitely the hardest video I've yet to make.

[@janoskovacs9406]

Thank you very much! I needed to rewind parts of the video to fully digest the idea, but finally i did. :) Please, keep up the quality work!

[@BrendanGalea]

Ya Ive gotten some good feedback that maybe I went a little fast 😅 but really glad you were able to get it! This has definitely been the hardest video for me to make so far

[@zdspider6778]

12:33 The vertical FOV is actually the angle from the center to the top (or center to the bottom, same thing). It's a half angle. Hence the theta/2.

[@everestgjonaj]

Also the the height he marks at the illustration should be height/2.

[@madelinevandergriff2408]

Time to drop everything for the new Brendan Galea video, love your stuff man!

[@user-fy4iq6if4z]

Man you saved my life. I was cramming my 3D book and completely lost in projection stuff then found your vid. Thank you so much aa

[@user-ri2ms2mm7w]

My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life kzbin.info/aero/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B- kzbin.info/www/bejne/bZWUZ3amjNVgpc0

[@mohsinmd2662]

so clear and crisp. Thanks for this video

[@BrendanGalea]

Glad you liked it!

[@gvcallen]

Honestly I've probably watched this video around 4 times. Best explanation on the topic and a super good reference to hold on to. Thanks a lot!

[@BrendanGalea]

Wow, thanks!

[@animationmann6612]

This is a Beautiful Explanation

[@YashBhalgat]

I have been searching for a clear explanation of NDC system and the projective transformation. This is such a good video with illustrations that explain the concepts beautifully!

[@leiwang1566]

It's very useful and eay to understand, thank you for your work.

[@void-fp9hq]

Thank you! Amazingly helpful video!!!

[@samarthtandale9121]

Nice Explanation video !!! Somehow I had to watch in 2x to not get distracted and understand better

[@Director414]

Awesome video! Thanks a lot for this!

[@reengineer2497]

thank you so much for the explanation. thanks to you I also understood new things about z-fighting.

[@MKULTRA_Victim_]

Currently learning basic math, but I watched this whole video for some reason and I really hope that someday I can understand it.

[@ntsakoselaule8646]

Love your explanations man, you make the documentation easy S/o

[@SreyRc]

Excellent video! Can't thank you enough!

[@anonymous-random]

This is a masterpiece. Thx!

[@kodesafi]

I really really needed this video. Thank you !!!!!

[@theRPGmaster]

Thanks again for continuing the series! I have something important (potentially) to add about future 3D animations: the cube demo ran poorly for me, low framerate, creating an ugly jitter. Unusual, since my hardware is fairly fast. Fixed it by disabling the "apidump" layer I had defined in the "VK_INSTANCE_LAYERS" system environment variable. However, this made my framerate skyrocket to above 3000 FPS, making the spinning cube become an imperceivable blur. Your demo implementation is framerate dependent, I have since created a lightweight way to measure timing (delta) and adjust the speed of animations to any system. Just thought I should remind you of this just in case. I can provide my implementation if you'd like to use it. Edit: having apidump enabled will not only slow you down, but also spam the console on every api call. Hope it helps. Sorry for the long post!

[@BrendanGalea]

Ya this is something I should have mentioned. The easiest way to limit frame rate for now is update the chooseSwapPresentMode function in the swapchain by commenting out the mailbox present mode, and using FIFO (vsync). Your solution is the ultimately correct one, where the game loop needs some timing mechanism. We'll cover the different methods of game loop timing pretty soon.

[@ajaykumar-qw3qr]

Very nice and detailed explanation. Thank you for the video.👏

[@BrendanGalea]

Glad it was helpful! Thanks!

[@joyceji6400]

Amazing video to learn 3d projection!!

[@AmaanHasanDilawar]

This is what I was looking for...thank you.♥♥

[@torljungberger2486]

I think i finally get it! Thank you for this video

[@openroomxyz]

Thanks! Very good explenation.

[@tomaszstanislawski457]

I think that both OpenGL and Vulkan use right-handed coordinate axis. The coordinates just differ by rotation by 180 degrees around x axis.

[@stef8776]

I’m working on a career change right now, going back to school in the near future for an MS in CS with hopes to be a graphics engineer. I’m reviewing all the prerequisite math right now so having an understanding of what kind of math I’ll need to learn in the future is very helpful. Even though I didn’t really understand 95% of this it was still a great video 😁

[@BrendanGalea]

Thanks and best of luck with your studies!!

[@codejunki567]

Just a question, do you have relevant CS or math experience? Because just going "I want to program 3d space vectors" is not something the average Joe just gets up and does?

[@stef8776]

@@codejunki567 no I don’t, that’s why I’m going back and taking all the prerequisites before doing the grad program. I have to do calc 1, 2, linear algebra, physics, C++ programming, etc. I’ve done a lot of Python, R and SQL but that’s not really relevant. Believe me, I understand how difficult is and how much I have left to learn. I probably won’t be able to start the grad program for another year and half or so.

[@Dom-zy1qy]

​@@stef8776honestly if it's something you're interested in it's not very difficult. I think people who complete certain qualification programs try to oversell it's difficulty. As a complete average Joe myself I haven't really had problems with the maths side of 3d graphics. It's actually helped me improve my intuitive understanding of certain concepts. If you enjoy the work, you'll be thinking about it enthusiastically when you're out and about, going on a walk, doing whatever. When things are on your mind out of pure intrigue, you are probably in the right field and will definitely succeed.

[@rouge97r]

Just wow, thank you so much!!

[@tokyospliff]

That intro omfg 🔥🔥🔥🔥

[@blender6895]

thanks for great explaination

[@Lazullien]

this is the type of stuff you understand when you're learning it, but when you're programming it makes you wanna die

[@BrendanGalea]

Hahaha ya exactly. The main problem is that you rarely get useful error messages as feedback. Things will just look wrong. My first school project working on projection matrices I was passing in the matrix in row major order and didn’t realize OpenGL wanted column major… took me hours to figure that one out

[@jayocaine2946]

@@BrendanGalea any tips for getting my brain to actually work with all these different orderings? It's 99% of what i spend my time debugging in my brain

[@robertomontalti3064]

I don't know why, but every explanation online of how and why the z order is preserved ends up in a shady and not very explanatory way. At 10:35 you literally made me get it instantly! thank you for this great content, keep it up brother!

[@Ryco117]

Thank you very much for making this quality tutorial series, I've been following along slowly for some time and really appreciate the detail and clarity. I will say that, for me personally, the videos which focus more on the mathematical background are less useful, but I may not be representative of the audience; just thought the feedback could be useful. Looking forward to more lessons!

[@BrendanGalea]

Thank you! And I appreciate the feedback. It's good to know what's working for some people and what isn't. Well, some good news then is this is the most math heavy video for the foreseeable future. Tutorial 14 will have just a bit more and then it's predominantly coding after that point.

[@dexterman6361]

@@BrendanGalea Might wanna do a poll, I enjoyed this video! Helps me understand why I put in a matrix, without blindly copy-pasta-ing it :) I personally prefer both!

[@BrendanGalea]

@@dexterman6361 - ya that's a good idea. Something I'll have to think about

[@user-ri2ms2mm7w]

My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life kzbin.info/aero/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B- kzbin.info/www/bejne/bZWUZ3amjNVgpc0

[@silver-zero]

thank you so much for this

[@alonsomartins712]

Era exatamente esse tipo de vídeo que eu tava procurando. Muito obrigado!! 👏😁 ➕1 subscriber!

[@Sir_Pickle]

Amazing again, was nice to get to some theory.

[@user-ri2ms2mm7w]

My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life kzbin.info/aero/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B- kzbin.info/www/bejne/bZWUZ3amjNVgpc0

[@Enaku]

Brendan, you're a gem.

[@BrendanGalea]

thank you :)

[@soolydooly1234]

finally a tutorial that actual explains the maths

[@pavelperina7629]

Interesting. I've tried to derive it using transform to ortogonal frustum first (far goes to one, near goes to near/far) and then solving equations to open the wedge and map right, near point to 1,0. One slight inaccuracy is z buffer precision. It's from times when z buffer was stored as fixed point number. Today it can be switched to floating point. Also range can be both -1 to 1 or 0 to 1 in modern OpenGL and z test can be negated.

[@stellarmagic3316]

All that Math Library Programmed in the Engine is Insane. Layers and Layers of Codes writing New Codes to render the output, Just like Neural Network!

[@rutvikpanchal466]

New video, Awesome!

[@Dannnneh]

Valuable videos.

[@abdullahpatel2120]

Right... Definitely understood what being said.

[@simgamedev6423]

Thank you so much!

[@ESCAcarlos]

awesome video!

[@aj-uo3uh]

This is actually a general derivation that holds for a skewed view frustrum. The picture shows a regular frustrum. Skewed frustra you need when you want to make a huge picture so that you have to chop up the initial regular frustrum.

[@pontosinterligados]

I have a 3x4 matrix obtained from a camera calibration method. The matrix includes the intrinsic and extrinsic of the camera but not really a model view projection matrix. How do I dare to convert this raw matrix into a 4x4 projection matrix?

[@florisvanderhout2675]

wow this is amazing thank you very much

[@user-ri2ms2mm7w]

My friends, search for your life purpose, why are we here?? I advise you to watch this series and this video 👇 as a beginning to know the purpose of your existence in this life.. kzbin.info/aero/PLPqH38Ki1fy3EB-8xmShVqpbQw99Do2B- kzbin.info/www/bejne/bZWUZ3amjNVgpc0

[@musikosv]

Great video, ty!!!

[@sha-2564]

Good job man

[@tapewormerbinkosti3141]

Thank you so much! You explained everything I needed.

[@xanderlinhares]

If you use the right-hand rule with X pointing to the right and Y pointing up, positive Z goes into the screen and -Z goes out. If that matches up, it is right-handed. Otherwise it is left.

[@DaminGamerMC]

Amazing!

[@BrendanGalea]

Thanks!

[@romanshvayko211]

Amazing

[@jungwisely1137]

Great video, making LA way more interesting to me personally

[@The_Airbender_]

Awesome thanks 👍

[@ColonelJ1]

I'm a bit confused. In the first example, the origin "orthographic view volume" is in world-space? Therefore this is valid for a camera looking down the z-axis, but if you wanted to have a camera looking towards anywhere you would need to add a rotation matrix in there right?

[@dslbit2]

omg, you are awesome!

[@MurderByProxy]

I’ve always been impressed by people who can break down problems and explain each step easily… loved the video and will have to see it 100 times more…

[@user-gc6bb1tk9h]

This is amazing!! I always wondered what was behind it, great Job!. Love your explanations man, you make the documentation easy S/o.

[@lakshraja4831]

You are the best!!!

[@sto2779]

Nice, I can use these math for making LiDAR sensors.

[@flameofthephoenix8395]

Great! I think I'll not watch it though; I've been for a few years just figuring this out mostly on my own, I don't really want to spoil anything for myself! I think I've figured out UV mapping or at least something akin to it, I figured that if you know which triangle a point is being projected from, then you can change the central point, in other words shift every 3d object's coordinates aside from Z position and reproject the point again, if you do this twice making sure that you're projecting to central positions such that the orthographic projection of the de-projected 3d point can't be on a line from the two 2d projected points, then you can now draw a line from the two projected positions to the point that was their central position, find the line intersection and you'll now have the X and Y coordinates of the de-projected point, all that's left is to simply figure the Z coordinate based on X and Y coordinates.

[@EnupGames]

But what's that?! It's homogeneous coordinates to the rescue! Love it!

[@baconinvader]

I understand now! thanks

[@stachowi]

Damn this was good.

[@_general_error]

Good explanation of the frustrum matrix... but I can imagine total newbies would not take much out of it, unless you show them some code and what effects can be achieved with manipulating the frustrum parameters. One thing which tripped me up is what are n and f values and how they are chooses... but this is probably just me not paying attention.

[@adygadgets]

nice video really

[@tapewormerbinkosti3141]

Except 1 thing that I don't understand: from the final output of perspective projection matrix, do you need to divide x and y by w from the final output of perspective projection matrix to get the correct x and y? Thanks for potential answer 😊

[@hawns3212]

No, in the shader when you set gl_position it will automatically divide by the w component.

[@Aurora12488]

This is almost the perfect derivation of perspective projection (especially since it includes both the off-axis and FOV versions!). However, I wish you would have talked more about why applying the first perspective transform gets us into a space that allows for trivially transforming with the orthographic projection matrix. It doesn't seem intuitive to me. Firstly, I think it would have helped to emphasize that multiplying by n and dividing by z actually turns the slanted edge of the frustum into the right-angled straight edge of the desired rectangular prism because everything on that will end up landing at the exact same x value (or y, for the top, respectively). Secondly, however, we should note that what we get after multiplying with our perspective matrix is *not* a 3D rectangular prism, since its components haven't been divided by the w component (i.e. z value) yet. It's a seemingly somewhat arbitrary 4-dimensional entity, and so it's unclear from my limited linear algebra skills why doing the perspective divide and then applying the orthographic projection is equivalent to doing the orthographic projection and then the perspective divide. There seems to be a deeper "truth" to homogeneous coordinate equivalence that is more than just a notational convenience. If you had touched a little on that, this would have been the de-facto, perfect explanation.

[@Aurora12488]

Hmm, trying to figure it out more, it seems like the "divide-by-w" can really just be considered to be multiplying the matrix-vector multiplication by the scalar 1/w. And if you have a scalar a and a matrix-vector multiplication M times v, M[av] = a[Mv] (as well as [aM]v). I think that's the last piece of the puzzle.

[@australianoz]

One question, the vertical field of view as you display considers the view center point to be at Y = 0 and not at h/2, thus theta is incorrect in your statement, or did i miss something? (Coming from a bloke who used to have to draw these projections on paper back in the early 80's).