noclipping into MP4 files

[@GSTChannelVEVO]

Hey everyone! This is my first sponsored vid. It's kinda short but it's what I was working on when I got the email. :P It's neat to be approached by a sponsor I actually use lol anyway, here's the link and boilerplate again: kzbin.info?cc=gstchannel& Monthly paid subscription. Price per month varies. First 2 months free. Terms apply. Cancel anytime. If you subscribe through the link in this post or the banner appearing in this video, I may get a commission.

[@saltedmutton7269]

what would happen if you ran this video through the no-clipper??? like especially at 4:13 where it's already unrecognisable from the original image

[@GSTChannelVEVO]

@@saltedmutton7269 oh god that's a cool idea. i think it'd just dissolve into soup but it'd probably be fun

[@saltedmutton7269]

​@@GSTChannelVEVOmaybe with a high-resolution video the first time round it might give some interesting results? almost sounds to me like 4D but i'm not a Physics Man (yet) so i'm not sure... do you have a link to the program that you could put in the description?

[@GSTChannelVEVO]

@@saltedmutton7269 good idea. I've added a link to the description

[@bs_blackscout]

I got the popup! Ty

[@Mittzys]

For anyone wondering this technique is called Gaussian Splatting

[@GSTChannelVEVO]

lol, busted. i've been infatuated with what you can do with gaussian splatting. and even just COLMAP structure-from-motion

[@JohnVance]

Thanks! I've heard of that but had no idea this is what it was in practice.

[@therealquade]

What tools do you use to do this?

[@GSTChannelVEVO]

​@@therealquade graphdeco-inria/gaussian-splatting! nothing fancy beyond careful framing.

[@MiniKodjo]

​@@GSTChannelVEVO Hoo I have friends working at INRIA 😲

[@makhnobux]

i have no idea what is happening. this must be what it feels like for old people to open a PDF

[@solidshrek]

lmao

[@arthurbarbosa8204]

The program he's using, "Gaussian Splatting", allows the user to inspect a single frame of a video file by "rendering" it as if it were a three-dimensional environment. However, it isn't magic. Data not fully visible in the frame being inspected will either be misrepresented, missing, or in the case of larger amounts of invisible areas, blurred out violently. It's why he can look around the corner of the room almost flawlessly, but turning around or moving a lot to look at what isn't in the video anymore results in an abstract painting.

[@cupofdirtfordinner]

@@arthurbarbosa8204 great explanation, tysm!

[@icarusgaming6269]

bro said "noclipping into MP4 files" and then actually did it

[@PhillipAmthor]

Brand new sentence

[@r.g.thesecond]

This is the opposite of an educational video, where you keep trying to figure out what the fuck the creator means.

[@GSTChannelVEVO]

I've done so many well-researched and educational videos. I thought it was time for a change. Time to undo all of that. To uneducate the world.

[@Scypek]

"You can't noclip into videos because it's not possible" *Repeatedly shows footage of actually doing it* There may have been some deliberately misleading phrasing there, but the whole thing barely feels like a lie.

[@SianaGearz]

A deducational video?

[@real-horse]

​@@GSTChannelVEVOkeeping mystery alive

[@kormannn1]

​@@SianaGearz video that makes you forget what you were taught, wow!

[@borstenpinsel]

The irony of people WITH youtube premium sitting through a two minutes Ad about youtube premium hearing you say "you will never see ads"😂

[@JB52520]

And I never do. 🤔

[@ilusions4]

sponsorblock + ublock origin is the real youtube premium

[@sillypinkmoth]

@@JB52520 wowe u passed out from 2:00 to 3:20

[@Jecobe]

sponsorblock?

[@GSTChannelVEVO]

yeah that occurred to me, lol I tried to clearly mark where to skip to, and failing that, sponsorblock works wonders ;)

[@tverdyznaqs]

This video feels like it's from a parallel universe for 2 seperate reasons.

[@Scoped21]

Which are? I'm curious now.

[@ZphyZphyer]

​@@Scoped211. Noclipping into mp4s and not games 2. This video has near no explanation of whats happening, though the creator of the video normally makes educational vids

[@axelprino]

This explains surprisingly little despite the fact that almost every spoken line seems to be framed as if it were an explanation, kinda impressive.

[@borstenpinsel]

Basically an algorithm tries to create a 3D scene from the still frames of the video. Like a crude 3D scan. This data is then explorable in a 3D environment.

[@torphedo6286]

@@borstenpinsel it's actually a pretty advanced 3D scan, I was reading about it for school last year. if you take your own photos from multiple angles or find a high-quality video to work with, you can get almost perfectly photorealistic renders in real-time from most angles of the subject. it's cool stuff

[@BoundaryBreak]

This video is Palworld and I'm Nintendo standing at the edge of the line with my legal team. Jk

[@GSTChannelVEVO]

ruh roh :D

[@Laundry_Hamper]

They're always just there a few thousand units away from everything else, waiting to be teleported in when someone hits the Shigeru trigger

[@the_crum]

“This video is Palworld and I’m Nintendo edging the line with my legal team”

[@Abdega]

So you are standing at the edge of a *_boundary?_*

[@CombustibleLemon77]

@@GSTChannelVEVO spirits?

[@Leftysrev3nge]

Gaussian splatting reminds me of those art displays using random trash and detritus to create an image when viewed from the right angle.

[@PixyEm]

a lot of neon feathers and pipe cleaners!

[@RushFreak]

I love the premise and dead pan delivery, this a comedically genius application for the technology lmao

[@theblankuser]

Gaussian Splatting renders are just braindances from Cyberpunk but right now

[@KevinD-nf8jv]

exactly what i was thinking

[@PhillipAmthor]

Not to mix up with the rapper gaussian that is also a mathematican who does gaussian spitting

[@TheTimmynatoRex]

I was literally about to make a comment about how this looks a lot like braindances, but it seems like you beat me to it.

[@Slash0mega]

oh man, calling this noclipping and implying that you could find stuff out of bounds was really getting my goat. seems this is just a gag though.

[@GSTChannelVEVO]

hehe, yeah. I might have a strange idea about what constitutes a "joke". sorry about your goat. :P

[@generallyunimportant]

unfortunately theres only shitty unprotected ram out of bounds, so you couldnt get your goat even if you could noclip.

[@noaag]

@@generallyunimportant ranching simulator game but the animals are rogue AIs, and if you die in jpeg you die irl swordartonline style

[@MDPToaster]

Huh, never thought KZbin would sponsor a video.

[@meILM]

They seem to be sponsoring a whole bunch of medium sized KZbinrs all of the sudden.

[@GSTChannelVEVO]

@@meILM yeah that sounds right. i was given a recommendation to drop this video at noon today (i was late lol) so i suspect some people received a wall of youtube-sponsored videos in their subscription feed today

[@jlco]

It's a better approach to shifting people to premium than what they've _been_ doing...

[@rockpie.squashfs]

I only saw 2 youtubers ever get sponsored by KZbin: Hunter R. and him.

[@theawesometeg219]

I got recommended this video instead of sponsored

[@westingtyler1]

would be cool to do this with movies, like that intro city shot in blade runner

[@GSTChannelVEVO]

it's technically possible, but the results are quite finnicky. at glance, I'm not sure it'd look very good in that particular case.

[@AROAH]

The math going on to make this work is so far beyond my comprehension it might as well be magic.

[@amdradeon3863]

this was one of my favourite gameplay elements in cyberpunk 2077

[@Catishcat]

The weird thing is that many of us can model these places in our heads, even from an extremely low quality recording. In fact, the quality may not even matter up to a certain point. This means that this concept is actually possible, just not very possible *yet*.

[@madghostek3026]

I think our brain is really good at making us feel like it knows the structure, but it doesn't because it's not needed, try rotating a more complex 3d model around, for example of human head, you can't predict the position of features super precisely until you see them (for example draw a dot where it should end up). But it still feels normal and good

[@albertnortononymous9020]

But that would mean advancing AI and that could lead to dangerous consequences.

[@TheBcoolGuy]

@@madghostek3026Counterpoint: the consistency of shape and perspective in dreams, which is not spoiled by lucid dreaming. Things morph, but you can move around no problem.

[@Quitobito]

​@@TheBcoolGuyHow reliable is your source of information? Yeah, not at all.

[@HCR_]

@@Quitobito we're talking about theoretical things wtf did you expect

[@Name_Pendingg]

title: "noclipping into mp4s" me: oh like, de-compiling mp4s to figure out how they work CTS: *starts noclipping through mp4s*

[@SnipeyFox]

heh, welcome back to boundary bre- how in the hell

[@featherpoof]

I'm sorry I was expecting you to go in-depth on like what mp4 codec is and see what the files store inside behind the scenes and visualize it. I was not expecting you to GO IN FIRST FUCKING PERSON and just BEGIN FLYING AROUND. HOLY SHIT LOL

[@jhxc64]

I thought this was gonna be a shitpost at first, but it turned out to be a super cool look into gaussian splatting. S-tier Boundary Break parody.

[@EmergencyTemporalShift]

Theoretically, pulling data from a low resolution video should be possible because of the motion and extracting the data from variation in the pixel brightness

[@GSTChannelVEVO]

I've heard of this but I've never actually worked with it 🤔

[@weaved-matters]

The sponsor popup looks so alien, I've never seen a KZbin video sponsored by KZbin wild

[@sroach275]

God its like a glitched out braindance, great stuff

[@rynesherman9099]

Its like watching those super long fractle videos that just keep diving deeper into something

[@InsanityPrevails]

0:54 *Entire world collapses as the still image turns into a video and my tiny babbie brain can't comprehend this magic*

[@276-]

absolutely... AWESOME!! i find gaussian splatting and other new technologies awesome, and i really enjoyed how this concept of using it on real videos was executed! props to you, GST. you just earned yourself a sub :D

[@SylphDS]

It seems obvious in hindsight, but I had no idea you were a wizard.

[@gumballegal]

haha for whatever reason i tought you were gonna literally noclip the camera into the video like if it were a 2D plane. Great stuff

[@sdg1]

I'ma be 100% honest I have no clue how I got on this video I set my phone down on Instagram then all of a sudden this was playing but I'm too invested to leave so take my sub

[@GSTChannelVEVO]

welcome! this is pretty unlike any of my other videos, except in chill vibe. so... beware? :P

[@JackieBright]

Bro noclipped into your phone

[@sdg1]

@@GSTChannelVEVO I'll be sure to check those other vids out then!

[@scriptguru4669]

Now I'm thinking how to noclip a mp3

[@rockpie.squashfs]

4:04 POV: Posy's close ups

[@Mark_Rober]

posy fan alert

[@Owoshima_]

POSY MENTIONEDDDD 🔥🔥🔥🔥

[@Breadn]

this genuinely feels like magic wtf

[@alfiowo]

This is probably how it would feel like for a 4th dimensional being to visit our 3rd dimension universe

[@Yngdady]

KZbin's first industry plant?!

[@madskillz808]

I once had a dream where i was in a park area next to a place i used to work (in real life). I suddenly fell though the floor, and was sudden carted sideways like i had landed on a fast moving conveyor belt, being able to look up and observe the land i used to occupy. I woke up suddenly after a couple of moments, as the effect left me feeling like i couldn't breath. I wish my experience of noclipping was nearly as colourful as this.

[@Super.Whimsy]

Using Gaussian Splats to explore into videos as a “no clip” is brilliant. 🤯

[@chuckerchuckerton7046]

Reminds me of the "enhance" scene in Bladerunner where Deckard is peering around corners in a photograph.

[@nostalgia_junkie]

4:04 loooooove how the pixels almost shimmer

[@MuteObserver]

Fantastic concept for one of these -most people have no idea this is possible Cheers ^^

[@Auberginej]

4:08 breakcore album art

[@skylark.kraken]

Inside a JPEG artefact world someone is taking a photo of someone else “make sure to get my good side”, the good side being the very specific angle they aren’t a disjointed blur spread across a large distance

[@OnyeNacho]

This is what I call trickster tech. It was never designed to truly generate interactivity in video clips, it was only designed to emulate it from the outset, literally emphasizing and digitizing the "fake it until you make it" phrase. A gimmick; nothing more.

[@sparx0s]

What kind of trickery is this

[@jerry_blueberry]

really took crawling through the screen literally

[@sortaspicey9278]

This reminds me a lot of cyberpunk 2077's braindance editing environment

[@DOCTORdoctor-j3u]

this is what combing through peoples memories will be like

[@folddyy]

it's the first time I've seen somebody actually get sponsored by KZbin.. interesting

[@EveBatStudios]

Great to see people appreciating this. I've been doing this with AGIsoft and Reality Capture for years and LumaAI's Gaussian splatting model does a great job on those cases where the data is too compressed to provide consistent data for photogrammetry.

[@tundraosu]

This is like Cyberpunk all over again! 😭

[@Ohio.Gozaimasu]

The other day I found a video I shot with a Nokia 6300 on an old HDD. It's 176 x 144px and 60kbps (yep). We've come a LONG way. Do you know RAW (DNG) image format? You can do wonders with a phone that supports it (Galaxy S line of phones, for instance). Just load that file in Lightroom or Adobe Camera Raw and you can literally make invisible things visible. Also, it's free of post-processing that butchers regular JPGs.

[@GSTChannelVEVO]

I stumbled across that format when looking for a way to store images with a more-than-8-bit color depth per channel, but haven't actually used it in a camera. seems magical

[@Ohio.Gozaimasu]

@@GSTChannelVEVO It is. You can take a picture of a musician on stage in harsh blue light, remove the blue tint and make his face look like a face again. You can take a picture of a dark alley at night and make it look almost decent. There's no noise removal or edge contrast enhancement unless you add it yourself. Everything's inherently sharper. Just make sure that the phone or camera shoots in this format because not all of them do.

[@he2a]

TBH even though I know techniques like gaussian splatting is present, I never imagined that it can technically be used to noclip through videos. Thats pretty impressive. Would be really interested in more videos like these.

[@Imdrekaiser]

Gaussian splatting is so cool, I wish there was more stuff that used it

[@benjaminzuniga5750]

You are a total artist, you always inspired me to make videos, to upload my music, but in the end I never do more than nothing... Anyways, love your content since Early Sega Genesis Mixes

[@mushroomjuise2349]

it feels like this video isn’t real, as if to understand what is happening you need context that never existed in this world

[@arthasmenethil2201]

This is like braindances on Cyberpunk

[@TheBoysAreBackInTown22]

This completely messed with my head. It’s like that enhance scene in Blade Runner but in real life. I’m not sure if I should be scared or impressed how soon such a thing has become a reality. Edit: I should clarify what I mean by it being scary. It’s more so the fact that archived footage has been used to be turned into a 3D space. It gives it this uncanny feeling that I cannot describe. I am well aware this technology is nothing new. In fact, Lidar has been around for a long ass time. But that still doesn’t stop me from being impressed on what it’s capable of.

[@GSTChannelVEVO]

It doesn't show any data that you don't already have, so it's not truly magic or scary, but it's still really cool imo!

[@umbreonben]

This is incredible holy

[@emanuel3617]

It's actually crazy that we live in a time where noclipping videos is possible soon it will be reality itself

[@blackespectral]

the internet never fails to amaze me

[@drama3587]

Dude this is just like a BD in cyberpunk wtf. My head hurts.

[@brandontylerburt]

Like a Hermetic treatise, this video has much to say in the depths of its poetry which isn't immediately obvious on the surface of its prose.

[@progect3548]

woah thats absolutely sick!

[@AlexGeek]

Didn't know what to expect from the title, but I got a beautiful and interesting video.

[@turkeyleg72]

4:06 bro entered a Galaxy

[@desu38]

Okay, this is pretty cool use for nerfs

[@icy76z29]

This feels like something out of cyberpunk

[@themyth1990]

Gaussian Splatting is so fun to mess with

[@torusx8564]

open source Gaussian splatting generator? NIIIIIICEEE

[@mel7876]

we got cyberpunk braindancing before gta 6

[@alex15095]

There's new research coming out like WildGaussians, SpotlessSplats, etc which are more robust to people in the scenes

[@GSTChannelVEVO]

ough that's cool. i haven't really been keeping up but i love to see it

[@therealgretzki]

you reminded me of the cool old clip called CHOP CUP with parallax perspective illusions. very cool stuff with mp4!

[@user-og7tm4me2n]

this is the first time ive seen a youtube sponsor lol

[@onethreeseventhree]

gaussian splatting is like the RTX of lidar scanning

[@channingcheese2]

That one SCP article about the anomalous DVD of "The Sopranos"

[@basiliotornado]

Love how i (think i) know how you did this!! Gives me a feeling of superiority 😁

[@user9999-z]

thats not good

[@quantumblauthor7300]

Isn't this black-box stuff???

[@basiliotornado]

@@quantumblauthor7300 What do you mean by black-box? The software to do this is freely available

[@hhhsp951]

can't wait for desinc to copy this

[@orangnejones]

Now I REALLY want some kind of game where you can scroll through and explore someone else's memories in this sort of style

[@RafiRo1]

VR + this = u are inside of a mp4

[@emjanti2258]

im not high enough for this

[@simulatedrealitycheck]

I learned about paid promotions unexpectedly

[@Odd-Bike]

scrumptious

[@ukigaminghd1393]

This is actually mad impressive. Wow.

[@smolbrendan5978]

this video scares me. deeply.

[@ZinTDT]

Never tought id see someone noclip into videos. I want to do it myself lol

[@acdbrn2000]

So I might be off with this thought, but is this substituting the idea that a video is images over time and changing it to images over area. Stitching together the frames like how a panoramic shot works. So moving objects/people would cause blurring and other artifacts along with any issues caused by not having a stable frame of reference.

[@C_Corpze]

I love how this looks, this is such a neat use for gaussean splatting.

[@carlomazzoli]

This video is crazy but even the KZbin sponsor?? What? 😭

[@siniarskimar]

0:55 nah my brain just bluescreened at this exact moment

[@SynthesisOnee]

So this is the stuff I saw with that one guy noclipping into mirrors and weird reflective stuff.

[@ammarmeeps4139]

The music at 4:02 is so different from what I generally hear, that it almost seems otherworldly, though I dont think thats the right word for it. Reminds me of the Vintage storys OST

[@0KylePrior0]

This video features either NeRFs or gaussian splatting. I recommend checking out Two Minute Papers for brief explanations. It's pretty neat stuff! The reason why things get blurry in video footage when there are people is because the people are usually walking around. It's hard to quantify what a moving object should look like when it's in a different position and place from to frame. Current techniques are really only meant for static imagery instead of video footage with people moving around.

[@sihtam7612]

Impressive tech!

[@gaebreeuhl]

oh dude i remember seeing this from corridor crew! thank god for the guy in the comments reminding us that it's gaussian splatting

[@twisstedtoast]

How does one get sponsored by KZbin ITSELF

[@priwncess]

oh man such a dope idea

[@NoVIcE_Source]

this is amazing

[@YINHE2.0]

its like a beta version of braindance from cyberpunk

[@frokfrdk]

Damn that's pretty cool

[@batlin]

Did you just pull a "zoom and enhance" on us? Yo this isn't Bladerunner man!