How to get nearest neighbor/billinear HYBRID scaling in AE - and why we need it!

On the one hand, this is super nerdy and niche.
On the other hand, it seems like all major NLEs should have already had this ages ago.
Update: Apparently, this is usually called "sharp bilinear."
AEP file: www.dropbox.com/s/27aumftdqa2...
Image it references: www.dropbox.com/s/aog7iql4nwq...
Another image: www.dropbox.com/s/oq5r610e4k7...
Taran's massive list of feature requests and bug reports for creative software:
docs.google.com/spreadsheets/...

[@Camhin1]

Linus is so lucky to have you as an editor!

[@ComicMelon]

So sharp-billinear? This technique is fairly common in pixel-art titles. Sonic Mania is a prime example.

[@DutchmanDavid]

Y'all can vote for this issue here: adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/37024072-nearest-neighbor-scaling-sampling-simple-and-v

[@TaranVH]

Maybe. Is that the real name for this?

[@ComicMelon]

@@TaranVH I've seen it called many things, that just seems to be the most popular term.

[@NicolasHanna]

I don't understand or going to use this but there is just something so interesting about whatever Taren talks about!!

[@babylonian]

found my way here from that (super frustrating) Adobe forum thread -- thanks so much for this, Taran!

[@JohnPaulBuce]

hi nick

[@JuhoSprite]

Sup, ur a great creator I love ur vids

[@TaranVH]

Go vote for this feature request! adobe-video.uservoice.com/forums/911233-premiere-pro/suggestions/37024072-nearest-neighbor-scaling-sampling-simple-and-v

[@livinlicious]

Why do I wander down these rabbitholes. Macros, Color, Scaling. Obsession.

[@TaranVH]

Just wait until I release my video on color profiles. COLOR PROFILES. They are HORRIFYING

[@RamkrishanYT]

Taran is slowly becoming the Louis Russman of Adobe products

[@cranknlesdesires]

Taran just wants all this for his up and coming Terraria lets play

[@goeiecool9999]

Taran you must be such a headache for the adobe developers lol.

[@simplywhy]

Taran you're a life saver for all, I've learnt so much from you. Thanks a lot, you're the best.

[@thogameskanaal]

This is actually perfect. Not just for video editing, but also non-integer stretching on emulators, so the screen can be 4:3 without it looking distorted.

[@EposVox]

TIL After Effects has Nearest Neighbor at all... I don't know AE well.

[@hlfan]

Your "fancy math" breaks under 100 percent/shouldn't you scale down bilinear? If you do the following: slider = *_SLIDER_* /100; deduct = 100*Math.ceil(slider); answer = 100*slider/Math.ceil(slider); …it works with a slider in percent and avoids the infinite values under 100

[@TaranVH]

Sooooooo, I tried this, and it doesn't work. Can you give me the entire code sample? Here's what I have: yeah = comp("Fancy_method").layer("Null 3").effect("Slider Control")("Slider"); deduct = yeah/100; deduct = Math.floor(deduct); deduct = deduct*100; answer = yeah-deduct; answer = ((answer/deduct)*100)+100; [answer,answer]; ---- And here's the code on the nested object: temp = comp("Fancy_method").layer("Null 3").effect("Slider Control")("Slider"); temp = temp/100; temp = Math.floor(temp); temp = temp*100; [temp, temp];

[@hlfan]

​@@TaranVH I have to test this myself back at home tomorrow but it should go like: yeah = comp("Fancy_method").layer("Null 3").effect("Slider Control")("Slider"); answer = yeah/100 answer = Math.ceil(answer); deduct = answer*100; answer = yeah/answer; [answer,answer]; and *swap out* the Math.floor for a Math.ceil on the nested object. That should make objects scalable even below 100%. > shouldn't you scale *down* bilinear?

[@TaranVH]

@@hlfan woah I hadn't even thought of that. Yeah, scaling DOWN seems like it would be slightly superior. Good thinking.

[@hlfan]

@@TaranVH I just discovered that you'll need a bigger subcomp for scaling down **

[@Akselmoi]

I'm not video editor but this is still really fascinating.

[@simplywhy]

Linus should give you a raise for making the vids more awesome looking.

[@TaranVH]

Maybe he already has ;)

[@wobblysauce]

@@TaranVH What about another one on top...

[@mr2octavio]

@@TaranVH Congratulations! You deserve it after blowing up Waifu2x

[@Roalethiago]

@@mr2octavio i'm actually using this program af to increase my fotos and remove jpeg compression noise

[@mr2octavio]

@@Roalethiago me too, it's magic software

[@rootdirectory56]

this is my version, if you want the aep then let me know Bilinear scale: var scale = thisComp.layer("Control").effect("Scale")("Slider"); var nearest_scale = comp("Nearest Neighbor").layer("Image").transform.scale[0]; var altered_scale; if (scale

[@JacoxNovak]

You bring me information that I didn't know I wanted and every time I am so happy I clicked on your video! Thanks!

[@jared2b]

I always enjoy Tarans' random videos they teach me something and they are a welcome surprise.

[@kyuuketsuki-san7684]

I am happy to there somebody thinking seriously and create great solutions about this hard problems who isn't a developer.

[@VisualBasic6]

Taran-Van-Hybrid

[@Napert]

never been so early that the youtube throws out an error when trying to watch this and allow me to finally watch this when it's done processing to 1080p

[@wobblysauce]

watched at x2 480p.

[@OrangeC7]

Taran, just coin the unit of measurement "ppp" (pixels per pixel) already.

[@danwat1234]

Nice work on the 2019 Gigabyte Aero 15x video port-tiltage

[@prototk5186]

"I know I ask for a lot" * shows giant spreadsheet of requests* Thought that was great

[@galdocstutorials]

I use AE for a completely different purpose -- I did a bunch of animations for a Doom mod (Project Brutality). So, something like this would be great, because it deals with highly pixelated stuff getting scaled and rotated a bunch. Thanks for this!

[@zahidshabir4038]

how the freak am I not subbed to you (the newest video I have seen before coming back to this channel after accidentally finding it was the Letter to Corsair video) I have known of your channel since like 4 or 5 years ago and actually remember subbing to you I probably got unsubbed by youtube in a glitch or something as you have added so much videos and I have missed them all

[@cat-boy1357]

Taran: makes a video. Adobe devs: Ah s***, here we go again.

[@SaltyMaud]

Yup. This is Taran alright. Never stop being Taran.

[@HyperDash]

KZbin needs some better scaling for watching videos 38 seconds after the upload when there's only 360p to go off of.

[@drone_video9849]

What I don't understand is that when you upload in 4k, the first and only video available is 360... why isn't is the 4k that you uploaded available right away?

[@TaranVH]

@@drone_video9849 youTube re-compresses it using v9 to save space. It takes time to do.

[@drone_video9849]

Taran Van Hemert that makes sense. Why isn’t premier within its KZbin settings compressing to VP9 then? Saves everyone some time ?

[@ItWasEnder]

@@drone_video9849 Because youtube would literally re-compress it after you upload it lol.

[@drone_video9849]

@@ItWasEnder its likely. It would make sense to encode to VP9 4K or 1080p and upload that. I'm waiting on the h264 encode anyways, I wonder if the vp9 encode is really slow when done locally?

[@tychowerner]

You tell me problems I didn't know I had.

[@AyushBakshi]

Better wait for CC 2025.

[@troyy99]

i really need this for my pixel animations!

[@XOIIOXOIIO]

This week on Taran's mental breakdown: Scaling! Also I feel like I should start trying after effects instead of sony vegas so I can get some useful info from these videos, just sucks to change after so long and being used to how a ui works.

[@Fry981]

Quick heads up: After Effects is not equivalent to Vegas. Vegas is an editing software while After Effects is for compositing and motion graphics. Trying to edit a video in AAE would be a suicide. If you really wanna switch your editing software then what you're looking for is Adobe Premiere. Or... you could just stick with Vegas as your editing tool and combine it with After Effects for compositing. Admittedly, that production pipeline isn't as straight forward as it is in combo with Premiere but from personal experience I know it's doable.

[@beachmobjellies]

Not just for pixel art. This actually is important for 3D compositing (depth pass and motion vectors must not be anti-aliased). I end up setting the layer quality to "draft" when i have to scale as a workaround. requesting features at adobe is like talking to a wall....a very very thick wall....with no one behind, they are all out to party cause they earn more money with their omniture purchase big data shit

[@Maks-ji7de]

I bet after effects devs will implement it using this exact piece of code

[@aaaa9r]

This level of insanity blurs the line of "Advanced technical stuff" and memes dangerously. All to just move, resize and rotate screenshots like if it was million dollar fine art. Think about it. And I've been watching all of this stuff, I mean color correction; upscaling; the bugged e-bike while going to the supermarket, religiously. Taran, you're fucking crazy and we love that. Just eat and sleep properly, please. We want more Taran.

[@TaranVH]

Working out, (and eating and sleeping properly) is actually the first chapter of the upcoming part 2 of the 4hr tutorial

[@aaaa9r]

That sounds great! I would like to hear general_life opinions and advices from you. I'll stay tuned

[@robertreitze3192]

For the final showcase of the zoom and transform you could have added comparisons to current capability.

[@GreenBoxMedia]

I am not 100% sure about the technical side but "RedGiant Instant 4k" could be something you want to look into.

[@feha92]

I prefer the left version at 1:10 as even though the lines end up looking wobbly, its better than the blur on the right imo. Aside from being blurry even when paused, the right also feels sort of like the camera is shaking really small really fast.

[@dylzzgaming4423]

Finally i am early to a video.

[@foodice11]

Basically how it should look in the ui, is have nearest neighbor enable an additional dropdown called remainder that lets you change how the partials look.

[@AxisAngles]

Antialiased nearest neighbor. Pixel integration!

[@TheA_Gamer554]

they added hybrid scaling to Obs Studio. It's called Angle filtering i believe

[@Atnas1010]

Would it be helpful if there was a plugin that you gave position rotation and scale as input, and it transformed the layer correctly?

[@ReikazeRambles]

personally I prefer the look of nearest neighbor over your hybrid with either the rotation and the zooming but that's a preference I guess. It's more faithful to the original image. But the hybrid seems pretty flexible and probably more aesthetically pleasing for most people.

[@zyad48]

I think it's also a matter of scaling lower res pixelart _up_ where this mix of bilinear and nearest neighbor will help the most and look the least crappy, the original scaling he showed with terraria is working with a pretty low resolution, so it ends up looking a little jank, but when you're scaling up for a 1080p youtube video, it should look a decent amount better.

[@SkyyySi]

Basicly you want to scale the material a lot mor then you need, then scale it down to what you want need.

[@JohnPaulBuce]

cool stuff

[@anfanger4]

OMG OMG OMG thanx! Developers pleeeease!

[@theslimbin]

you should look into the Hermite filter it seems to preserve pixels in cemu, but that is the only place I've seen that

[@AaronTheGerman]

Is there some sort of anti-aliasing effect in AE you could apply to the footage after the nearest neighbor filtering? I don't have After Effects, but I do spend a lot of time with graphics programming and what would be a straight forward way to solve the problem

[@TaranVH]

Proper anti-aliasing is based upon knowing where the underlying line actually is... which is essentially what I'm doing by using bilinear like this.

[@MatheusPratta]

So basically you use nearest-neighbor scaling to get to the closest size to your target (either by dividing or multiplying by integer numbers), making sure the value is either exactly the target or a little higher, then you use bilinear (maybe bicubic?) scaling to scale down to your target size, since you still get to keep details and it will fit perfectly.

[@zombiedude347]

This sharp-bilinear scaling looks just like the scaling algorithm I was thinking of programming for myself. I just didn't know the name of it. The first step was to nearest-neighbor upscale the image to the least common multiple of the input and output sizes. The second step is to downscale to the output size, by averaging the pixels, optionally using alternate color spaces.

[@TaranVH]

you are correct. I actually did it wrong by using bilinear for the remaining upscale, when I believe I shoud only ever upscale with nearest neighbor, and DOWNscale with bilinear from there.

[@GaviLazan]

No one tell Taran about bilinear color space.

[@FandomEZ]

Is there any reason to use different scalling menthods outside of sprite art? Thanks

[@Werdna12345]

he also showed screenshot examples

[@rootdirectory56]

zooming in on screen capture looks wayyy better with hybrid scaling

[@Thespikedballofdoom]

I have tried so hard to find someone talking about this

[@stellated]

I've never used it but Inkscape (and maybe Illustrator too) can trace a pixel art to a vector. Edit: In Inkscape use the normal Trace Bitmap option because Trace Pixel Art is either wack or I'm just stupid. Edit2: Yeah, I'm just stupid.

[@asphaltwhale]

Well, CorelDraw does it pretty good, but we are talking bout Adobe products so..

[@PixlRainbow]

The trace pixel art option is intended for unscaled pixel art, with pixels that are actually 1:1. It also requires that you tell it to actually expect square pixels, because it tries to curve the pixels by default for some reason as if it was some sort of HQ2X filter.

[@UnPuntoCircular]

This would obviously be a workaround, but couldn't you pre-scale your screencapture in Photoshop to the max size you're intending to zoom in (with nearest neighbor), and then just zoom in and out in AfterEffects?... The only issue, as you said, would be the huge size of the image file if you're intending to zoom-in until a pixel is 400px wide.

[@TaranVH]

That is precisely the issue. Anthony actually did this for some hour-long screencapture, and all it did was make the file 4x larger, and 4x laggier to scrub through.

[@TaranVH]

And I do pre scale static images in photoshop all the time. So much, I have it assigned to F12 as an action. The problem is when you want to do it for footage.

[@UnPuntoCircular]

Yeah, *and* it wouldn't work for video, unless you're willing to make a png sequence of pre scaled images, which would be hell. In the past, I've done what I call "fake zoom". Not for nearest neighbor upscaling, but to do huge zoomings (like 1000% or more) and what I do is to basically re-create just the final zoomed screen and put it on the layer above, so I end up with a sort vector-zooming effect of a raster image. Motion blur usually hide the fakeness of it all.

[@yousefslimani99]

I Have a Question can we remove video Bilinear and change it to nearest Neighbor While during Output the Video on Adobe Premiere Pro??

[@DiminuendoBlue]

Dumb question but couldn't you use the mosaic effect if you know the pixel size and resolution of the video/image?

[@FunctionGermany]

What infuriates me by far the most about this is that for all these years, AE never exported what i *actually* set in my composition - by default. I don't remember any time if has this as a visible issue, but with industry standard software i expect my program to be WYSIWYG. >:( genuinely mad about this

[@TaranVH]

Welcome to the club. This issue is not just with Adobe software, but ALL software. You never REALLY know what it's doing under the hood. This is why you test things one small component at a time. Learn all the most basic, fundamental things you can. Often, you will realize that you need to do this when things just aren't adding up or making sense, in odd situations, and you don't know why. Just be glad that NOW, you DO know this new thing. This is why I had to go down the rabbit hole of digital color for several months... I emerged out the other side with a deep understanding of how and why things work the way they do. And now I can say with certainty when one should and should NOT be using color management. It's a real gotcha if you don't understand how it all works. Gonna make a video about that. We at LMG have been improperly calibrating our monitors for years.

[@aure_eti]

I'm sure i will one day understant this thing :) and what's going on here

[@jurisdumagan4138]

Hey Taran! by any chance can u make a tutorial on how to make your own macro keyboard (if its possible)

[@TaranVH]

i did. search for "taran HASU usb"

[@SuperDerek]

Video topic is Nearest Neighbor Scaling? You can count on me being here for support and thumbs-ups. This needs to happen, Adobe. And give it to me in Premiere Pro as well, while you're at it.

[@tutorialshop7670]

Thats good

[@MysteryFinery]

why isnt this in premiere >:(( damn, adobe needs to do something already.

[@SrNibb]

if anyone still comes around here and need stuff for pixel art I found Autokroma's Visual PixelPerfect which might be useful for some peeps

[@galaxybiscuit]

Nearest neighbour looks the best at rest. - Jonas D 2020

[@munhib7360]

Hi Taran, how can I make a programmable keyboard for OBS studio without sacrificing letters (ex. I want to use my keyboard to type and do hotkeys at the same time without triggering hotkeys when I want to type) hope this makes sense.

[@TaranVH]

It does not make sense. Are you talking about making a 2nd keyboard? Or adding macros to your single keyboard?

[@munhib7360]

Making a second macro keyboard for OBS

[@TaranVH]

@@munhib7360 Well, there are several methods. I currently use QMK with AHK. The 2nd keyboard doesn't trigger normal keystrokes, because AHK will block them. kzbin.info/aero/PLH1gH0v9E3ruYrNyRbHhDe6XDfw4sZdZr

[@munhib7360]

Thanks so much. Your help is well appreciated!

[@KeithYipKW]

Try upscale the image nearest to neighbours to a very large size than you need, e.g. 4x first. Rasterize, then bilinear downscale and rotate to whatever you need.

[@TaranVH]

Did you watch the video? This is what I do. But it only works for images, not video.

[@KeithYipKW]

@@TaranVH Yes but I guess I wasn't paying attention. I watch it again and find that you show it in the video. You may convince adobe to implement it by having many of your viewers to upvote the feature. I do not use after effect and premiere pro so I do not understand why it does not work for videos. What is the problem of prescaling a video first? Is it sometimes too large to import just like an image?

[@HerbaMachina]

Damn never noticed this lol, for real tho why isn't this a default scaling option tho.

[@davidjohansson1416]

Integer scaling nvda driver related to this?

[@iuoful]

Hi Taran because i didnt find a direct way to contact you i try this way. Thanks to your videos i now also use ahk at my workplace. My Questions ist how many of your coworker use it? In many videos of ltt i only spot you using mutiple keyboards.

[@TaranVH]

I'm the only one to use AHK so extensively. Some of my co workers have asked for a script or two, which I have set up for them. To my knowledge, they don't write their own scripts.

[@iuoful]

@@TaranVH thank you very much for the answer

[@World_Theory]

I believe most strongly, that the scaling method you are looking for is pixel-mixing. Here is a page that does an awesome job at describing it: entropymine.com/imageworsener/pixelmixing/

[@linawhatevs8389]

8:44 deduct = deduct;

[@Brenna_stubbs]

You should go to Adobe weave Linus and have a look around

[@gudenau]

When will someone make a worthy FOSS version of this stuff? 🤔

[@dezg5791]

If only there were more Tarans to support the complicated Foss project AE would be.

[@gudenau]

@@dezg5791 I could probably make a video editor base at least. Wouldn't be very good though.

[@AkagamiStreams]

Are you still with Linus Tech..I have not seen you in those videos for long time

[@tverdyznaqs]

If I needed to animate a zoom into a pixilated image I'd just do nearest neighbor and mask it with heavy motion blur. But I understand that this is not the way of our video-editing lord and savior the Taran

[@drgabi18]

destroy hdds by recording stuff in uncompressed avi and i believe if you scale with only integers it will not subtract pixels but hey that's just a theory

[@LaskyLabs]

Shimmer is not just a trouble for you... Anyone who has to scale old games into higher resolutions knows the pains of this. You want sharp pixels, but then things move and it starts to shimmer. It's a mess.

[@Ghennesph]

This aught to be the default for scaling tbqh.

[@hatdog703]

Hey review the rk825 keyboard please

[@hatdog703]

I know that you might like it

[@s8wc3]

I bet you they'll just take what you've made and just make it an animation preset

[@ABaumstumpf]

Uhm.... you already explained the solution why this is not needed at all. If you want pixel-perfect scaling then simply have the source at the needed higher resolution and just downscale it as needed. Good old 16x16 sprite, scale it up to say 256x256 or whatever you need with nearest neighbour and then you can scale it down to whatever size you want. And the little slider you made - yes, that is what those features are for.

[@TaranVH]

you weren't paying attention. That doesn't work for large images, like a 4K screencapture. It'd be enormous - 38400 by 21600.

[@ABaumstumpf]

@@TaranVH "you weren't paying attention." - i did..... you weren't paying attention to what i said.

[@regeman100]

i dont even use ae, why am i watching this

[@DanLMH]

Adobe Devs to busy working on other things lol

[@SRoyTheComfy]

360p? Not 4k?

[@TaranVH]

one of these days I'll upload a video that's ACTUALLY 360p

[@SRoyTheComfy]

@@TaranVH and I'll watch that on my 4k tv :)

[@telotawa]

yeah=yeah+100

[@TaranVH]

yeah

[@DemiImp]

I feel like you don't understand what is going on. These are SAMPLING a texture. What you are asking is for it to scale the texture it's sampling from before it samples it for the transformation... which means that it's not a pretty big deviation from how sampling formulas work. A sampling formula uses a source texture, 4 source pixels, and a math formula describing how to weight each of the 4 source pixels to determine what the output color is. What you are describing can be either accomplished by scaling the source texture... or by adding an additional math formula to determine what the 4 source pixels are. And I don't think they are going to add additional formula/parameters to the sampling functions to make this happen because of how nonstandard that would be.

[@TaranVH]

I feel that YOU are the one who doesn't understand what's going on. Nearest Neighbor is not "nonstandard." It's literally the least processor-intensive, easiest, laziest possible way to upscale or downscale anything. "Just use the exact same pixels that are already there." It's the default, lowest common denominator. It's what we used back when computers couldn't handle anything else. And it still makes computer graphics and pixel art look great.

[@DemiImp]

@@TaranVH It already has nearest neighbor. He showed it in the video. He is asking for something more complex than nearest neighbor.

[@vintagemotorsalways1676]

adobe please do it.

[@hotdoge1393]

Are you Dutch

[@Chronophylos]

i c terraria i upvote

[@fffklan3986]

Try waifu shit works

[@gordonou7065]

If you want it to be more smooth, use warp stabilizers.

[@wesleykronmiller390]

oof, 360p

[@ineedanewhobby3669]

Taran it's 11:00 go to sleep

[@kalazakan]

Who goes to sleep at 11? lmao