Bicubic Interpolation - Computerphile

Scaling images is usually smoother using bicubic interpolation. Dr Mike Pound explains why.
More on bicubic: bit.ly/Computerphile_bicubic1
Wikipedia bicubic article: bit.ly/computerphile_bicubic_wiki
Resizing Images (Bilinear & Nearest Neighbour): • Resizing Images - Comp...
Slow Loris Attack: • Slow Loris Attack - Co...
Quantum Computing: • Quantum Computing 'Mag...
Babbage's Analytical Engine: • Babbage's Analytical E...
Sorting Secret: • Sorting Secret - Compu...
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com

[@whuzzzup]

We need a video on the "CSI: Crime Scene Investigation" interpolation.

[@ezadviper]

Enhance !

[@Madsy9]

Pan! Uncrop!

[@aajjeee]

look behind the camera

[@frollard]

Can you get a face from that reflection in the coffee mug?

[@dealloc]

There is a project on GitHub (alexjc/neural-enhance), which uses neural network to 'enhance' images; it's based on the joke itself.

[@chlendimounir2834]

i like this dude

[@displayoff]

he's dank

[@funkysagancat3295]

I like the explanations etc. but I don't like his voice :/ I know that it is something completely personal and that is not going to chance soon, but I don't like it

[@KingfisherTalkingPictures]

Very educated Frodo Baggins

[@DuncanEllis]

thanks for ex-splining that.

[@robinw77]

Duncan Ellis Come on, you can do bezier than that

[@otakuribo]

Duncan Ellis I should've node better

[@reblogo]

You're really ahead of the curve on these puns.

[@oresteszoupanos]

You are all setting a bad ex-sample.

[@nkag545]

Mind if I join you, hermit-e-s?

[@javierbg1995]

Dr Mike Pound is probably the best communicator in the channel. Always happy to see a new video featuring him!

[@vladqoq]

OMG, i just love the way how Dr Mike Pound sounds. Computerphile, please, make more videos with Dr.Mike Pound. HE IS AWESOME.

[@TheArcher92]

I really dig this guy. He's got humour and a clear way of explaining stuff. more please! :)

[@budkin]

I could watch Dr. Mike explain literally anything and find it engaging.

[@KnownNever]

Nothing like a cup of Dr. Mike to start the morning off right... great vid as always!

[@NorbertHarrer]

Ahh, I love it when Spiderman does computer science :-D

[@Chrisymcmb]

lol

[@TimeHarmonic]

I really like the videos with Dr Mike Pound. They are the best

[@lewisb8634]

I agree.

[@rouuuk]

i didnt know his name. but i agree

[@synchronos1]

The name of the interviewee is always in the description text of the video.

[@oresteszoupanos]

He's... sound as a Pound! ^_^

[@hellterminator]

I do like Dr. Pound, but professor Brailsford is better.

[@yandodov]

I like your videos with Mike. Make some more videos with Mike, please. Thanks!

[@ScienceSeance]

What a great teacher. Pint deserved for this dude.

[@8eck]

I'm impressed of how much this man knows. Every time i'm googling some complex stuff explanation, i see his tutorials in the results. :D

[@jiaming5269]

Very well explained, quite enjoyed it!!!

[@TheHardSteppa]

Thank you for this. Now some of the other articles ive read actually make sense!

[@SazLowify]

Thanks for this video, it explained a lot of problems I had with interpolation in the past.

[@lawrencedoliveiro9104]

7:17 If you operate in high-dynamic range formats like EXR, which has floating-point pixel components, you have room to both overshoot and undershoot. Then you can renormalize the image using a filmic response curve or something without clipping.

[@notnullnotvoid]

Even then you can have overshoot past black into *negative* brightness values, which is still a problem.

[@WIImotionmasher]

I'd like to say, that relatively new effect in these videos, where the piece of paper is distorted to look like a top down view, is cool.

[@Soundole]

After watching this video, I now have an explanation for some of the artifacts that appear when I'm resizing pixel-art. Thanks for helping me to better understand the world

[@userou-ig1ze]

keep up the awesome work. this channel and vids are a gem!

[@NewLayer]

Theoretically, should scaling an image down using bicubic interpolation ALWAYS result in a sharper image? I have been told the answer is yes. Resizing something from 2000 pixels wide to 1000 pixels wide, this obviously makes sense to me, but lets say I have a 2001 pixel wide image, and shrink it to 2000 pixels wide. How is that possible without losing quality/sharpness?

[@lawrencedoliveiro9104]

Yes, worth talking about Lanczos. Also worth talking about the general principle of image/signal reconstruction using the sinc function, because all resampling techniques are basically shortcuts to avoid the high computational expense of sinc.

[@alanturingtesla]

Such an awesome video. Exactly enough theory, exactly enough drawing and exactly enough maths :D

[@unekdoud]

Please make a playlist for Dr Mike Pound's Computerphile videos, they're great.

[@SomethingUnreal]

Eventually, this series of interpolation videos will come a full circle back to convolutional neural networks when it reaches the Waifu2x upscaling software.

[@TheMasonX23]

I use Hermite interpolation (x * x * (3 - 2x)) in my games all the time. It eases in and out evenly near 0 and 1 cheaply, and makes blending look so much better in most circumstances

[@britishcuisineasmr4045]

What is also interesting with respect to this technique is that CNN-based superresolution models perfrom *significantly* better if the training input has been downscaled using bicubic interpolation rather than cruder methods.

[@Nautilus1972]

I didn't think I'd be happy to come across a bicubic interpolation video today, but here we are.

[@kelvin.salton]

The best explanation I've ever seen.

[@logicalfundy]

"auto" most likely changes the resizing algorithm being used when sizing up vs sizing down, or when resizing is minimal (1.1 times) vs when it is a large change (5 times or so).

[@MrSkizzzy]

Awesome Video on Image processing, hope we See more in the Future :)

[@DimaDesu]

Great explanation. Super clear! Cheers!

[@AJ-et3vf]

Very nicely well-explained. Thank you!

[@0x8080]

Today at work (I'm a software engineer), a co-worker was talking to another one of my co-workers about animating an element within the software we are working on. I know that some forms of animation use linear interpolation, so I got on Wikipedia and read a little bit about it to brush up on the subject (I usually study things that just come to mind :P). Anyways, as I was studying, I noticed Bi-Cubic Interpolation, something I had never heard of, as I haven't taken a high enough math class to really get into that. It was so damn strange to just get on KZbin and see that you made a video about something that discreet. Freaks me out hahahaha.

[@tomkriek]

Mike Pound videos, instant like

[@lewisb8634]

Yep, same!

[@endreman0727]

Is he "Like Pound" then?

[@GtaRockt]

that's such a cool name

[@lawrencedoliveiro9104]

2:15 Originally, a “spline” was an actual rod made of some flexible material, that a draughtsperson would use to draw curves on paper with a pencil. Then the term was carried over to mathematical operations for doing the same sort of thing.

[@tycoinreno]

one example where interpolation (of any kind) is terrible is resizing barcodes. Those should be resized using nearest-neighbor (sometimes called "none" when selecting interpolation method).

[@DarkAngelEU]

These videos have made digital photography so much more interesting than my five years at the Art Academy, which basically made me turn into a film snob lol

[@skeetabomb]

Computerphile are single-handedly keeping the tractor-feed dot-matrix printer paper manufacturers in business :P

[@neoqueto]

Do a video on the Lanczos filter

[@Samboy_Chips]

Yes! More on video/picture scaling algorithms!

[@styleisaweapon]

you would think that lazy people would only want to downscale

[@bytefu]

swifterik Not exactly, but I'm too lazy to explain anything to you.

[@EconaelGaming]

Then we'll finally know how to pronounce it :D

[@Sjobban112a]

+swifterik, I google stuff like that all the time! :D

[@djdeluxe76]

Explained very well. Thanks!

[@LianParma]

Scaling pixel art and imagens based on 8-bit aesthetics are good examples of situations where nearest neighbor works better than bilinear or bicubic interpolation. The final image should keep the hard edges.

[@G4mm4G0bl1n]

I can see all your pictures covered in Moiré.

[@SeezSantos]

No wonder AE to PR upscaling can take so long when taking a clip from 1080 to 4K. Thank you sir !

[@Schattengewaechs99]

Every single frame has to be converted into a wave and the quantized to get a pixel image again.

[@OneAndOnlyMe]

In Photoshop you can explicitly tell it which algorithm to apply when resizing up or down, so in auto mode the code would use the resizing up/down to determine which algorithms is best for scaling an image up/down.

[@gameoverwehaveeverypixelco1258]

I used to use an old paint program, I found that the option to resample image instead of resizing gave a smoother less jagged edge image, looked smooth as silk. less newspaper dot effect or wavy lines in images made up of visible dots or grain like scans of old photos.

[@TechyBen]

WOW. I need to know this equation. You just told me it...... THANK YOU!

[@Czeckie]

he's a really great educator

[@10100rsn]

For images, I've found that xBRZ scaling looks better and preserves the edges making the result sharper. This is better when you want to scale an image up for print or if you want to vectorize it after scaling. Or if you just wish to increase the resolution without blurring the edges.

[@gizmoguyar]

Photoshop has "Bicubic sharper" and "Bicubic smoother" I don't know what the difference is, but I do know that the "auto" setting uses smoother when upsampling, and sharper when downsampling. In both cases it uses some sort of bicubic algorithm. It also has a plain "Bicubic" setting. Adobe says this is the "most precise", but the slowest.

[@10vid5]

The overshoot of the cubic interpolation is mostly due to the fact that the derivative at a point is taken as the arithmetic mean of the two slopes that unite with the neighboring points: derivative_at_point = ( prev_slope + next_slope ) / 2 This overshoot can be mitigated by averaging the *angles* of those slopes instead of the slopes themselves: derivative_at_point = TAN ( ( ATAN (prev_slope) + ATAN (next_slope) ) / 2 )

[@contingenceBoston]

7:42 I've noticed that some optimized images look as though they've been filtered with an unsharp mask (or something of the sort) on the scale of a pixel or two. Could this unintentional contrast be a paradoxical byproduct of the "overshooting" and "undershooting" which can occur with bicubic interpolation?

[@airraidsiren]

So, I know this is not Numberphile, but there are distinctly different things called "cubic interpolation", and I think it's worth mentioning because in this video Mike is talking generally about cubic spline interpolation where you have a different polynomial between each pair of points. Normally this is done in image editing programs with Hermite splines that join end to end to make a C1 continuously differentiable function. But you can also make piecewise cubic functions that are not continuously differentiable (non-osculatory), which is what is shown on the first page linked to in the description. This is not normally done on images, and I personally wouldn't recommend anyone implementing this in 2D for photos. Cubic Hermite spline interpolation is also flexible, and you can adjust for more smoothness or more overshoot. I mention this because it was asked below what the difference is between Photoshop's bicubic (smoother) and bicubic (sharper), and the answer is the tension parameter of the splines which determines how sharp or gradual your slope approximations are at the sample (control) points.

[@5ilver42]

now this needs to be appended with explaining bicubic smooth and bicubic sharp

[@primostyles]

Knows a lot for a guy with only 10 books lol

[@LoisoPondohva]

Plasmaboo or these are only books he didn't have time to read yet.

[@MrCOPYPASTE]

I think he uses bicubic interpolation of those books contents, then uses some permutation, combinatory and use case statistics to refine his response further down .

[@RolopIsHere]

Those are the books that he is reading that day.

[@lawrencedoliveiro9104]

I have that C++ book. Not as useful as the actual language reference.

[@lawrencedoliveiro9104]

“WPF In C# 2010” ... I sincerely hope that is not being used much any more ...

[@AlexanderKrivacsSchrder]

In addition to the ones dealt with so far, GIMP has a resize mode called "Sinc(Lanczos3)." Any plans for explaining that one?

It's a signal-processing type approach. One way of thinking about it, might be like this: whereas cubic interpolation fits a third-degree polynomial curve by using the 4 nearest pixels, Lanczos interpolation is more like fitting a Fourier series to a much larger area (ideally infinite, but in practice not). That's not what actually happens (it's actually using a "Lanczos kernel", which has an effect similar to attaching a Sinc-windowed Sinc wavelet (yes, two sincs) to each source pixel, and then summing up all the wavelets at each destination point), but at least to me it's simpler to think of it that way. So it's even smoother than bicubic, but may very well cause overshoot and sharp-edge artifacts worse than bicubic.

[@AlexanderKrivacsSchrder]

@ I appreciate you taking the time to explain this to me. Unfortunately, a lot of those words carry no meaning for me with my current knowledge, and so I'd presumably have to do a lot of additional reading in order to fully understand your explanation. When asking about an explanation for the mode, I was hoping for more of a visual explanation in the style of these videos since I find those a lot easier to, if not necessarily *understand* what's going on, at least quickly get a sense of what's going on.

@@AlexanderKrivacsSchrder Yeah, I was trying to say a couple of things: 1) A video explaining the mathematics behind Lanczos interpolation in a simple way does not seem very likely to me, and 2) if you don't care about the mathematics, and don't care about the mathematical difference between a polynomial curve and a Fourier curve, then you probably don't even need such a video. It works just like a bicubic curve, it just uses many more points at a time, and tries to fit a smooth curve over them using a more complicated formula based on sums of sine curves. Beyond that, you can think of them as pretty much the same. As mentioned, both methods even handle sharp edges somewhat poorly. But if you are interested in some mathematics, like the sums-of-sine-curve aspect, then maybe check out a video or two on wavelets. Alternatively, there's a Mike Pound video on JPEG DCT. You can watch it, then imagine that to scale something, the source image is first converted with such a DCT (Discrete Cosine Transform). Sine/cosine curves are smooth, after all. Then, once you have the DCT matrix, you now consider the cosine curves to be continuous, so that they can be sampled at the in-between points when you create the scaled image. That should give a reasonable idea of the principle, I think.

[@georgikostov6982]

Site on bicubic is great ! Tnx :)

[@vishu226]

awesome!

[@starkgate]

Very interesting video, thanks ! Can we get a video on other scaling algorithms ? (eg the ones that rely on fractals)

[@ButzPunk]

Can you do a video about reticulating splines?

[@user-xn4lw5ux2i]

What about sinc- and Lanczos-Interpolation? As far as I know, the latter is used to reduce the ringing artefacts.

[@postvideo97]

maybe also talk about xBR, HQ and Sparse Coding methods for upscaling too?

[@drunkensailor775]

also neural network real time scalers, NNDI3 and other very interesting things about video rendering of today

[@postvideo97]

oh boy I could talk about ANNets all day.

[@airraidsiren]

When people talk of sinc interpolation w.r.t. images, they usually mean a Lanczos type windowed sinc interpolation. It is very smooth, which works well for some images, but the tradeoffs are the much large kernel size (thus slower computation) and also the fact that optimally smooth is not always perceptually the best result. A bit of overshoot gives increased acutance, which can make the image look "sharper" or crisper. This is often desirable, especially when downsizing. Also, FWIW, you can do bicubic Hermite spline interpolation without ringing, or with lots of ringing, to taste, depending on how you calculate your slopes. It's fully adjustable via the "tension" parameter of cardinal splines.

[@George70220]

​@@airraidsiren how did you learn all this? I'd love to dive

[@cubicinfinity2]

Does it make a difference what order you interpolate in: Horizontal then vertical, rather than vertical then horizontal?

[@ericsbuds]

LIKED! ok time to watch the video now :D

[@bruno_sjc_]

this guy rocks!

[@DaVince21]

I just realized that 'reticulating splines' basically means that Sim City is busy determining the splines between each tile to form natural hills and so on.

[@ppVon]

Is it possible to create a program that mixes the methods on a case to case basis? I think that would be interesting to see.

[@j7ndominica051]

Trusting the "auto" preset isn't always the best thing. Some versions of photoshop made a "sharper" version of bicubic the default, which gave horible ringing halos. They have probably improved the defaults, but in that version (I believe it was 8.0) the sharper had absolutely no application. If you were to clip the range then you wouldn't actually observe the overshoots, and have no problem. They do show up often on screenshots or textual captions or dark trees against light sky.

[@mawest]

legend

[@RABWA333]

thanks, was very helpful , I have data in form of (time stamp), these that collected by RFID sensors carried by persons walking through the town , now , i want to connect the routes which the people take using the time stamp i have . but i found some missed reading (some RFID tags did not work). How i can complete these missed reading ? which method or algorithm should i use? any suggestion

[@loloioi]

Wish I watched this before doing my quiz a few months ago on cubic spline interpolation... :(

[@geeky_explorer9105]

U are awesome thanks a lot

[@willhdq]

Can you tell me what is Cardinal Interpolation please? Would it be possible to make a video on it?

[@thomassynths]

I'm always amazed when I see people draw X's by drawing two curves butted against one another (as opposed to drawing two intersecting lines). To each his own.

[@rebeccachoice]

Thomas Eding quite common when writing mathematical formulae: it's to differentiate the cross product × from the algebraic variable x (which often appears in this curved style in italics)

[@thomassynths]

Interesting. Thanks for the replies.

[@fergochan]

It's also pretty much how you write a cursive x, without taking your pen off the page. It's how I was taught it in primary school (in Australia).

[@dume85]

that's weird why wouldn't they just say the gradient at the max and min (256 and 0) are 0 like they teach everyone in calc one? maybe it takes too much computing power to override the linear estimate of the gradient. Now i'm really curious what it would look like if you went way overboard with this and started estimating inflection points ect and a bunch of other stuff like that.

[@SharonMau]

Aloha . . thank you!

[@Ziggletooth]

I need some help. Whats the best way to scale down a raster image? I have some images I want to send to print but I'm concerned it may be too pixelated. Is there a better, sharper way than just using the standard scale function in gimp?

[@edwarddoernberg3428]

are there any systems designed to work with high contrast edges between regions of limited variability. the most obvious example being text. I am thinking first detect lines between regions, create a function to define the line including identification of sharp corners. scale up the region boundary lines so they remain crisp then scale up each region without considering any information from other regions.

[@WWayneB]

Is there a technique to tell how inaccurate the algorithms are and compare them? For example with bilinear and bicubic magnify by 2.3 and then shrink with the algorithm again to get the same original image size. Are there standard ways to give a value showing which one on the "round trips" is closer to the original and by how much with a quantitative analysis?

[@airraidsiren]

Yes. Do the process you describe, and then diff the result of the round-trip interpolated version with the original, pixel by pixel. The difference should be small if you enlarge then shrink, but it will obviously be considerable if you do it the other way round. In Photoshop, you can do this difference by putting the two images in separate layers and setting the top one to blend type "difference" (which is the absolute value of their subtraction). Because the resulting differences are small, you'll need to either merge the layers and normalize (auto levels), or use a level adjustment layer on top of them to see it clearly.

[@catlolis]

IM PINCHING AND ZOOMING

[@gavindownes4541]

half way from X1 and X2 it will have a greater value than either one. Wouldnt this produce bright spots ?

[@johndue2366]

Paper from a line printer, - that takes me back :-)

[@jezzbanger]

I'm aware you'd get diminishing returns but couldn't you do a bi-(nth degree polynomial)-ic interpolation for any odd n using progressively higher derivatives, capped at a point so you don't use too much computational resources?

[@Superphilipp]

Oooh, somebody got a fancy new high framerate camera.

[@joerivde]

Binge watching this

[@glenneric1]

I would differ on his statement that nearest neighbor(and only nearest neighbor) doesn't create information. All three types(nearest neighbor, bilinear, bicubic) guess at what the middle values are. It's just that nearest neighbor is a very simplistic guess. So they all create data without truly discovering new information.

[@Tahgtahv]

Is there a way to do this while keeping sharp edges sharp? Imagine you have an area of black meeting an area of white. Is there a way to scale this with smooth edges without introducing intermediate color? I don't think this is exactly nearest neighber. The best I can think of, and it sounds complicated, is vectorizing the image (somehow), scaling it, and then re-rasterizing it again. I'd imagine something that did this would take some sort of threshold. Colors closer together than the threshold would be scaled together using something like bicubic, but further apart would be considered essentially seperate images sharing a (possibly curved) seam.

[@natfailsyoutube8163]

There are other algorithms... maybe you want to look at algorithms intended to upscale pixel art (maybe see Wikipedia article "Pixel art scaling algorithms" which describes several). Understand though that whatever the algorithm you create pixels that aren't in the source data, the result is never going to be "perfect".

[@Kaepsele337]

I think it should work if you create two images, one by nearest neighbor, the other by bicubic, then combine them, where you weigh each pixel by the output of an edge detector.

[@RealMcDudu]

thanks. Quick question - why do you use these old 80s printers pages as drafts ?

[@BEP0]

Nice.

[@rrajpuro]

[5:50] - Why is there an interpolation of Spatial coordinates? Isn't it supposed to be an interpolation of intensity values

[@redstonechoi]

thanks

[@runhigh123]

Is it possible to use BiCubic as default, and use BiLinear only at those sharp step changes??

[@MikeMcCollister]

I'm wondering if there are interpolation techniques that will vary throughout the image. For example, most of the image might be best suited for bi-cubic interpolation but portions might be better for bi-linear interpolation.

[@fluffycarrot42]

i was wondering that aswell

[@florianh.6256]

You can always use more calculation-effort and try to predict better. For example: You could set some threshhold and look further. If the values are within the threshold you would use more points to define the gradient -> Essentially weighting the curve to respect larger areas more. Or more visual: Use the example at 2:00. The 4th point is on the same level as 2 and 3. This meets your threshold and your gradient would be more horizontally - essentially having the resulting curve be steeper and farther out on the right side. This probably creates more problems with "overshooting", thugh you can try to counteract this, because you know that you were within the threshold and act accordingly.

[@airraidsiren]

There are hybrid algorithms that are more sophisticated and do things like treating hard edges differently from gradual gradients, but I've never seen one that includes bilinear in the mix. Bilinear is most often a comprise that's made for quality vs processing time (like for realtime interpolation when bicubic is too costly for the hardware to do). For typical use cases on photos and video, bilinear is rarely the most appealing option visually.

[@Mister1Slime]

Nice

[@C-Swede]

I love that he’s using dot matrix printer paper

[@salmamoora6048]

Hi sir, I want to ask you about bicubic convolution interpolation in image to addopt on pixel estimation on wide missing pixels. Is it possible to use 8x8 neighbour pixels to perform bicubic convolution interpolation? How about the kernel and the interpolation formula?

[@puellanivis]

With the number of people taking screenshots of text, then pasting it in, and saving it as a jpeg, I'm surprised that you didn't cover this situation. Like, when the text of something has become so ridiculously blurry and spread out, that it starts to have sharp edges tens of pixels away... Compounding error. :P

[@michaelhaddad2190]

But how do you reduce an image size? How you decide what is NOT important and can be neglected?

[@wilsonc2250]

How about bi-quadratic interpolation? It is a better way to do image scaling?

[@synchronos1]

I believe Photoshop always uses bicubic resizing algorithm with the Auto setting. The Auto just picks Bicubic Sharper when shrinking the image size and Bicubic Smoother when enlarging.

[@AlphaYellow]

Yup exactly

[@aolson5795]

Does image rotation work any differently from rectangular resizing? Because if you rotate some number of degrees, some places may move less than one pixel's length, and not in a straight line...

[@airraidsiren]

Bitmap rotation uses the interpolation methods discussed in these videos, just as for rectangular resizing. You're still mapping each pixel of your result "target" image back to where its values come from in the source image, you're just replacing the x and y scaling with a rotation matrix. You transform your target pixel coordinates back to where they source from, and because these are non-integer values for all but the trivial case of 90 degree rotations, you use one of the interpolation methods to get your final value. Nearest neighbor is pretty unsightly, with a grainy look and jaggy edges, but bilinear and bicubic look nice and give the "anti-aliased" edge look to rotated lines.

[@Mobin92]

Shouldn't that visualization at 5:52 be 3-dimensional? Otherwise it seems kinda confusing to me.