Throttling: New Developments in Application Performance with CPU Limits - Dave Chiluk, Indeed

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Throttling: New Developments in Application Performance with CPU Limits - Dave Chiluk, Indeed
Are you seeing excessively long tail response times from your applications running on containerized clouds (Kubernetes, Docker, Marathon)? Have you ever seen an application be throttled even though it’s no where near its CPU limit? Up till now, the answer has always been simply turn off hard-limits, but that has potentially nasty performance implications in shared environments. Now there's another option! This session will explain the real root cause of what has been happening. We'll introduce the kernel mechanisms that Kubernetes and other Container Orchestrators rely on to enforce CPU limits. We'll then show how they were broken, how we fixed them, and what those changes mean for you and your clouds. By the end of this session you'll understand exactly what you are getting when you set the CPU limits on your pods.
sched.co/Uae1

[@ThomasBurton-c3y]

Thank you!!!!! CPU throttling disappeared after using integer values for limits

[@Kavinnathcse]

At 4.30 - you have mentioned that minimum drops to 44 when new request comes in. I am still not clear about that. As per my understanding the min should be 87 right? 1000m (milicores) = 1 core

[@JoelSchopp]

Nice find. Great presentation as well.

[@davefar2964]

Thanks a lot for the talk @Dave Chiluk. Your explanation at the beginning makes me think that an application with CPU limit x < 1 will be throttled for most periods (i.e. throttle percent will be almost 100% when requests >> x * period). Correct? Then your suggestion to aim for at most 10% throttle percent can only be achieved with integer CPU limits (or at least close to 1, 2, 3...). Correct?

[@DavidChiluk]

I'm not 100% certain I understand what you are asking, but the way you ask it implies to me that you are misunderstanding. If you are guaranteed to always have requests waiting to be processed you will always be throttled 100% (assuming your code is sufficiently multi-threaded). That's not necessarilly a bad thing. For example if you had a limit of 4 think of it as if you had a 4 cpu node with each CPU running 100%. In that case you could have gone faster if you had 5 or more CPUs but you were throttled by the physical limitation of having 4 CPUs. This CFS mechanism approximates your 4 CPU machine on a 16 CPU node, by allowing your application to use 1/4 of the cpu time available on those 16 CPUs. In both cases you are throttled 100% of the time. The same would be true if you could create a 3.256 CPU machine *(actually this is now kinda a thing with non-homogenous core architectures). Integer vs non-integer limits is a widely misunderstood topic. Kubernetes however does have a guaranteed scheduling policy where if requests = limits = whole value it will pin the application to a subset of CPUs. In reality that may actually slow many of your requests down as they now need to be serialized onto one of those pinned CPUs rather than being serviced immediately by idle cores on large nodes. I hope this helps?

[@shonlev-ran8439]

Why did you set CPU limit in the first place?

[@yaoyuanbill]

Hi, does using the whole quota mean specifying limits of CPU to integer values for a container? Does it make sense to specify request low e.g., 50m, if we know that the application will not use CPU for more than 5ms in a 100ms period? So we end up with 1 CPU limit and 50m CPU request, is this a reasonable setting?

[@srikiraju]

This guy has seen some shit, battle scars man