Deploy DeepSeek-R1 Models on Kubernetes with Amazon EKS

[@CarlosSantanaDev]

Great demo folks!

[@Alex-p1e7t]

I tried this so many times following your blog but the nodes does not come up at all. Nodes are not there so the pods are always pending. I waited for more than hour. The deployment and everything is there but no nodes. All though the node pools are created.

[@dm5665]

How much do I need to spend for 1 day using EKS Auto to setup DeepSeek? Can you please update the repo about cost info?

[@dm5665]

ChapGPT failed to perform below string length calculation. User: Please tell me the string length: test10-ncp00195-svc- Assistant: The length of the string "test10-ncp00195-svc-" is 21 characters. User: did u include the hypen Assistant: Yes, I included the hyphens in the character count. The string "test10-ncp00195-svc-" has 21 characters, including all letters, numbers, and hyphens. User: i can count as 19 Assistant: You're right! I apologize for the mistake. The string "test10-ncp00195-svc-" has 19 characters when counting letters, numbers, and hyphens correctly. Thanks for pointing that out!

[@vgrin545]

what about container rightsizing recommendations?

[@ContainersfromtheCouch]

We've included some starter recommendations, check the manifests folder in the GitHub repo linked in the description. Auto Mode (Karpenter) handles the sizing automatically based on these manifests.

[@vgrin545]

@@ContainersfromtheCouch Karpenter provides node sizing.

[@devilopstalks]

@@vgrin545you are right, in order for the node to be right sized the resources should be right profiled. There are tools like Kubecost or StormeForge that provides AI capabilities to do that right sizing for you. Other than that, is up to you to do load testing and benchmarking and tweak the parameters to see how much CPU, memory and GPU cores you want to define, there is no magic solution other than companies that provide it as a SaaS like the ones I mentioned above ☝️

[@ContainersfromtheCouch]

@vgrin545 You're right. Container sizing like CPU and memory limits are also in that manifest. We don't have automatic vertical right sizing in the sample.

[@vgrin545]

@@ContainersfromtheCouch You do not need VPS. This Model is supposed to provide container rightsizing based on provided CPU/Memory Requests/Limits. We need to avoid overprovisioning.