RailsConf 2017: 5 Years of Rails Scaling to 80k RPS by Simon Eskildsen

RailsConf 2017: 5 Years of Rails Scaling to 80k RPS by Simon Eskildsen
Shopify has taken Rails through some of the world's largest sales: Superbowl, Celebrity Launches, and Black Friday. In this talk, we will go through the evolution of the Shopify infrastructure: from re-architecting and caching in 2012, sharding in 2013, and reducing the blast radius of every point of failure in 2014. To 2016, where we accomplished running our 325,000+ stores out of multiple datacenters. It'll be whirlwind tour of the lessons learned scaling one of the world's largest Rails deployments for half a decade.

[@Sonsequence]

You said that flash sale traffic pattern is that one shopify storefront will drive much more traffic than all the rest combined. Is this a problem with your "pod" approach? i.e. you must target the pod housing that storefront ("a little shopify" as you said) and vertically scale its "little" database shard, redis, etc. As you said, the overhead that needs to be maintained on the other pods means you must hard limit how much one pod can grow while leaving the others potentially under-utilized while one gets overloaded. It seems undisastrous but is somewhat special to a company like your's which has 377 thousand separate audiences, where the heavy use of one resource doesn't correlate closely with an ease up on others*. For most companies, all resources under one roof has the nice side effect that they hedge against each other (if all your users suddenly start playing over "here" they're not simultaneously playing over "there"). Of course, you must shard in some way at scale but as you said before, the less the better. Or do you already have a policy of scaling up each pod as far as capacity reasonably allows? * It occurs to me that actually you possibly address so much of the market for boutique-online shops that you *do* see a bounded glob of core users swilling around between brands they don't realise are running the same system under the hood. Is that so?

[@pauldacus4590]

Wow, this is an extremely well-rehearsed, on-point and informative talk. If all conference talks were this good, I'd think about going to them again.

[@omarmoataz]

The people you meet.

[@piyushkatariya1040]

Really good talk, especially about building resiliency (fault tolerance)

[@alexdunae]

Thanks that was great

[@damianpolan5776]

Amazing presentation. Nice work

[@mp8889]

Nice presentation, very informative

[@rahls7]

15:02: Quick sort is N square, right? I think he meant merge sort

[@Simon-xi8tb]

No, bubble sort is n squared. Quicksort is n squared only in worst case, but on average it's much faster (n* log n)

[@kennethmarete5329]

awesome talk, concise and clear.

[@kleinesmaccify]

Really good talk. Wish the guy do stay in place for a second...he's zigzag movement really annoying.

[@az9048]

This is a really fantastic talk. Thanks Shopify and Simon!

[@leonidaspreston3310]

I know im randomly asking but does any of you know a method to log back into an instagram account..? I was dumb lost my login password. I would appreciate any help you can offer me

[@angeldominik650]

@Leonidas Preston Instablaster =)

[@leonidaspreston3310]

@Angel Dominik I really appreciate your reply. I got to the site thru google and Im in the hacking process now. Seems to take quite some time so I will reply here later with my results.

[@leonidaspreston3310]

@Angel Dominik it did the trick and I now got access to my account again. I'm so happy:D Thank you so much, you really help me out !

[@angeldominik650]

@Leonidas Preston No problem =)