Keynote: How Spotify Accidentally Deleted All its Kube Clusters with No User Impact - David Xia

[@MrRidwanbejoz]

I believe Spotify is company that appreciate the engineering team so much. Culture of Learning is new thing.

[@manipal2011]

2 Teams - Kubernetes [Cluster Operators and Cluster Users]

[@emmadoyle4157]

If the internal slack channel was "eerily quiet" it's probably because teams don't have enough alerting set up to notify them that their applications/services aren't running in production.

[@sreejitkar7981]

This is like one of those water cooler conversations you get to have from that seasoned architect at your work who has had enough interesting mistakes ! Also I feel isolating your bundles with your infra can actually help avoid these errs

[@unfathomablej]

This is super entertaining. Sorry you guys had to deal with a mangled tfstate file in production. It's a terrible rite of passage.

[@bojackhorsingaround]

Wonderful test case even for a beginner like me. Good talk!

[@mj-np9sy]

Go through your envs and protect those clusters from deletion now that you can!

[@fredow123456]

let's do it before bad things happen 😂

[@jasonquek8279]

Ouch this was really painful. I guess you were running -auto-approve or no manual review of the tf plan before application.

[@sachinkadam4742]

ya, Guess so...quite an ignorant and not recommended approach for Prod.

[@anandkrishna6687]

loved it, honest , great learning

[@dm5665]

yes many times deleted k8s cluster accidentally...

[@eusfrasiuspatrickmarshall]

:clappepe:

[@rayoroderik]

Patrick Marshall :pepego: