@IndeedEng: From 1 To 1 Billion: Evolution of Indeed's Document Serving System

Video with Audio Description: • [Audio Description] @I...
Transcript: go.indeed.com/...
Slides: engineering.ind...
@IndeedEng Tech Talk #1, February 27, 2013:
As Indeed's traffic has grown to its current level of over 3 billion job searches per month worldwide, we have evolved our job data storage and serving architecture in order to maintain high levels of reliability and performance, including an average retrieval time per document of 31ms. This talk describes that evolution, from the initial solution to a dedicated service and custom data store built around a log-structured merge-tree (LSM-Tree) implementation.
Speakers:
Jack Humphrey is director of the engineering teams that build Indeed's job search and resume products. Since joining Indeed in 2009, he has helped build the service architecture that now handles over 3 billion job searches monthly.
Jeff Plaisance is a software engineer at Indeed focused on data storage infrastructure and analysis tools, including the datastore that serves up billions of jobs daily for Indeed's search results.

[@shubhammalik5875]

Quite insightful.

[@marcspehlmann9044]

I think the algorithm at 41:57 should read `while newIndex.size