Julien Nioche - URL Frontier, an open source API and implementation for crawl frontiers

This talk will present URLFrontier, an API and service implementation of a crawl frontier. After an introduction to how it fits in a distributed crawl architecture, we will go in more details on what the project provides, how it has been used so far and future works.
More: pretalx.com/bbuzz22/talk/D3TKUX/
Web: 2022.berlinbuzzwords.de/
Twitter: / berlinbuzzwords
LinkedIn: / berlin-buzzwords