From a naive point of view it&#x27;s easy. But it has to scale and launching a new process for every request works only for a pet project.<p>So a crawler based on a headless browser that consumes little memory and runs for weeks is a major achivement.