Abstract
Many developers entertain the notion that they could write a 'curl' clone in a weekend. After all, how hard could it be to send a network request and print the result?
In reality, they vastly oversimplify the intricacies involved.
My command-line tool lychee, an async Rust link checker, initially anticipated as a weekend venture, has now extended into two years and still counting.
This journey unveils the deceptive complexity of what appears to be a rudimentary task on the surface.
Throughout the developmental process, I encountered a plethora of challenges that added layers of complexity. Some of these included handling rate limiting, navigating around incorrect status codes, mastering the art of URL parsing, dealing with timeouts, and grappling with client-side rendering using JavaScript.
We will go through each of these challenges step by step, and explore the Weirdness of the Web.
Furthermore, we delve into the essential skills required in the development process, exploring the often-overlooked aspects of funding open-source projects and the relentless world of open-source maintenance. These insights shine a light on the harsh realities and the hidden rewards of this crucial part of software development.
The rabbit hole goes deep. How deep, you ask?
Join the talk to find out.
