Designing Python APIs for Data You Don’t Control
Speaker
Saurav Jain
Saurav Jain, Apify's Developer Community Manager, excels in community building and devrel. With a history of growing Amplication's community to 40K, he now enhances Apify's developer engagement. An international speaker, he has contributed to more than 100 conferences from remote places in Africa to events in Silicon Valley. His work bridges developers globally, fostering innovation and collaboration within the tech ecosystem. His expertise and passion for technology make him a pivotal figure in nurturing tech communities.
Abstract
The web isn’t an API, but Python developers often treat it like one. This talk explores how to design Python interfaces for unstable data sources, focusing on schema evolution, defensive parsing, and protecting downstream users.
Description
Python developers are used to working with APIs that have documentation, versioning, and some expectation of stability. Websites offer none of that — yet many Python systems depend on web data every day. This talk reframes web scraping and extraction as an API design problem under extreme uncertainty. Instead of focusing on selectors or parsing techniques, we’ll focus on how to design Python-facing interfaces that can survive change. We’ll explore topics such as optional fields, backward-compatible schema changes, defensive parsing, and meaningful error semantics. We’ll also discuss how breaking changes affect downstream users, especially when your data feeds analytics pipelines, dashboards, or automated systems. By the end of the session, attendees will have a clearer mental model for treating web data as a volatile dependency — and practical strategies for designing Python APIs that protect users from inevitable change.