EGTL data-processing model prototype using Python

Discover how EGTL (Extract, Generate, Transfer, Load) extends traditional ETL by adding a “generate” step powered by GenAI. In this talk, I’ll demonstrate how Python pipelines on top of data warehouse can automatically extract data, generate new insights, and deliver optimized transformations. We’ll explore practical workflows, real-world use cases, and best practices—equipping you to apply EGTL in your own data projects.

data engineering, automation, big data application develpment

Experience a novel approach to data ingestion pipelines with EGTL (Extract, Generate, Transfer, Load), a practical evolution of the standard ETL process. By incorporating a “generate” step powered by GenAI into the workflow, EGTL unlocks new potential in data transformation, advanced data analytics, and automation. In this talk, I’ll walk you through the core components of EGTL, demonstrating how Python-based data pipelines can leverage generative AI to produce enriched datasets on the fly before transferring and loading them into downstream systems. Using real-world use cases, I’ll illustrate how EGTL benefits both data engineers and scientists by reducing manual overhead, accelerating iterative development, and unveiling unexpected insights. You’ll learn implementation best practices—covering everything from tool selection and architecture design to error handling and governance. This talk will highlight key strategies for integrating GenAI directly into your pipelines.