Write-Audit-Publish Pattern in Modern Data Pipelines

Name: Write-Audit-Publish Pattern in Modern Data Pipelines
Start: 2024-04-05T08:30:00+00:00
End: 2024-04-05T08:55:00+00:00
Location: SMK

Talk 25 min April 05, 2024 at 08:30

Tomas Peluritis

Data is new oil, and one of the ways is leakage and poisoning the surrounding environment. What happens if you pollute one of the datasets used in some decision makers facing dashboards? In this talk, I will explain the reemergence of the Write-Audit-Publish pattern and how you can achieve it using Apache Iceberg and Apache Spark.