Migrating billions records from SQL to NoSQL using continuous migration technique with PySpark and DataProc.

Talk 25 min April 23, 2025 at 10:30

Piti Champeethong

The batch mechanism is challenging when handling continuous data migration with DataProc. However, I'm introducing a new approach for continuous data pipelines enabled by PySpark. The participants will learn new methods to handle data consistency and reserve data completeness in a million-scale migration from SQL database into NoSQL, MongoDB.