Driving down the Memray lane - Profiling your data science work
Room: Saphire B - PyData
Date: 2023-05-19
Time: 11:00 - 11:25
Abstract
In this talk, we will be exploring what memory profiling is, and how it can help with data science work. We will start the talk with a basic explanation of how Python arrange memories for various objects. This lays the foundation explanation of why we need a special tool to memory profile Python programs.
Then we will be going through a data science use case where we memory profiles some part of the process with the Memray Jupyter plug-in. This would be a use case that a data science practitioner or learner would be familiar with and they can see how memory profiling could be useful.
We will then explain how to interpret the frame diagram in Memray, a commonly used diagram in memory profiling to understand how much memory a process and its sub-process uses. This is something that for a new user, it could be hard to understand and not know what to look into. From this example, audiences can see what they can learn about from the frame diagram.
Cheuk Ting Ho
Before working in Developer Relations, Cheuk has been a Data Scientist in various companies which demands high numerical and programmatical skills, especially in Python. To follow her passion for the tech community, Cheuk is now the Developer Advocate at Anaconda. Cheuk also contributes to multiple Open Source libraries like Hypothesis and Pandas.
Besides her work, Cheuk enjoys talking about Python on personal streaming platforms and podcasts. Cheuk has also been a speaker at Universities and various conferences. Besides speaking at conferences, Cheuk also organises events for developers. Conferences that Cheuk has organized include EuroPython (which she is a board member), PyData Global and Pyjamas Conf. Believing in Tech Diversity and Inclusion, Cheuk constantly organizes workshops and mentored sprints for minority groups. In 2021, Cheuk has become a Python Software Foundation fellow.
