Select ML from Databases

[Music] currently i work as a developer advocated couch base uh so for those of you who are not familiar with couchbase uh it's basically a document database that is highly performant and well suited for highly distributed applications and last year uh in cloud space we introduced the possibility for you to run your machine learning models within your database and so i took this opportunity to like explore the machine learning capabilities provided by couchbase and also provided by some of the other existing products on the market uh so this is just a really uh high level overview of machine learning i'm pretty sure all of you know about this uh so the idea is that a lot of these companies they have a lot of data right now and there is a lot of data that is being continuously collected from all the applications that we use and they are trying to solve the problems uh or they are trying to get some meaningful value from these data so there are things like personalized offers for example uh your taxi writing apps or you have things like coupons that are sent when they detect that you are you might be churning or there are lots more other examples like anomaly detection for credit card for frauds and so on and uh this is like one of the common workflows that you see in when you deploy a model so you start with uh training you're getting some training data and typically your applications would be uh data would be in transactional databases like postgres or even like more sequel databases and uh you typically don't use the same databases for your analytical workloads because they are uh they are more time consuming and like they they shouldn't affect your applications that access the operational database and so typically you have etl jobs that or extract transform and load jobs that take this data from the operational database and put it into your analytics state analytical databases and then you have your classical feature engineering where you experiment with different features for the data that is stored in your analytical database and sometimes you also create like features tools so that the features can be reused and then you have the classic training where you run machine learning algorithms or deep learning algorithms like support vector machines or random forest or neural nets and then comes the process of deploying this model and normally you tend to package this model uh into a by using a simple uh api framework like flask or fast api and then build a small micro service around it uh which is which is then accessed by your applications that are calling this machine learning use case uh well the good thing is it works quite well and you can also have a lot of flexibility in scaling your services and also doing whatever you like in the models however some of the issues that i've seen in practice is that you end up with a lot of duplication of data especially when you have people who work on let's say local copies of the data and you do some experiments then there's a high likelihood that you tend to deviate from the standard data in your operational database and also it's not a trivial job to maintain the ctl jobs especially with all those fault handling and retries and all those kind of things and also this updating these models so data is a combined effort of both the application team and also the data scientists or data engineers in most cases so coming to uh like some of the options that are provided by machine learning options for machine learning and databases so currently what you see is like the modern databases they have adapted to the needs of the industry and they are adding machine learning algorithms inbuilt into the database for example you have bigquery or redshift where you can directly call tensorflow models or even uh they they also have a lot of custom or not not custom i mean they have an input uh support for a lot of the algorithms like support vector machines gradient boost and so on similarly there is uh there are like third-party additions like mines db which you can uh plug into your existing data and they create like a copy of your data and then work on it but it's completely invisible to you then you also have databases like sql server or oracle or firebase or cultures where you have support for writing your own models for machine learning applications and these are typically done in python.r uh one thing is like in most of these cases you would write your influence in terms of queries yeah so as i explained earlier uh so this is like a subset of all the uh algorithms that are provided out of the box in bigquery and yeah it even supports uh importing like tensorflow models and the workflow is like you write a statement create model and you specify the parameters of the model and you can just call select model with your inputs to get your inference uh yeah with my instantly you have support for a lot of applications uh but what they do under under the hood is they create like a copy of your data uh yeah and they work on this but you basically use mines db to create uh your predictor and then yeah you basically do your inference in a sequel like statement and coming to the custom machine learning models uh so in this case the workflow is like you start building your model in your developer machines mostly in python or r then you create like a package and then most of the cases you have like a custom procedure in the database which is then calling this package and yeah the inference is also like sql like uh yeah i'll show this in a demo yeah but just to summarize some of the uh advantages here is like you don't uh have a lot of variation between your analytical data and your operational data and one more thing here is that your data doesn't leave the database uh so this might be important in some uh like uh let's say uh it's more for like legal purposes like there are some laws in for some data that they shouldn't be processed in certain areas and yeah it's also quite nice if you don't have data scientists to get some out of the box models and the model update part is also quite stable so you just develop your or retrain your model and then deploy and you won't need a micro service to deploy your model i mean there are some drawbacks as usual like uh like one thing you might want to keep in mind is that it might not be suitable to for all our models like uh for example if your model is highly dependent on gpu maybe the inference is not that performant when using uh databases because in most of the cases you don't use a gpu in the database although there are some databases that use a gpu yeah so with that said i'll just do a quick demo using couch base so how we do this in cloud spaces we have a concept of services uh so we have like your data service which is typically your data or like operational data where you store your uh transactions and then you have an analytic service which is tracking your operational data in real time so this is creating all those uh uh or it is copying all the data from your data service and this is also completely independent from your operational database uh yeah and you can also scale this up and down however you like and this is also like massively parallel processing so depending on how many nodes you are running your analytic service you get like that kind of linear performance and yeah you basically write your udfs in python for your machine learning uh so i just want to give you like a real world example so this is like a website that i found online but what they do is like they take a few parameters from you and based on that they give you a code for your insurance and uh so i found like a data set on kaggle which does like which estimates the medical costs based on these features and it's like a really small data set and it's just for like checking out what is possible and yeah so essentially you put values for all these features like any gender bmi smoking status or region and children and based on that you get like a estimated medical cost and based on this you can get your insurance code and all this is done in real time and this is a workflow that you do so you train your model locally in python then you package it along with all the dependencies into a small package and this is then uh deployed into course-based analytics and you create a user-defined function that would call this that would be called in your sql statements yeah i'll just quickly jump to the demo so this is like a simple query workbench where you can type in your query so i already have some query uh which is doing like an insurance estimate for someone age 40 40 female uh bmi and all so let's just run it yeah so you see that like within a one second you have a inference which is saying that the medical cost is like 19 000 for this person for a year uh this is not all so you can also do this within your application so i just built like a simple application and here you can put in some values uh yeah and to like an estimate uh so what is happening under the food is this data is now getting stored into your into my data service and then it is uh getting copied into the analytic service and you are getting this inference eighteen thousand dollars and all this is happening in like 11 seconds which is quite fast yeah and i can also show you like the uh simple model so the model is not doing anything so it's just like a simple random forest with uh one with car with encoding for the uh categorical features yeah this is that and you are just like dumping this into a pickle file and here you are just reading the special file and then there's a function that is just doing an inference it's that simple yeah uh so this was like a simple example so i also tried like another example using deep learning uh so one of the common use cases that you see is like real time uh machine translation for example on amazon if you're coming uh if you're accessing amazon from a different country you still get like the reviews from other countries and you can also translate them to your language in your profile the same thing happens on twitter so for example you can see our tweets in german and you can click on a button and it gets translated so this is something which is quite cool and i said okay let's see if we can do it and the good thing is uh open source is great so you have all these pre-trained models that are available on hugging phase so i just uh checked it out and there is like a library which is the one which is used by microsoft translator and it has support even for multiple languages and it's like just 300 400 mb per model and there are over 1000 models so you could theoretically do this for any combination of languages like yeah and yeah so that's what i tried to do so i created like a a simple application that would just uh get those messages and do it real-time translation so let me just go back to the demo so i have this running on my local machine but you can also join this so by scanning this qr code if you like and type in a message and we can test this model so let's just get some message yeah i i don't speak lithuanian so i just uh i'm just trying a translator let's see if it works yeah this would take a bit more time but hopefully it would work so yeah just wait in the meanwhile if you like you can try this yeah so it's kind of accurate i would say i mean although i don't know the language but at least it's similar to what i typed in for the original and yeah if you are joining the chat like i can also try to get some of the messages so i mean this whole thing took like less than 20 seconds uh yeah so uh yeah let's just try to get the last 10 messages and if some of you have sent some message hopefully we'll see there yeah let me just also show you the code that is uh doing this so this is all that i had to do so this is just a pre-trained model and there's nothing out of the ordinary like you just take this pre-trained model from which is published by helsinki university and yeah uh you have this text that you're getting from query and you are doing a translation and returning it yeah this is still taking a bit of time but that's fine uh yeah you can do the same thing here so this is just like the query workbench like we saw earlier so let's just test it with a different language yeah german oh yeah oh okay there was a server error live demos okay but let's see if this one worked okay wow let's see okay yeah it's basically coming from the database so maybe a spawn we where there was just too much traffic let's see if we can see the data in the uh yeah okay yeah i see some messages so let's just try to try it once again ah okay i see what is happening yeah so there was yeah the analytic source crashed for some reason i mean yeah so as you can see on top this is still like in a developer preview mode uh yeah and this is like we are still working on it and there are like one or two edge cases where you could have these issues let's see if yeah so now it's back uh let's see yeah let's try this okay no it's still not working but you've had to trust me this was working yeah uh but yeah i mean you get the idea that you do like uh you can just create like custom models in python and you can run the inference directly uh using uh like sequel like i mean it's not officially sequel this is like quite similar to sql uh yeah we call it sql plus plus because we are working on json documents not like relational documents and you can also access this using both or like sdk in any of your languages or you there's even like a rest endpoint which you can use and yeah you can also scale just the service independently right now all this is running just on my machine inside docker so this is like two cpus i guess and like 5 gb of ram yeah but essentially i can eliminate all the micro services that you have that i have for deploying my jobs and also get rid of etl jobs especially in this case well uh to conclude i would say machine learning is becoming more and more a part of your database architecture like all these big companies are providing support for them natively and also there's a lot of support that is coming for like custom models so not just models that are provided by the databases and but as you saw earlier like what we did with the deep learning example is kind of on the border like it might not it's quite heavy for something that you're running inside your database like the model is around 800 megabytes in memory yeah and yeah this is all so to summarize i would say if you are a company and you don't have like data scientists maybe uh this is something uh you can use some of these um algorithms out of the box it works and yeah this is something which you should definitely consider if you're trying to run your machine learning applications yeah and yeah all the code is uh public so you can find it on github uh yeah i'll also share the slides and yeah if you have any questions please reach out yeah and yeah that's all thanks [Applause]