Data Science with Python — Past, Present and Future

so first of all kill for coming today to my talk and topic that I will be focusing on is basically the data science in Python how it all started and so on and so on so first before I start I would like to thank also organizer for this amazing event and for having me here so we're really like great organization great venues are really nice to be here and I really loved villainess so during my talk I will be this is simply the structure of my talk and during it I would like to be like more interactive with you I will use this something called slide oh how many of you know so I don't use it so use it also this time the hashtag is Python healthy and constantly during the presentation I will be asking you some questions so make it like more interactive so a few words about me I'm economist for education and as for profession or after my graduate study I simply started as analyst data scientist and I'm doing consulting for about 10 years mostly for with focus on public sector but few clients are also from private one I'm also a founder of Pi Deacon Bratislava local organizer and also leader are and tableau community and recently we started also julia meetups as for my journey towards the python i started during my studies in matlab so only after that after some experience with stated Stata and SAS I switch to open-source and yeah first one was are then tightened and out of out of Python or out of this data science world I also play football lead one team so basically that's another topic in case you meet me I'm definitely into it discussing football and so on and also I founder of effective altruism so vacuum these are the books that I have been working on or not as an outer but I helped hurt as for the quality of of it and currently writing one but I cannot say more and basically teaching data camp which is like a global provider of the education within data science and they're not some local ones go learn to code it's within Slovakia and Czech Republic okay so first I would like to ask you how is how many of you are developers here and how many see yourself as data scientist okay the Python out pelty as in the under siding so basically this conference hashtag official okay so we can stop it here so majority see themselves as a data scientist so that's you are on the right place and PI data threat so what is this was just a test this was just like a warm-up before the other questions so what basically is data science in recent years we can like see in the private sector but also in academia like booming of these data size puzzle world it basically become buzzword and some may like get the impact or might get the impression that this was something that was invented recently one of the one of the persons who helped to like push this data science like combination of work work and simply made this data science as buzz word is DJ Patil who is like mostly cited as the one who with a jeff hammer burger coined the term data science the problem with this claim is that the only source that can like confirm this from the scientific perspective is his own book so if you look like harder or deeper into the past you will see that this is simply not true so I I don't want to diminish his like contribution to the profession but it simply starts with statistician named John Tukey and the beginnings of data science itself is connected with Bell Labs and simply when there was like the beginning of the computer science itself so basically connected also with another very famous person Jonathan Newman in 50s and later in 70s with this Bell Labs and John Tukey who simply created the the first like concept how we should analyze data how the computers should process them and so on and so on so if we want to define a data science it's basically collection of the scientific results and methods that transform the raw form of data into the meaningful information and knowledge and hopefully at the end of the wisdom that will help us to make a better decision so also in public sector also in private ones also in our private lives there is also one of common misconception some people took the terms dicta science and big data as as equal this is not true big data is just one of the problems that data science like tries to solve and what data science is is simply interdisciplinary like combination of computer science statistics and one of the one of the professions that really helped to like push data science forward where also economy or econometrics mostly within the time series analysis ok if you want to go further into this topic the history of data science itself great source is David Dunn over from Stanford the papers from 2015 there is doing okay so what as I mentioned the basic the basic concept is to take the raw data or the signal and we try to identify this signal within the raw data and then give it some context it becomes information and after that hopefully knowledge there is like role of the data scientists and after the purpose part when there is the decision needed it's mostly out of the hands of data scientists so basically it's your client or somebody that you work for for example policy a decision maker and so on so at the end the change should be like based on data or based on the data driven approaches simply so this is Co like the size pyramid and this is the process or the workflow of data science so simply you will start with a definition of problem this is very important because if you don't define the problem in a well manner you simply don't find the right data and cannot continue so in the middle as you see it's like iterative process or the the process from cleaning transformation visualization modeling and hopefully at the end it should be like the insight there in case that we don't find or don't identify inside we have to return back or collect more data relevant to the problem solving if problem is solved then we have to simplify it to client simply explain to the decision-maker what we like date or what are the results and at the enter there is also deployment so basically the the model should be implemented and useful for the for the client okay so if we like after the beginning of data science itself we have seen recently and this is the main topic of my talk is basically data science in Python if you check the history of Python Python is not ideal language for data science it was basically founded as the very easy language that was designed for kids simply so it started with Guido van Rossum and his basic idea I did at the beginning was the language ADC so simply like the the the easiest way how to teach even preschool or pre elementary school kids how to program so basically Guido's view was that programming should be like part of the part of the areas that should be included into into the like computing into the general knowledge of of the young kids so later this was not successful project but later on he started this during the break in the work it was before Christmas in 1980 and this was like the first version then he left from from Netherlands and he moved to to us and then it begins like its own journey the journey of Python one of the disadvantages of Python itself is that it's really slow and really not so well for for processing of data so I I'm like I mean the core of Python or the Python without the additional libraries later on it began it began useful also for processing a lot of data but it was in the core designed for teaching simply teaching kids the name is derived from Monty Python flying circles which you probably know ok as I mentioned Python is not ideal for processing data but there is one programming language that is ideal for data or was a design at the beginning it's called art and it's also connected with the bell labs in the dimension before with John Tukey so it basically starts started as a programming language s and it's connected with John Chambers as the founder and later on the motivation simply to create art was to avoid the fees for the license of this as language that later become like commercial version of the original s which was open source and foundation of the other languages connected with the University of Auckland in New Zealand ok so this is the another question what was your first data science programming language subjects level and I'm interested in what were your beginnings okay Python c-plus well okay okay 75 okay so we can see that the clear winners Python so simply even the vision of the window at the beginning was that it was like first programming language that everybody should learn and even data scientists or the majority among you like picked it out so how the Python begin the programming language for data science it was basically the effort collective effort of many very clever minds at the start it's connected with now focus itself which is a nonprofit organization in us and the the name now focus is connection of numpy and focus on numpy simply and the most important name in this context is Travis Olaf and who was simply founder or creator of this numpy library and his basic idea was to create some tool that can process the data in a similar way like MATLAB so basically open source alternative to MATLAB and he connected three other libraries under the this name vampire later on he also created a site pi which is also two guru numbers and by as matlab alternative another important name is john hunter the creator of matplotlib which is dicta visualization library and the core of data science is done within Python world in library called pandas so you probably heard about it and the founder is vesnik the one on the right at the top so basically the all these data processing tools or libraries are very effective and it's the the syntax is pythonic or Python but in the core the effectiveness of these libraries is because they are implemented in something called seitan or the or the there is a translation from Python to C and this is why it's so quick later on all these the data stack or by DeCosta become included in one of the library site I was the first like connection of these data tools later on it was also created one a Python distribution which is called anaconda so simply currently if you like who want to enter the designs world in Python you simply install anaconda as one of the distribution where you have all these tools and all the results of the work and PhDs and so on from the most talented people in the world and all selected like thousand libraries in one distribution together with Titan oh there are there is greater network of PI data conferences and meetups this is one of the place here in Vilnius and we found it also the local branch in Bratislava Slovakia as for the of the local meetups within the world ok so these are the simply tools so that are mostly user I will recommend as for the Python world so as for IDs I'm probably the mostly who uses Jupiter notebooks or before it was called ipython notebooks and it simply is so like language agnostic it started as item project but currently you can find a lot of kernels for all the programming languages that you mentioned before during the during the slider slider checking so you can use Jupiter notebooks with any language that you like like if you want to program professionally or if you want to use advanced tools for interaction with Python or for programming in Python there is great to buy jump rings called buy charm so basically it's it works well also with Jupiter notebooks and also you can basically write scripts and like train them and create packages so this is like the area of classical software development is mostly used to that I would recommend and there is also another like evolution of the Jupiter notebooks which is called Jupiter lab and this is basically the idea of the future it's currently still not in like final versions it's still in development but it's like combination of so Java also other JavaScript and a lot of other programming languages but it simply works as your browser and you can interact with the sub module which is the Jupiter notebook itself for data collection in past we mostly used like classical imports in pandas but there is currently like the switch towards the completely new for which is like the common result of the most like advanced data scientist within Python world and our world from patent world des makini its outer of pandas and from our world Hydra we come and they basically create this like new tool for a representation of the data within memory so one is within the ROM and one is simply a representation on your disk so one Apache arrow and then a feather for data visualization you probably all know matplotlib but based on this Seabourn is the most advanced and if you want to create something interactive booker is the new like a vision for data visualization itself and plotly is also cross language for data cleaning and transformation data cleaner blazed dusk is for parallel computing so if you want to paralyze your quote for data modeling like classical statistical modeling is mostly using models within Python and then by CN statistics by MC 3 and for deep learning care us together with in the or PI torch okay so there is like a great rivality between the classical or the typical the science community which is basing our language and current leader which is the Python PI data stack and it simply doesn't make sense because even in the anaconda distribution that I mentioned it simply recently began even distribution of not just peyten but also art itself and basically we can like see that these different worlds or this different areas of data science are colliding and in the future we will not like make the distinction in such way as it was in past so it's simply the most of the innovation is created still in our community or in the most advanced - for the design but the Python world or the PI data PI data stack is developing in rapid range so it simply is inspired by the art world and also the other way around ok switch this and there is another like voting which ID you mostly use for the majority of your work for charm nice okay so Jupiter notebooks so also good choice but consider also the Jupiter lap in the future because it simply combines Jupiter not boobs with other technologies so keep it in mind so I will end it here okay so what is the future of data science itself some say that it's the pythonic way or the PI data stack that will lead to innovation but as I mentioned the most of the innovation within the designs is still in the art community or in the art world it's tradition also from the past but the bye-bye Lida world is improving in rapid range but there are still some limitations within the Python that we cannot like overwrite or we simply are limited by this so at least from my point of view the real innovations or the real pushing the boundaries of data science itself is currently happening in julia community and also as we heard many times the future is female so maybe at least from my point of view or from point of view of many others within the julia community this is the way forward so what are the advantages of julia first of all i have to mention that also Julia is financed by now focus and it basically started as project at MIT by these four very clever people and the vision was basically to take all the advantages of all other programming languages and to combine them in something that will that will like be very fast very easy syntax as peyten and have all the advantages of over the other if you check for example the Jupiter not book z2 you selected for the IDS even the name itself Jupiter is like combination of free programming languages it starts with Julia GU is from there and Python and are the the order is not like random so it's basically even from from this point of view it's seen the Julia as the next logical step towards the future of data signs itself if I select or if we agree that the pythonic way is the way forward there are some lot of improvements in area for example the technology called ray by outers from rice lab the outers of spark so definitely if you want to check or hand get the snake because for the future check for example project Modine which is simply like pandas on like advanced or triple the speed of pandas so if you want to improve the speed of pandas check out project model and it's based on the rate technology and the ray itself is basically used also in machine learning and so far Python has really the best libraries within machine learning then there is plant the pandas 2.0 using the Apache arrow technology and so on so it also there will be a lot of improvements there so definitely check also this out and another voting what's the future this was like my vision or vision that I presented but what your opinion on this okay so it's logical we are on byte and Conference yeah but at least okay I will take it as it is that definitely check out though so Julia because it greatly combines also the classical software development with the data science world and this is simply what we need because like classical data scientists are not so well in software development but this is something that combines this world and we need it for the for the future so some few words I'm over times sorry for that this is a few words about the PI data as the global movement or the conferences as meetups as I already mentioned within Slovakia we started in 2016 in December as 40 globally after few days after us in LA was created and within the region or the v4 countries we were the second one after Poland where there already had in Barcia rocks lava and Krakow and in Vienna few months before us they started after I tried to push in Shakya and it took two years till they found it and one year after we found it also Hungary in Budapest the local meetup was created as you see from the map there is there is more the PI data a local meet up outside the US than in u.s. itself so this is simply something that was not expected at the beginning it simply started as an educational project for with a new focus so it really like it's great to see this grow but there are a lot of problems connected to it and the first one in Europe was founded by young here so PI data London was the first one that drive is only fund I think he hand-picked you to like push it first teary okay so yeah and what are the goals of now focus basically so how many of you know the lady on the left or no this picture okay so a lot of you that's great but some thing that you probably don't know is that also the dis like a breakthrough in science was done by mostly technologies financed by now focus or financed via via this educational program and support for developers so basically these are the goals of now focus and these are the goals of Pi data itself simply to support the talented developers and to push the science forward yeah and these are some activities I will skip this this is about thousand the conferences that we cooperate this is one of our activities if you don't know one of the very talented data journalists and so that here was killed because of his work so this was one of our activities that we made together with one of the biggest the IE universities and it has a lot of impact and this is more about us so we started about three years ago it's the organization that I mainly work for is data Institute and it's nonprofit and so we are in partnership with now focus an organizer of the PI data Bratislava and from the day one we started this with transparent account and all the these are the basically rules also our values of openness and transparency these are the this is the link to the slide and if you want to support us you can use the QR code and this is me two years ago that's not important it was in Berlin and you can rate also my talk so I'm interesting how it okay nice it's not that hopefully it is informative to us okay so it's about date so thank you okay this is contact to me we have also like that if you want to learn more about us and these are the winter accounts did I manage and yeah and this is the last thing that I will mentioned and I would like to invite you all to gratis lava later this year first conference that we will organize will we focus on our language and these are the already confirmed keynote speakers one of the founders of data science La Silla Kafka and Chester is mine who is like curriculum lead in big data and he hand-picked me to decamp so so check out this and hopefully in their future but we have to discuss with Nava booth first set the date but hopefully later this year there will be also a PI theta conference or track in other conference and you are all welcome to join and come also to visit us as I came here so thank you for your attention and sorry for the delay but yeah you