Feature store: the missing data layer in ML pipelines?
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Hammar, Kim
Data may be the new oil, but refined data is the fuel for AI. Machine learning (ML) systems are only as good as the data they are trained on and getting the data in the right format at the right time is a challenge. ML systems are trained using sets of features, a feature can be as simple as the value of a column in a database entry, or it can be a complex value that is computed from diverse sources. A feature store is a central vault for storing documented and curated features, ideally with support for access control. A feature store enables automatic feature analysis and monitoring, feature sharing across models and teams, feature discovery, feature backfilling, and feature versioning. The feature store is a data management layer that fills an important piece in the modern machine learning infrastructure, it empowers enterprises to scale their machine learning workflows and make full use of their investment in machine learning. In this talk, we will present key points on how to take your machine learning workflow to the next level using a feature store, and demonstrate how the feature store fits into the larger machine learning pipeline. We will introduce HopsML, an open-source, end-to-end machine learning pipeline built on the world's most fastest and most scalable Hadoop distribution, Hops Hadoop. With HopsML you can build production-ready machine learning pipelines using open source software, where features are stored in a shared feature store that is automatically backfilled as new data arrive, where machine learning models can be trained on datasets in the order of billions examples using distributed deep learning, where data scientists can follow engineering principles by using versioned and reproducible experiments, and where models can be automatically deployed in an elastic manner using auto-scaling.
 He היא Can I sit here? There's one. There's one. Keep an eye on the room and we'll see you in a fight. Thank you. So thank you very much and welcome to this lightning talk. My name is Kim and I'm a software engineer at a startup called Logi-Lux. We are based in Stockholm and we are developing a platform for doing big data analysis and machine learning at large scale called Hopesworks. And this presentation I'll be talking about feature stores and try to convince you why you need a feature store to manage the data in your machine learning pipelines. So it will be slightly different from the previous talks. Now, if you're a traditional machine learning person, the way that your problems are set up is that you have these X and Y values and then your task is to kind of model this data and learn the relationships between your variables if you are a data scientist. But these X and Y values don't just show up clean by themselves. Rather, they have to come from somewhere. And that somewhere tends to be a script that some data engineer have written to extract the data, maybe from a Kafka topic or from a production warehouse somewhere. And what we're seeing now increasingly when machine learning is getting adoption in industry is that this process of extracting the X and Y values and getting the data in the right format and the right time for just doing machine learning is a very big task. And some companies even argue that this is the hardest problem in machine learning right now. Uber, for example, are claiming this. And they are arguably one of the companies in the world that apply machine learning at a larger scale. So what they're saying is kind of interesting. But even if you don't necessarily agree with everything, Uber is assessing this quote. We at larger clocks agree with Uber on one point, and that is that there should not be a question mark up here. Rather, it should be a standard as interface for how you feed the features into your model for training and serving. A standard as interface between data engineering and data science. And that is the feature store, which I'll be explaining in the remainder of this talk. So looking at the definition of a feature store is actually surprisingly simple. It is just a storage location within your organization where you store documented and curated features. But the power of the feature store really comes from the strong semantics that we can enforce on top of this feature store and abstractions that we can build on top of it. So some of the things that we're trying to achieve with a feature store are things like feature versioning, automatic feature documentation and analysis, and also feature backfilling. But the main point of the feature store, at least in my opinion, is this first one in the list, which is that by using a feature store, it becomes much simpler to reuse features across different models and across different teams. And let me elaborate on a little bit why there is such a big improvement for many organizations today. So what we argue at Logi-Logi-O clocks is that by investing in your feature store, the cost of your machine learning products will start to taper off. And our point here is that instead of doing new big investments in each machine learning product you take on and build custom pipelines, you can do kind of one-time investments in the lower layers of your data science stack, such as the feature store. And as you build up that feature store with more and more high quality features, the cost of building new models on top of the feature store will start to go down. And let me now look at an example that further makes this a point of feature reuse ability. So this is like a typical architecture at an organization today for doing machine learning, where you have this kind of siloed machine learning pipelines, one for each product. And this is a fairly simple workflow, but when you have this architecture, it's very hard to do things like reusing features across different pipelines. So typically what you have to do, actually, what I've seen at organizations is that if you have a feature here that you want to reuse here, you just copy the definition and recompute it again. And if you want to do more complex things, like for example feature backfilling or automatic feature analysis and documentation, it's basically impossible to use in this architecture. So what we suggest instead is to use what is called a feature store as the common interconnect between all of these pipelines. This means that a feature is no longer tied to a coupled specific model, rather a feature is now an independent, versioned and reproducible artifact that can potentially span many different models. So we no longer have this tight coupling between model and feature, rather they are kind of disconnected now. And this gives us a lot of freedom and flexibility as engineers or data scientists. For example, one big benefit of this is that now we have this consistency that no matter if you're using a feature for training or for serving or for a neural network or for a decision tree, you're always going to use exactly the same definition of that feature. Because we're not redefining it over and over again, we're just reusing the same feature again. And this also means that we can do nice things like here, we have all features centralized, so we can easily analyze them, look for their correlations. We can also do things like feature backfilling, so we can be more efficient in how we compute the features. And that is kind of the promise of what the feature store will give you. And now I'll just very briefly go into the more technical details on how we can build out this thing. And I will speak now from experience of building a feature store on our platform. So internally, a feature store consists of five different components, starting with the storage layer where we store the actual feature data. This tends to be predominantly numeric data like floats and integers, but it can also be categorical data or binary data, for example. Then we have the metadata layer. And this is, in my opinion, the most important layer to get right with a feature store. And you'll see in our implementation, we explore this quite a lot. So this is where we store things like the version of a feature, the ownership of the feature, which models are used, which job is with computer feature and from which data set does it originate. This means that we can automatically backfill features when necessary. We also store a lot of statistics about our features for helping our data scientists understand the features, like the distribution of all features over time, the correlation with each other, so Spearman and Pearson correlations, and we also store descriptive statistics and clustering analysis and so on. Then we have the two final components of the feature store, which is the feature registry and the API. And this is kind of the client-side interface to the feature store. And the registry, you can think of it as the front-end to our feature store. It is a place where we can publish our features, we can search for them, we can browse their metadata, or the computation of features, and publish our own work, basically. So you can think of it in a little way like an app store for machine learning. So it's a place where you can discover new features within the organization and you can also publish your own work so that other people can reuse it. And then the final component is the API. This is what we use inside our machine learning pipelines to read and write to the store. And this is just regular client-side libraries that we use. And I'll show you a quick example of how our API looks like. So here we have two examples of the API, one for reading from the store and one for writing to the store. And as you can see, code-wise, I hope you agree with me, it's quite a simple API. So to read from the store, you have to provide a list of feature names. And to write to the store, you have to provide a data frame of features. And to make this API so simple, what we have done is that we have abstracted out of our heavy lifting to our client-side libraries. So we actually have a query planner that will analyze your query and give you the result. And now my time is up. But if you're interested, I have several references on the final slide where you can read more. And just to summarize this presentation, machine learning is a very powerful tool, but it also comes with a very high technical cost. And in the beginning of this presentation, I mentioned this complexity of getting the data in the right format at the right time for doing machine learning. And our solution to this problem is to invest in data management layers specifically designed for machine learning called a feature store. And the world's first open-source feature store is available on our platform called Hopseworks. Thank you. APPLAUSE OK, so well done. Next speaker is setting up one question from the audience. Anyone wants to break? Could you show me just for natural language processing, what would be just your data? Repeat the question. Yes. So the question is, what type of features would it be for natural language processing? And typical features could be, for example, word embeddings are very common. But it can also be features such as, is this word a noun or a synonym or those kind of features. But word embeddings, I think, is the main feature we're using in the NLP right now. So would it be during the party feedback and the demo? Yes, you can do that as well. It depends on the task you're trying to solve, but that's also an option. But you can add more features to the better in most cases. APPLAUSE
 He היא  Can I sit here?  There's one.  There's one.  Keep an eye on the room and we'll see you in a fight.  Thank you.  So thank you very much and welcome to this lightning talk.  My name is Kim and I'm a software engineer at a startup called Logi-Lux.  We are based in Stockholm and we are developing a platform for doing big data analysis  and machine learning at large scale called Hopesworks.  And this presentation I'll be talking about feature stores  and try to convince you why you need a feature store to manage the data  in your machine learning pipelines.  So it will be slightly different from the previous talks.  Now, if you're a traditional machine learning person,  the way that your problems are set up is that you have these X and Y values  and then your task is to kind of model this data  and learn the relationships between your variables if you are a data scientist.  But these X and Y values don't just show up clean by themselves.  Rather, they have to come from somewhere.  And that somewhere tends to be a script that some data engineer  have written to extract the data, maybe from a Kafka topic  or from a production warehouse somewhere.  And what we're seeing now increasingly when machine learning is getting  adoption in industry is that this process of extracting the X and Y values  and getting the data in the right format and the right time  for just doing machine learning is a very big task.  And some companies even argue that this is the hardest problem  in machine learning right now.  Uber, for example, are claiming this.  And they are arguably one of the companies in the world  that apply machine learning at a larger scale.  So what they're saying is kind of interesting.  But even if you don't necessarily agree with everything,  Uber is assessing this quote.  We at larger clocks agree with Uber on one point,  and that is that there should not be a question mark up here.  Rather, it should be a standard as interface for how you feed the features  into your model for training and serving.  A standard as interface between data engineering and data science.  And that is the feature store,  which I'll be explaining in the remainder of this talk.  So looking at the definition of a feature store  is actually surprisingly simple.  It is just a storage location within your organization  where you store documented and curated features.  But the power of the feature store really comes from the strong semantics  that we can enforce on top of this feature store  and abstractions that we can build on top of it.  So some of the things that we're trying to achieve with a feature store  are things like feature versioning, automatic feature documentation  and analysis, and also feature backfilling.  But the main point of the feature store, at least in my opinion,  is this first one in the list,  which is that by using a feature store,  it becomes much simpler to reuse features across different models  and across different teams.  And let me elaborate on a little bit why there is such a big improvement  for many organizations today.  So what we argue at Logi-Logi-O clocks is that  by investing in your feature store,  the cost of your machine learning products will start to taper off.  And our point here is that  instead of doing new big investments in each machine learning product  you take on and build custom pipelines,  you can do kind of one-time investments in the lower layers of your data  science stack, such as the feature store.  And as you build up that feature store with more and more high quality features,  the cost of building new models on top of the feature store will start to go down.  And let me now look at an example  that further makes this a point of feature reuse ability.  So this is like a typical architecture at an organization today  for doing machine learning, where you have this kind of siloed machine learning pipelines,  one for each product.  And this is a fairly simple workflow,  but when you have this architecture,  it's very hard to do things like reusing features across different pipelines.  So typically what you have to do, actually,  what I've seen at organizations is that if you have a feature here that you want to reuse here,  you just copy the definition and recompute it again.  And if you want to do more complex things,  like for example feature backfilling or automatic feature analysis and documentation,  it's basically impossible to use in this architecture.  So what we suggest instead is to use what is called a feature store  as the common interconnect between all of these pipelines.  This means that a feature is no longer tied to a coupled specific model,  rather a feature is now an independent, versioned and reproducible artifact  that can potentially span many different models.  So we no longer have this tight coupling between model and feature,  rather they are kind of disconnected now.  And this gives us a lot of freedom and flexibility as engineers or data scientists.  For example, one big benefit of this is that now we have this consistency  that no matter if you're using a feature for training or for serving  or for a neural network or for a decision tree,  you're always going to use exactly the same definition of that feature.  Because we're not redefining it over and over again,  we're just reusing the same feature again.  And this also means that we can do nice things like here,  we have all features centralized, so we can easily analyze them,  look for their correlations.  We can also do things like feature backfilling,  so we can be more efficient in how we compute the features.  And that is kind of the promise of what the feature store will give you.  And now I'll just very briefly go into the more technical details  on how we can build out this thing.  And I will speak now from experience of building a feature store on our platform.  So internally, a feature store consists of five different components,  starting with the storage layer where we store the actual feature data.  This tends to be predominantly numeric data like floats and integers,  but it can also be categorical data or binary data, for example.  Then we have the metadata layer.  And this is, in my opinion, the most important layer to get right with a feature store.  And you'll see in our implementation, we explore this quite a lot.  So this is where we store things like the version of a feature,  the ownership of the feature, which models are used,  which job is with computer feature and from which data set does it originate.  This means that we can automatically backfill features when necessary.  We also store a lot of statistics about our features  for helping our data scientists understand the features,  like the distribution of all features over time,  the correlation with each other, so Spearman and Pearson correlations,  and we also store descriptive statistics and clustering analysis and so on.  Then we have the two final components of the feature store,  which is the feature registry and the API.  And this is kind of the client-side interface to the feature store.  And the registry, you can think of it as the front-end to our feature store.  It is a place where we can publish our features, we can search for them,  we can browse their metadata, or the computation of features,  and publish our own work, basically.  So you can think of it in a little way like an app store for machine learning.  So it's a place where you can discover new features within the organization  and you can also publish your own work so that other people can reuse it.  And then the final component is the API.  This is what we use inside our machine learning pipelines to read and write to the store.  And this is just regular client-side libraries that we use.  And I'll show you a quick example of how our API looks like.  So here we have two examples of the API,  one for reading from the store and one for writing to the store.  And as you can see, code-wise, I hope you agree with me, it's quite a simple API.  So to read from the store, you have to provide a list of feature names.  And to write to the store, you have to provide a data frame of features.  And to make this API so simple, what we have done is that we have abstracted out  of our heavy lifting to our client-side libraries.  So we actually have a query planner that will analyze your query and give you the result.  And now my time is up.  But if you're interested, I have several references on the final slide where you can read more.  And just to summarize this presentation, machine learning is a very powerful tool,  but it also comes with a very high technical cost.  And in the beginning of this presentation, I mentioned this complexity of getting the data  in the right format at the right time for doing machine learning.  And our solution to this problem is to invest in data management layers  specifically designed for machine learning called a feature store.  And the world's first open-source feature store is available on our platform called Hopseworks.  Thank you.  APPLAUSE  OK, so well done. Next speaker is setting up one question from the audience.  Anyone wants to break?  Could you show me just for natural language processing, what would be just your data?  Repeat the question.  Yes. So the question is, what type of features would it be for natural language processing?  And typical features could be, for example, word embeddings are very common.  But it can also be features such as, is this word a noun or a synonym or those kind of features.  But word embeddings, I think, is the main feature we're using in the NLP right now.  So would it be during the party feedback and the demo?  Yes, you can do that as well. It depends on the task you're trying to solve,  but that's also an option.  But you can add more features to the better in most cases.  APPLAUSE
Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_0_1751.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_1752_2891.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_2892_8759.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_8760_12131.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_12132_13187.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_13188_13631.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_13632_13907.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_13908_14159.png Feature_store__A_Data_Management_Layer_for_Machine_Learning_Data_Management_for_ML_86rrEJqRLO4_14160_14273.png