In this talk, we’ll share our experience building and deploying Efesto, a scalable Data Feature Store. This Python and PySpark software solution empowers data scientists and engineers to focus on solving real business problems by eliminating the need for developers to “reinvent the wheel” with repetitive feature engineering tasks. We’ll explore how Efesto leverages Apache Airflow’s orchestration capabilities to manage dependencies and ensure data quality for hundreds of deployed data features.​

​

Here’s a deeper dive into what we’ll cover:​

• Technical Deep Dive: We’ll explore the design and architecture of Efesto, focusing on how we created a Python library for building data features.​

• Airflow as the Standard: We’ll showcase how we built an Airflow-based toolkit to standardize process execution across all teams.​

• Unify and Conquer: Discover strategies for unifying data across silos and enforcing feature consistency within your organization’s data pipelines.​

• Real-World Results: Witness concrete examples of how Efesto empowers data science teams to build better models faster, improve collaboration, and unlock the full potential of their data.