Netflix improves its recommendation algorithms and predicts the success of new shows by using massive amounts of data. Because managing machine learning pipelines in large scale is hard and time-consuming, Netflix developed a tool called Metaflow to boost the productivity of data scientists. Codemate has partnered up with Netflix to make Metaflow more powerful and usable by designing and developing its first graphical user interface.
Why Metaflow was created
Data scientists help Netflix address complex business problems that would be nearly impossible to solve without large amounts of data. They use Metaflow to develop algorithms meant to improve a host of business decisions. Netflix’s ML practitioners identified the need for a better user interface for Metaflow to make it easier to keep track of their work. They also needed a better and faster way to manage numerous data science projects.
Codemate has been Netflix’s development partner for Metaflow since spring 2020. Codemate developed Metaflow’s first-ever Graphic User Interface (GUI) with the specific needs of data scientists as a starting point. Now Netflix data scientists can focus on modeling and use the Metaflow GUI to track and manage their ML pipelines. Codemate designed the GUI to run on AWS. We also developed clear documentation of the GUI project enabling a smooth open-source release. Initial feedback from Netflix’s data scientists has been very positive, as the GUI is seen to save a significant amount of user time.
Making of the perfect product
Netflix had defined high-level goals for the project. For example, they needed a GUI that answers “What is happening or has happened when ML models are ran”? “The GUI should feel snappy and it should be able to handle a run with 100K+ task and an archive of up to 1M runs.” Additionally it should be easy to adopt and easy to deploy. We used these goals as our guiding principles to define, design and deliver the product.
We assembled an agile team with responsibility of designing and developing the product that met these criteria. With a close collaboration and frequent feedback loops with Netflix ML team we incrementally developed the UI to meet their exact needs. During the 16 months project some of the features were pivoted based on initial feedback and iterated until they were perfect.
Metaflow UI release
As Codemate was the main partner for designing and developing the Metaflow UI, Netflix decided to organize the release event together with us. In this video Brett Rose from Netflix shows the Metaflow UI in action, Codemate’s designer Teemu Kemppainen goes through the user flow and design process. Ville Tuulos from Outerbounds then gives important information and practical tips on how to get started with Metaflow and extend it through plugins.
Results
As Metaflow is an open source project, the whole data science community in the world is now able to use the tool. The feedback has been astonishingly positive and the Netflix ML team has boosted their productivity significantly. This means better matching show recommendations for you and me.
“I’d like to give a really big thanks and shout-out to Codemate. They’ve been an amazing partner. The design work was excellent and through several development iterations they really delivered an excellent product.”
Brett Rose
Manager of ML infrastructure at Netflix
Want to learn more?
Ask more about ML/AI, data science or open source development from Toni Piirainen.
