At this point, few if any Fortune 500 companies have missed the memo on the advantages of building custom AI. They’re hiring data scientists at a record pace, a sign that Machine Learning has moved beyond Tech into every industry and sector. Enterprises had already internalized that “every company is a software company” and today they’re wrestling with how to become AI companies as well. Better productivity, competitive advantages, and outsized returns await.
The peril, though, is that developing custom ML models is hard — and takes more than simply pointing teams of data scientists at a business problem. Even sophisticated Tech companies continue to wrestle with streamlining ML development workflows. Added to that is regulatory pressure to also be able to explain how their models work (once they work).
Scale was an early investor in JFrog and CircleCI at a point in time when DevOps was gaining early traction inside enterprises. We see parallels in the state of enterprise AI today, with a push toward “MLOps” to formalize and automate how enterprises build custom AI. DevOps drove the development of new tools and processes and created large new software companies along the way; MLOps is on a trajectory to do the same.
Today we’re announcing our investment in the Series A for Comet. Comet’s MLOps platform lets data scientists manage, debug, monitor, and share ML models. The company is also announcing a new product aimed at models in production, more on that below.
Comet Does for MLOps What Github Did for Code
Comet’s co-founder and CEO, Gideon Mendels, shared a story that explains how the company came to be. As a new team member at Google, Gideon was asked to improve an ML model already in production — but without access to records about how it was made. It was like asking a baker to improve on a cake without also sharing its original recipe.
Data scientists use Comet to track Machine Learning experiments throughout the iterative process of training with various inputs (data, hyperparameters, algorithms, etc.) to achieve a desired output. Comet provides the logs and visualization tools to understand which inputs lead to which outputs — basically, documenting the recipe as it evolves over time.
The company uses the tagline “Doing for ML what Github did for code” to highlight how it is replacing ad hoc tools and processes with products that work the way data scientists prefer to work. It is clear from our conversations with enterprise data scientists that there is a real need for Comet in the ML tool stack.
Comet supports teams as small as a single data scientist running experiments in the cloud to large teams running them on-prem, enabling them to:
- Manage data sources and experiment results
- Debug models to see what’s not working
- Share results with other data scientists and adjacent teams (e.g. data labelling team)
- Monitor model performance, including errors and skews
Users can easily compare experiments — code, hyperparameters, metrics, predictions, dependencies, system metrics, and more — to understand differences in model performance and thus build better models faster through increased productivity, collaboration and explainability.
With today’s announcement of Comet Model Production Monitoring (MPM), the platform now provides visibility across a model’s entire lifecycle. It’s a big step forward in consolidating the tools and records that data scientists need to continuously build, deploy, and improve ML models.
Strong Team Focused on a Massive Market
Comet’s explosive growth to date has been driven by an efficient bottom-up sales motion from a small team. This is a key signal, because in general product teams have not previously been an important class of software buyers. But this is changing with the pressure to leverage AI for competitive advantage and the increasing prominence of data science inside companies. Comet’s growth is a strong signal that they’re addressing the needs of these buyers and adding value to their workflows. The fact that Uber’s teams replaced their in-house tools with Comet further strengthens the argument.
We’ve enjoyed getting to know Comet’s co-founders. Gideon Mendels, CEO, was previously a deep learning researcher at Google as well as Columbia University’s Spoken Language Processing Group. He is complemented by Nimrod Lahav, CTO, who held senior R&D and development roles at Wix and VMWare.
If their customers have anything to say about it, Comet is on a path to become a key system of record for data science, adding value to how custom AI is built inside the enterprise.
Eric Anderson contributed to our investment in Comet.ml.