I agree, but I think there is a difference between metrics and having data to investigate. A metric is a value you are tracking over time, but when something changes you will likely need to do an investigation. That investigation might be the raw data, it might be some dynamic queries or it might be other metrics.

Tracking metrics because you think they might be useful in the future for diagnosing other problems doesn't make sense with modern systems as dynamic queries are so fast.

Ha, and I bet there is a metric for that too!

Yeah, it's hard to generalize these kinds of things since companies can operate so differently. I've seen very large companies where everyone can use the same 5 metrics and others where each team needs their own set of 5. I think the key thing is that no team is using anymore than absolutely necessary.

But it's not always clear what is "green"? For example, New Users going up can be a good thing but New Users going up by a lot can represent a fraud attack. Metrics always require some interpretation to provide value which is why we have dashboards instead of alerts.

There are some metrics that are binary good/bad and I agree that in those cases you should just have an alert.

YMMV but this is based on my experience building some of the largest analytics platforms like Flurry and Outlier.ai which were used by hundreds of thousands of companies. The only dysfunctional company I worked at was Verizon and they... don't really use metrics.

That would be true if you didn't assign a cost to each metric that you add. It was when the cost of tracking metrics plummeted that this started to become a problem, as there was no external friction in collecting more. If you assign a cost, and respect that cost, you shouldn't continuously add more.

YMMV but this was based on dozens of conversations with companies where they spent hours each week reviewing those dozens of dashboards. I was specifically talking about the metrics used by any given team, as you are right that different teams might use different metrics.

"The key is that any given person shouldn’t be using any more metrics than absolutely necessary to do their job well."

I've seen too many companies waste time on metrics theatre, tracking hundreds of metrics while knowing less and less about how the business is actually doing. The more metrics you have, the less you know.

There are two types of metrics you need: Success and Velocity. One tells you where you are going and the other tells if you'll get there.

Hi! (Founder of Outlier.ai here) You are right, our platform is designed to produce the most important insights from massive amounts of data, without requiring human supervision/configuration. It is most useful in applications when there is too much data to set up guardrails, or the teams don't know what guardrails to create. Our typical customers are very large consumer businesses who have data spread across dozens of systems and need to ensure they never miss important emerging trends or problems.

We are not an alerting or monitoring system, so I don't think you'd use us for the same applications as Orbital. The typical users of Outlier are the business users ranging from executives to business operations who want to make sure they are asking the right questions about the business.

Orbital looks like a great product, good luck in building your business!