Home Artificial Intelligence The Data ROI Pyramid: A Method for Measuring & Maximizing Your Data Team Calculating data ROI Calculating data investments Levers to optimize data investments Levers to maximise data product return Calculating data downtime Levers to reduce data downtime Calculating Data ROI Is Hard — But It’s Value It

The Data ROI Pyramid: A Method for Measuring & Maximizing Your Data Team Calculating data ROI Calculating data investments Levers to optimize data investments Levers to maximise data product return Calculating data downtime Levers to reduce data downtime Calculating Data ROI Is Hard — But It’s Value It

0
The Data ROI Pyramid: A Method for Measuring & Maximizing Your Data Team
Calculating data ROI
Calculating data investments
Levers to optimize data investments
Levers to maximise data product return
Calculating data downtime
Levers to reduce data downtime
Calculating Data ROI Is Hard — But It’s Value It

Struggling to articulate the worth of your data team? Learn how one can calculate your data team’s return with the Data ROI Pyramid.

Towards Data Science
Image by writer.

As recently as a yr ago, about half of the info leaders I spoke with felt the business value of their team sold itself. Today, maximizing and measuring data team ROI is near the highest of each data leader’s agenda.

Most data team ROI formulas concentrate on some version of the next calculation:

Lift / investment = ROI.

And while there’s definitely value in its simplicity, it doesn’t capture the total value of the info team. For instance, how do you capture the worth of:

  • A customer churn dashboard
  • A dataset that supports ad hoc queries of engagement behaviors
  • A migration to a brand new data stack that supports faster, more scalable compute
  • Increasing data adoption by 30% because of this of a knowledge quality initiative

It’s challenging! Grizzled data industry veterans who’ve successfully turned the wild world of customer acquisition funnels right into a predictable science start squirming when looking inward.

During the last six months, I’ve spoken with data leaders and iterated on a wide range of ROI formulas, all in an attempt to achieve some extent that, if not capturing the exact value of a knowledge team, can get us slightly closer.

And the results of those conversations has been none aside from a brand recent data ROI pyramid. Yes, I’m aware of all the nice pyramids and conjoined triangles which have come before but this one is different. Kind of.

The goal of this pyramid is aimed squarely at helping data leaders

  • Catch up with to the business
  • Balance competing priorities
  • And concentrate on the proper metrics to generate value for his or her stakeholders.

So, with the preamble out of the best way, let’s take it from the highest!

Image by writer.

As a general rule of thumb, the upper your report in a company, the less and more encompassing your metrics will develop into. The CEO doesn’t care what number of dashboards you support or your data freshness SLA adherence percentage.

They need to know what their investors need to know: “Am I getting a return on my investment?”

The info ROI pyramid tackles this query with an analogous formula to the one within the introduction:

(Data product value — data downtime) / data investment = ROI

…but there are two key differences. The primary is the more expansive definition of “data product revenue” (more on that later) and the second is the introduction of knowledge downtime.

The downtime variable is essential to incorporate because as more data teams drive higher levels of revenue–via machine learning models, customer-facing apps, data democratization, and other initiatives–the more severe the implications of downtime becomes by way of lost time, revenue, and trust.

This also makes reducing data downtime one among the three key strategies for data leaders to extend ROI: you possibly can either increase revenue, you possibly can decrease your investments, or you possibly can reduce your data downtime. And one among those fruits hangs quite a bit lower than the others.

So, now that now we have a framework for calculating ROI, let’s dive deeper into how we discover the variables.

Image by writer.

The formula is simple — investment = people + solutions.

However it’s also easy to overcomplicate.

Some contracts are annual. Others usually are not. Some solutions charge based on usage. Others don’t. My advice here is to maintain this component relatively easy. Follow an aggregate projection of costs divided evenly across a time period (typically a month or quarter).

Relating to optimizing your data investments, the secret is efficiency. To get essentially the most out of your data investments, it’s worthwhile to improve the speed at which those investments deliver value.

Below are three levers you possibly can pull to enhance efficiency in your data systems, your data teams, and your data consumers.

  • System optimization — The associated fee of virtually all modern data solutions relies on usage. The metrics it would be best to regulate listed below are the full variety of items that drive these costs (tables, queries, VMs, warehouses, etc) and the outliers at the highest (expensive queries) or the underside (unused tables). Just a few ways to grasp and control system costs might be through assigning ownership for domains, cleansing up unused assets and expensive queries, and even consolidating your data stack around central tooling.
  • Time to construct and maintain — The time it takes to construct and maintain your key data assets, including data products and machine learning capabilities, is a key lever that measures your data team’s productivity. While larger upfront investments could also be required to develop an efficient data platform to supply, simplifying the workflows for constructing and maintaining data pipelines can have a dramatic impact on data team efficiency.
  • Time to insight (or motion) — This lever is concentrated on the time it takes your data consumers to understand value. In other words, how effectively has the info team enabled your data consumers? Discoverability and self-service can each work to enhance time-to-insight for consumers, while micro-batch infrastructures could make that data available at the bottom possible latency for machine learning, analytics, and reporting.
Time spent enabling data self-service is normally well-spent, however the pay-off have to be greater than the hassle spent to construct and maintain it. Image courtesy of Shane Murray.

Calculating data product return is by far essentially the most complex step in our ROI calculation. That’s because as the info industry continues to advance, the range and complexity of knowledge product use cases has continued advancing right together with it.

Fortunately, data products can generally be split into one among three primary categories: analytical data products, operational data products, and customer-facing data products. These can exist in the shape of dashboards, ML models, experiments leveraging data insights, and — in fact — generative AI. While the latter could also be a bit more complex to construct, genAI remains to be a knowledge product at its core and its value can still be calculated using the methodologies we’ll outline below.

To actually answer this query, the formula for data product return must be inclusive of all data team activities, whether or not they produce value directly (for instance revenue generated via a paywall ML model) or not directly (for instance a customer churn dashboard).

The formula I might propose for calculating data product return is that this:

Value from Analytical Data Products + Operational Data Products + Customer Facing Data Products = Data Product Return

As you possibly can see, this a part of the equation presents essentially the most variables for our ROI calculation. Since this a part of the equation is by far essentially the most complicated, we’ll be spending nearly all of our time here. So, with that in mind, let’s take a take a look at each subcategory (or use case) in a bit more detail.

Analytical Data Products

First, let’s dig into the worth of essentially the most traditional — and customary — data use case: analytics.

Analytical data products are a combination of the critical dashboards, ML models, and experiments conducted and supported by your data team to deliver insights for decision making.

Whether we’re talking in regards to the marketing dashboards or necessary metrics like customer LTV, analytical data products play a fundamental role within the day-to-day operations of any business. But in the identical way that every one data products aren’t created equal, the best way you’ll calculate their value isn’t either. And, in fact, some might be easier to calculate than others.

First, let’s take a look at experiments:

Measuring incremental impact

Measuring absolute impact is one among the only ways to grasp return from any data product. And calculating the worth of an experiment conducted because of this of your data team’s research and analytical insights might be quickly estimated by understanding the delta between test versus control and translating those numbers into dollars earned/saved every month.

For a more conservative approach, you could possibly estimate value by computing the return against a random or average decision to higher represent decisions made without support out of your data team.

Combining tens or lots of of those experiments per yr offers you a ballpark figure for the incremental value delivered by the experimentation platform and the analytical work surrounding those experiments.

Measuring value-to-stakeholders

But what about dashboards? Rarely are these initiatives so easily measured by a controlled or natural experiment.

To account for these data products, we’ll must take a more nuanced approach to estimating value. On this case, we’ll be translating qualitative data into something representative by tapping into the consumers themselves.

Imagine it or not, your enterprise users and data consumers are literally quite knowledgeable about how worthwhile your dashboard is (or isn’t) to them. And their responses might be quantified. While this will likely not seem rigorous enough at first glance, this is definitely just like the method MIT economists used to find out how free services contribute value to the national GDP. For instance, they asked respondents how much they might need to be paid not to make use of Facebook or Google Maps for a yr. ($40 to $50 a month for Facebook when you are curious).

For crucial dashboards, data teams can go a step further by making a benchmark for respondents, like “We estimate the associated fee of maintaining this dashboard to be about $5,000 last quarter. In your estimation did it add that level of value to your work over that time period?” For a baseline, here is how our survey of 200 data professionals revealed they judged their data consumers would value their dashboards:

  • Lower than $500k: 5%
  • 500k-1m: 11%
  • 1m-10m: 49%
  • 10m-25m: 32%
  • 25m+: 5%

Customer Facing Lift

Here I’m referring specifically to data that’s customer facing, not the ML models powered by data. This data use case generally is available in two flavors.

The primary is when data IS the product. There are a major number of companies that ingest, transform, after which sell data to other firms. It might be a knowledge mining company compiling insights from web scraping eCommerce sites or a television manufacturer that sells viewership data to advertisers.

On this case, calculating is pretty straightforward: the revenue of the info product is the revenue of the sale. Whenever you find ways to counterpoint this data, you make it more worthwhile and thus increase the sale price.

Nevertheless, what about cases where data is just a part of the product being offered? For instance, some extent of sale system providing insights back to the merchant on their foot traffic patterns? Or a video player that breaks down views across audience segments by time?

In some cases, the info might be a pleasant to have. In other cases, it’ll be a major factor on customer acquisition and retention. Luckily, data teams have been experimenting and measuring the impact of features on retention for some time now.

Image courtesy of Shane Murray.

Operational Lift

I define operational data use cases as activities that MUST happen. Examples would come with reporting to the board or an airline re-accommodating passengers of a delayed flight.

If the info systems went down, these activities would still occur, but they might be considerably more painful. A company could have to manually collect and aggregate data from across the business for its report or passengers may have to go to the client service desk fairly than have an app mechanically present their options for re-accommodation.

In these cases the worth is often best determined by the hours saved between the more automated and more painful process. In some situations, alternative impacts similar to tremendous avoidance or poor customer satisfaction might be calculated as well.

Image courtesy of Shane Murray.

You’ll be able to optimize data product value by increasing how effective it’s and the range of its impact. Some broad proxies to measure this across a big selection of use cases and industries are adoption, reach, and velocity.

  • Adoption and Reach — The more use a knowledge product sees, the more value it might ostensibly provide. So driving improved reach and adoption may dramatically increase the incremental value your data products will provide.
  • Velocity — One in all the largest drivers of experimentation value is velocity: what number of meaningful experiments can the organization execute over a time period? More velocity can mean improved productivity, a more mature central platform, and even improved enablement for data consumers.
Image by writer.

Finally, we want to grasp how data downtime impacts ROI.

In other articles, we’ve discussed how one can calculate data downtime using your incident and response times. Here’s what that formula looks like:

Variety of incidents x (average time to detection + average time to resolution)

This is useful in measuring how your overall data product reliability is trending. But on this case, we aren’t as serious about the combination data downtime or the efficiency of the team (yet).

What we wish to seek out out here is the operational cost for the info downtime of specific data products. To do that, you’ll need data lineage with a view to understand the impact a knowledge issue on an upstream table has across various downstream data products.

Since we’ve already calculated the revenue generated from each of our data products, including key dashboards, we will now subtract the operational cost of that downtime from the revenue.

For this component of the ROI calculation, I like to recommend only specializing in downtime that violates your data SLAs. If a dashboard that’s checked every day has a knowledge freshness issue that only persists for a couple of hours before being resolved, that downtime is unlikely to have an operational impact on the organization (and your data team shouldn’t be penalized for it).

Improvements in data downtime can have a dramatic impact in your data team’s ROI calculation, especially to be used cases where the info is so central to the product offering that data downtime is comparable to operational downtime.

Red Ventures custom built SLA adherence dashboard. Image courtesy of Red Digital at Red Ventures.

We’ve covered the metric the CEO cares about and the strategic metrics data leaders should consider. Now let’s talk in regards to the tactical levers your team can pull to maximise your outcomes across these strategic metrics.

SLA Adherence

SLA adherence, the period of time the SLA has been breached divided by the period of time it has been upheld, may also help data teams maintain a granular understanding of what number of data quality incidents are having an antagonistic impact on specific data assets — and what measures to take top protect the worth of those products.

Coverage%

Minimizing data downtime can often be directly correlated to how well your data products are covered with monitors and custom tests from raw ingestion to final resulting table. Data pipelines and systems are radically interdependent. Higher coverage percentage typically will translate into higher time to detection and backbone.

Status Update%

The teams which might be the very best at documenting their incident history are almost at all times those with the bottom downtime. High status update percentages encourage a lower time to resolution and may even help prevent incidents by providing higher insight into overall data health.

In case your data team has a low status update percentage it’s either due to alert fatigue or because your team is unable to quickly triage incidents.

While I’m under no illusions that calculating the ROI of a knowledge product is either a precise science or a fast win, I’ll ardently uphold that it’s a worthwhile endeavor — and maybe most significantly, an achievable destination.

I’m also confident there are few activities more existentially critical for data leaders.

By higher quantifying and optimizing the info team’s value, we will higher earn our colleagues trust and recognition of the worth we add to the business. At that time, the sky’s the limit for motivated, well-led data teams.

LEAVE A REPLY

Please enter your comment!
Please enter your name here