Why we must always move to a world where analyses are the default first step
It’s been nearly 2 years since Seth Rosen’s trashboard tweet, and yet the trashboard epidemic remains to be quite real. We still encounter firms which are curiously comfortable with having 1000’s of dashboards, their analysts left to spelunk deep caverns of ad hoc requests with naught but an IDE.
There are good intentions throughout, after all — ad hoc work will be painful, and an automatic solution sounds appealing. What’s more, dashboarding is a secure bet — we still live in a world where the dashboard remains to be accepted as The Default Tool For Analytics, in spite of everything.
Let’s discuss how we came, why this needs to vary, and where we must always go from here. Briefly, in what follows I’ll argue that:
- A long time of heritage have brought us to a weird place where dashboards have primacy.
- But dashboards aren’t at all times the perfect solution to provide value to your org, especially after the primary few.
- When you consider the 2 statements above, your beliefs, your processes, your tooling must fundamentally change.
Dashboards vs. analyses, by the use of cooking
So don’t misunderstand — dashboards are incredible, they’re just blunt. They will serve countless needs, but in doing so, rarely do they serve particular needs perfectly. It’s the Heisenberg uncertainty principle for data: the more data you expose, the less sharp your insight. Dashboards, by nature, prioritize data exposure. They provide data by itself, and interpretation is left to the buyer. A certain variety of core dashboards are definitely required, after all. Every business unit has core metrics and events that it tracks, and these need to be surfaced somewhere — the dashboard here acts because the heartbeat of the business. But beyond this, value greatly diminishes after each nth dashboard.
Analyses, alternatively, are sharp. They expose interpretation, enabling more precise relevance to business problems. If dashboards are like salad bars, analyses are made-to-order meals. And just as specific meals fit cravings higher, so too will analyses at all times fit specific business needs higher. However, incremental salad bar items (more dashboards) have diminishing returns — they’ll never quite satisfy a specific craving, but they’ll crudely solve most hunger-related problems. You possibly can profit greatly from having just a few, but you don’t must proceed constructing them endlessly.
Then why do dashboards still have primacy?
Although I’ve found folks generally agree on that sentiment, we live in a peculiar time in history where dashboarding still retains its primacy. It made sense when data was slow, data was hard to handle, and dashboards were cumbersome to make — core dashboard maintenance was everything of an analyst’s job. But because the warehouse sped up, as analytics needs modified, because the returns on additional dashboards diminished, our mindset never adjusted accordingly.
We’ve in some way turn out to be trapped by some perversion of the sunk cost fallacy: we’d allocated all incremental resources to dashboarding, and so we began to consider dashboards were of the utmost importance.
I’ll concede that there’s an inherent attractiveness to dashboards: dashboards allow us to construct with high coverage in mind. They sell the dream that our ad hoc request volume will drop to 0 in the future. Sooner or later, given enough dashboards, we’ll have addressed all questions and all follow-up questions which are to come back.
But unfortunately, running a business is never so formulaic. Power law distributions proliferate. Preferential attachment begets pockets of value and worthlessness, change and stagnation, accumulation and void. Not all things are equal. It’s the world of the Pareto principle, where avalanches drive change. Where 20% of the trouble drives 80% of the outcomes.
And on this world, analyses aren’t any exception. When you were to rank order all of the work you’ve done by impact, I think you’ll find most of your value add got here from just just a few key analyses. And on this world, the dashboard flounders — what we gain in exhaustiveness, we rob from the avalanche.
I’ve long been a proponent of elevating the role of the analyst. But I seldom do analytics work nowadays, and after I recently got my first ad hoc request in months, I’ll admit I jumped directly into the info. I had a robust urge to reply the query as quickly as possible — I viewed the work as a hindrance, not a possibility. It was an try and diminish the work required of me, but in doing so, I fell from Co-founder to human API. I fell victim to the bad loop ruining analytics, whilst a self-proclaimed curate of this borderline analytics religion.
“The whole lot is a self portrait. The whole lot is a diary. Whenever you care about something it becomes fairly obvious to everyone else. Whenever you care, your devotion or discipline is directional. Most individuals can intuitively feel it.”
– Chuck Palahniuk
Ergo: we want to vary our defaults — our beliefs, our processes, our tools — or risk falling back into the standard habits.
Belief: value beyond data pulls.
Step one in driving any kind of behavior change is to merely internalize that there’s a transparent reason why you’re doing what you’re doing. We want to consider that our worth add extends beyond getting data. By all means, when you consider that your organization’s primary need is a core set of dashboards, then set those up and focus your efforts on that. But when you’ve got an inkling that you would be able to provide leverage outside of this — by helping folks make higher decisions, or by even keeping others intellectually honest within the conversions — then internalize this and proceed under the belief that it’s true.
Processes: analysis-by-default, not dashboard-by-default.
To some extent, I consider all of us wish to have more impactful roles, drive more business value, take part in higher levels of strategic problem-solving. But beyond simply recognizing that, the remaining hurdles are still substantial. In my estimation, probably the most debilitating hindrance henceforth is that it’s far too easy to fall into bad habits. We want to construct systems that make it easy to do the correct things and harder to do the incorrect ones.
“Goals are good for setting a direction, but systems are best for making progress.”
– James Clear, Atomic Habits
I’d start by considering a move to an analysis-by-default world, particularly when you’ve already invested heavily into self-service systems and a set of core dashboards. If we spend our IC time attempting to maximize the usefulness of our work (evaluation), somewhat than attempting to maximize the scalability of our work (dashboards), it follows then that our work will find yourself being more useful, by definition.
Beyond that, construct templates, standards around how work ought to be done. Write up hardened operating principles to your team. You would possibly even set rigid policies to implement higher behavior: analyses don’t count unless you set them in our centralized, agreed-upon place, as an illustration. Adopt rituals that reinforce good behavior: have every day stand-ups where you present this work; bake ritual compliance into your performance reviews. Do whatever it takes to ensure folks do the things that will make them higher.
If we consider in analysis-by-default, we want a set of tools that inverts common working patterns and pushes us to construct analyses first, then dashboards, not vice versa. Definitely I’m biased — this has been the philosophy behind constructing Hyperquery — but even when you’re not sold on our vision, even ensuring you don’t jump into Tableau for each evaluation is a worthy first step. At minimum, narrative-first is a must. By simply forcing yourself to default to words as deliverables, somewhat than data, you’re already breaking out of a ruinous habit.
Once I was at Airbnb, I might often get questions from my stakeholders. And for a very long time, my first instinct was to leap to SQL, immediately. We’ve all done this: in the need to offer quick value, we go full Minority Report, live coding. There’s a time and place for this, but we’re far too desirous to go there when we must always pondering as an alternative — synthesizing and drawing on learnings from the wealth of past experiments and deep dives we’ve already conducted. We fail to offer leveraged value, and as an alternative seek approval for our flashy (unrivaled, sure!) technical prowess. And this quietly becomes the worth others expect of us.
Narrative-first forces alignment, relevance, value over immediacy. Narrative-first elevates the conversation beyond technical contributions to strategic contributions. Narrative-first nudges to your stakeholders that your value lies in narratives, not code.
Analytics is ripe for change. Even when you don’t buy the arguments I’ve laid out, it’s undeniable that our industry has modified astronomically within the last decade. And with latest cars come latest roads, latest laws. At minimum, I hope this post pushes you to take into consideration what that latest world ought to appear like.