One of the more common mistakes made by companies engaging on the data and reporting strategy journey is taking a top down approach. Making technology and process choices based on assumption, rather than necessity or requirement. 

Uncover your "Actual Needs" 

More often than not companies implement a solution driven approach that is a cookie cutter model often not delivering on their actual needs. If its good for one business, it must work for my organisation, right?. Wrong. The most important thing to do as part of your Data and Reporting Stratgey journey is to become completely intimate with the business and its needs in terms of data and reporting and execute a plan that suits your organisation and its goals.

At the essence of success within any company data and reporting strategy is understanding the actual business needs driving the change and the strategy itself. Taking a bottom up approach will deliver your organisation architectures and solutions that suit your specific needs and will essentially drive forward progress within the organisation. The trick to defining your strategy is allowing for change and the ability to pivot when required. You won't ever plan and execute on a single strategy, that will see you through till retirement. As the business and its strategy change and adapt, your data and reporting strategy needs to also be able to adapt easily to match the business needs. If you have the right architectures and processes in place - Pivoting and change will be a much easier thing to do over time.

Eliminate Waste

Throughout the process of identifying your businesses "actual needs", you need to openly and honestly flag areas of inefficiencies in your existing process. Examples of these are; manually producing reports when they can be automated, manually extracting data from multiple source systems (happening widely across the business), staff not willing to change or adapt to improve, doing things because "thats the way they've always been done". Call it early and act on eliminating the waste, otherwise it will linger for years to come and you'll burn many hours and dollars by not addressing the problem..

It's amazing how many people hours add up for all the manual and inefficient processes companies accrue over the years. The key is to no longer ignore these inefficiencies, and act on them as part of continuous improvement, built into your data and reporting strategy.

To eliminate waste (not in one big hit), adopt a kaizen philosophy. By continuously improving standardised activities and processes, kaizen aims to eliminate waste. When used in the business sense and applied to the workplace, kaizen refers to activities that continually improve all functions and involve all employees from the CEO to each individual staff member. It also applies to processes that cross organisational boundaries.

Its easy to ignore the obvious and tackle the bigger problems, but the smaller wins tend to add up and change the way an organisation operates from within.

Traditional Data Warehouse Architecture

Old School Microsoft/Oracle...What's wrong with this approach, and what can we learn from mistakes made?

Traditional data warehousing architectures generally point to a single monolithic data store, usually sitting on (expensive) physical kit in a (expensive) data centre. You see source system databases talking directly to complex logic within the ETL process (tightly coupled), the DW approach doesn’t encourage source system data cleansing as the poor data quality is masked within the ETL process, it limits the ability to ingest huge data sets due to physical storage costs, direct schema access (trust me, thats not a good thing), overloaded responsibilities to collect every speck of data, arrange the data, store history and querying, it's a single component that when changed impacts many consumers, it's a complex ecosystem as well as being seen as one system to rule them all.

This approach generally leads to the formation of isolated Data Warehouse teams who spend most of their time optimising database tables and stabilising performance over spending time with actual business users focusing on actual needs.. The result is a big Data Warehouse "Pet" that is very difficult to pivot from and ends up growing and being maintained with little value.

Before I get attacked by the traditionalists, I've lived both worlds and am not the "anti-datawarehouse" guy. I see value in certain components of traditional data warehousing, but used in conjunction with a loosely coupled architecture that supports decoupling systems that make things easy to move and change when it needs to. The emergence of cloud based data solutions, has no doubt changed the data storage and warehousing landscape for the better.

With the emergence of BIG DATA and its explosion across application and solution providers, we no longer have the old days of a ORACLE vs. Microsoft decision, or the need for businesses to shell out (top end of) hundreds of thousands of dollars up front for a solution and its hardware - We now have the complexity of deciphering the myriad of data store/analytics solution vendors, pimping their wares in the cool "new world of big data". Choice and variation is a good thing, but it also adds complexity for organisations defining and executing a relevant data and reporting strategy that suits "their needs". Defining different approaches and new tool sets to achieve your expected results can be a hard journey without the right knowledge and guidance. 

Decouple and Simplify - Target Architectures

Over the many years I've spent in data and analytics, one of the key successes I've adopted is decoupling systems and making things flow more freely - I'm talking independent data stores, service based architecture and end points that users can connect to, extract what they need and store it for use in their own bounded context. In the event of a system (imagine a change from an old CRM to something like Salesforce) or business change (buying or selling a new business and the related integrations), things are much easier to manipulate and manage in a decoupled world over the traditional data warehouse, ETL type scenario. 

A Service based data architecture with independent data stores create a much clearer visibility of component custodianship and it broadens data availability to the wider business. When adopting a service based approach it enables the ability to vary technology methods (Data stores / Platforms) based on the individual requirement – Suiting the needs. The approach does not lend it self to one technology to rule them all. You essentially adopt technology and methodologies that is suited to a specific need at any point in time. 

Independently built and deployed services / data stores, deployed through interfaces promotes data cleansing at source systems and data usage in its own bounded context. My philosophy is to move away from having one single monolithic database (data warehouse) as the only option, to independent data stores (in conjunction with some type of data warehousing - Redshift, as an example)

One Reporting Tool to Rule them all?

I think not. I spent a great deal of my earlier career trying to force people into using the corporately chosen "tool of choice" for internal reporting, and it was a hard gig. You'd inevitably spend large amounts of cash on Business Intelligence platforms that "can do everything" then spend months trying to figure out why people across the business don't use the shiny new tool the company implemented. The key to success in this space is to let your teams choose the tool that suits them, to get the job done in the most efficient manner. I don't by any means, suggest adopting hundreds of tools and trying to support them, but get the right mix of 5 or 6 different options that cover the bases internally for what people are looking to do and let them run with it.

One of the pitfalls of having HUGE Business Intelligence systems that do everything, is that they are usually quite complex to operate and tend to need specialist skills to get any real value from them. Simplicity is key, again use a bottom up approach and define what the actual needs are before jumping into solution mode. More often than not, a simple and easy to use solution will achieve the results you're looking for.

Decentralise Reporting Capability 

One of the key factors to success in terms of achieving tangible results around reporting (that actually get used or looked at) and gaining insights from data is FREEEEEEDOM...and decentralisation of the "reporting capabilities". For a company to succeed in this space, the people working in the different business units, who need the data and insight, need to gain the skills to self serve and build, maintain and update their own reports. The moment you have a separate, disconnected (centralised) team that are the "reporting specialists", you've already lost the fight. Having some specialist skills embedded across the business works well, but as soon as they become a centralised team the requirement vs output is often misaligned. This is where you end up with many manual, additional processes to patch and work in conjunction with the reports developed by a centralised team - essentially you end up doubling up on the work and effort. Not a good outcome!

Give your teams and staff the freedom to access, create and interrogate the business data an information they need, in the most efficient manner possible.

Now, as mentioned above, a TOTAL cookie cutter approach is not what I recommend for any type of strategy, application, or data & reporting approach - so, some of what's written above, may not suit your specific business needs or circumstance. There will certainly be parts that can be adopted, and adjusted to your operations. You don't need to reinvent the wheel, but you need to apply the right lens when adopting certain ways of doing things and decipher if they are right for your business. The key to success in this space, is spending the time to truly understand your business, its overall strategy and how a data and reporting strategy fits into that bigger picture. Be true to your business needs and execute a strategy that will deliver your organisation and its staff, the specific results your business needs.

Data and Reporting strategies are not an easy thing to get right, but when it happens, you'll see such a positive difference in how data is used and valued within your organisation. I'll be writing a lot more in this space in the hope of showing you some of the mistakes I've made, and lessons learned that have essentially guided me to a better place in terms of Data and Reporting over the years.

Drop us a line here at Vivify Labs to discuss how we can work with you on embedding a suitable strategy for your organisation. This email address is being protected from spambots. You need JavaScript enabled to view it.