Large amounts of data have never been more accessible—and it’s forcing utilities to take a digital-first mindset to effectively use it
The amount of data stored around the world is growing at a breakneck pace. It’s expected that by 2025, we will have collectively amassed more than 180 Zettabytes or 167 trillion GB of data (that’s a lot).
Within utilities, data collection is also rapidly outpacing its business usage. When looking at collecting data from meters through AMI (Automated Metering Infrastructure), where new readings are being done every hour (or even more frequently), it’s easy to imagine the growing mountain of data available to utilities.
The amount of data is invaluable—but only when this information is well governed, managed, and protected, which requires a highly skilled and competent workforce aligned to defined processes.
Consider a distribution utility that wants to improve the way outages are being managed. By leveraging the right internal and external data, the company could predict when and where outages occur—and allocate resources accordingly.
To do this, it needs historical outage data to analyze the circumstances of past outages, combined with data on utility assets, staffing, and environmental data (like weather). All this information is often spread out over different systems, spreadsheets, and reports with little or no standardization. Months of data discovery would be required to locate and analyze this data before even considering or predicting any future outages.
It’s no wonder 68% of most companies’ data isn’t used successfully for strategic purposes. If utilities don’t know where the data lives, it’s impossible to derive value from it. That underscores the importance of utilities having proper data maturity and—more importantly—data governance.
Data maturity is the level at which the company incorporates data into its decision-making. In 2016, utilities had the lowest data maturity of all industries. But increased digital operations and use cases around sustainability, outage management, and smart meters have put a spotlight on data usage for insights and advanced analytics. We’ve witnessed significant advancements in the past few years through the development of analytics platforms and the leveraging of cloud-based solutions.
The problem? Increasing enterprise data maturity requires more than the implementation of technology. The more significant factor is data governance, defined as a collection of processes, roles, policies, standards, and metrics that ensure the effective and efficient use of information. Moreover, proper data governance includes key organizational roles—with well-defined responsibilities—to ensure decisions and processes are being appropriately utilized.
Data management and governance in utilities are often viewed as an IT responsibility—with no acknowledgement that data is mainly created and used by the business. Consider the example of predicting future outages: The data of previous outages wasn’t created by IT, and it won’t derive value from that data. Unlocking the value of data and leveraging it for use cases benefits the business, which means the business has the largest stake in adequate data management and governance.
But data literacy across utilities tends to be low, suppressing the democratization of data analytics. Without value-added insights, it’s challenging to garner executive level support for data initiatives, leaving a vicious circle of limited data-driven decision-making.
Data governance is a large undertaking that requires broad organizational support, training, and change management. Even when appointing a data governance manager, it’s often challenging to know where to start.
The first step to effectively using data is knowing where to find it, what it looks like, and who is responsible for its quality. Early questions to ask include:
The best way to achieve this is by creating a data catalog—a service that lets analysts, data scientists, and developers register, enrich, discover, understand, and consume data sources.
Many utilities might already have a data catalog tool implemented—but its full potential can’t be reached without the proper governance around it. An effective and value-adding data catalog needs a data organization to populate and maintain it—it’s an excellent driver of data governance, analytics, and data literacy.
A proper data catalog should have flexible searching and filtering options to allow users to quickly find relevant sets of data or browse meta data based on a technical hierarchy of data assets.
The benefits of a data catalog are simple to understand:
Beyond the high-level benefits, implementing a data catalog to inventory your data has the ability to accelerate data governance efforts. Consider again the outage prediction example: With a well-maintained catalog, users are able to search keywords such as outage, response crew, weather, or feeder to discover the available data that can feed into a prediction.
The catalog should be able to tell where to find the data, what it looks like, and how up to date it is. This both saves considerable time from manually locating the data and suggests data sets that the user might not have been aware of or considered.
It’s important to understand that this information doesn’t simply appear in the catalog as soon as the tool is implemented.
The right roles need to be appointed for populating and maintaining the information. These roles are often within the business, opening the door for a more organized and engaged data governance framework.
But when the catalog is successfully implemented, and when the right people are put in the right roles, these steps have significant impact on supporting data governance:
There is no magic wand when it comes to implementing a data catalog and building the subsequent data governance organization around it. Roles need to be appointed and rules must be defined, and utilities need a commitment from those resources who fill multiple roles with limited time—but even that doesn’t guarantee success. It’s a large undertaking with potential roadblocks that need to be accounted for.
Implementing a data catalog is more than an IT project. It requires business involvement and support to ensure success—a lack of shared sponsorship will doom the initiative.
What’s more, indexing data is a significant undertaking, meaning value will only come over time. Start with valuable use cases and POCs to show rapid value and gain support.
Last, using a data catalog and driving analytics requires training and awareness of data culture. Data literacy needs to increase to successfully implement a catalog and data governance organization.
Managing and governing data within a utility is an ongoing journey. A data catalog is a great way to kickstart that process. Success isn’t guaranteed, and it needs the right support, training, usage, and ownership to accelerate value driven decision-making. But that’s exactly why it serves as an excellent driver for many utilities that are looking to democratize the use of data and start building greater insights.