A New Use for Autoencoders in the Utility Industry
In This White Paper
Each day, utility companies use smart meters to collect an enormous amount of data on energy usage. In this white paper, you will learn how to cluster customers based on patterns in their electricity usage, as well as be presented with potential service enhancements utility companies may be able to make based on results.
Smart meters used by electric utilities collect an enormous amount of data on energy usage each day. Leveraging this data, we can better understand how individual consumers use their electricity by clustering customers according to their pattern of electricity usage. Clustering, however does not perform well on high-dimensional data such as meter readings. To overcome this, a type of deep neural network called an autoencoder can reduce the dimensionality of the meter reading data to a representative set of relevant patterns. Clustering then can be performed on the lower dimensional, representative data to group together consumers with similar patterns of usage. This clustering is a fully unsupervised technique. In general, this process of reducing dimensionality through auto-encoding and subsequent clustering may be used to characterize a variety of high-dimensional data sources other than electric meter readings.
In collaboration with a utilities industry consortium (the Consortium) and a regional utility company (the Company), WWT has been working on a project to classify electricity consumption behavior. The analysis presented lays a foundation for several use cases: categorizing general patterns of electricity demands, detecting specific electric loads the customer owns, identifying distributed energy resources, and predicting future energy usage.
Smart meters are electronic devices which collect information about utility usage, generally at a frequency of at least once an hour, and report this data back to a central repository. Utilities have been collecting this data for many years. With hundreds of thousands of customers, and thousands of records per customer, big data methods must be used to analyze this large source of high-dimensional information.
Artificial intelligence offers a new way of using raw data to understand energy consumption behavior of individual residences and businesses. This analysis will aid in estimating the future impact on the grid of each business and residence. With the rise in solar panels, electric vehicles and home batteries, and the increasing abundance of electric devices, leveraging these insights will be necessary to prevent outages, reduce energy waste and keep costs down.
The analysis described in this paper takes a 3-step approach to classify electricity consumption behavior in the Consortium data set:
- Reduce dimensionality of the image leveraging an auto-encoding technique based on neural networks.
- Cluster the lower-dimensional meter reading images using a k-means clustering technique.
- Analyze and classify energy usage patterns of the cluster.
The raw data in this study comes from an AMI (Advanced Metering Infrastructure) dataset of 150,000 meters. The data is stored in a table with columns holding:
- Meter number.
- Time (at an hourly frequency).
- Meter reading (integer kWh).
A separate dataset references the transformer, feeder and substation associated with the meter, and the approximate latitude and longitude of the meter.
Some of the meter data is corrupt or missing. This study only used meters for which location data is available, and which have a meter reading for over 90% of the hours over two years. Approximately 80,000 meters were used in all.
The raw data is stored in a Hadoop lesystem and preprocessed using HiveQL. The autoencoder is trained on a NVIDIA DGX-1, using four Tesla V100 GPUs. The autoencoder so ware is written in python jupyter notebooks using numpy and keras packages. Images are plotted using matplotlib, Microsoft Excel and Tableau.
Meter readings as an image
Figure 1: A 1D representation of raw meter readings over two years.
Figure 2: A meter reading image. Lighter pixels represent higher usage.
To better understand how to interpret the image in Figure 2, the following are key dimensions:
- Pixel brightness indicates higher energy usage.
- X-axis represents hours in a given week.
- Y-axis represents weeks in a given year over two years.
This creates the 168×104 image that represents energy usage by one customer over two years by day of week. Patterns by the season and day are noticeable to a human eye.
Representing the meter data in an image helps to understand why AI has an advantage for the problem of segmenting consumer energy usage. A neural network can be trained to find relevant patterns in the meter readings, just as a human can see in the image. The image also aids in our final analysis, as we can compare visually which meters the neural network decides to group together.
Deep autoencoder training
To reduce dimensionality of the meter reading data, a type of deep neural network called an autoencoder is used. The autoencoder is fed a sample of data from several thousand meters and trained to compress the full set of meter readings down to just 5-10 representative values. These encoded values can be thought of as representing the most relevant features or patterns of each meter.
The autoencoder works by training a feed-forward neural network to match the input data with itself. This sounds like a simple task, but the network is forced to hold the information in a bottleneck, where the hidden layer in the middle of the network has very few values. This hidden layer holds the compressed representation of the meter data.
Figure 3: A simple autoencoder with one compressed hidden layer.
The autoencoder has two parts, as shown in Figure 3: the encoder, which generates the compressed representation; and the decoder, which takes the compressed values and reconstructs the meter readings. The neural network is trained to reduce the difference between the input and the reconstruction.
Figure 4: Three designs for the autoencoder, increasing in complexity. Each box shows the number of nodes in each layer. The first and last layer have the dimension of the input data, 17496.
A deep autoencoder has more hidden layers and more nodes in each layer, increasing the complexity of the information the neural network can encode. Figure 4 shows three designs for the autoencoder tested on the meter data. A deeper neural network takes longer to train and having too many nodes can affect how the neural network converges. Having too many layers or nodes can also decrease performance overall, as the neural network has a greater chance of overfitting.
The size of the middle layer is important. A larger middle layer allows for more traits to be encoded. However, a smaller middle layer gives a clearer way to compare two meters and allows for a more robust clustering algorithm. After some experimentation we settled on a middle layer of 10 values for this study.
The hidden layers of the neural network in this study were densely connected layers with LeakyReLU activation functions.
Unsupervised clustering algorithms perform poorly with data that is high-dimensional, like the meter readings, which each have over 17000 values. Clustering the meters using the encoded representations, which have a much smaller number of dimensions, separates distinct groups of meters more efficiently.
For the initial clustering analysis, a k-means algorithm was used. To show this method as a proof of concept and easily identify general classifications of behavior, consumers were only clustered into a small number of groups in this analysis.
The autoencoder trained on 50K sample meters compresses the meter reading data from 17496 raw values down to 10 real numbers. Evaluating a separate test set of 20K samples, the meters can be reconstructed with an average loss of 0.97. This means that on average, the reconstructed meterreadings have an error of 0.97 kWh in comparison to the original input.
Figure 5 shows one example of the reconstructed reading. Notice the reconstructed reading image provides the "essence" of the raw reading and does it when only given 10 real values. This allows for only those 10 values to be used in the clustering step, drastically reducing dimensionality and allowing the clustering to more easily converge. In comparison, simply using the k-means algorithm on the high-dimensional raw data takes much longer, and results in one group which contains almost every meter.
FIGURE 5: TOP: RAW METER READINGS. BOTTOM: OUTPUT AFTER COMPRESSION AND RECONSTRUCTION.
Figure 5: Top: raw meter readings. Bottom: output after compression and reconstruction.
We explored clustering meters into several different numbers of groups. Having many clusters may separate out specific behaviors, but makes it takes more effort to analyze. Clustering the meters into four groups results in distinct patterns among each group, which better displays the overall benefit of this approach.
The distinctions between clusters can be seen simply looking at the average monthly usage of each cluster. In Figure 6, the clusters labeled 0, 1, and 3 all have a similar pattern of usage, having higher readings in the summer than the winter. These may be similar types of consumers segmented by size of the residence, as a larger residence likely uses more electricity. Cluster 4, however, has a much different pattern of usage, having higher usage in the winter than in the summer. Consumers in this cluster may be more likely to use electric heating over the others. Most residences use gas for heating their homes, which does not affect electricity usage. Consumers using electric heaters would have a spike in usage in colder months when heaters turn on.
Figure 6: Average monthly usage by cluster.
Certain locations have higher concentrations of one type of cluster over another. This may occur in areas with similar types of residences. The map in Figure 7 shows an area on the left which predominantly has consumers in clusters 2 and 3, those with high and moderate usage and more during the summer. On the right side, most of the residences fall into cluster 4, the consumers with more usage in the winter. A variety of reasons, such as the age of the homes or economic factors, could account for these regional traits.
Figure 7: A sample of meter locations, plotted by cluster type. Clusters 1,2,3, and 4 are purple, blue, red, and green respectively.
Knowing which areas have higher concentrations of loads like electric heaters can be useful for utilities. For this initial investigation, only a simple clustering process was used which identified broad patterns of usage. Other clustering techniques could help utilities classify more specific consumer behavior. With additional information and a more sophisticated clustering process, it is possible that traits such as solar power generation or electric vehicle usage could be distinguished.
A novel approach to segmenting residences by their energy usage was developed. The 3-step process of reducing the dimensionality of meter data using a neural network, clustering and analyzing the resulting clusters proved to be a valuable method for defining behavioral patterns of energy consumers. Though only a few clusters were analyzed, broad categories of behavior were discovered. Clear patterns of usage emerged based on the season and amount of energy consumption. The information gained would aid utilities seeking to classify customers based on raw meter data alone. It has the potential to distinguish specific loads such as electric heaters. Segmenting consumers with similar usage patterns could inform utilities which customers are at the highest risk for outages or equipment failures.
Continuing this effort in the future, a finer clustering method could be employed and additional information about the customers could be leveraged to classify more specific types of consumers, such as those generating their own solar energy or using an electric vehicle. The current work presented uses only an unsupervised method; with more information about weather or solar electricity generation, supervised learning techniques could improve the process as well.
The results of this clustering may also be valuable for predicting future energy usage. With more knowledge about the types of loads and usage patterns, more accurate prediction of energy usage can be achieved. Utilities already collect information to classify customers and use this to aid their energy usage forecasts; the novel classifications found by our clustering method could help to improve upon these predictions.
As another area of exploration, the autoencoder design could be optimized. Using the meter reading image representation, it is possible that convolutional neural networks could be utilized, which are known to improve image recognition.
The collective understanding of neural networks and autoencoders is increasing rapidly. These advances in AI have the potential to unlock many new uses for the vast amounts of data being collected. The research in this paper offered a new use for autoencoders in the utility industry, but the approach has viable applications in other realms as well. Many sources of high-dimensional data have the potential to benefit from an autoencoder: text documents, medical images or data in a time series like the meter readings. An autoencoder would reduce the size of these pieces of data, identifying only the most relevant patterns and allowing for a more productive analysis.