The focus of this case study of non-intrusive load monitoring (NILM) algorithms by four vendors at 11 non-randomly selected homes in the SDG&E region was to evaluate the rough accuracy of each algorithm in an effort to better understand the state of that industry. Whole building electrical power and energy use data was granted to the vendors at four approximate frequencies of 10-seconds, 1-minute, 15-minutes, and 1-hour. There were three data sources for this whole building data: two Rainforest Automation Eagle gateways obtaining high frequency data from the utility electricity smart meter, different only in firmware; and SDG&E Green Button Connect data. The average sampling and recording rate of the Eagle gateways was 10-seconds and the researchers also up-sampled this data to 1-minute and 15-minute intervals. The Green Button data was at 1-hour intervals for most homes and 15-minute intervals at two homes based on the homes’ electricity rate schedule.
In addition, zip code location was granted to all vendors at the beginning of the study. Home appliance survey data was granted near the end. The vendors were asked to provide disaggregated predictions based on up to all four frequencies prior to and after receiving the appliance survey data. In respect of their valuable time, they were not required to provide all. Incidentally, most vendors used only the 10-second gateway data and the Green Button data and some only provided pre-appliance-survey predictions. The researchers suggested to the vendors to provide hourly and daily predictions but they were allowed to give predictions at higher or lower frequencies. The researchers reviewed all vendor predictions but focused the analysis on the daily predictions if provided by each vendor for comparison and simplicity’s sake. In cases that only monthly predictions were provided, those were analyzed. Abbreviated hourly and minute interval results are provided in section Accuracy Calculations.
The researchers found that various complexities made it difficult to confidently calculate disaggregation accuracy across all vendor predictions and all measured major appliances at each home. First, measuring at only the home breaker-level meant that in numerous cases other plug loads or lighting were in the same data streams as major appliances. Second, some homes had multiple instances of appliances such as refrigerators and only some of those breakers were measured due to measurement and verification (M&V) equipment and installation cost and risk constraints. Third, there were occasional gaps in the whole building meter data and the breaker level data. For all of these reasons, the focus of the accuracy calculations are on those appliances that the researchers found to be on dedicated breakers.