Smart Speakers Assessment

The primary goal of SCE’s Smart Speaker Demonstration Project was to test customers’ ability to optimize connected energy loads in their homes using an Amazon Smart Speaker (voice commands) as the interface. Participants were given an Amazon Echo Dot, smart switches, smart lights, an Ecobee thermostat, and Universal Devices, Inc. (UDI) ISY gateway that unites smart technologies installed and commissioned in the home. These devices are connected to the smart speaker and SCE smart meter through the gateway. The demonstration project also sought to determine the most desired voice interactions (“how much is my bill,” “what rate am I on,” etc.) that helped customers better understand and manage their energy usage. SCE contracted with an Measurement and Verification (M&V) Consultant to evaluate the project’s efficacy. The M&V Consultant’s assessment helped quantify (meter-based, home-by-home) any energy usage impacts. This assessment helps inform the Demand Response (DR) and or Energy Efficiency (EE) potential of home smart speaker devices. SCE also worked with its Customer Experience team to develop a qualitative understanding of the overall customer experience, and to identify desired voice interactions to improve how customers manage their energy usage. Customers asked energy-related questions and set device optimization preferences using their smart speakers. Universal Devices, Inc.’s (Automation and Energy Management Firm) algorithms used the customers’ Time-of-Use (TOU) rates, energy use, and preferences to optimize connected devices, which had their settings adjusted to run less during peak times. An Alexa skill named “Energy Expert” initiated interactions and commands between customers and their Echo Dots.  UDI’s ISY 994i ZW pro was the load controller at the center of the smart speaker home automation setup used in this demonstration project. All ISY models were OpenADR 2.0a/2.0b Certified Virtual End Nodes (VENs). In California, the latest revision of Title 24 mandated the installation of a Certified OpenADR device in every new and retrofit venue starting in 2020. These are key findings and recommendations pertaining to load impacts from the Smart Speaker Demonstration Project: The evaluation was limited by low counts of active participants. Additional recruitment of participants and promotion of skills and features would result in a more robust evaluation. Load impacts were evident for certain subsets of treatment customers and time periods, but the mechanism(s) leading to the effects could not be definitively attributed without a larger participant population. A large portion of the initial set of participants was inactive by the time of the evaluation. Skill log and device level data was sparsely populated. More stringent Quality Control (QC) of data the collection would have allowed for a more comprehensive analysis and a more refined attribution of observed effects. The specific devices with the most use included smart appliances, refrigerators, thermostats, and interior lighting. Additional education and focus on these end uses could provide the most value to future iterations of this program.   These are key process evaluation findings and recommendations stemming from discussions with program and implementation staff: Existing frameworks of the optimization algorithm and initial Alexa skills benefitted the project rollout, but additional skills were developed later, increasing the time between installation and treatment. Since these were developed and tested, any future iterations of the program would benefit from the foundation. The project design required a third party to install the equipment during the COVID-19 pandemic, resulting in low participation. The implementer cited customer reluctance to allow people into their homes. Installers could have benefitted from further training and support due to the complexity of connecting multiple devices with the meters and gateways. The implementer received frequent requests for additional support from the installers, particularly when the SCE program manager was not present on site to oversee the installation. The program implementer reported home equipment (for example, thermostats) was occasionally installed incorrectly and did not communicate with SCE’s smart meter. Additional validation that equipment was accurately installed and communicating with other equipment may have been beneficial. Also, presetting home equipment to the largest extent possible and considering a cloud-based approach instead of hardwires could have also been effective. Customer outreach seemed to be beneficial in improving engagement with the devices, particularly after the launch of the additional smart speaker skill capabilities. Providing regular outreach and education on program updates would have likely increased program impacts. Given the typical customer profile for the demonstration project was” engaged with home automation topics and technologies”, they could have potentially been instructed on equipment installation as well, though perhaps with a smaller subset of the components involved in this project.