Optimal Maintenance Policies for Automated Demand Response Devices

Demand side participation is now widely recognized as being extremely critical for satisfying the growing electricity demand in the US. The primary mechanism for demand management in the US is demand response (DR) programs that attempt to reduce or shift demand by giving incentives to participating customers via price discounts or rebate payments. Utilities that offer DR programs rely on automated DR devices (ADRs) to automate the response to DR signals. The ADRs are faulty; however, the state of the ADR is not directly observable -it can only be inferred from the power consumption during DR events. The utility loses revenue when a malfunctioning ADR does not respond to a DR signal; however, sending a maintenance crew to check and reset the ADR also incurs costs. In this paper, we show that the problem of maintaining a pool of ADRs using a limited number of maintenance crews can be formulated as a restless bandit problem, and that one can compute a very good policy for this problem using Whittle indices. We show that the Whittle indices can be efficiently computed using a variational Bayes procedure even when the demand reduction magnitude is noisy and when there is a random mismatch between the clocks at the utility and at the meter. The results of our numerical experiments suggest that the Whittle-index based approximate policy is within 3.95% of the optimal solution for all reasonably low values of the signal-to-noise ratio of the meter readings.