This paper addresses issues in closed loop performance monitoring. Particular attention is paid to detecting whether an observed deviation from nominal performance is due to a disturbance or due to a system change. This is achieved by introducing a novel performance measure that allows feasible application of a standard CUSUM change detector. The paper includes explicit results on the probability of mistaking a disturbance for a system change and demonstrates the algorithm with a simulation study.