Accuracy against the answer key
Every detection the operator raised is matched to the world's planted incidents (incidents_truth). Precision: how many raised were real. Recall: how many planted were caught.
precision96%553 of 576 detections real
recall95%494 of 520 planted caught
handled without a person95%of 806 actions
actions reverted1undone by a person
mean time to resolve0.6 min564 resolved
decided by rules / model573 / 0model asked only where policy allows
Weakest detector: wrong feeder meter, precision 85% on 39 detections. It is named here, not hidden: see its definition and known gaps.
Per incident kind
| incident kind | runs | raised | precision | planted | recall | age when caught | time to resolve |
|---|---|---|---|---|---|---|---|
| crew to restored area | realtime | 7 | 100% |
10 | 70% |
3.2 min | 0.0 min |
| estimated bill | daily 06:00 | 57 | 100% |
60 | 95% |
4,207.6 min | 0.0 min |
| outage cluster | realtime | 242 | 93% |
196 | 94% |
5.6 min | 0.0 min |
| planned work collision | realtime | 10 | 100% |
11 | 91% |
1.6 min | 11.4 min |
| priority customer in outage | realtime | 102 | 99% |
86 | 100% |
23.2 min | 1.6 min |
| reverse flow unregistered | daily 06:00 | 32 | 100% |
32 | 100% |
8,436.4 min | 0.0 min |
| transformer overload | hourly | 37 | 100% |
37 | 92% |
120.0 min | 2.7 min |
| voltage out of band | realtime | 10 | 100% |
10 | 100% |
42.0 min | 0.0 min |
| wrong feeder meterweakest | hourly | 39 | 85% |
36 | 92% |
42,354.1 min | 0.0 min |
| zero consumption | daily 06:00 | 40 | 100% |
42 | 95% |
5,340.0 min | 0.0 min |
Daily checks run at 06:00, so the storm window (17:00 to 02:00) holds their planted cases but none of their runs; the week view counts them. Age when caught: how long the fault had been true when the detector first saw it. Drift: green (No drift: answers usable, confidence steady, few reverts.)