Accuracy against the answer key
Every detection the operator raised is matched to the world's planted incidents (incidents_truth). Precision: how many raised were real. Recall: how many planted were caught.
precision96%332 of 347 detections real
recall67%276 of 413 planted caught
handled without a person93%of 542 actions
actions reverted1undone by a person
mean time to resolve0.7 min335 resolved
decided by rules / model344 / 0model asked only where policy allows
Weakest detector: wrong feeder meter, precision 77% on 13 detections. It is named here, not hidden: see its definition and known gaps.
Per incident kind
| incident kind | runs | raised | precision | planted | recall | age when caught | time to resolve |
|---|---|---|---|---|---|---|---|
| crew to restored area | realtime | 7 | 100% |
9 | 78% |
3.2 min | 0.0 min |
| estimated bill | daily 06:00 | 0 | n/a |
25 | 0% |
||
| outage cluster | realtime | 207 | 94% |
166 | 95% |
5.6 min | 0.0 min |
| planned work collision | realtime | 7 | 100% |
8 | 88% |
0.2 min | 5.6 min |
| priority customer in outage | realtime | 86 | 100% |
71 | 100% |
26.5 min | 1.9 min |
| reverse flow unregistered | daily 06:00 | 0 | n/a |
32 | 0% |
||
| transformer overload | hourly | 23 | 100% |
20 | 100% |
120.0 min | 2.9 min |
| voltage out of band | realtime | 4 | 100% |
4 | 100% |
30.0 min | 0.0 min |
| wrong feeder meterweakest | hourly | 13 | 77% |
36 | 28% |
11,381.6 min | 0.0 min |
| zero consumption | daily 06:00 | 0 | n/a |
42 | 0% |
Daily checks run at 06:00, so the storm window (17:00 to 02:00) holds their planted cases but none of their runs; the week view counts them. Age when caught: how long the fault had been true when the detector first saw it. Drift: green (No drift: answers usable, confidence steady, few reverts.)