Feeder Vellmoor Distribution · control room Synthetic data Sun 02:00
Ten minutesFullKnown gaps

Feeder: architecture

The full design behind the ten-minute version. The reference implementation in this repository runs every part described here on a synthetic network in one DuckDB file; the production notes say what changes against a real utility's systems. Known gaps lists what the synthetic world cannot show.

1. Sources

FactSystemFreshness
Last gasp, power restoreHead-end system of the smart meters (AMI)seconds
Daily import, export, interval reads receivedMeter data management (MDM)daily, after the overnight run
Feeder head load, end-of-line voltageSCADA historian15 minutes
Transformer loading (monitored units)SCADA historianhourly
Outage ticketsOutage management system (OMS)seconds
Work orders, crew movementsWorkforce management (WFM)seconds
Feeders, transformers, tie points, meter connectivityGIS network modelnightly
Accounts, billing day, priority register, solar registrationsCustomer information systemhourly
CallsContact centreseconds
Booked planned workWorks planninghourly

The reference world holds these as tables (feeder/world/schema.sql); the data map in systems and fact_sources says which system owns which column.

2. The definitions

Each term is a YAML file in feeder/dictionary/definitions/: a title, owner, version history, parameters, SQL templated on {now} and the parameters, and tests. A change to a parameter is appended to a history file and bumps the version of every variant that uses it; the ledger records the version each decision ran on.

Detector terms return one row per incident with a subject (transformer, feeder, meter, work order, planned work), a start time and an evidence object. The ten detectors:

TermCadenceSubjectWhat it sees
outage_clusterevery ticktransformer or feedermeters dark three minutes or more with no ticket or planned work covering them
priority_customer_in_outageevery tickmetera priority-register premise dark in an unplanned outage
crew_to_restored_areaevery tickwork ordera crew still driving to a ticket whose meters are eight in ten back
voltage_out_of_bandevery tickfeederthree readings in a row outside 0.94 to 1.06 per unit
planned_work_collisionevery tickplanned workbooked work within twelve hours on a feeder whose own or tie feeder has an open fault
transformer_overloadhourlytransformertwo hours or more above 120 % of the rating
wrong_feeder_meterhourlymeterlost and regained supply with another feeder's cluster while its own stayed on
zero_consumptiondaily 06:00meterthree complete days of zero at an occupied premise that used power before
reverse_flow_unregistereddaily 06:00metertwo days of export with no solar registered
estimated_billdaily 06:00meterthree days with no reads and the billing close within five days

Sector terms that Ask answers: outage, sustained outage (two meanings: regulator, over three minutes; network operations, over five), restored, affected customers, priority customer, feeder load.

3. Decisions

3.1 From incident to proposal

The runner deduplicates incidents (same kind and subject within fifteen minutes, already in the queue, already overridden by a person, or still inside the last action's cooldown) and asks a worker for one proposal each. The worker uses the fixed rule when the runbook leaves one action, or when the policy says rules decide that kind; otherwise it may ask a model, validates the answer against the policy and the evidence, and falls back to the rule if the answer is unusable.

3.2 The controller

For each proposal, in order: the action exists, is not on the never list, and is allowed for this kind; its arguments are known, well-formed and name only ids that the evidence names; its written conditions hold (a crew is only recalled when eight in ten of the ticket's meters are back); an ask-first rule does not apply (a notice promising more than twelve hours); the tier is not 2; no cooldown is running; the feeder's hourly budget, the network's per-minute limit, or the metering desk's own limit is not used up. The first failing check decides the outcome: execute, queue for a person, wait for the next minute, or decline, each with a plain-English reason.

3.3 Runbooks that chain

A new outage ticket is followed by the notice to its customers, with an estimated restoration time from a fixed table: two hours for one transformer, three for a section, one more for a hundred customers or more, plus time for the open tickets across the network.

3.4 Ledger and undo

Every outcome is a ledger row; approvals, rejections and rollbacks are new rows pointing at the row they act on. Each executed tool records the rows it changed and how to neutralise the ones it created (a ticket is withdrawn, a notice cancelled, a crew stand-down reversed by a compensating event). A rollback refuses if somebody changed the same row since.

Anatomy of a ledger row: time, kind (detection, proposal, action, approval, rejection, rollback, housekeeping), subject type and id, action and endpoint, tier, status, the human involved (if any), a note, the parent row it acts on, and the evidence (definition key and version, the rows that matched, and a stake note in customers). Undo adds a new row whose parent is the original.

3.5 People

The approval queue holds tier-2 items. In a replay, a scripted control-room shift (control engineers, a planner, a crew lead; synthetic, and labelled so on every row they write) approves, rejects or reverts by written rules after a realistic delay. Items older than two hours are decided by the shift's backlog rules, so the queue does not grow for days.

4. Scale

The reference world has 200,000 meters on 1,200 feeders from 90 substations, about 200,000 meter events over 90 days, 920,000 feeder readings, 2.8 million daily meter rows, and around 1,170 planted faults. A replay of the storm evening on a 30-second tick, after six days on a five-minute tick, takes a few minutes on a laptop with rules only. Detector queries stay in tens of milliseconds because each reads a bounded time window.

In production the realtime detectors move to stream jobs on the head-end and OMS feeds (the definitions compile to both), daily ones stay in batch on the MDM, and the ledger lives in a relational store with the same append-only rules.

5. Measurement and rollout

6. Failure modes

7. The map and play mode: routes