ECS for Control: Observations – Implementation

The observation machinery in code: counters at the source, the distiller and supervisor Systems, and the calibration computed and locked at world creation.

Note

Exploratory / evaluation. Third document of the Observations set. It assumes the concepts document (vocabulary) and the calibration document (the declaration and its derived quantities); the ECS ground – store, Systems, manifests, the derived schedule – is the store concepts and store implementation documents' subject. Everything shown here is implemented and running in the EcsControlWatch evaluation program; the verification document tests it on target. This document shows only what runs – no machinery for relations the program does not use. All cited code, the library modules included, is the example's own: library modules such as Observations are quoted from the example-local lib directory, frozen with the example – they may not represent the modules of the main framework library.

The Program and the Path

EcsControlWatch watches one condition: the clip of a print buffer. Its print partition contains the complete observation path, from the counted event to the corrective command:

watched mechanism       PrintBuffers        lib algorithm module
event source counters   CS pbClipsTally     Tally record, in the store
distiller               PrintClipSystem     System
evidence transport      CS pbClipsObsPort   observation port, in the store
supervisor              AdaptDwellSystem    System
commanded loop          DrainSystem         System, drains the buffer to a UART
command transport       CS drainCmdPort     command port, in the store
calibration inputs      Vprint              authored values
calibration results     CLprint             locked components

The loop the path closes: usage Systems print through a buffered writer; DrainSystem empties the buffer to a serial terminal at a commanded period; when demand exceeds what the current period drains, puts clip – events; the distiller condenses them into a level; the supervisor compares the level against the calibrated thresholds and commands the drain period. Everything is partition-local, and every piece is an ordinary store item or System – the observation machinery required no kernel change of any kind.

One liberty of the layout is worth naming up front: within one partition, under one scheduler, the command port is not strictly necessary – a plain store token with the right access class would carry the commands, and the same holds for the observation port and its readings. The ports are used here for example and test purposes; they become necessary where such a path crosses a partition or schedule boundary.

The Library Module

The whole accounting lives in one library algorithm module, Observations. It owns the record types and the procedures over them, holds no state of its own, and is payload-blind – it never learns what an event means:

TYPE
  (* at event source *)
  Tally* = RECORD
    events*: INTEGER;
    ops*: INTEGER
  END;

  (* observation distiller -> supervisor via port *)
  Distilled* = RECORD
    level*: INTEGER;
    lastTick*: INTEGER;
    ops*: INTEGER  (* cumulative operations at distillation: evidence odometer *)
  END;

  (* observation distiller owned CS *)
  Account* = RECORD
    cursorEvents*: INTEGER;
    cursorOps*: INTEGER;
    dist*: Distilled
  END;

Tally is the source's counter pair – the two monotonic streams of the calibration document's counting regime. Account is the distiller's working state: the cursors remember the counter values of the previous pass, so each pass processes exactly the delta; Distilled is the published reading – the level, its recency, and the operations counter at distillation time (the evidence odometer the supervisor's dwell reads, below).

Two procedure duos operate on them – an Init and an Update each:

PROCEDURE* UpdateTally*(VAR tally: Tally; events: INTEGER);
BEGIN
  INC(tally.events, events);
  INC(tally.ops)
END UpdateTally;


PROCEDURE* UpdateAccount*(VAR acct: Account; calib: DistCalib; tally: Tally; tickNow: INTEGER);
  VAR deltaE, deltaO, level: INTEGER;
BEGIN
  deltaO := tally.ops - acct.cursorOps;
  IF deltaO > 0 THEN
    deltaE := tally.events - acct.cursorEvents;
    acct.cursorEvents := tally.events;
    acct.cursorOps := tally.ops;
    level := acct.dist.level;
    level := level + (deltaE * calib.weight) - (deltaO * calib.decay);
    IF level < 0 THEN
      level := 0
    ELSIF level > calib.cap THEN
      level := calib.cap
    END;
    acct.dist.level := level;
    acct.dist.lastTick := tickNow;
    acct.dist.ops := tally.ops
  END
END UpdateAccount;

UpdateTally is one operation entering the books: its event count (possibly zero) and its own increment of the operation counter, together – the counting regime's atomic pair. UpdateAccount is one distiller pass: the deltas since the last pass applied as one net quantity, the level clamped to 0 .. cap. This is the calibration document's level machinery, line for line; a pass that finds no new operations changes nothing – batching invariance in the guard deltaO > 0.

Tallying at the Source

The join point is PrintBuffers.PutString – the put operation itself:

PROCEDURE PutString*(VAR state: State; VAR buf: ARRAY OF CHAR;
          s: ARRAY OF CHAR; numChar: INTEGER; VAR tally: Tally);
  VAR i, free, failed: INTEGER;
BEGIN
  IF numChar > LEN(s) THEN numChar := LEN(s) END;
  free := RingBuffer.Free(state);
  failed := 0;
  IF numChar > free THEN
    numChar := free;
    failed := 1
  END;
  Observations.UpdateTally(tally, failed);
  i := 0;
  WHILE i < numChar DO
    buf[RingBuffer.PutIndex(state, i)] := s[i];
    INC(i)
  END;
  RingBuffer.Publish(state, numChar)
END PutString;

Three of the concepts document's claims are visible as code:

only the mechanism can tally: the free-space comparison is the one place in the program where the clip is knowable, and the counting happens right there, inside the caller's own operation.
absorption: a put that finds insufficient space delivers what fits and returns – the event is handled gracefully, counted, and the caller runs on. The procedure has no error return; the operation completes the same way whether it triggered or not.
the layered vocabulary: the buffer's local variable is named failed – at the source, a clip is a failure of the put, and the source says so. What crosses into UpdateTally is a neutral event count.

The tally the mechanism writes into is not the mechanism's own: it lives in the store (CS.pbClipsTally, one Tally per print buffer), handed in by the caller's call chain. Evidence is store state – visible, snapshotable, owned like any other Component. The hand-in is easy to point to – PrintBufAdapt, the thin adapter that makes the buffered path Texts.Writer-compatible, is the whole of it:

PROCEDURE PutString*(handle: INTEGER; s: ARRAY OF CHAR; numChar: INTEGER);
(* handle = CS array index *)
  VAR S: CS.Store;
BEGIN
  S := CS.S;
  PrintBuffers.PutString(S.pbBuf[handle].state, S.pbBuf[handle].buf, s, numChar, S.pbClipsTally[handle])
END PutString;

One procedure resolves the store and names the buffer's state, its character array, and its tally, side by side – the library module below it receives everything through its signature and owns nothing. The seam where store-resident evidence meets a store-blind library is exactly one line wide.

The Distiller System

PrintClipSystem is the distiller – one System, one pass per period over every print buffer's tally:

PROCEDURE runSystem(VAR clipAcct: CS.PbClipsAcctArray; tally: CS.PbClipsTallyArray;
                    VAR obsPort: CS.PbClipsObsPortArray);
  VAR bix: INTEGER; obsVal: ObservationPorts.Value;
BEGIN
  bix := 0;
  WHILE bix < LEN(clipAcct) DO
    Observations.UpdateAccount(clipAcct[bix], CL.L.pbDistCalib, tally[bix], Kernel.TickCount);
    obsVal.dist := clipAcct[bix].dist;
    obsVal.timestamp := Kernel.TickCount;
    ObservationPorts.Put(obsPort[bix], obsVal);
    INC(bix)
  END
END runSystem;


PROCEDURE Run*;
  VAR S: CS.Store;
BEGIN
  S := CS.S;
  runSystem(S.pbClipsAcct, S.pbClipsTally, S.pbClipsObsPort)
END Run;

BEGIN
  Name := SystemName;
  CLEAR(Manifest);
  Manifest.Cx := {T.L.pbClipsTally};
  Manifest.O := {T.L.pbClipsAcct};
  Manifest.P := {T.L.pbClipsObsPort}
END PrintClipSystem.

Each pass: update the account from the tally deltas, then publish the fresh Distilled reading – level, odometer, recency – on the buffer's observation port. The distiller applies the locked weights (CL.L.pbDistCalib) and concludes nothing: no threshold appears in this System.

An eagle-eyed reader will notice an asymmetry: one drain System, but a distiller iterating over an array of buffers. In this program every one of these arrays – buffer, tally, observation port, account – holds exactly one element. The complication is deliberate and self-inflicted: the machinery is written and exercised for the general case, several buffers each with its own complete observation path, rather than for the single-buffer case whose simplicity could hide issues the general case would surface.

(The running program's runSystem additionally prints the pass's deltas and the resulting level to the console – the instrumentation that produces the record the verification document decodes; it is omitted here.)

The periods follow the calibration document's period rules. The usage System producing the print load runs at 600 ms; the distiller runs at the same 600 ms, and the derived schedule places it after the usage System in the tick – evidence is distilled in the tick it arises. Both periods are authored in one place (the program-wide period module) as milliseconds over the 50 ms tick.

Calibration at Creation

The declaration is authored in the print partition's value module, Vprint – tuning data, a record populated at load:

VAR
  pbClipsCalibDef*: Observations.CalibDef;

BEGIN
  (* accept 1 clip per 8 ops *)
  pbClipsCalibDef.eventsAccept := 1;
  pbClipsCalibDef.opsAccept := 8;
  (* trigger with 1 clip per 6 ops *)
  pbClipsCalibDef.eventsTrig := 1;
  pbClipsCalibDef.opsTrig := 6;
  (* one event per clip *)
  pbClipsCalibDef.eventsPerOp := 1;
  (* trigger with a consecutive burst of 6 clipping operations *)
  pbClipsCalibDef.opsBurst := 6;
  (* clear at 34% of Tset, cap at 200% of Tset *)
  pbClipsCalibDef.clearPct := 34;
  pbClipsCalibDef.capPct := 200
END Vprint.

The partition's locked-components module, CLprint, derives and locks the results in its own init – one call replaces any hand-maintained constant chain:

(* print clipping observations *)
Observations.Calibrate(V.pbClipsCalibDef, L.pbDistCalib, L.pbAdaptCalib);

The division of homes is the point: V authors, CL binds and locks, CS runs. The declaration is tuning data and lives with the other tuning values; the derived results are locked components – written once at load, read-only to every importer thereafter; the store holds only the running state. The derivation itself:

PROCEDURE Calibrate*(def: CalibDef; VAR distCalib: DistCalib; VAR adaptCalib: AdaptCalib);
  VAR q, weight, decay, tset, tclear, cap: INTEGER;
BEGIN
  (* validity *)
  ASSERT(def.eventsAccept >= 1, Errors.PreCond);
  ASSERT(def.eventsPerOp >= 1, Errors.PreCond);
  ASSERT((def.eventsTrig * def.opsAccept) - (def.opsTrig * def.eventsAccept) > 0, Errors.PreCond);
  ASSERT(def.opsBurst * def.eventsPerOp > def.eventsAccept, Errors.PreCond);
  ASSERT(def.clearPct < 100, Errors.PreCond);
  ASSERT(def.capPct >= 100, Errors.PreCond);

  (* scaling: level units per event/op, iterated to the resolution floor *)
  q := 1;
  tset := def.opsBurst * ((def.eventsPerOp * def.opsAccept) - def.eventsAccept);
  WHILE tset * q < TsetMin DO
    q := q * 10
  END;

  (* level-denominated results, carry q *)
  weight := def.opsAccept * q;
  decay := def.eventsAccept * q;
  tset := tset * q;
  tclear := (def.clearPct * tset) DIV 100;
  cap := (def.capPct * tset) DIV 100;

  distCalib.weight := weight;
  distCalib.decay := decay;
  distCalib.cap := cap;

  adaptCalib.tset := tset;
  adaptCalib.tclear := tclear;

  (* ops-denominated results, q-invariant *)
  adaptCalib.recOps := (tset - tclear + decay - 1) DIV decay;
  adaptCalib.capRecOps := (cap - tclear + decay - 1) DIV decay
END Calibrate;

This is the calibration document's derivation, realised in integers – and the three places where the integer realisation adds something are exactly the ones that document deferred here:

the validity checks run as load-time assertions. A declaration that fails them refuses the world at creation – through the framework's ordinary error path, before a single tick runs. The relations get an enforcement point.
the q iteration spends the scaling freedom. The calibration document showed that any common scaling of the level-denominated quantities preserves every relation; Calibrate uses that freedom for integer resolution, scaling Tset up to an authored floor (TsetMin) so that the percentage-derived thresholds do not truncate into distortion. The calibration document's row runs at q = 1 – the scaling is armed but not needed.
the recovery horizons take the ceiling form. (tset - tclear + decay - 1) DIV decay is "clean operations to reach the clear threshold or below" under truncating integer division – exact for every decay, where a plain quotient would be exact only at decay = 1.

The two result records also split along their consumers: DistCalib carries what the distiller needs (weight, decay, cap – mechanism, no thresholds), AdaptCalib what the supervisor needs (tset, tclear, recOps, capRecOps – thresholds and horizons, no weights). Each side receives exactly its own vocabulary.

The Supervisor: Adaptation with a Dwell

AdaptDwellSystem is the supervisor – the print partition's adaptation loop, closing on the drain period:

PROCEDURE Run*;
  VAR S: CS.Store; cmd: DrainCmdPorts.Command; obsVal: ObservationPorts.Value; cmd0: INTEGER;
BEGIN
  S := CS.S;
  ObservationPorts.Get(S.pbClipsObsPort[CL.L.drainPrintBufIx], obsVal);
  IF obsVal.dist.ops - S.adaptRun.opsAtCmd >= CL.L.pbAdaptCalib.recOps THEN
    cmd0 := S.adaptRun.drainPeriodCmd;
    IF obsVal.dist.level >= CL.L.pbAdaptCalib.tset THEN
      IF S.adaptRun.drainPeriodCmd < CL.L.drainCfg.maxCmdNo THEN
        INC(S.adaptRun.drainPeriodCmd)
      END
    ELSIF obsVal.dist.level <= CL.L.pbAdaptCalib.tclear THEN
      IF S.adaptRun.drainPeriodCmd > 0 THEN
        DEC(S.adaptRun.drainPeriodCmd)
      END
    END;
    IF S.adaptRun.drainPeriodCmd # cmd0 THEN
      S.adaptRun.opsAtCmd := obsVal.dist.ops
    END
  END;
  cmd.cmdRef := S.adaptRun.drainPeriodCmd;
  cmd.timestamp := Kernel.TickCount;
  DrainCmdPorts.Put(S.drainCmdPort, cmd)
END Run;

BEGIN
  Name := SystemName;
  CLEAR(Manifest);
  Manifest.C := {T.L.pbClipsObsPort};
  Manifest.O := {T.L.adaptRun};
  Manifest.P := {T.L.drainCmdPort}
END AdaptDwellSystem.

The policy reads top to bottom:

the dwell gates everything. obsVal.dist.ops is the evidence odometer – cumulative operations at distillation. A decision is allowed only when at least recOps operations of fresh evidence have passed since the last command change (opsAtCmd): one decision per correction's measurable effect, the recovery horizon serving as the dwell. Denominated in operations, the gate is idle-proof by construction – quiet time moves no odometer.
the ladder steps by one. Level at or above tset: one step up (shorter drain period), clamped at the repertoire's top. Level at or below tclear: one step down, clamped at the bottom. Between the thresholds – the hysteresis band – the command stands.
the command is a reference, not a value. What crosses to the drain is cmdRef, an index into the drain's own authored period repertoire (CL.L.drainCfg.periods – 12, 6, 3 ticks). The supervisor knows the repertoire's top index (maxCmdNo), never its contents; an invalid period is not rejected but unrepresentable.
the command is re-published every pass – current command, fresh timestamp – so the port always carries the standing order, not only its changes.

A note against the concepts document: de-escalation here reads the trouble level's cleared band – the direction-neutral slack observation of the concepts document (an evidence stream of its own, against the drain's capability) is not implemented; the dwell plus tclear do the relaxing on the trouble record's quiet alone.

The Policy Alternatives

The dwell policy is the shipped one, but not the only one the program carries: three alternatives live beside it, in PrintPartition/adapt-variants/, and any of them becomes the supervisor by changing one aliased import in the partition module. All four share the manifest and the port protocol – deliberately, so that the derived schedule is invariant across the exchange: swapping the policy changes the policy and nothing else. The verification suite stands on exactly this property.

AdaptHold0 and AdaptHold1 are not adaptation policies at all – they hold a fixed command, forever. They exist as the verification's controls: the instrument is validated under a constant plant before any closed loop runs. Each reads the observation port and discards the value, keeping the manifest identical to the live policies':

PROCEDURE Run*;
  VAR S: CS.Store; cmd: DrainCmdPorts.Command; obsVal: ObservationPorts.Value;
BEGIN
  S := CS.S;
  ObservationPorts.Get(S.pbClipsObsPort[CL.L.drainPrintBufIx], obsVal); (* read, discard *)
  S.adaptRun.drainPeriodCmd := HoldCmd;
  cmd.cmdRef := S.adaptRun.drainPeriodCmd;
  cmd.timestamp := Kernel.TickCount;
  DrainCmdPorts.Put(S.drainCmdPort, cmd)
END Run;

(AdaptHold0 with HoldCmd = 0, the slowest drain period; AdaptHold1 with HoldCmd = 1 – otherwise identical.)

AdaptLadder is the bare closed loop: the same hysteresis ladder as the shipped policy, level-triggered, one step per pass – and no dwell:

PROCEDURE Run*;
  VAR S: CS.Store; cmd: DrainCmdPorts.Command; obsVal: ObservationPorts.Value;
BEGIN
  S := CS.S;
  ObservationPorts.Get(S.pbClipsObsPort[CL.L.drainPrintBufIx], obsVal);
  IF obsVal.dist.level >= CL.L.pbAdaptCalib.tset THEN
    IF S.adaptRun.drainPeriodCmd < CL.L.drainCfg.maxCmdNo THEN
      INC(S.adaptRun.drainPeriodCmd)
    END
  ELSIF obsVal.dist.level <= CL.L.pbAdaptCalib.tclear THEN
    IF S.adaptRun.drainPeriodCmd > 0 THEN
      DEC(S.adaptRun.drainPeriodCmd)
    END
  END;
  cmd.cmdRef := S.adaptRun.drainPeriodCmd;
  cmd.timestamp := Kernel.TickCount;
  DrainCmdPorts.Put(S.drainCmdPort, cmd)
END Run;

The difference to the shipped policy is exactly the dwell gate and the opsAtCmd bookkeeping – nothing else. And that difference is a design statement: the ladder acts on the level whenever it stands beyond a threshold, including on passes whose evidence still predates its own last correction – it can command a second step before the first has had any measurable effect. The dwell adds the one gate the calibration document's recovery horizon exists for: no further decision until the correction's effect is measurable in fresh operations. What each behaviour looks like on target – the ladder's built-in second step at every trip, the dwell holding it – is the verification document's subject.

AdaptDwell (§ The Supervisor, above) is the ladder plus that gate – the shipped configuration.

The Commanded Loop: Drain Timing

DrainSystem is the corrected loop – not part of the observation machinery, but the subject of its commands, and one timing decision in it matters to everything downstream:

PROCEDURE Run*;
  VAR bix, uartHandle, delta: INTEGER; ch: CHAR; S: CS.Store; cmd: DrainCmdPorts.Command;
BEGIN
  S := CS.S;
  DEC(S.drainRun.ticker, Kernel.ElapsedTicks);
  IF S.drainRun.ticker <= 0 THEN
    bix := CL.L.drainPrintBufIx;
    uartHandle := CL.L.drainUartHandle;
    WHILE ~(PrintBuffers.Empty(S.pbBuf[bix].state) OR UARTdrain.Full(uartHandle)) DO
      PrintBuffers.Get(S.pbBuf[bix].state, S.pbBuf[bix].buf, ch);
      UARTdrain.Put(uartHandle, ch)
    END;
    DrainCmdPorts.Get(S.drainCmdPort, cmd);
    delta := CL.L.drainCfg.periods[cmd.cmdRef];
    S.drainRun.ticker := delta - ((Kernel.TickCount - 1) MOD delta); (* re-lock to grid *)
  END
END Run;

The drain runs every tick and self-times with a ticker, so its period can follow the command without kernel involvement. The reload line is the decision:

ticker := delta - ((TickCount - 1) MOD delta)

Instead of adding the period to wherever the ticker happened to land, the reload locks the next firing to a fixed grid anchored at the kernel's first tick. Two properties follow. Runs are phase-reproducible: however the period was reached – including through a burst of command changes – the firing pattern depends only on the current period, never on history; the verification document's character-exact repeatability stands on this line. And period changes re-lock cleanly: the repertoire is authored as a divisor chain (12, 6, 3 – each period dividing the previous), so stepping to a shorter period lands on a grid the longer one already occupied – escalation is transient-free, and only de-escalation pays a single re-alignment hop.

Tokens and Manifests

Every store item on the observation path is a declared token, and each System's manifest names its access – the declarations the derived schedule and the creation checks read:

Heartbeat (usage)     Px  pbBuf, pbClipsTally      deposits: prints, and clips counted
Drain                 Cx  pbBuf                    collects the buffered chars
                      Cp  drainCmdPort             consumes the previous command
                      O   drainRun                 owns its ticker
PrintClip (distiller) Cx  pbClipsTally             collects the counter deltas
                      O   pbClipsAcct              owns the accounts
                      P   pbClipsObsPort           produces the readings
AdaptDwell (superv.)  C   pbClipsObsPort           consumes the readings
                      O   adaptRun                 owns command state and dwell reference
                      P   drainCmdPort             produces the commands

The classes are the store implementation document's, in brief: P produce and C consume within a tick, Cp consume the previous tick's value, O own privately, and the deposit pair Px/Cx for spaces many parties write into during their own runs and one party collects. The deposit pair is the manifest form of tallying at the source: the usage System declares Px on the tally – it deposits events while printing – and the distiller declares the collecting side.

Two absences are as deliberate as the declarations:

calibration appears in no manifest. CL reads are locked components – constant after creation, unable to carry dataflow – and stay manifest-invisible by design. The declarations describe the running dataflow, nothing else.
the supervisor never touches the account, the tally, or the buffer. Its manifest is three tokens: readings in, own state, commands out. The narrowness is the concepts document's role separation, machine-checkable.

From these declarations the schedule is derived: usage before distiller before supervisor, the command consumed by the drain one tick later (Cp – the deterministic unit delay). The period rules' "zero added latency" ordering is not arranged by hand; it falls out of what the Systems declared.

The verification document runs this machinery through a sixteen-run verification suite – instrument validation under the hold policies first, then the closed loop under the ladder and the dwell – and confirms the calibration document's numbers on target, character-exact.