Runs and feedback API

Method	Path	Auth	Purpose
POST	/runs/:id/feedback	Token	Label a run correct or wrong (or clear it with null), optionally with a reason, a note and a missed-page URL. Returns what Watcher learned.
POST	/runs/:id/feedback/interpret	Token	Retry reading a run's feedback note after a "deferred" result. 503 without a model key.
GET	/feedback/stats	Token	Rolling precision, recall and F1 with recent false positives and negatives, split by alert tier. ?days=N for a window.
GET	/watches/:id/feedback/stats	Token	The same accuracy summary for one watch.
GET	/watches/:id/learning	Token	What the watch learned from feedback (rules, suggestions) plus proposals computed from its labels.
PUT	/watches/:id/learning	Token	Pause or resume the learned rules ({ "paused": true }). Nothing is deleted.
POST	/watches/:id/lessons	Token	Add a rule by hand (kind + text).
PUT	/watches/:id/lessons/:lessonId	Token	Edit a rule, or switch it on or off. Activation respects the per-watch limit.
DELETE	/watches/:id/lessons/:lessonId	Token	Delete a rule.
DELETE	/watches/:id/learning/suggestions/:suggestionId	Token	Dismiss a suggestion the feedback interpreter made.
POST	/watches/:id/learning/dismiss	Token	Dismiss a computed proposal ({ "proposalId": "…" }) so it stops resurfacing.
GET	/collection/stats	Token	Rolling collection metrics, including the extraction rate.

Run history itself is read through the watches API: GET /watches/:id/runs.

The run record

{
  "id": "0f1c…",
  "watchId": "framework-releases",
  "ownerId": "local",
  "timestamp": "2026-09-10T06:00:04.120Z",
  "outcome": "notified",
  "summary": "Version 2.0 was released as stable.",
  "evidenceUrl": "https://example.com/blog/v2-ga",
  "evidenceQuote": "v2.0 is now generally available",
  "reasoning": "[gemini-…] The post states general availability…",
  "sourceErrors": [],
  "collection": { "attempted": 2, "collected": 12, "extracted": 11, "failed": 0 },
  "consultedModels": ["gemini-2.5-flash-lite", "openai/gpt-oss-20b"],
  "corroboratingTier": "official",
  "alertTier": "strong",
  "confidence": 0.91,
  "grounded": true,
  "eventFingerprint": "5d2e…",
  "watchVersion": "a41c9e0b2f7d1c33",
  "lessonsApplied": ["8b1f…"],
  "feedback": null
}

Field	Use it for
outcome	Which stage the run reached. See Run outcomes
reasoning	Why the model decided what it did. Diagnostic only — never delivered in an alert
evidenceQuote / evidenceUrl	The verified basis for the alert
sourceErrors	Per-source failures, when any
collection	attempted / collected / extracted / failed counts
consultedModels	Primary, and the second opinion when it was asked
corroboratingTier	The highest trust tier among the sources that carry the evidence
redelivered	How many previously-undelivered alerts this run replayed
evaluationStats	Chunking and truncation, when the batch was large
alertTier	strong (confirmed) or weak (unconfirmed) for a run that fired
grounded / disputed / viaFailover	Whether the gate or the model shaped the verdict
watchVersion	A hash of the condition and thresholds the run was judged under
lessonsApplied	The learned rules that were in the evaluator’s prompt
heldBack	Why a triggered alert was not sent: kind (outdated, repeat, off-topic), reason, by, and eventDate when known. The run keeps a normal outcome
reviewed	true when the final review passed the alert, false when it couldn’t run and the alert went out as unconfirmed
evidenceDate / publishedAt	When the event happened, as stated, and the source’s publication date
feedbackReason / feedbackNote / feedbackMissedUrl	The detail of your label, when you gave one

Labelling a run

curl -s -X POST https://<your-host>/runs/0f1c…/feedback \
  -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{"feedback":"wrong","reason":"not-confirmed","note":"a status page is not the admit card"}'

feedback is "correct", "wrong", or null to clear it (which also clears the detail). The detail is optional:

Field	Meaning
reason	Why. It must fit the run: a fired alert takes off-topic, not-confirmed, already-known, duplicate, wrong-source, bad-evidence or other; a quiet run takes missed or other; a correct unconfirmed alert takes should-be-confirmed. Anything else is a 400
note	Your own words, up to 500 characters. Required with other
missedUrl	With missed: the page the event was on

The response is the updated record plus what Watcher learned from it:

{
  "run": { "...": "the updated record" },
  "learning": {
    "status": "interpreted",
    "understood": "Only an actual admit-card download counts.",
    "effects": ["Removed this alert from the watch's memory, so it can't hide the real event."],
    "lesson": { "id": "8b1f…", "kind": "clarify", "text": "A status page is not the admit card…" }
  }
}

status is applied (a rule from the reason, or a state fix), interpreted (your note was read), deferred (the note couldn’t be read right now — retry with POST /runs/:id/feedback/interpret; the label itself is already saved) or none. A run belonging to another account returns 404.

Note

Labels are how tuning becomes evidence rather than guesswork. A run that fired and was wrong is a false positive; a run that did not fire and was wrong is a false negative. Both feed the statistics below.

Accuracy statistics

curl -s https://<your-host>/feedback/stats -H "authorization: Bearer $TOKEN"

{
  "labeled": 24,
  "counts": { "truePositives": 9, "falsePositives": 2, "falseNegatives": 1, "trueNegatives": 12 },
  "precision": 0.818,
  "recall": 0.9,
  "f1": 0.857,
  "falsePositives": [ { "...": "recent examples" } ],
  "falseNegatives": [ { "...": "recent examples" } ]
}

Computed across every labelled run you own, with the most recent false positives and negatives included so you can see what went wrong rather than only how often. The response also carries byTier (precision of confirmed vs unconfirmed alerts), reasons (how often each reason was given) and collectionLabels (labels on runs that judged nothing, kept out of precision and recall). Add ?days=30 for a rolling window, or use GET /watches/:id/feedback/stats for a single watch.

What a watch learned

curl -s https://<your-host>/watches/framework-releases/learning -H "authorization: Bearer $TOKEN"

Returns the watch’s learning document — its lessons (each exclude, include or clarify, with where it came from and how many runs used it), interpreter suggestions, and whether learning is paused — plus proposals computed from your labels (a trust rule, or a change to apply through the revise flow). Rules are added with POST /watches/:id/lessons, edited or switched with PUT, removed with DELETE; proposals and suggestions can be dismissed. None of these routes change a watch’s settings — that always goes through your approval. See The feedback loop.

Collection statistics

curl -s https://<your-host>/collection/stats -H "authorization: Bearer $TOKEN"

{ "runs": 120, "attempted": 240, "collected": 1180, "extracted": 940, "failed": 6, "extractionRate": 0.797 }

extractionRate is the share of collected items that yielded substantive text. A rate well below 1 means pages are being fetched but not read — the classic JavaScript-portal symptom. See Sources return nothing.