Runs and feedback API

Method	Path	Auth	Purpose
POST	/runs/:id/feedback	Token	Label a run correct or wrong, or clear the label with null.
GET	/feedback/stats	Token	Rolling precision, recall and F1 with recent false positives and negatives.
GET	/collection/stats	Token	Rolling collection metrics, including the extraction rate.

Run history itself is read through the watches API: GET /watches/:id/runs.

The run record

{
  "id": "0f1c…",
  "watchId": "framework-releases",
  "ownerId": "local",
  "timestamp": "2026-09-10T06:00:04.120Z",
  "outcome": "notified",
  "summary": "Version 2.0 was released as stable.",
  "evidenceUrl": "https://example.com/blog/v2-ga",
  "evidenceQuote": "v2.0 is now generally available",
  "reasoning": "[gemini-…] The post states general availability…",
  "sourceErrors": [],
  "collection": { "attempted": 2, "collected": 12, "extracted": 11, "failed": 0 },
  "consultedModels": ["gemini-2.5-flash-lite", "llama-3.3-70b-versatile"],
  "corroboratingTier": "official",
  "feedback": null
}

Field	Use it for
outcome	Which stage the run reached. See Run outcomes
reasoning	Why the model decided what it did. Diagnostic only — never delivered in an alert
evidenceQuote / evidenceUrl	The verified basis for the alert
sourceErrors	Per-source failures, when any
collection	attempted / collected / extracted / failed counts
consultedModels	Primary, and the second opinion when it was asked
corroboratingTier	The highest trust tier that backed the verdict
redelivered	How many previously-undelivered alerts this run replayed
evaluationStats	Chunking and truncation, when the batch was large

Labelling a run

curl -s -X POST https://<your-host>/runs/0f1c…/feedback \
  -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{"feedback":"wrong"}'

feedback is "correct", "wrong", or null to clear it. The updated record is returned. A run belonging to another account returns 404.

Note

Labels are how tuning becomes evidence rather than guesswork. A run that fired and was wrong is a false positive; a run that did not fire and was wrong is a false negative. Both feed the statistics below.

Accuracy statistics

curl -s https://<your-host>/feedback/stats -H "authorization: Bearer $TOKEN"

{
  "labeled": 24,
  "counts": { "truePositives": 9, "falsePositives": 2, "falseNegatives": 1, "trueNegatives": 12 },
  "precision": 0.818,
  "recall": 0.9,
  "f1": 0.857,
  "falsePositives": [ { "...": "recent examples" } ],
  "falseNegatives": [ { "...": "recent examples" } ]
}

Computed across every labelled run you own, with the most recent false positives and negatives included so you can see what went wrong rather than only how often.

Collection statistics

curl -s https://<your-host>/collection/stats -H "authorization: Bearer $TOKEN"

{ "runs": 120, "attempted": 240, "collected": 1180, "extracted": 940, "failed": 6, "extractionRate": 0.797 }

extractionRate is the share of collected items that yielded substantive text. A rate well below 1 means pages are being fetched but not read — the classic JavaScript-portal symptom. See Sources return nothing.