Models and keys

Watcher uses models for several different jobs, and treats them as separate budgets rather than one shared pot.

Use-case categories

Category	Job	Provider
Evaluation	The primary judge: is the condition met?	Google
Second opinion	Cross-family confirmation of every trigger, and the failover tier	Groq
Embedding	Semantic deduplication of notified events	Google
Drafting	The AI watch drafter, reviser and explainer	Google
Search	Web search and page extraction	Tavily
Reader	The extraction tier for JavaScript-heavy pages	Jina

Each category has its own key pool. A key exhausted for chat evaluation therefore does not disable embeddings, even when it is physically the same key — cooldowns are tracked per model, while a revoked key is benched everywhere.

Note

Configure a single GOOGLE_GENERATIVE_AI_API_KEY and it seeds the evaluation, embedding and drafting pools at once. Categories only diverge when you add category-specific keys.

Rotation and health

Every model call goes through its category’s pool:

• Rate limited → the key cools down (honouring the provider’s retry hint when given, otherwise 60 seconds) and the pool advances to the next key.
• Invalid or unauthorised → the key is benched for the life of the process.
• Transient network or 5xx → one quick retry on the same key, then rotate.
• Anything else → surfaced immediately, because rotating would not help.

Model clients are configured not to retry internally, so a rate limit fails fast and rotation happens instead of waiting.

Pool health is in-memory: after a restart, a cooling key is tried again.

Choosing the model per category

Models are chosen from a curated allowlist per category — you cannot point a category at an arbitrary model id, which keeps a typo from silently disabling a capability.

PUT /admin/categories/evaluation
{ "provider": "google", "modelId": "gemini-2.5-flash" }

The change is probed with a live call before it is saved. With no active key to probe with, it is saved and flagged as unverified rather than rejected.

Keep the second opinion in a different family

The value of the second opinion is that it fails differently from the primary. Configuring both categories to the same provider family removes most of that value; the admin screen warns when you do.

Where keys can live

Environment variables — simplest, and all that a scheduled CI job needs.

The encrypted store — keys added through the admin API or panel are:

• probed live before being saved. A broken key is rejected with a reason; a valid but currently rate-limited key is saved on cooldown, because rejecting a hot key would defeat the point of pooling.
• encrypted at rest with a master key that exists only in the environment (CONFIG_ENCRYPTION_KEY). The database never holds plaintext.
• never returned. The API returns a masked preview such as AIza…3f9, never the value.

If CONFIG_ENCRYPTION_KEY is not set, stored keys are ignored with a warning and environment keys are used instead.

Usage metering

Every pooled call is metered locally into a per-key, per-day ledger: requests, successes, rate limits, auth failures and token totals, kept for about 90 days. GET /admin/keys/:id/metrics returns a 30-day series.

Note

These are Watcher’s own counts, not a bill from the provider. They are an estimate — useful for seeing which pool is under pressure, not for accounting.

Privacy of watch content

Free second-opinion tiers may use submitted prompts for training. For a watch over private or sensitive sources, set:

allowSecondaryModel: false

That watch then runs primary-only: no second opinion, and no failover to the secondary model.

Going deeper

• Model and quota errors — what failures look like.
• Costs and quotas — what runs out first.
• Admin API — the routes behind the panel.