Skip to content
Watcher Docs

Sources return nothing

Two different symptoms, two different causes:

Outcome Meaning
collection-failed Every source threw an error. Something is broken or unreachable.
no-candidates Sources ran fine and returned nothing. Usually extraction, not connectivity.

The run record stores a per-source error message for the first case, and collection counts (attempted, collected, extracted, failed) for both.

collection-failed — errors

Section titled “collection-failed — errors”
Error text Cause Fix
404, 403 The URL moved, or the site blocks automated fetches Update the URL; try the feed or a mirror
UND_ERR_CONNECT_TIMEOUT, ETIMEDOUT The site is slow or down Usually transient — the watch stays due and retries at the next opportunity
ENOTFOUND Bad hostname Fix the URL
Anything mentioning legacy renegotiation or SSL An old portal with outdated TLS Set hints: { legacyTls: true } on that source
TAVILY_API_KEY is required A search source with no key configured Add the key, or change the source type

no-candidates — nothing extracted

Section titled “no-candidates — nothing extracted”

This is the JavaScript problem in most cases.

How a page becomes text Page extraction is a ladder of four tiers, tried in order and only when the previous tier returned less than 200 characters: a clean extract service, article extraction from the fetched HTML, a whole-page tag strip, and finally an opt-in reader tier that renders JavaScript-heavy pages. 1 · Extract API clean parse, needs a key 2 · Article parse readability on the HTML 3 · Strip tags saves notice boards 4 · Reader tier opt-in: hints.reader under 200 chars? under 200 chars? under 200 chars? Result: one candidate item — a URL plus its readable text A tier is only adopted if it produced more text than the tier before it. Watcher does not run a browser. A page that builds its content entirely client-side may need tier 4 — or a different source.
Each tier is only tried when the previous one returned under 200 characters.

Work through these in order:

1. Look at what a plain fetch returns.

Terminal window
curl -sL "https://example.com/notices" | head -c 2000

If the interesting text is not in that output, Watcher cannot see it either — it does not run a browser.

2. Enable the reader tier. It renders JavaScript-heavy pages into clean text:

- type: page
  value: "https://example.com/notices"
  hints: { reader: true }

It is opt-in per source because it draws on a shared free tier.

3. Find a better source. In order of preference: an RSS feed, a JSON or API endpoint the page itself calls, a printer-friendly or archive version, or a search source that finds the news wherever it is published.

4. For feeds that only publish a summary, fetch the linked article too:

- type: rss
  value: "https://example.com/feed.xml"
  hints: { followLink: true }

Checking extraction health over time

Section titled “Checking extraction health over time”

GET /collection/stats — and the panel’s collection dashboard — aggregate recent runs into attempted, collected, extracted and an extraction rate. A rate well below 1 means items are being collected but yielding too little text, which is precisely the JavaScript-portal symptom.

Search sources returning nothing

Section titled “Search sources returning nothing”
  • The query is too narrow. Test it by hand in a search engine.
  • officialDomains pins results to those domains — if the news is published elsewhere, the search cannot see it.
  • recencyDays is too tight for a slow-moving story.

The panel’s draft preview (POST /watches/draft/preview) runs the search once and shows you the top results, which is the fastest way to test a query without waiting for a run.