- a valid Retry-After: 0 means retry at once but was treated as absent
(`or 5.0`) and waited 5s; honour it exactly now
- jitter our own headerless fallback and the inter-subreddit pacing so several
analyses sharing an IP don't retry in lockstep and re-collide on the limit;
keep the single-retry ceiling (more retries can't fix an exhausted IP budget) #1193
- crypto reached StockTwits as Yahoo's BTC-USD (404) instead of BTC.X, and Reddit
searched the dashed pair that barely matches; both now resolve the base via a
shared crypto_base() helper, restoring crypto sentiment
- also fixes a StockTwits resilience test class that pytest never collected #1113
The JSON search endpoint is reliably WAF-blocked (403) for public clients, so
probing it on every call doubled request volume against Reddit's per-IP rate
limit and tripped 429 on the RSS fallback, blanking the sentiment feed. Fetch
the Atom/RSS feed directly (JSON kept as an opt-in path that still degrades to
RSS on 403), back off once on a 429 honouring Retry-After, and pace requests a
little wider. Also broaden the error handling to catch http.client chunked
transfer errors (IncompleteRead/BadStatusLine) alongside OSError, which on their
own slipped through and crashed the pipeline.
Reddit blocks the anonymous JSON search endpoint, which silently emptied
the sentiment analyst's Reddit source. Fall back to the public RSS search
feed when JSON fails. RSS lacks score/comment counts, so those posts are
marked "via RSS feed" rather than shown with fake zeros.
#862
