feat: add declarative site parser engine
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# RAA deployment
|
||||
|
||||
Last updated: 2026-08-10.
|
||||
Last updated: 2026-08-11.
|
||||
|
||||
RAA is not a separate code project anymore. The source of truth for code and
|
||||
documentation is:
|
||||
@@ -96,20 +96,15 @@ All source records, JSON configs, schedules, errors, raw posts, and media state
|
||||
remain in the RAA database. The worker is managed by Docker Compose from
|
||||
`/opt/site-parser` and should be healthy as `site-parser-worker`.
|
||||
|
||||
PopularAirsoft source id `72`:
|
||||
PopularAirsoft source id `72` uses `https://popularairsoft.com/` and the full
|
||||
v1 config from `site_parser_worker/README.md`. Its RSS is not used for
|
||||
discovery because it mixes short video pages with full articles. HTML discovery
|
||||
selects the `The Latest News` cards and the worker then opens every `/news/...`
|
||||
detail. Verified live 2026-08-11: nine details parsed, zero errors.
|
||||
|
||||
- URL: `https://popularairsoft.com/rss.xml`
|
||||
- Config: `{"format":"rss","access":"cloudflare","max_items":10,"min_text_length":50}`
|
||||
- `access='cloudflare'` deliberately starts the browser/RuCaptcha flow before
|
||||
fetching the RSS.
|
||||
- Keep `follow_links` disabled for this feed. Nine current entries have no
|
||||
body, only title/video, and are intentionally filtered before media upload.
|
||||
- Cloudflare cookies and browser User-Agent are reused by `media-uploader` for
|
||||
protected images.
|
||||
- Admin image previews use authenticated Telegram-backed
|
||||
`/raw/media/{media_id}` URLs.
|
||||
- Verified 2026-08-10: source `ok`; raw post `102` is `storage_ready`; four
|
||||
images are `uploaded`. The AI later rejected the content as non-target.
|
||||
Cloudflare session state is returned to the main app and is also reused by
|
||||
`media-uploader` for protected images. Admin previews remain authenticated
|
||||
Telegram-backed `/raw/media/{media_id}` URLs.
|
||||
|
||||
## RAA Categories
|
||||
|
||||
|
||||
Reference in New Issue
Block a user