docs: document external site parser

STATE.md

@@ -1,6 +1,6 @@
 # N8 Parser: current state
 
-Last updated: 2026-08-03
+Last updated: 2026-08-10
 
 This file is the short handoff state for future Codex threads. Read this first before touching the project, so the whole chat history does not need to be carried forward.
 
@@ -37,6 +37,8 @@ Current runtime is local infrastructure, not the old VPS.
 
 1. Sources are configured in admin.
 2. VK parser reads enabled sources and stores raw posts.
+   The same worker also dispatches `platform='site'` sources to the external
+   site-parser module one at a time.
 3. Media/storage uploader sends raw post copies to Telegram storage and stores media metadata.
 4. AI qualifier scores raw posts and marks accepted/rejected/maybe.
 5. AI writer rewrites accepted posts into editor-ready drafts.
@@ -75,6 +77,60 @@ canonical git repository as FN-8. Code work for both projects must happen in
   (`raa-sender`) remains a valid legacy VK reposting app and is not the shared
   RAA parser/writer/poster app.
 
+## External Site Parser
+
+As of 2026-08-10, website parsing is integrated into the normal raw-post
+pipeline. The main application owns sources, per-source JSON configs,
+schedules, settings, database state, deduplication, media jobs, and errors.
+The external module is stateless: it receives one source request, fetches it,
+and returns normalized JSON.
+
+- Runtime: LXC `108`, IP `192.168.1.113`, port `8080`.
+- Management: Docker Compose in `/opt/site-parser`.
+- Repository compose file: `site_parser_worker/docker-compose.yml`.
+- The module is protected by `WORKER_TOKEN`; the value is stored only in
+  deployment settings/secrets.
+- RuCaptcha is stored in the main app setting
+  `site_parser_rucaptcha_token`; never copy the key into git documentation.
+- Main settings are under `/workers` -> `Site Parser`: worker URL/token,
+  RuCaptcha token, request timeout, global interval, and default minimum body
+  length.
+- Website sources are added on `/sources` with `platform='site'` and their own
+  JSON config. A missing or invalid config produces a visible source error and
+  does not run silently.
+- Due website sources are processed sequentially, not launched together.
+- `access='cloudflare'` enables the Playwright/RuCaptcha path for that source.
+  Plain RSS/sites do not use a browser.
+- Browser cookies plus the exact browser User-Agent are returned to the main
+  app and reused by `media-uploader` when downloading protected images.
+- Uploaded photos are previewed in the admin through authenticated
+  `/raw/media/{media_id}` URLs backed by Telegram storage, not by the original
+  Cloudflare-protected URL.
+
+Current RAA test source:
+
+```json
+{"format":"rss","access":"cloudflare","max_items":10,"min_text_length":50}
+```
+
+- Source: `Popularairsoft`, `https://popularairsoft.com/rss.xml`, id `72`.
+- Do not add `follow_links` for the current feed: one of ten entries has a real
+  body and four photos; the other nine are title-plus-video entries and are
+  intentionally skipped by the body-length threshold.
+- Verified on 2026-08-10: source status `ok`, raw post `102` reached
+  `storage_ready`, and all four photos were uploaded. Its later rejection is
+  an AI non-target decision, not a parser/media failure.
+- Verified RAA deploy: commit `f866a13a8a83598e26a7a6a3406b8bc1c01c9a80`.
+
+LXC 108 operations:
+
+```bash
+cd /opt/site-parser
+docker-compose up -d --build
+docker-compose ps
+docker-compose logs -f
+```
+
 ## Important Text Rules
 
 - AI writer output text must be clean: no physical hashtags at the end.