# N8 Parser: current state

Last updated: 2026-06-17

This file is the short handoff state for future Codex threads. Read this first before touching the project, so the whole chat history does not need to be carried forward.

## Server

- Production host: `109.120.156.205`
- Domain: `https://sw.exostring.xyz`
- App directory on server: `/opt/vk-parser`
- Timezone: `Asia/Yekaterinburg`
- Main services:
  - `vk-parser-admin.service`
  - `vk-parser.service`
  - `vk-storage-uploader.service`
  - `ai-qualifier.service`
  - `ai-writer.service`
  - `site-poster.service`
- Useful commands:
  - `cd /opt/vk-parser`
  - `systemctl status vk-parser-admin.service ai-qualifier.service ai-writer.service`
  - `journalctl -u ai-writer.service -n 100 --no-pager`
  - `journalctl -u ai-qualifier.service -n 100 --no-pager`
  - `PYTHONPATH=src .venv/bin/python -m py_compile src/vk_parser_app/admin.py src/vk_parser_app/workers/ai_writer.py`

## Local Project

- Local workspace: `D:\DEVELOPMENT\new_vk_parser`
- Main app package: `src/vk_parser_app`
- Admin routes: `src/vk_parser_app/admin.py`
- Templates: `src/vk_parser_app/templates`
- Workers:
  - VK parser: `src/vk_parser_app/workers/parser.py`
  - Telegram media/storage uploader: `src/vk_parser_app/workers/vk_storage_uploader.py`
  - AI qualifier: `src/vk_parser_app/workers/ai_qualifier.py`
  - AI writer: `src/vk_parser_app/workers/ai_writer.py`
- DB migrations: `db/migrations`
- Extended project documentation: `PROJECT.md`

## Pipeline

1. Sources are configured in admin.
2. VK parser reads enabled sources and stores raw posts.
3. Media/storage uploader sends raw post copies to Telegram storage and stores media metadata.
4. AI qualifier scores raw posts and marks accepted/rejected/maybe.
5. AI writer rewrites accepted posts into editor-ready drafts.
6. Editor page allows review, editing, category/source tag selection, media upload/removal, accept/reject.
7. Site poster automatically publishes accepted media-ready posts to Ghost.

## Important Text Rules

- AI writer output text must be clean: no physical hashtags at the end.
- `raw_posts.final_text` must also stay clean.
- Category and source hashtags are separate fields:
  - category tag: `final_category_tag` / `rewrite_category_tag`
  - source tag: `final_source_tag` / `rewrite_source_tag` / `sources.tag`
- Publication preview may show hashtags by composing:
  - clean text
  - blank line
  - `#category #source`
- The editor textarea must show only clean editable text.
- The list preview in `/editor` should show the publication preview, including composed hashtags.
- `build_publication_text()` in `src/vk_parser_app/text_utils.py` is display/composition helper only. Do not use it before writing `final_text` to DB.

## Recent Hotfix: Hashtags

On 2026-06-17 the hashtag flow was fixed:

- `ai_writer.py` now stores clean `final_text=$2`, not text with hashtags.
- `admin.py` prepares:
  - `review_text`: clean text for textarea
  - `publication_preview_text`: composed display text with hashtags
- editor accept/save store clean text.
- existing DB rows with trailing `#category #source` in `final_text` were cleaned.
- server check after fix: `remaining_hashtag_tail: 0`.

## Admin Pages

- `/sources`: source management.
- `/raw`: raw post feed.
- `/editor`: editor queue.
- `/workers`: worker controls and settings.
- `/prompt-test`: AI writer prompt test page.
- `/users`: admin users.
- `/logs`: audit/log page if enabled.

## Prompt Architecture

- Prompts are split into editable human part and technical contract part.
- User should be able to edit human prompt safely without breaking JSON schema.
- Contracts should contain JSON-only/output-schema requirements.
- Writer should return category ID, not free-form tag, where possible.
- Categories are managed in worker/admin settings and have stable IDs.

## Security Notes

- Secrets should not be committed.
- `.env` should contain only secrets and deploy-specific values.
- Runtime settings that are not secrets should live in DB/admin settings.
- Admin has login protection and server has firewall/fail2ban hardening from previous iterations.

## Working Rules For Future Codex Runs

- Prefer small targeted file reads with `rg` and narrow ranges.
- Avoid dumping long post texts from DB unless explicitly needed.
- For server checks, return counts/statuses first.
- Be careful: both local and server worktrees may be dirty.
- Do not `git reset --hard` or revert user/server hotfixes.
- When fixing production, patch server and then mirror the same change locally.