new_vk_parser/STATE.md

N8 Parser: current state

Last updated: 2026-06-17

This file is the short handoff state for future Codex threads. Read this first before touching the project, so the whole chat history does not need to be carried forward.

Server

• Production host: 109.120.156.205
• Domain: https://sw.exostring.xyz
• App directory on server: /opt/vk-parser
• Timezone: Asia/Yekaterinburg
• Main services:
  • vk-parser-admin.service
  • vk-parser.service
  • vk-storage-uploader.service
  • ai-qualifier.service
  • ai-writer.service
  • site-poster.service
• Useful commands:
  • cd /opt/vk-parser
  • systemctl status vk-parser-admin.service ai-qualifier.service ai-writer.service
  • journalctl -u ai-writer.service -n 100 --no-pager
  • journalctl -u ai-qualifier.service -n 100 --no-pager
  • PYTHONPATH=src .venv/bin/python -m py_compile src/vk_parser_app/admin.py src/vk_parser_app/workers/ai_writer.py

Local Project

• Local workspace: D:\DEVELOPMENT\new_vk_parser
• Main app package: src/vk_parser_app
• Admin routes: src/vk_parser_app/admin.py
• Templates: src/vk_parser_app/templates
• Workers:
  • VK parser: src/vk_parser_app/workers/parser.py
  • Telegram media/storage uploader: src/vk_parser_app/workers/vk_storage_uploader.py
  • AI qualifier: src/vk_parser_app/workers/ai_qualifier.py
  • AI writer: src/vk_parser_app/workers/ai_writer.py
• DB migrations: db/migrations
• Extended project documentation: PROJECT.md

Pipeline

1. Sources are configured in admin.
2. VK parser reads enabled sources and stores raw posts.
3. Media/storage uploader sends raw post copies to Telegram storage and stores media metadata.
4. AI qualifier scores raw posts and marks accepted/rejected/maybe.
5. AI writer rewrites accepted posts into editor-ready drafts.
6. Editor page allows review, editing, category/source tag selection, media upload/removal, accept/reject.
7. Site poster automatically publishes accepted media-ready posts to Ghost.

Important Text Rules

• AI writer output text must be clean: no physical hashtags at the end.
• raw_posts.final_text must also stay clean.
• Category and source hashtags are separate fields:
  • category tag: final_category_tag / rewrite_category_tag
  • source tag: final_source_tag / rewrite_source_tag / sources.tag
• Publication preview may show hashtags by composing:
  • clean text
  • blank line
  • #category #source
• The editor textarea must show only clean editable text.
• The list preview in /editor should show the publication preview, including composed hashtags.
• build_publication_text() in src/vk_parser_app/text_utils.py is display/composition helper only. Do not use it before writing final_text to DB.

Recent Hotfix: Hashtags

On 2026-06-17 the hashtag flow was fixed:

• ai_writer.py now stores clean final_text=$2, not text with hashtags.
• admin.py prepares:
  • review_text: clean text for textarea
  • publication_preview_text: composed display text with hashtags
• editor accept/save store clean text.
• existing DB rows with trailing #category #source in final_text were cleaned.
• server check after fix: remaining_hashtag_tail: 0.

Admin Pages

• /sources: source management.
• /raw: raw post feed.
• /editor: editor queue.
• /workers: worker controls and settings.
• /prompt-test: AI writer prompt test page.
• /users: admin users.
• /logs: audit/log page if enabled.

Prompt Architecture

• Prompts are split into editable human part and technical contract part.
• User should be able to edit human prompt safely without breaking JSON schema.
• Contracts should contain JSON-only/output-schema requirements.
• Writer should return category ID, not free-form tag, where possible.
• Categories are managed in worker/admin settings and have stable IDs.

Security Notes

• Secrets should not be committed.
• .env should contain only secrets and deploy-specific values.
• Runtime settings that are not secrets should live in DB/admin settings.
• Admin has login protection and server has firewall/fail2ban hardening from previous iterations.

Working Rules For Future Codex Runs

• Prefer small targeted file reads with rg and narrow ranges.
• Avoid dumping long post texts from DB unless explicitly needed.
• For server checks, return counts/statuses first.
• Be careful: both local and server worktrees may be dirty.
• Do not git reset --hard or revert user/server hotfixes.
• When fixing production, patch server and then mirror the same change locally.