new_vk_parser/STATE.md

N8 Parser: current state

Last updated: 2026-07-28

This file is the short handoff state for future Codex threads. Read this first before touching the project, so the whole chat history does not need to be carried forward.

Runtime / Deploy

Current runtime is local infrastructure, not the old VPS.

• Infrastructure source of truth: D:\DEVELOPMENT\.infra.md.
• Git deploy remote: gitea -> http://192.168.1.135:3000/exostring/new_vk_parser.git.
• Deploy path: push to gitea/main -> Gitea webhook -> Coolify -> Docker build/deploy.
• Coolify + Gitea LXC: 192.168.1.135 (CTID 102).
• Application LXC: 192.168.1.105 (CTID 105, new-vk-parser).
• Public admin URL: https://panel.f-n8.ru (also https://admin.f-n8.ru when DNS is current).
• Coolify repository URL must be reachable from the deployment container; use http://192.168.1.135:3000/exostring/new_vk_parser.git or another resolvable Gitea URL, not http://coolify.lan:3000/....
• Do not deploy this project through sw.exostring.xyz, 109.120.156.205, /opt/vk-parser, or manual systemctl unless the user explicitly says the old VPS is being restored.
• Old sw.exostring.xyz / 109.120.156.205 notes are stale for this project.

Local Project

• Local workspace: D:\DEVELOPMENT\new_vk_parser
• Main app package: src/vk_parser_app
• Admin routes: src/vk_parser_app/admin.py
• Templates: src/vk_parser_app/templates
• Workers:
  • VK parser: src/vk_parser_app/workers/parser.py
  • Telegram media/storage uploader: src/vk_parser_app/workers/vk_storage_uploader.py
  • AI qualifier: src/vk_parser_app/workers/ai_qualifier.py
  • AI writer: src/vk_parser_app/workers/ai_writer.py
• DB migrations: db/migrations
• Extended project documentation: PROJECT.md

Pipeline

1. Sources are configured in admin.
2. VK parser reads enabled sources and stores raw posts.
3. Media/storage uploader sends raw post copies to Telegram storage and stores media metadata.
4. AI qualifier scores raw posts and marks accepted/rejected/maybe.
5. AI writer rewrites accepted posts into editor-ready drafts.
6. Editor page allows review, editing, category/source tag selection, media upload/removal, accept/reject.
7. Site poster automatically publishes accepted media-ready posts to Ghost.

Important Text Rules

• AI writer output text must be clean: no physical hashtags at the end.
• raw_posts.final_text must also stay clean.
• Category and source hashtags are separate fields:
  • category tag: final_category_tag / rewrite_category_tag
  • source tag: final_source_tag / rewrite_source_tag / sources.tag
• Publication preview may show hashtags by composing:
  • clean text
  • blank line
  • #category #source
• The editor textarea must show only clean editable text.
• The list preview in /editor should show the publication preview, including composed hashtags.
• build_publication_text() in src/vk_parser_app/text_utils.py is display/composition helper only. Do not use it before writing final_text to DB.

Recent Hotfix: Hashtags

On 2026-06-17 the hashtag flow was fixed:

• ai_writer.py now stores clean final_text=$2, not text with hashtags.
• admin.py prepares:
  • review_text: clean text for textarea
  • publication_preview_text: composed display text with hashtags
• editor accept/save store clean text.
• existing DB rows with trailing #category #source in final_text were cleaned.
• server check after fix: remaining_hashtag_tail: 0.

Admin Pages

• /sources: source management.
• /raw: raw post feed.
• /editor: editor queue.
• /workers: worker controls and settings.
• /prompt-test: AI writer prompt test page.
• /users: admin users.
• /logs: audit/log page if enabled.

Prompt Architecture

• Prompts are split into editable human part and technical contract part.
• User should be able to edit human prompt safely without breaking JSON schema.
• Contracts should contain JSON-only/output-schema requirements.
• Writer should return category ID, not free-form tag, where possible.
• Categories are managed in worker/admin settings and have stable IDs.

Security Notes

• Secrets should not be committed.
• .env should contain only secrets and deploy-specific values.
• Runtime settings that are not secrets should live in DB/admin settings.
• Admin has login protection and server has firewall/fail2ban hardening from previous iterations.

Working Rules For Future Codex Runs

• Before any deploy/server operation, read D:\DEVELOPMENT\.infra.md first.
• For this project, push deployable changes to gitea/main; Coolify handles deployment.
• Prefer small targeted file reads with rg and narrow ranges.
• Avoid dumping long post texts from DB unless explicitly needed.
• For runtime checks, use the current Coolify/Gitea/LXC 105 path from D:\DEVELOPMENT\.infra.md.
• Do not git reset --hard or revert user/server hotfixes.