Initial RAA parser poster copy

This commit is contained in:
Your Name
2026-08-03 16:18:56 +05:00
commit d01890ff2c
148 changed files with 18924 additions and 0 deletions
+27
View File
@@ -0,0 +1,27 @@
APP_ENV=production
APP_SECRET_KEY=change-me
ADMIN_SITE_TITLE=RAA Редакторская
ADMIN_APP_TITLE=RAA Редакторская
ADMIN_BOOTSTRAP_LOGIN=admin
ADMIN_BOOTSTRAP_PASSWORD=change-me-long-password
DB_HOST=localhost
DB_PORT=5432
DB_NAME=raa_parser
DB_USER=raa_parser_user
DB_PASSWORD=change-me
VK_ACCESS_TOKEN=
VK_GROUP_ACCESS_TOKEN=
VK_API_VERSION=5.199
VK_STORAGE_GROUP_ID=0
TG_BOT_TOKEN=
TG_MEDIA_CHANNEL_ID=
LOCAL_BOT_API_URL=
TELEGRAM_API_ID=
TELEGRAM_API_HASH=
ADMIN_HOST=0.0.0.0
ADMIN_PORT=8080
LOG_LEVEL=INFO
+5
View File
@@ -0,0 +1,5 @@
* text=auto eol=lf
*.bat text eol=crlf
*.cmd text eol=crlf
*.ps1 text eol=crlf
+15
View File
@@ -0,0 +1,15 @@
.env
.env.*
!.env.example
__pycache__/
*.py[cod]
.venv/
logs/
uploads/
original_project/
/ai_worker.py
/media_worker.py
/parser.py
/raw_feed_worker.py
vkdev_main.js
sample_photo.jpg
+59
View File
@@ -0,0 +1,59 @@
FROM aiogram/telegram-bot-api:latest AS telegram-bot-api
FROM python:3.12-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
ca-certificates \
postgresql-client \
curl \
ffmpeg \
git \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir -p /usr/local/share/ca-certificates/russian-trusted \
&& curl -fsSLk https://gu-st.ru/content/lending/russian_trusted_root_ca_pem.crt \
-o /usr/local/share/ca-certificates/russian-trusted/russian_trusted_root_ca_pem.crt \
&& curl -fsSLk https://gu-st.ru/content/lending/russian_trusted_sub_ca_pem.crt \
-o /usr/local/share/ca-certificates/russian-trusted/russian_trusted_sub_ca_pem.crt \
&& update-ca-certificates
# Copy the local Telegram Bot API binary and its musl runtime from the official aiogram image.
COPY --from=telegram-bot-api /usr/local/bin/telegram-bot-api /usr/local/bin/telegram-bot-api
COPY --from=telegram-bot-api /lib/ld-musl-x86_64.so.1 /lib/ld-musl-x86_64.so.1
COPY --from=telegram-bot-api /usr/lib/libssl.so.3 /usr/lib/libssl.so.3
COPY --from=telegram-bot-api /usr/lib/libcrypto.so.3 /usr/lib/libcrypto.so.3
COPY --from=telegram-bot-api /usr/lib/libz.so.1 /usr/lib/libz.so.1
COPY --from=telegram-bot-api /usr/lib/libstdc++.so.6 /usr/lib/libstdc++.so.6
COPY --from=telegram-bot-api /usr/lib/libgcc_s.so.1 /usr/lib/libgcc_s.so.1
# Install python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy source code
COPY src/ src/
COPY scripts/ scripts/
COPY db/ db/
COPY docker-entrypoint.sh /usr/local/bin/vk-parser-entrypoint
RUN chmod +x /usr/local/bin/vk-parser-entrypoint \
&& mkdir -p /var/lib/telegram-bot-api /tmp/telegram-bot-api
COPY favi.png .
COPY .env.example .env
ENV PYTHONPATH=/app/src
ENV PYTHONUNBUFFERED=1
ENV LOCAL_BOT_API_URL=http://127.0.0.1:8081
ENV TELEGRAM_WORK_DIR=/var/lib/telegram-bot-api
ENV TELEGRAM_TEMP_DIR=/tmp/telegram-bot-api
ENV TELEGRAM_HTTP_PORT=8081
ENV TELEGRAM_LOCAL=1
EXPOSE 8000
# Start uvicorn without any workers
ENTRYPOINT ["vk-parser-entrypoint"]
CMD ["uvicorn", "vk_parser_app.admin:app", "--host", "0.0.0.0", "--port", "8000"]
+1728
View File
File diff suppressed because it is too large Load Diff
+34
View File
@@ -0,0 +1,34 @@
# RAA Parser/Poster
RAA is a separate deployment copied from the FN-8 parser/poster codebase.
## Runtime
- Local workspace: `D:\DEVELOPMENT\raa-parser_poster`
- Public admin domain: `https://raa.panel.f-n8.ru`
- Site poster must stay disabled until a site adapter is configured.
- The current site-poster adapter boundary is `site_poster_provider`; only `ghost`
exists in code today. WordPress should be added as a code adapter, not as
arbitrary executable code stored in settings.
## First-Run Settings
Set real secrets in Coolify/env, not in git:
- `APP_SECRET_KEY`
- `ADMIN_BOOTSTRAP_LOGIN`
- `ADMIN_BOOTSTRAP_PASSWORD`
- `DB_*`
- `TG_BOT_TOKEN`
- `TELEGRAM_API_ID`
- `TELEGRAM_API_HASH`
- `VK_ACCESS_TOKEN` or `VK_GROUP_ACCESS_TOKEN`
Set runtime settings in the admin UI/DB:
- `TG_MEDIA_CHANNEL_ID`
- `tg_poster_bot_token`
- `tg_poster_chat_id`
- `vk_poster_owner_id`
- `site_poster_enabled=false`
- `site_poster_provider=""`
+199
View File
@@ -0,0 +1,199 @@
# VK Parser New Core
Первый чистый слой проекта:
- админка с источниками, пользователями, воркерами и raw-лентой;
- VK parser worker;
- VK storage uploader worker;
- PostgreSQL job queue.
Старый проект лежит в `original_project/` как справочник. Новый код живёт в `src/vk_parser_app/`.
## Что нужно на сервере
Ubuntu 22.04/24.04:
```bash
sudo apt update
sudo apt install -y python3 python3-venv python3-pip postgresql postgresql-contrib nginx git curl
```
Для будущей работы с видео:
```bash
sudo apt install -y ffmpeg
```
## БД
Пример:
```bash
sudo -u postgres psql
```
```sql
CREATE DATABASE vk_parser;
CREATE USER vk_parser_user WITH PASSWORD 'CHANGE_ME';
GRANT ALL PRIVILEGES ON DATABASE vk_parser TO vk_parser_user;
\c vk_parser
GRANT ALL ON SCHEMA public TO vk_parser_user;
```
## Установка проекта
```bash
cd /opt
sudo git clone <repo-url> vk-parser
sudo chown -R $USER:$USER /opt/vk-parser
cd /opt/vk-parser
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
```
Заполнить `.env`:
```env
APP_SECRET_KEY=long-random-string
ADMIN_BOOTSTRAP_LOGIN=admin
ADMIN_BOOTSTRAP_PASSWORD=long-admin-password
DB_HOST=localhost
DB_PORT=5432
DB_NAME=vk_parser
DB_USER=vk_parser_user
DB_PASSWORD=...
VK_ACCESS_TOKEN=...
VK_STORAGE_GROUP_ID=239548476
```
Важно: для `wall.post`, `wall.edit`, `photos.saveWallPhoto` нужен VK user token пользователя с правами в storage-группе. Community token для этого контура не подходит.
## Миграции
```bash
PYTHONPATH=src .venv/bin/python scripts/apply_migrations.py
```
Первый админ создаётся из `ADMIN_BOOTSTRAP_LOGIN` / `ADMIN_BOOTSTRAP_PASSWORD` при старте админки.
## Проверка VK storage
Без фото:
```bash
PYTHONPATH=src .venv/bin/python scripts/vk_storage_spike.py
```
С фото:
```bash
cp /path/to/photo.jpg sample_photo.jpg
PYTHONPATH=src .venv/bin/python scripts/vk_storage_spike.py
```
Скрипт должен вывести ссылку вида:
```text
https://vk.com/wall-239548476_123
```
После этого надо руками проверить:
- пост создан от имени storage-группы;
- админ/редактор группы может открыть и отредактировать пост;
- вложенное фото видно в браузере.
## Запуск локально
Админка:
```bash
PYTHONPATH=src .venv/bin/uvicorn vk_parser_app.admin:app --host 0.0.0.0 --port 8080
```
Парсер:
```bash
PYTHONPATH=src .venv/bin/python -m vk_parser_app.workers.parser
```
VK storage uploader:
```bash
PYTHONPATH=src .venv/bin/python -m vk_parser_app.workers.vk_storage_uploader
```
## systemd units
Минимальный `vk-admin.service`:
```ini
[Unit]
Description=VK Parser Admin
After=network.target postgresql.service
[Service]
WorkingDirectory=/opt/vk-parser
Environment=PYTHONPATH=/opt/vk-parser/src
ExecStart=/opt/vk-parser/.venv/bin/uvicorn vk_parser_app.admin:app --host 0.0.0.0 --port 8080
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
```
`vk-parser.service`:
```ini
[Unit]
Description=VK Parser Worker
After=network.target postgresql.service
[Service]
WorkingDirectory=/opt/vk-parser
Environment=PYTHONPATH=/opt/vk-parser/src
ExecStart=/opt/vk-parser/.venv/bin/python -m vk_parser_app.workers.parser
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
`vk-storage-uploader.service`:
```ini
[Unit]
Description=VK Storage Uploader Worker
After=network.target postgresql.service
[Service]
WorkingDirectory=/opt/vk-parser
Environment=PYTHONPATH=/opt/vk-parser/src
ExecStart=/opt/vk-parser/.venv/bin/python -m vk_parser_app.workers.vk_storage_uploader
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
## Текущий pipeline
```text
sources(vk)
-> vk-parser
-> raw_posts + raw_post_media
-> jobs(type='vk.storage.copy')
-> vk-storage-uploader
-> wall.post в storage-группе
-> raw_posts.storage_post_url
```
Фото копируются в storage-группу. Видео в первой версии сохраняются как `link_only`/исходное VK-вложение, чтобы сначала стабилизировать основной контур.
+142
View File
@@ -0,0 +1,142 @@
# N8 Parser: current state
Last updated: 2026-08-03
This file is the short handoff state for future Codex threads. Read this first before touching the project, so the whole chat history does not need to be carried forward.
## Runtime / Deploy
Current runtime is local infrastructure, not the old VPS.
- Infrastructure source of truth: `D:\DEVELOPMENT\.infra.md`.
- Git deploy remote: `gitea` -> `http://192.168.1.135:3000/exostring/new_vk_parser.git`.
- Deploy path: push to `gitea/main` -> Gitea webhook -> Coolify -> Docker build/deploy.
- Coolify + Gitea LXC: `192.168.1.135` (CTID 102).
- Coolify app UUID: `n6cnr60anmruiwkiey0pukye` (`FN-8 admin`). API token is only in `D:\DEVELOPMENT\.infra.md`; do not copy it into repo docs.
- Application LXC: `192.168.1.105` (CTID 105, `new-vk-parser`).
- Public admin URL: `https://panel.f-n8.ru` (also `https://admin.f-n8.ru` when DNS is current).
- Coolify repository URL must be reachable from the deployment container; use `http://192.168.1.135:3000/exostring/new_vk_parser.git` or another resolvable Gitea URL, not `http://coolify.lan:3000/...`.
- Do not deploy this project through `sw.exostring.xyz`, `109.120.156.205`, `/opt/vk-parser`, or manual `systemctl` unless the user explicitly says the old VPS is being restored.
- Old `sw.exostring.xyz` / `109.120.156.205` notes are stale for this project.
## Local Project
- Local workspace: `D:\DEVELOPMENT\new_vk_parser`
- Main app package: `src/vk_parser_app`
- Admin routes: `src/vk_parser_app/admin.py`
- Templates: `src/vk_parser_app/templates`
- Workers:
- VK parser: `src/vk_parser_app/workers/parser.py`
- Telegram media/storage uploader: `src/vk_parser_app/workers/vk_storage_uploader.py`
- AI qualifier: `src/vk_parser_app/workers/ai_qualifier.py`
- AI writer: `src/vk_parser_app/workers/ai_writer.py`
- DB migrations: `db/migrations`
- Extended project documentation: `PROJECT.md`
## Pipeline
1. Sources are configured in admin.
2. VK parser reads enabled sources and stores raw posts.
3. Media/storage uploader sends raw post copies to Telegram storage and stores media metadata.
4. AI qualifier scores raw posts and marks accepted/rejected/maybe.
5. AI writer rewrites accepted posts into editor-ready drafts.
6. Editor page allows review, editing, category/source tag selection, media upload/removal, accept/reject.
7. Site poster automatically publishes accepted media-ready posts to Ghost.
- As of 2026-08-03, site publishing is provider-based: `site_poster_provider=ghost`
for FN-8, with `site_poster_base_url` and `site_poster_secret_file` copied from
legacy Ghost settings by migration `040_site_poster_provider_settings.sql`.
- New installs default `site-poster` to disabled and must explicitly set provider,
base URL, secret, and `site_poster_enabled=true`.
## Important Text Rules
- AI writer output text must be clean: no physical hashtags at the end.
- `raw_posts.final_text` must also stay clean.
- Category and source hashtags are separate fields:
- category tag: `final_category_tag` / `rewrite_category_tag`
- source tag: `final_source_tag` / `rewrite_source_tag` / `sources.tag`
- Publication preview may show hashtags by composing:
- clean text
- blank line
- `#category #source`
- The editor textarea must show only clean editable text.
- The list preview in `/editor` should show the publication preview, including composed hashtags.
- `build_publication_text()` in `src/vk_parser_app/text_utils.py` is display/composition helper only. Do not use it before writing `final_text` to DB.
## Recent Hotfix: Hashtags
On 2026-06-17 the hashtag flow was fixed:
- `ai_writer.py` now stores clean `final_text=$2`, not text with hashtags.
- `admin.py` prepares:
- `review_text`: clean text for textarea
- `publication_preview_text`: composed display text with hashtags
- editor accept/save store clean text.
- existing DB rows with trailing `#category #source` in `final_text` were cleaned.
- server check after fix: `remaining_hashtag_tail: 0`.
## Admin Pages
- `/sources`: source management.
- `/raw`: raw post feed.
- `/editor`: editor queue.
- `/workers`: worker controls and settings.
- `/prompt-test`: AI writer prompt test page.
- `/users`: admin users.
- `/logs`: audit/log page if enabled.
## Recent Admin UI State
As of 2026-07-29:
- Admin title comes from env `ADMIN_APP_TITLE`; fallback is `Редакторская`. The old hardcoded `Parser Admin` title and lightning SVG are gone.
- Browser/page title comes from env `ADMIN_SITE_TITLE`; fallback is `Редакторская`.
- Admin sidebar is collapsible and collapsed by default; collapsed state shows icons only.
- Theme toggle supports dark/light themes. Tailwind `app.*` colors use RGB-channel CSS variables (`rgb(var(--app-...)/<alpha-value>)`), so do not revert them to hex variables or opacity utilities will break. Light-theme hover contrast is patched in base CSS.
- `/editor` desktop filter panel is collapsible and collapsed by default; the feed expands into the freed width.
- Mobile editor status tabs scroll horizontally instead of overflowing the viewport. `/sources` and `/raw` have mobile layout fixes to avoid page-level horizontal scroll.
- In editor cards, post meta/status/source info is above the post, not in a side column.
- `Оригинальный текст` is a spoiler directly above `Финальный текст`.
- `AI заметки / Резолюция` is a separate spoiler. It contains AI notes/resolution and an AI rewrite preview below it, so editors can compare their changes with the original AI rewrite.
- `Финальный текст` textarea must autosize to fully fit text without internal scroll. Current implementation uses Alpine `ResizeObserver`, trims trailing whitespace in the browser/template, and keeps `min-h-[150px]`.
- Media hiding is handled by the editor media X button: it adds hidden `delete_media_ids`; backend marks media `editor_hidden=TRUE`, so hidden media is not published.
- Reject action has no confirmation popup.
- Save/accept/return actions use targeted HTMX partial updates for status/actions where possible, so open spoilers and the post body should not collapse after clicking action buttons.
- `/editor/{post_id}/return` returns accepted/rejected posts to review. The return button should exist on both accepted and rejected tabs.
## Recent AI Writer State
- Current live writer model setting checked on 2026-07-29: `ai_writer_model = anthropic/claude-sonnet-4-6`.
- Sonnet 4.6 can return `rewrites` as a JSON string instead of a list. `validate_rewrites()` in `src/vk_parser_app/workers/ai_writer.py` now parses stringified `rewrites` before validation.
- 2026-07-30 incident: post `2348` was retried 983 times by AI writer because automatic claim included `rewrite_status='failed'`; estimated internal cost was `$16.465977`. Live setting `ai_writer_enabled` was switched to `false` manually to stop spend. Automatic AI qualifier/writer claims must not include `failed`; failed posts require an explicit manual reset to `pending`. AI writer/qualifier keep running after a failed batch, show the error on `/workers`, and send a Telegram alert via daily-report/TG-poster bot recipients.
- Sonnet 4.6 may return `category_id=18` (`Не целевой контент`) with empty `text` and the explanation in `notes`. This is valid AI rejection, not a writer error; `validate_rewrites()` allows short/empty text only for non-target category.
- AI qualifier has exactly one automatic retry: after the first failed qualification batch, affected posts are returned to `qualification_status='pending'`; after the second failed batch for the same post, they stay `failed` and are not claimed again automatically.
## Prompt Architecture
- Prompts are split into editable human part and technical contract part.
- User should be able to edit human prompt safely without breaking JSON schema.
- Contracts should contain JSON-only/output-schema requirements.
- Writer should return category ID, not free-form tag, where possible.
- Categories are managed in worker/admin settings and have stable IDs.
## Security Notes
- Secrets should not be committed.
- `.env` should contain only secrets and deploy-specific values.
- Runtime settings that are not secrets should live in DB/admin settings.
- Dangerous FN-8 defaults were removed from runtime defaults on 2026-08-03:
no default TG publication chat, no default VK storage group, no default Ghost URL/key path.
- Docker image now copies `db/migrations`, and `docker-entrypoint.sh` runs
`python /app/scripts/apply_migrations.py` before starting the app.
- Admin has login protection and server has firewall/fail2ban hardening from previous iterations.
## Working Rules For Future Codex Runs
- Before any deploy/server operation, read `D:\DEVELOPMENT\.infra.md` first.
- Before pushing or deploying, run `git pull --rebase gitea main`.
- For this project, push deployable changes to `gitea/main`; Coolify handles deployment.
- Prefer small targeted file reads with `rg` and narrow ranges.
- Avoid dumping long post texts from DB unless explicitly needed.
- For runtime checks, use the current Coolify/Gitea/LXC 105 path from `D:\DEVELOPMENT\.infra.md`.
- Do not `git reset --hard` or revert user/server hotfixes.
+194
View File
@@ -0,0 +1,194 @@
CREATE TABLE IF NOT EXISTS admin_users (
id BIGSERIAL PRIMARY KEY,
login TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'admin' CHECK (role IN ('admin','editor','viewer')),
is_active BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS admin_sessions (
token_hash TEXT PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES admin_users(id) ON DELETE CASCADE,
csrf_token TEXT NOT NULL,
expires_at TIMESTAMPTZ NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_admin_sessions_user_id ON admin_sessions(user_id);
CREATE INDEX IF NOT EXISTS idx_admin_sessions_expires_at ON admin_sessions(expires_at);
CREATE TABLE IF NOT EXISTS sources (
id BIGSERIAL PRIMARY KEY,
platform TEXT NOT NULL DEFAULT 'vk',
name TEXT NOT NULL,
url TEXT NOT NULL,
external_id TEXT,
external_owner_id BIGINT,
active BOOLEAN NOT NULL DEFAULT TRUE,
status TEXT NOT NULL DEFAULT 'new',
status_msg TEXT,
parse_from TIMESTAMPTZ,
last_checked_at TIMESTAMPTZ,
last_parsed_at TIMESTAMPTZ,
priority INTEGER NOT NULL DEFAULT 100,
tags TEXT[] NOT NULL DEFAULT ARRAY[]::TEXT[],
settings_json JSONB NOT NULL DEFAULT '{}'::jsonb,
created_by BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
archived_at TIMESTAMPTZ
);
CREATE UNIQUE INDEX IF NOT EXISTS ux_sources_platform_url_active
ON sources(platform, lower(url))
WHERE archived_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_sources_active_platform ON sources(active, platform, priority, id)
WHERE archived_at IS NULL;
CREATE TABLE IF NOT EXISTS raw_posts (
id BIGSERIAL PRIMARY KEY,
source_id BIGINT NOT NULL REFERENCES sources(id) ON DELETE CASCADE,
platform TEXT NOT NULL,
external_post_id TEXT NOT NULL,
external_owner_id BIGINT,
original_url TEXT NOT NULL,
storage_owner_id BIGINT,
storage_post_id BIGINT,
storage_post_url TEXT,
raw_text TEXT NOT NULL DEFAULT '',
raw_json JSONB NOT NULL DEFAULT '{}'::jsonb,
text_hash TEXT NOT NULL,
content_hash TEXT NOT NULL,
posted_at TIMESTAMPTZ,
copied_at TIMESTAMPTZ,
status TEXT NOT NULL DEFAULT 'raw_saved',
skip_reason TEXT,
error_reason TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(source_id, external_post_id)
);
CREATE INDEX IF NOT EXISTS idx_raw_posts_status_created ON raw_posts(status, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_raw_posts_source_posted ON raw_posts(source_id, posted_at DESC);
CREATE INDEX IF NOT EXISTS idx_raw_posts_content_hash ON raw_posts(content_hash);
CREATE TABLE IF NOT EXISTS raw_post_media (
id BIGSERIAL PRIMARY KEY,
raw_post_id BIGINT NOT NULL REFERENCES raw_posts(id) ON DELETE CASCADE,
platform TEXT NOT NULL,
media_type TEXT NOT NULL CHECK (media_type IN ('photo','video','link','doc','unknown')),
original_url TEXT,
original_attachment_id TEXT,
storage_attachment_id TEXT,
storage_url TEXT,
preview_url TEXT,
width INTEGER,
height INTEGER,
duration_sec INTEGER,
sort_order INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'pending',
attempts INTEGER NOT NULL DEFAULT 0,
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_raw_post_media_post ON raw_post_media(raw_post_id, sort_order, id);
CREATE INDEX IF NOT EXISTS idx_raw_post_media_status ON raw_post_media(status);
CREATE TABLE IF NOT EXISTS jobs (
id BIGSERIAL PRIMARY KEY,
type TEXT NOT NULL,
entity_type TEXT NOT NULL,
entity_id BIGINT NOT NULL,
payload_json JSONB NOT NULL DEFAULT '{}'::jsonb,
status TEXT NOT NULL DEFAULT 'pending',
attempts INTEGER NOT NULL DEFAULT 0,
max_attempts INTEGER NOT NULL DEFAULT 5,
next_run_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
locked_by TEXT,
locked_at TIMESTAMPTZ,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE UNIQUE INDEX IF NOT EXISTS ux_jobs_open_entity_type
ON jobs(type, entity_type, entity_id)
WHERE status IN ('pending','retry','in_progress');
CREATE INDEX IF NOT EXISTS idx_jobs_claim ON jobs(type, status, next_run_at, id);
CREATE TABLE IF NOT EXISTS worker_controls (
name TEXT PRIMARY KEY,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
settings_json JSONB NOT NULL DEFAULT '{}'::jsonb,
updated_by BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS worker_heartbeats (
name TEXT PRIMARY KEY,
heartbeat_at TIMESTAMPTZ NOT NULL,
status TEXT NOT NULL DEFAULT 'running',
current_job_id BIGINT,
meta_json JSONB NOT NULL DEFAULT '{}'::jsonb,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS app_settings (
key TEXT PRIMARY KEY,
value_json JSONB NOT NULL,
value_type TEXT NOT NULL DEFAULT 'str',
title TEXT NOT NULL DEFAULT '',
description TEXT NOT NULL DEFAULT '',
category TEXT NOT NULL DEFAULT 'General',
updated_by BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS audit_log (
id BIGSERIAL PRIMARY KEY,
actor_id BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
action TEXT NOT NULL,
entity_type TEXT NOT NULL,
entity_id BIGINT,
before_json JSONB,
after_json JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES
('vk-parser', TRUE, '{}'::jsonb),
('vk-storage-uploader', TRUE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('parser_interval_sec', '300'::jsonb, 'int', 'Parser interval, sec', 'Pause between parser cycles.', 'Parser'),
('parser_new_source_lookback_days', '14'::jsonb, 'int', 'New source lookback, days', 'How far back to read a new source.', 'Parser'),
('parser_reparse_overlap_minutes', '120'::jsonb, 'int', 'Reparse overlap, min', 'Overlap window for known sources.', 'Parser'),
('parser_min_text_length', '50'::jsonb, 'int', 'Minimum text length', 'Posts with shorter text are treated as junk.', 'Parser'),
('parser_skip_reposts', 'true'::jsonb, 'bool', 'Skip reposts', 'Do not save reposts from VK copy_history.', 'Parser'),
('parser_skip_empty_text', 'true'::jsonb, 'bool', 'Skip empty text posts', 'Treat posts without text as junk.', 'Parser'),
('parser_skip_no_media', 'true'::jsonb, 'bool', 'Skip posts without media', 'Treat posts without media as junk.', 'Parser'),
('parser_skip_text_too_short', 'true'::jsonb, 'bool', 'Skip short text posts', 'Treat posts shorter than parser_min_text_length as junk.', 'Parser'),
('parser_store_skipped_posts', 'false'::jsonb, 'bool', 'Store skipped posts', 'Keep junk posts in raw_posts instead of dropping them.', 'Parser'),
('parser_dedupe_content_hash', 'true'::jsonb, 'bool', 'Dedupe by content hash', 'Skip posts whose text and media hash already exists in raw_posts.', 'Parser'),
('parser_source_pause_sec', '0'::jsonb, 'float', 'Source pause, sec', 'Pause between VK sources inside one parser cycle.', 'Parser'),
('vk_requests_per_second', '3'::jsonb, 'int', 'VK requests per second', 'Global VK API rate limit per worker.', 'VK'),
('vk_wall_page_size', '50'::jsonb, 'int', 'VK wall page size', 'wall.get count per page.', 'VK'),
('vk_api_timeout_total_sec', '15'::jsonb, 'int', 'VK total timeout, sec', 'Total timeout for one VK API request.', 'VK'),
('vk_api_timeout_connect_sec', '5'::jsonb, 'int', 'VK connect timeout, sec', 'Connection timeout for VK API.', 'VK'),
('vk_rate_limit_sleep_sec', '1'::jsonb, 'float', 'VK rate limit sleep, sec', 'Sleep after VK API rate limit error code 6.', 'VK'),
('vk_api_retry_attempts', '3'::jsonb, 'int', 'VK retry attempts', 'Attempts for temporary VK API/network errors.', 'VK'),
('vk_api_retry_min_delay_sec', '2'::jsonb, 'float', 'VK retry min delay, sec', 'Initial retry delay for temporary VK errors.', 'VK'),
('vk_api_retry_max_delay_sec', '10'::jsonb, 'float', 'VK retry max delay, sec', 'Maximum retry delay for temporary VK errors.', 'VK'),
('uploader_interval_sec', '5'::jsonb, 'int', 'Uploader idle interval, sec', 'Pause when no jobs are available.', 'Uploader'),
('uploader_download_timeout_sec', '45'::jsonb, 'int', 'Download timeout, sec', 'Timeout for source media download.', 'Uploader')
ON CONFLICT (key) DO NOTHING;
+27
View File
@@ -0,0 +1,27 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS tg_storage_chat_id BIGINT,
ADD COLUMN IF NOT EXISTS tg_storage_thread_id BIGINT,
ADD COLUMN IF NOT EXISTS tg_storage_message_ids BIGINT[] NOT NULL DEFAULT ARRAY[]::BIGINT[],
ADD COLUMN IF NOT EXISTS tg_storage_meta_message_id BIGINT;
ALTER TABLE raw_post_media
ADD COLUMN IF NOT EXISTS tg_file_id TEXT,
ADD COLUMN IF NOT EXISTS tg_file_unique_id TEXT;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('tg_media_channel_id', '""'::jsonb, 'str', 'Telegram media storage chat', 'Chat id or chat_id:topic_id for raw media storage.', 'Telegram'),
('local_bot_api_url', '""'::jsonb, 'str', 'Local Telegram Bot API URL', 'Optional local Bot API server URL.', 'Telegram'),
('telegram_media_group_max_items', '10'::jsonb, 'int', 'Telegram media group max items', 'Maximum media items per Telegram media group.', 'Telegram'),
('telegram_media_upload_delay_sec', '1'::jsonb, 'float', 'Telegram media upload delay, sec', 'Pause between Telegram upload operations.', 'Telegram'),
('telegram_retry_max_attempts', '4'::jsonb, 'int', 'Telegram retry attempts', 'Retry attempts for Telegram calls.', 'Telegram'),
('telegram_retry_backoff_max_sec', '15'::jsonb, 'int', 'Telegram retry max backoff, sec', 'Maximum retry backoff for Telegram calls.', 'Telegram'),
('telegram_caption_limit', '1024'::jsonb, 'int', 'Telegram caption limit', 'Caption character limit for media messages.', 'Telegram'),
('telegram_message_limit', '4096'::jsonb, 'int', 'Telegram message limit', 'Text message character limit.', 'Telegram'),
('telegram_text_overflow_marker', '"Продолжение следующим сообщением."'::jsonb, 'str', 'Telegram overflow marker', 'Caption marker when text does not fit.', 'Telegram'),
('media_post_job_pause_sec', '0.2'::jsonb, 'float', 'Post job pause, sec', 'Pause after each processed media storage job.', 'Uploader'),
('video_max_size_mb', '49'::jsonb, 'int', 'Video max size, MB', 'Maximum video size to upload to Telegram.', 'Uploader'),
('video_max_duration_sec', '300'::jsonb, 'int', 'Video max duration, sec', 'Maximum video duration to upload to Telegram.', 'Uploader'),
('uploader_yt_dlp_timeout_sec', '300'::jsonb, 'int', 'yt-dlp timeout, sec', 'Timeout for VK video downloads.', 'Uploader'),
('uploader_max_media_attempts', '3'::jsonb, 'int', 'Media max attempts', 'Maximum attempts per media item.', 'Uploader')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,16 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('parser_skip_reposts', 'true'::jsonb, 'bool', 'Skip reposts', 'Do not save reposts from VK copy_history.', 'Parser'),
('parser_skip_empty_text', 'true'::jsonb, 'bool', 'Skip empty text posts', 'Treat posts without text as junk.', 'Parser'),
('parser_skip_no_media', 'true'::jsonb, 'bool', 'Skip posts without media', 'Treat posts without media as junk.', 'Parser'),
('parser_skip_text_too_short', 'true'::jsonb, 'bool', 'Skip short text posts', 'Treat posts shorter than parser_min_text_length as junk.', 'Parser'),
('parser_store_skipped_posts', 'false'::jsonb, 'bool', 'Store skipped posts', 'Keep junk posts in raw_posts instead of dropping them.', 'Parser'),
('parser_dedupe_content_hash', 'true'::jsonb, 'bool', 'Dedupe by content hash', 'Skip posts whose text and media hash already exists in raw_posts.', 'Parser'),
('parser_source_pause_sec', '0'::jsonb, 'float', 'Source pause, sec', 'Pause between VK sources inside one parser cycle.', 'Parser'),
('vk_api_timeout_total_sec', '15'::jsonb, 'int', 'VK total timeout, sec', 'Total timeout for one VK API request.', 'VK'),
('vk_api_timeout_connect_sec', '5'::jsonb, 'int', 'VK connect timeout, sec', 'Connection timeout for VK API.', 'VK'),
('vk_rate_limit_sleep_sec', '1'::jsonb, 'float', 'VK rate limit sleep, sec', 'Sleep after VK API rate limit error code 6.', 'VK'),
('vk_api_retry_attempts', '3'::jsonb, 'int', 'VK retry attempts', 'Attempts for temporary VK API/network errors.', 'VK'),
('vk_api_retry_min_delay_sec', '2'::jsonb, 'float', 'VK retry min delay, sec', 'Initial retry delay for temporary VK errors.', 'VK'),
('vk_api_retry_max_delay_sec', '10'::jsonb, 'float', 'VK retry max delay, sec', 'Maximum retry delay for temporary VK errors.', 'VK')
ON CONFLICT (key) DO NOTHING;
+28
View File
@@ -0,0 +1,28 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('parser_skip_empty_text', 'true'::jsonb, 'bool', 'Skip empty text posts', 'Treat posts without text as junk.', 'Parser')
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET value_json='50'::jsonb,
description='Posts with shorter text are treated as junk.',
updated_at=NOW()
WHERE key='parser_min_text_length'
AND value_json='0'::jsonb;
UPDATE app_settings
SET value_json='false'::jsonb,
description='Keep junk posts in raw_posts instead of dropping them.',
updated_at=NOW()
WHERE key='parser_store_skipped_posts'
AND value_json='true'::jsonb;
UPDATE app_settings
SET description='Treat posts without media as junk.',
updated_at=NOW()
WHERE key='parser_skip_no_media';
UPDATE app_settings
SET description='Treat posts shorter than parser_min_text_length as junk.',
updated_at=NOW()
WHERE key='parser_skip_text_too_short';
+50
View File
@@ -0,0 +1,50 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS qualification_status TEXT,
ADD COLUMN IF NOT EXISTS qualification_score INTEGER,
ADD COLUMN IF NOT EXISTS qualification_decision TEXT,
ADD COLUMN IF NOT EXISTS qualification_reason TEXT,
ADD COLUMN IF NOT EXISTS qualification_model TEXT,
ADD COLUMN IF NOT EXISTS qualification_prompt_hash TEXT,
ADD COLUMN IF NOT EXISTS qualification_batch_id BIGINT,
ADD COLUMN IF NOT EXISTS qualified_at TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_raw_posts_qualification_claim
ON raw_posts(status, qualification_status, created_at)
WHERE status='storage_ready';
CREATE TABLE IF NOT EXISTS ai_qualification_batches (
id BIGSERIAL PRIMARY KEY,
status TEXT NOT NULL DEFAULT 'processing',
provider TEXT NOT NULL,
model TEXT NOT NULL,
prompt_hash TEXT NOT NULL,
posts_count INTEGER NOT NULL DEFAULT 0,
accepted_count INTEGER NOT NULL DEFAULT 0,
rejected_count INTEGER NOT NULL DEFAULT 0,
maybe_count INTEGER NOT NULL DEFAULT 0,
request_json JSONB NOT NULL DEFAULT '{}'::jsonb,
response_json JSONB,
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
completed_at TIMESTAMPTZ
);
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('ai-qualifier', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('ai_qualifier_enabled', 'false'::jsonb, 'bool', 'AI qualifier enabled', 'Enable qualification requests to AI API.', 'AI Qualifier'),
('ai_qualifier_provider', '"openrouter"'::jsonb, 'str', 'Provider', 'LiteLLM provider: openrouter, openai, anthropic, gemini, openai_compatible.', 'AI Qualifier'),
('ai_qualifier_model', '"openrouter/openai/gpt-4.1-mini"'::jsonb, 'str', 'Model', 'Model name in LiteLLM format.', 'AI Qualifier'),
('ai_qualifier_api_key', '""'::jsonb, 'secret', 'API key', 'AI provider API key.', 'AI Qualifier'),
('ai_qualifier_api_base', '""'::jsonb, 'str', 'API base', 'Optional base URL for OpenAI-compatible providers.', 'AI Qualifier'),
('ai_qualifier_batch_size', '30'::jsonb, 'int', 'Batch size', 'How many raw posts to classify in one request.', 'AI Qualifier'),
('ai_qualifier_min_score', '7'::jsonb, 'int', 'Minimum accepted score', 'Posts with score >= this value are accepted.', 'AI Qualifier'),
('ai_qualifier_interval_sec', '60'::jsonb, 'int', 'Idle interval, sec', 'Pause between qualifier checks.', 'AI Qualifier'),
('ai_qualifier_max_text_chars', '2000'::jsonb, 'int', 'Max text chars per post', 'Trim raw text before sending it to AI.', 'AI Qualifier'),
('ai_qualifier_temperature', '0'::jsonb, 'float', 'Temperature', 'AI sampling temperature for stable classification.', 'AI Qualifier'),
('ai_qualifier_timeout_sec', '120'::jsonb, 'int', 'Timeout, sec', 'AI request timeout.', 'AI Qualifier'),
('ai_qualifier_prompt', '"Ты квалификатор постов для проекта, который собирает посты производителей и магазинов милитари, тактической и airsoft-снаряги.\\n\\nОцени каждый raw-пост по пригодности для дальнейшего рерайта и публикации.\\n\\nПодходит: товарные посты, новинки, обзоры, производство, экипировка, тактическая одежда, броня, подсумки, рюкзаки, разгрузки, ремни, патчи, аксессуары, airsoft/страйкбольная снаряга, полезные анонсы производителя.\\n\\nМусор: мемы, поздравления, вакансии, политика, бытовуха, пустые посты, посты без конкретного товара/снаряжения, розыгрыши без предметного контента, чистая реклама скидки без понятного товара, нерелевантные новости.\\n\\nВерни только JSON-объект по схеме: {\\\"results\\\":[{\\\"id\\\":123,\\\"score\\\":8,\\\"decision\\\":\\\"accepted\\\",\\\"reason\\\":\\\"короткая причина\\\"}]}. score — целое число от 1 до 10. decision: accepted, rejected или maybe. Не добавляй markdown и пояснения вне JSON."'::jsonb, 'text', 'Prompt', 'Main qualification prompt.', 'AI Qualifier')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,47 @@
UPDATE app_settings SET title='Интервал парсера, сек', description='Пауза между полными циклами парсинга источников.' WHERE key='parser_interval_sec';
UPDATE app_settings SET title='Глубина нового источника, дней', description='Сколько дней назад читать посты при первом запуске источника.' WHERE key='parser_new_source_lookback_days';
UPDATE app_settings SET title='Перекрытие повторного чтения, мин', description='Насколько назад перечитывать источник, чтобы не пропустить посты.' WHERE key='parser_reparse_overlap_minutes';
UPDATE app_settings SET title='Минимальная длина текста', description='Посты с текстом короче этого значения считаются мусором.' WHERE key='parser_min_text_length';
UPDATE app_settings SET title='Пропускать репосты', description='Не брать VK-посты с copy_history.' WHERE key='parser_skip_reposts';
UPDATE app_settings SET title='Пропускать пустой текст', description='Не брать посты без текста.' WHERE key='parser_skip_empty_text';
UPDATE app_settings SET title='Пропускать без медиа', description='Не брать посты без фото/видео.' WHERE key='parser_skip_no_media';
UPDATE app_settings SET title='Пропускать короткий текст', description='Не брать посты короче минимальной длины текста.' WHERE key='parser_skip_text_too_short';
UPDATE app_settings SET title='Сохранять мусорные посты', description='Если включено, мусорные посты попадут в raw_posts со статусом skipped. Обычно выключено.' WHERE key='parser_store_skipped_posts';
UPDATE app_settings SET title='Дедупликация по контенту', description='Не брать пост, если такой текст и набор медиа уже есть в raw_posts.' WHERE key='parser_dedupe_content_hash';
UPDATE app_settings SET title='Пауза между источниками, сек', description='Задержка между чтением разных VK-источников внутри одного цикла.' WHERE key='parser_source_pause_sec';
UPDATE app_settings SET title='VK запросов в секунду', description='Ограничение частоты запросов к VK API.' WHERE key='vk_requests_per_second';
UPDATE app_settings SET title='Размер страницы VK wall.get', description='Сколько постов запрашивать за один вызов wall.get.' WHERE key='vk_wall_page_size';
UPDATE app_settings SET title='VK общий таймаут, сек', description='Общий таймаут одного запроса к VK API.' WHERE key='vk_api_timeout_total_sec';
UPDATE app_settings SET title='VK таймаут соединения, сек', description='Таймаут установки соединения с VK API.' WHERE key='vk_api_timeout_connect_sec';
UPDATE app_settings SET title='Пауза при VK rate limit, сек', description='Сколько ждать после ошибки VK API error_code=6.' WHERE key='vk_rate_limit_sleep_sec';
UPDATE app_settings SET title='VK retry попытки', description='Сколько раз повторять временно упавший VK-запрос.' WHERE key='vk_api_retry_attempts';
UPDATE app_settings SET title='VK retry минимум, сек', description='Начальная задержка между повторными VK-запросами.' WHERE key='vk_api_retry_min_delay_sec';
UPDATE app_settings SET title='VK retry максимум, сек', description='Максимальная задержка между повторными VK-запросами.' WHERE key='vk_api_retry_max_delay_sec';
UPDATE app_settings SET title='Интервал аплоадера, сек', description='Пауза, когда нет задач на загрузку raw-постов в Telegram.' WHERE key='uploader_interval_sec';
UPDATE app_settings SET title='Таймаут скачивания, сек', description='Таймаут скачивания исходного медиа.' WHERE key='uploader_download_timeout_sec';
UPDATE app_settings SET title='Telegram канал raw', description='ID Telegram-канала/топика для хранения raw-постов.' WHERE key='tg_media_channel_id';
UPDATE app_settings SET title='Локальный Bot API URL', description='Опциональный URL локального Telegram Bot API сервера.' WHERE key='local_bot_api_url';
UPDATE app_settings SET title='Задержка на медиа, сек', description='Пауза после отправки медиа, умножается на количество файлов.' WHERE key='media_upload_delay_sec';
UPDATE app_settings SET title='Пауза после поста, сек', description='Короткая пауза после полной обработки одного raw-поста.' WHERE key='media_post_job_pause_sec';
UPDATE app_settings SET title='Максимум медиа в группе', description='Максимум файлов в одном Telegram media group.' WHERE key='media_group_max_items';
UPDATE app_settings SET title='Максимум попыток медиа', description='Сколько раз пытаться скачать/загрузить отдельное медиа.' WHERE key='max_media_attempts';
UPDATE app_settings SET title='Telegram retry попытки', description='Сколько раз повторять временно упавший Telegram-запрос.' WHERE key='tg_retry_attempts';
UPDATE app_settings SET title='Telegram backoff максимум, сек', description='Максимальная задержка между retry Telegram-запросов.' WHERE key='tg_retry_backoff_max_sec';
UPDATE app_settings SET title='Максимум видео, МБ', description='Максимальный размер видео для загрузки в Telegram.' WHERE key='video_max_size_mb';
UPDATE app_settings SET title='Максимум видео, сек', description='Видео длиннее этого значения помечается как link_only.' WHERE key='video_max_duration_sec';
UPDATE app_settings SET title='yt-dlp таймаут, сек', description='Сколько ждать скачивания VK-видео через yt-dlp.' WHERE key='uploader_yt_dlp_timeout_sec';
UPDATE app_settings SET title='Включить AI-квалификатор', description='Разрешить воркеру отправлять raw-посты в AI API.' WHERE key='ai_qualifier_enabled';
UPDATE app_settings SET title='Провайдер LLM', description='Провайдер для LiteLLM. Для OpenRouter список моделей подгружается live.' WHERE key='ai_qualifier_provider';
UPDATE app_settings SET title='Модель LLM', description='Модель для квалификации. Можно выбрать из списка или вписать вручную.' WHERE key='ai_qualifier_model';
UPDATE app_settings SET title='API токен', description='Токен выбранного LLM-провайдера. Сохраняется в настройках.' WHERE key='ai_qualifier_api_key';
UPDATE app_settings SET title='API base URL', description='Опциональный base URL для OpenAI-compatible провайдера.' WHERE key='ai_qualifier_api_base';
UPDATE app_settings SET title='Размер пачки', description='Сколько raw-постов отправлять в один AI-запрос.' WHERE key='ai_qualifier_batch_size';
UPDATE app_settings SET title='Минимальная оценка', description='Пост принимается, если score больше или равен этому значению.' WHERE key='ai_qualifier_min_score';
UPDATE app_settings SET title='Интервал проверки, сек', description='Пауза между проверками новых raw-постов.' WHERE key='ai_qualifier_interval_sec';
UPDATE app_settings SET title='Максимум символов текста', description='Сколько символов raw_text отправлять в AI на один пост.' WHERE key='ai_qualifier_max_text_chars';
UPDATE app_settings SET title='Температура', description='Температура генерации. Для классификации обычно 0.' WHERE key='ai_qualifier_temperature';
UPDATE app_settings SET title='AI таймаут, сек', description='Таймаут одного AI-запроса.' WHERE key='ai_qualifier_timeout_sec';
UPDATE app_settings SET title='Промпт квалификатора', description='Основная инструкция для оценки пригодности raw-постов.' WHERE key='ai_qualifier_prompt';
+57
View File
@@ -0,0 +1,57 @@
ALTER TABLE ai_qualification_batches
ADD COLUMN IF NOT EXISTS prompt_tokens INTEGER,
ADD COLUMN IF NOT EXISTS completion_tokens INTEGER,
ADD COLUMN IF NOT EXISTS total_tokens INTEGER,
ADD COLUMN IF NOT EXISTS estimated_cost_usd NUMERIC(12, 6);
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS rewrite_status TEXT,
ADD COLUMN IF NOT EXISTS rewritten_text TEXT,
ADD COLUMN IF NOT EXISTS rewrite_notes TEXT,
ADD COLUMN IF NOT EXISTS rewrite_model TEXT,
ADD COLUMN IF NOT EXISTS rewrite_prompt_hash TEXT,
ADD COLUMN IF NOT EXISTS rewrite_batch_id BIGINT,
ADD COLUMN IF NOT EXISTS rewritten_at TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_raw_posts_writer_claim
ON raw_posts(qualification_status, rewrite_status, qualified_at, id)
WHERE qualification_status='accepted';
CREATE TABLE IF NOT EXISTS ai_writer_batches (
id BIGSERIAL PRIMARY KEY,
status TEXT NOT NULL DEFAULT 'processing',
provider TEXT NOT NULL,
model TEXT NOT NULL,
prompt_hash TEXT NOT NULL,
posts_count INTEGER NOT NULL DEFAULT 0,
ready_count INTEGER NOT NULL DEFAULT 0,
failed_count INTEGER NOT NULL DEFAULT 0,
request_json JSONB NOT NULL DEFAULT '{}'::jsonb,
response_json JSONB,
error TEXT,
prompt_tokens INTEGER,
completion_tokens INTEGER,
total_tokens INTEGER,
estimated_cost_usd NUMERIC(12, 6),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
completed_at TIMESTAMPTZ
);
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('ai-writer', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('ai_writer_enabled', 'false'::jsonb, 'bool', 'AI-райтер включён', 'Разрешает отправлять прошедшие квалификацию посты на рерайт.', 'AI Writer'),
('ai_writer_provider', '"anthropic"'::jsonb, 'str', 'Провайдер LLM', 'Через какого провайдера LiteLLM отправлять запросы райтера.', 'AI Writer'),
('ai_writer_model', '"claude-haiku-4-5-20251001"'::jsonb, 'str', 'Модель', 'Модель для рерайта. Для отладки используем дешёвую Haiku.', 'AI Writer'),
('ai_writer_api_key', '""'::jsonb, 'secret', 'API-ключ', 'Ключ провайдера для AI-райтера. Можно использовать тот же Anthropic-ключ, что и у квалификатора.', 'AI Writer'),
('ai_writer_api_base', '""'::jsonb, 'str', 'API base URL', 'Необязательный base URL для OpenAI-compatible провайдеров.', 'AI Writer'),
('ai_writer_prompt', '"Ты редактор Telegram/VK-канала про милитари, тактическую и airsoft-снарягу. Перепиши исходный пост производителя в готовый к публикации пост на русском языке.\\n\\nСтиль: живой, понятный, без канцелярита, без чрезмерной рекламы, без выдуманных фактов. Сохраняй конкретику: модель, материал, цвета, размеры, назначение, цену и условия, если они есть в исходнике. Не добавляй то, чего нет в исходном посте.\\n\\nФормат результата: 1 короткий заголовок первой строкой, затем 2-5 абзацев или аккуратный список, если исходник товарный. В конце не добавляй ссылки и хэштеги.\\n\\nВерни только JSON-объект по схеме: {\\\"rewrites\\\":[{\\\"id\\\":123,\\\"text\\\":\\\"готовый текст\\\",\\\"notes\\\":\\\"короткая заметка для редактора\\\"}]}. Без markdown вне JSON."'::jsonb, 'text', 'Промпт райтера', 'Главный промпт рерайта. Поддерживает обычные переносы строк.', 'AI Writer'),
('ai_writer_batch_size', '1'::jsonb, 'int', 'Размер batch', 'Сколько постов переписывать за один запрос. Для отладки лучше 1.', 'AI Writer'),
('ai_writer_max_text_chars', '3500'::jsonb, 'int', 'Максимум символов исходника', 'Обрезает исходный raw-текст перед отправкой в LLM.', 'AI Writer'),
('ai_writer_temperature', '0.4'::jsonb, 'float', 'Температура генерации', 'Ниже — стабильнее и суше, выше — свободнее и разнообразнее.', 'AI Writer'),
('ai_writer_timeout_sec', '180'::jsonb, 'int', 'Таймаут, сек', 'Сколько ждать ответ модели.', 'AI Writer'),
('ai_writer_interval_sec', '60'::jsonb, 'int', 'Интервал ожидания, сек', 'Пауза между проверками очереди writer, когда работы нет.', 'AI Writer')
ON CONFLICT (key) DO NOTHING;
+5
View File
@@ -0,0 +1,5 @@
ALTER TABLE ai_qualification_batches
ADD COLUMN IF NOT EXISTS prompt_text TEXT;
ALTER TABLE ai_writer_batches
ADD COLUMN IF NOT EXISTS prompt_text TEXT;
@@ -0,0 +1,6 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS qualification_reject_tag TEXT;
CREATE INDEX IF NOT EXISTS idx_raw_posts_qualification_reject_tag
ON raw_posts(qualification_reject_tag)
WHERE qualification_reject_tag IS NOT NULL;
@@ -0,0 +1,53 @@
ALTER TABLE sources
ADD COLUMN IF NOT EXISTS tag TEXT;
UPDATE sources
SET tag = lower(regexp_replace(COALESCE(NULLIF(external_id, ''), name), '[^[:alnum:]_]+', '_', 'g'))
WHERE tag IS NULL OR btrim(tag) = '';
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS rewrite_category TEXT,
ADD COLUMN IF NOT EXISTS rewrite_category_tag TEXT,
ADD COLUMN IF NOT EXISTS rewrite_source_tag TEXT;
CREATE INDEX IF NOT EXISTS idx_sources_tag ON sources(lower(tag))
WHERE archived_at IS NULL;
CREATE INDEX IF NOT EXISTS idx_raw_posts_rewrite_category ON raw_posts(rewrite_category)
WHERE rewrite_category IS NOT NULL;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('ai_writer_categories', '["защита","одежда","разгрузка","рюкзаки","airsoft","патчи","электроника","аксессуары","производство"]'::jsonb, 'text', 'Категории райтера', 'Список категорий, из которых AI-райтер выбирает одну. Можно писать через запятую или с новой строки.', 'AI Writer')
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET value_json = to_jsonb($prompt$You are an editor of a Russian Telegram channel about military, tactical and airsoft gear.
Write on behalf of the channel — as an expert observer, not the manufacturer.
Source posts come from manufacturers/shops; your job is to reframe them as channel news.
VOICE: concise, expert, matter-of-fact. Like telling a knowledgeable friend what dropped.
Example opener framing: "У [Producer] вышел...", "Wartech показали...", "[Producer] добавили в линейку..."
RULES:
- Use only facts from source text. Never invent specs, properties or claims.
- If source is thin — write a short post, do not pad it.
- No hype, no exclamation spam, no manufacturer PR tone.
- Post length: 3-6 lines. Short is better than padded.
- No bullet lists unless source has 4+ distinct specs worth listing.
- Do not add hashtags. The system will add category and producer hashtags separately.
CATEGORIES: pick exactly one from the categories field in input.
INPUT fields used: id, producer_name, producer_tag, text.
qualification_score and media_count are metadata; ignore them for rewrite unless they help understand context.
OUTPUT: JSON only, no markdown outside JSON.$prompt$::text),
description = 'Главный промпт рерайта. Модель выбирает category, но не пишет хэштеги.',
updated_at = NOW()
WHERE key = 'ai_writer_prompt'
AND (
value_json::text ILIKE '%хэштег%'
OR value_json::text ILIKE '%hashtag%'
OR value_json::text ILIKE '%#CATEGORY%'
);
+39
View File
@@ -0,0 +1,39 @@
UPDATE app_settings
SET title='Инструкция квалификатора',
description='Редактируемая человеческая инструкция: критерии оценки и мусора. JSON-контракт ответа добавляется кодом автоматически.'
WHERE key='ai_qualifier_prompt';
UPDATE app_settings
SET title='Инструкция райтера',
description='Редактируемая человеческая инструкция: стиль, факты, длина и голос канала. Категорию и JSON-контракт ответа контролирует код.'
WHERE key='ai_writer_prompt';
UPDATE app_settings
SET description='Модель для квалификации. Список берётся live из API провайдера, если доступен ключ; fallback помечается отдельно.'
WHERE key='ai_qualifier_model';
UPDATE app_settings
SET description='Модель для рерайта. Список берётся live из API провайдера, если доступен ключ; fallback помечается отдельно.'
WHERE key='ai_writer_model';
UPDATE app_settings
SET description='Токен выбранного LLM-провайдера. В админке показывается полностью, поэтому доступ к настройкам должен быть только у доверенных пользователей.'
WHERE key IN ('ai_qualifier_api_key', 'ai_writer_api_key');
UPDATE app_settings m
SET value_json = to_jsonb('anthropic/' || trim(both '"' from m.value_json::text))
FROM app_settings p
WHERE p.key = replace(m.key, '_model', '_provider')
AND p.value_json = '"anthropic"'::jsonb
AND m.key IN ('ai_qualifier_model', 'ai_writer_model')
AND m.value_json::text NOT LIKE '"anthropic/%'
AND m.value_json::text NOT LIKE '"%/%"';
UPDATE app_settings m
SET value_json = to_jsonb('gemini/' || trim(both '"' from m.value_json::text))
FROM app_settings p
WHERE p.key = replace(m.key, '_model', '_provider')
AND p.value_json = '"gemini"'::jsonb
AND m.key IN ('ai_qualifier_model', 'ai_writer_model')
AND m.value_json::text NOT LIKE '"gemini/%'
AND m.value_json::text NOT LIKE '"%/%"';
@@ -0,0 +1,15 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS qualification_model_decision TEXT;
UPDATE raw_posts rp
SET qualification_model_decision = item->>'decision'
FROM ai_qualification_batches b,
jsonb_array_elements(COALESCE(b.response_json->'results', '[]'::jsonb)) AS item
WHERE rp.qualification_batch_id = b.id
AND (item->>'id')::bigint = rp.id
AND rp.qualification_model_decision IS NULL
AND item ? 'decision';
CREATE INDEX IF NOT EXISTS idx_raw_posts_qualification_model_decision
ON raw_posts(qualification_model_decision)
WHERE qualification_model_decision IS NOT NULL;
+32
View File
@@ -0,0 +1,32 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS editorial_status TEXT,
ADD COLUMN IF NOT EXISTS final_text TEXT,
ADD COLUMN IF NOT EXISTS final_category TEXT,
ADD COLUMN IF NOT EXISTS final_category_tag TEXT,
ADD COLUMN IF NOT EXISTS final_source_tag TEXT,
ADD COLUMN IF NOT EXISTS editor_notes TEXT,
ADD COLUMN IF NOT EXISTS regeneration_prompt TEXT,
ADD COLUMN IF NOT EXISTS reviewed_by BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
ADD COLUMN IF NOT EXISTS reviewed_at TIMESTAMPTZ,
ADD COLUMN IF NOT EXISTS edited_at TIMESTAMPTZ;
UPDATE raw_posts
SET editorial_status = 'review'
WHERE editorial_status IS NULL
AND rewrite_status = 'ready';
UPDATE raw_posts
SET final_text = rewritten_text,
final_category = rewrite_category,
final_category_tag = rewrite_category_tag,
final_source_tag = rewrite_source_tag
WHERE rewrite_status = 'ready'
AND final_text IS NULL;
CREATE INDEX IF NOT EXISTS idx_raw_posts_editorial_status
ON raw_posts(editorial_status, rewritten_at DESC, id)
WHERE rewrite_status = 'ready';
CREATE INDEX IF NOT EXISTS idx_raw_posts_final_category
ON raw_posts(final_category)
WHERE final_category IS NOT NULL;
@@ -0,0 +1,43 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
(
'ai_qualifier_contract',
'"OUTPUT SCHEMA (return array matching input order):\n{\"results\":[{\"id\":123,\"score\":8,\"decision\":\"accepted\",\"reason\":\"до 10 слов на русском\",\"reject_tag\":null}]}\n\nRules:\n- Input is a JSON array of posts.\n- Return JSON only. No markdown. No text outside JSON.\n- score must be integer 1..10.\n- decision must be exactly one of: \"accepted\", \"rejected\", \"maybe\".\n- reject_tag (rejected/maybe only) must be one of: \"meme\", \"no_product\", \"politics\", \"discount_only\", \"off_topic\", \"vacancy\", \"low_content\", \"wrong_language\", \"injection\", \"weapon\", null.\n- Return one result for every input post id."'::jsonb,
'text',
'Технический контракт квалификатора',
'JSON-схема и строгие правила ответа. Обычно не меняется при правке смыслового промпта.',
'AI Qualifier'
),
(
'ai_writer_contract',
'"OUTPUT SCHEMA (return array matching input order):\n{\"rewrites\":[{\"id\":123,\"category\":\"разгрузка\",\"text\":\"готовый текст без хэштегов\",\"notes\":\"короткая заметка для редактора или null\"}]}\n\nRules:\n- Input is a JSON object with key \"posts\" containing accepted posts.\n- Each post includes producer_name and producer_tag. Use producer_name when it helps, but do not invent facts.\n- If post includes editor_regeneration_prompt, follow it as an additional editor instruction while still obeying facts and output schema.\n- Do not add hashtags to rewritten text. Pick exactly one category from the categories list in input.\n- Return JSON only. No markdown. No text outside JSON.\n- Return one rewrite for every input post id."'::jsonb,
'text',
'Технический контракт райтера',
'JSON-схема, выбор категории и запрет хэштегов в тексте. Обычно не меняется при правке стиля.',
'AI Writer'
)
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET title='Свободная инструкция квалификатора',
description='Роль, тон, критерии и примеры оценки. Технический JSON-контракт вынесен отдельно.'
WHERE key='ai_qualifier_prompt';
UPDATE app_settings
SET title='Свободная инструкция райтера',
description='Стиль, голос канала, правила фактов и примеры текста. Технический JSON-контракт вынесен отдельно.'
WHERE key='ai_writer_prompt';
CREATE TABLE IF NOT EXISTS admin_login_attempts (
id BIGSERIAL PRIMARY KEY,
ip TEXT NOT NULL,
login TEXT NOT NULL DEFAULT '',
success BOOLEAN NOT NULL DEFAULT FALSE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_admin_login_attempts_ip_time
ON admin_login_attempts(ip, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_admin_login_attempts_login_time
ON admin_login_attempts(login, created_at DESC);
+9
View File
@@ -0,0 +1,9 @@
ALTER TABLE raw_post_media
ADD COLUMN IF NOT EXISTS editor_hidden BOOLEAN NOT NULL DEFAULT FALSE,
ADD COLUMN IF NOT EXISTS editor_added BOOLEAN NOT NULL DEFAULT FALSE,
ADD COLUMN IF NOT EXISTS editor_uploaded_by BIGINT REFERENCES admin_users(id) ON DELETE SET NULL,
ADD COLUMN IF NOT EXISTS editor_uploaded_at TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_raw_post_media_editor_visible
ON raw_post_media(raw_post_id, sort_order, id)
WHERE editor_hidden = FALSE;
@@ -0,0 +1,54 @@
UPDATE raw_posts
SET rewrite_status='ready',
editorial_status='review',
regeneration_prompt=NULL,
updated_at=NOW()
WHERE COALESCE(editorial_status, '')='regenerating';
UPDATE app_settings
SET value_json = to_jsonb($prompt$
Ты редактор русского Telegram-канала про тактическую, милитари и airsoft-экипировку.
Пиши от лица канала: как экспертный наблюдатель, а не как производитель или магазин.
Задача: превратить исходный пост в короткую новостную публикацию для канала.
Стиль:
- коротко, уверенно, по делу;
- без рекламного восторга и без выдуманных характеристик;
- только факты из исходного текста;
- если фактов мало, делай короткий пост, не растягивай;
- текст должен быть структурированным, не простынёй.
Формат текста:
- 3-6 коротких строк или 2-4 компактных абзаца;
- не добавляй ссылки и хэштеги в text.
Важное правило про источник:
- producer_name — это источник поста: производитель, магазин или площадка.
- Не утверждай, что producer_name «выпустил», «сделал», «представил» товар, если из текста не ясно, что это именно производитель этого товара.
- Если источник выглядит как магазин/площадка или в тексте упомянут другой бренд, формулируй нейтрально: «у [producer_name] появился...», «[producer_name] показали/добавили...», «в продаже появился...».
- Если из текста явно видно, что источник сам производит товар, можно писать: «[producer_name] выпустили...».
Категория:
- выбери ровно одну category из списка categories во входных данных;
- если сомневаешься, выбирай ближайшую по назначению товара.
OUTPUT: JSON only.
$prompt$::text),
updated_at = NOW()
WHERE key='ai_writer_prompt';
UPDATE app_settings
SET value_json = to_jsonb($contract$
OUTPUT SCHEMA (return array matching input order):
{"rewrites":[{"id":123,"category":"разгрузка","text":"готовый текст без хэштегов","notes":"короткая заметка для редактора или null"}]}
Rules:
- Input is a JSON object with key "posts" containing accepted posts.
- Each post includes producer_name and producer_tag. Use producer_name carefully: it can be a manufacturer, shop, or publishing source.
- Do not add hashtags to rewritten text. Pick exactly one category from the categories list in input.
- Return JSON only. No markdown. No text outside JSON.
- Return one rewrite for every input post id.
$contract$::text),
updated_at = NOW()
WHERE key='ai_writer_contract';
@@ -0,0 +1,24 @@
CREATE TABLE IF NOT EXISTS content_categories (
id BIGSERIAL PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
tag TEXT NOT NULL,
is_active BOOLEAN NOT NULL DEFAULT TRUE,
sort_order INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_content_categories_active_order
ON content_categories(is_active, sort_order, name);
INSERT INTO content_categories(name, tag, sort_order)
SELECT value, regexp_replace(lower(value), '[^[:alnum:]_а-яё]+', '_', 'g'), ordinality::int
FROM app_settings s
CROSS JOIN LATERAL jsonb_array_elements_text(
CASE WHEN jsonb_typeof(s.value_json) = 'array' THEN s.value_json ELSE '[]'::jsonb END
) WITH ORDINALITY AS items(value, ordinality)
WHERE s.key = 'ai_writer_categories'
ON CONFLICT (name) DO UPDATE
SET tag = EXCLUDED.tag,
sort_order = LEAST(content_categories.sort_order, EXCLUDED.sort_order),
updated_at = NOW();
@@ -0,0 +1,25 @@
UPDATE app_settings
SET value_json = to_jsonb(
trim(
regexp_replace(
value_json #>> '{}',
E'\\n?OUTPUT:[[:space:]]*JSON only\\.[[:space:]]*',
E'\\n',
'gi'
)
)
),
updated_at = NOW()
WHERE key = 'ai_writer_prompt'
AND value_json #>> '{}' ~* 'OUTPUT:[[:space:]]*JSON only\.';
UPDATE app_settings
SET value_json = to_jsonb(
CASE
WHEN value_json #>> '{}' ~* 'Return JSON only\\.'
THEN value_json #>> '{}'
ELSE trim(value_json #>> '{}') || E'\n- Return JSON only. No markdown. No text outside JSON.'
END
),
updated_at = NOW()
WHERE key = 'ai_writer_contract';
@@ -0,0 +1,74 @@
UPDATE app_settings
SET value_json = to_jsonb($prompt$
РОЛЬ
Ты редактор профильного русского Telegram-канала про тактическую, милитари и airsoft-экипировку.
Ты эксперт в тактическом, военном и страйкбольном снаряжении. Последние 10 лет ты следишь за новостями про новинки тактического снаряжения.
Ты профессиональный копирайтер и эксперт по SEO-оптимизации текстов для сайтов и соцсетей.
Пиши от лица канала: как экспертный наблюдатель, а не как производитель или магазин.
ТВОИ ЗНАНИЯ
1. Ты профессионально разбираешься в материалах и технологиях, которые используются в производстве военного и тактического снаряжения: типы тканей, кордуры, стропы, фурнитура, интерфейсы креплений, системы быстрого сброса, мембраны, корсетные системы, баллистическая и противоосколочная защита.
2. Ты разбираешься в практическом применении тактического, военного и страйкбольного снаряжения. Можешь отличить, что подходит для страйкбола, что для гражданских стрелков, что для военных, а что универсально для всех направлений.
3. Ты профессионально разбираешься в истории тактического и военного снаряжения. По конструкции можешь отличить классические и популярные модели от новых разработок последних лет.
ЗАДАЧА
Превратить исходный пост в короткую новостную публикацию для канала.
РЕЗУЛЬТАТ
Создать грамотный, понятный, структурированный и оптимизированный для русскоязычной аудитории текст для публикации в Telegram и ВКонтакте.
Текст должен быть адаптирован для военных, гражданских стрелков и людей, которые интересуются военным, тактическим и страйкбольным снаряжением.
СТРУКТУРА ТЕКСТА
1. SEO-заголовок с обязательным названием компании/источника и названием продукта или предмета новости. Пример: "Новый подсумок XXX от компании YYY".
2. Черта-разделитель.
3. Краткое объяснение, чем интересна новость или новинка. Указывай конкретику, избегай общих слов. Пример: "Арс Арма выпустила подсумок" лучше, чем "Российский производитель снаряжения выпустил подсумок".
4. Тело новости: основные свойства, назначение, технические особенности. Лаконично, используя факты исходного текста, опиши все важные свойства или тезисы. Если блок получился слишком объёмным, сократи его без потери смысла.
5. Если есть в исходнике, кратко укажи материалы и цвета, в которых производится новинка.
6. Если есть в исходнике, укажи стоимость новинки.
7. Черта-разделитель.
8. Фраза: "Подписывайтесь на нас в МАКС, ТГ, ВК".
ФОРМАТ ТЕКСТА
- 3-6 коротких строк или 2-5 компактных абзацев;
- не добавляй ссылки и хэштеги в text.
СТИЛЬ
- коротко, уверенно, по делу;
- без рекламного восторга и без выдуманных характеристик;
- нельзя использовать абстрактные слова типа "успех", "секрет", "мотивация", "справишься" и подобные;
- только факты из исходного текста;
- нельзя придумывать факты;
- если фактов мало, делай короткий пост, не растягивай;
- текст должен быть структурированным, не простынёй;
- используй подход "Пиши, сокращай": конкретика лучше общих и абстрактных формулировок.
ВАЖНОЕ ПРАВИЛО ПРО ИСТОЧНИК
- producer_name — это источник поста: производитель, магазин или площадка.
- Не утверждай, что producer_name "выпустил", "сделал", "представил" товар, если из текста не ясно, что это именно производитель этого товара.
- Если источник выглядит как магазин/площадка или в тексте упомянут другой бренд, формулируй нейтрально: "у [producer_name] появился...", "[producer_name] показали/добавили...", "в продаже появился...".
- Если из текста явно видно, что источник сам производит товар, можно писать: "[producer_name] выпустили...".
КАТЕГОРИЯ
- выбери ровно одну category из списка categories во входных данных;
- если сомневаешься, выбирай ближайшую по назначению товара.
$prompt$::text),
updated_at = NOW()
WHERE key='ai_writer_prompt';
UPDATE app_settings
SET value_json = to_jsonb($contract$
OUTPUT SCHEMA (return array matching input order):
{"rewrites":[{"id":123,"category":"разгрузка","text":"готовый текст без хэштегов","notes":"короткая заметка для редактора или null"}]}
Rules:
- Input is a JSON object with key "posts" containing accepted posts.
- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.
- Pick exactly one category from the categories list in input.
- Do not add links or hashtags to rewritten text.
- Return JSON only. No markdown. No text outside JSON.
- Return one rewrite for every input post id.
- The "text" field must be a ready-to-publish Russian post with normal paragraph line breaks encoded as JSON string newlines.
- The "notes" field must be a short editor note in Russian or null.
$contract$::text),
updated_at = NOW()
WHERE key='ai_writer_contract';
+74
View File
@@ -0,0 +1,74 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS rewrite_category_id INTEGER,
ADD COLUMN IF NOT EXISTS final_category_id INTEGER;
WITH duplicate_categories AS (
SELECT id,
ROW_NUMBER() OVER (PARTITION BY sort_order ORDER BY is_active DESC, id) AS duplicate_rank
FROM content_categories
),
renumber AS (
SELECT id,
(SELECT COALESCE(MAX(sort_order), 0) FROM content_categories)
+ ROW_NUMBER() OVER (ORDER BY id) AS new_sort_order
FROM duplicate_categories
WHERE duplicate_rank > 1
)
UPDATE content_categories c
SET sort_order = r.new_sort_order,
updated_at = NOW()
FROM renumber r
WHERE c.id = r.id;
CREATE UNIQUE INDEX IF NOT EXISTS idx_content_categories_public_id
ON content_categories(sort_order);
UPDATE raw_posts rp
SET rewrite_category_id = c.sort_order
FROM content_categories c
WHERE rp.rewrite_category_id IS NULL
AND rp.rewrite_category IS NOT NULL
AND (
lower(c.name) = lower(rp.rewrite_category)
OR lower(c.tag) = lower(COALESCE(rp.rewrite_category_tag, ''))
);
UPDATE raw_posts rp
SET final_category_id = c.sort_order
FROM content_categories c
WHERE rp.final_category_id IS NULL
AND rp.final_category IS NOT NULL
AND (
lower(c.name) = lower(rp.final_category)
OR lower(c.tag) = lower(COALESCE(rp.final_category_tag, ''))
);
UPDATE app_settings
SET value_json = to_jsonb(
replace(
value_json #>> '{}',
E'- выбери ровно одну category из списка categories во входных данных;\n- если сомневаешься, выбирай ближайшую по назначению товара.',
E'- выбери ровно один category_id из списка categories во входных данных;\n- categories приходит списком объектов: id, name, tag;\n- если сомневаешься, выбирай ближайший id по назначению товара.'
)
),
updated_at = NOW()
WHERE key='ai_writer_prompt';
UPDATE app_settings
SET value_json = to_jsonb($contract$
OUTPUT SCHEMA (return array matching input order):
{"rewrites":[{"id":123,"category_id":2,"text":"готовый текст без хэштегов","notes":"короткая заметка для редактора или null"}]}
Rules:
- Input is a JSON object with key "posts" containing accepted posts.
- Input has key "categories" with objects: id, name, tag.
- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.
- Pick exactly one category from the categories list in input and return its numeric id as category_id.
- Do not add links or hashtags to rewritten text.
- Return JSON only. No markdown. No text outside JSON.
- Return one rewrite for every input post id.
- The "text" field must be a ready-to-publish Russian post with normal paragraph line breaks encoded as JSON string newlines.
- The "notes" field must be a short editor note in Russian or null.
$contract$::text),
updated_at = NOW()
WHERE key='ai_writer_contract';
@@ -0,0 +1,19 @@
UPDATE app_settings
SET value_json = to_jsonb($contract$
OUTPUT SCHEMA (return array matching input order):
{"rewrites":[{"id":123,"category_id":2,"text":"готовый текст без хэштегов","notes":"короткая заметка для редактора или null"}]}
Rules:
- Input is a JSON object with key "posts" containing accepted posts.
- Input has key "categories" with objects: id, name, tag.
- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.
- Pick exactly one category from the categories list in input and return its numeric id as category_id.
- Do not add links or hashtags to rewritten text.
- Return JSON only. No markdown. No text outside JSON.
- Never return an empty object. Never omit the "rewrites" key.
- Return one rewrite for every input post id.
- The "text" field must be a ready-to-publish Russian post with normal paragraph line breaks encoded as JSON string newlines.
- The "notes" field must be a short editor note in Russian or null.
$contract$::text),
updated_at = NOW()
WHERE key='ai_writer_contract';
@@ -0,0 +1,15 @@
DELETE FROM audit_log
WHERE created_at < NOW() - INTERVAL '30 days';
CREATE INDEX IF NOT EXISTS idx_audit_log_created_at
ON audit_log(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_log_action_time
ON audit_log(action, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_audit_log_actor_time
ON audit_log(actor_id, created_at DESC)
WHERE actor_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_audit_log_entity_time
ON audit_log(entity_type, entity_id, created_at DESC);
@@ -0,0 +1,15 @@
UPDATE app_settings s
SET value_json = to_jsonb(wc.enabled),
updated_at = NOW()
FROM worker_controls wc
WHERE (wc.name, s.key) IN (
('ai-qualifier', 'ai_qualifier_enabled'),
('ai-writer', 'ai_writer_enabled')
);
UPDATE raw_posts
SET rewrite_status='failed',
rewrite_notes=COALESCE(rewrite_notes, 'recovered stale writer processing state'),
updated_at=NOW()
WHERE rewrite_status='processing'
AND updated_at < NOW() - INTERVAL '30 minutes';
@@ -0,0 +1,21 @@
-- Keep the AI-writer divider deterministic.
-- Target separator length: 27 characters.
UPDATE app_settings
SET value_json = to_jsonb(
replace(
value_json #>> '{}',
'2. Черта с точным количеством символов ━━━━━━━━━━━━━━━━━━━━━━━━━━━',
'2. Черта-разделитель: отдельная строка ровно из 27 символов. Используй строго эту строку без изменений: ━━━━━━━━━━━━━━━━━━━━━━━━━━━'
)
),
updated_at = NOW()
WHERE key = 'ai_writer_prompt'
AND (value_json #>> '{}') LIKE '%Черта с точным количеством символов%';
UPDATE raw_posts
SET rewritten_text = regexp_replace(rewritten_text, E'(^━{3,}$)', '━━━━━━━━━━━━━━━━━━━━━━━━━━━', 'gm'),
final_text = regexp_replace(final_text, E'(^━{3,}$)', '━━━━━━━━━━━━━━━━━━━━━━━━━━━', 'gm'),
updated_at = NOW()
WHERE COALESCE(rewritten_text, '') LIKE '%━%'
OR COALESCE(final_text, '') LIKE '%━%';
+93
View File
@@ -0,0 +1,93 @@
ALTER TABLE raw_posts
ADD COLUMN IF NOT EXISTS publication_status TEXT NOT NULL DEFAULT 'pending',
ADD COLUMN IF NOT EXISTS publication_error TEXT,
ADD COLUMN IF NOT EXISTS published_at TIMESTAMPTZ,
ADD COLUMN IF NOT EXISTS tg_publication_chat_id BIGINT,
ADD COLUMN IF NOT EXISTS tg_publication_thread_id BIGINT,
ADD COLUMN IF NOT EXISTS tg_publication_message_ids BIGINT[],
ADD COLUMN IF NOT EXISTS tg_publication_url TEXT;
CREATE INDEX IF NOT EXISTS idx_raw_posts_publication_queue
ON raw_posts(publication_status, editorial_status, rewrite_status, published_at, reviewed_at, id);
CREATE INDEX IF NOT EXISTS idx_raw_posts_publication_rank
ON raw_posts(final_category_tag, final_source_tag, published_at DESC);
CREATE TABLE IF NOT EXISTS publication_runs (
id BIGSERIAL PRIMARY KEY,
poster TEXT NOT NULL,
schedule_id TEXT NOT NULL,
scheduled_for DATE NOT NULL,
scheduled_time TEXT NOT NULL,
planned_count INTEGER NOT NULL DEFAULT 0,
published_count INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'started',
error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(poster, schedule_id, scheduled_for)
);
CREATE INDEX IF NOT EXISTS idx_publication_runs_poster_date
ON publication_runs(poster, scheduled_for DESC, scheduled_time);
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('tg-poster', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('tg_poster_enabled', 'false'::jsonb, 'bool', 'TG poster enabled', 'Double safety flag for publishing accepted posts to Telegram.', 'TG Poster'),
('tg_poster_bot_token', '""'::jsonb, 'secret', 'TG poster bot token', 'Bot token for final Telegram publication. For test can be the same as upload bot token.', 'TG Poster'),
('tg_poster_chat_id', '""'::jsonb, 'str', 'TG publication chat id', 'Target Telegram channel/group id. Supports chat_id:thread_id for topics.', 'TG Poster'),
('tg_poster_schedule_json', '[{"id":"morning","time":"10:00","count":1,"enabled":true},{"id":"evening","time":"18:00","count":1,"enabled":true}]'::jsonb, 'json', 'TG publication schedule', 'EKB time schedule rows: id, time HH:MM, count, enabled.', 'TG Poster'),
('tg_poster_interval_sec', '60'::jsonb, 'int', 'TG poster interval, sec', 'Pause between poster checks.', 'TG Poster'),
('tg_poster_media_group_max_items', '10'::jsonb, 'int', 'TG media group max items', 'Telegram media group supports 2..10 media items.', 'TG Poster'),
('tg_poster_caption_limit', '1024'::jsonb, 'int', 'TG caption limit', 'Telegram caption limit for media messages.', 'TG Poster'),
('tg_poster_message_limit', '4096'::jsonb, 'int', 'TG message limit', 'Telegram text message limit.', 'TG Poster'),
('tg_poster_max_attempts', '3'::jsonb, 'int', 'TG post retry attempts', 'Retries for Telegram API errors.', 'TG Poster'),
('tg_poster_retry_backoff_max_sec', '30'::jsonb, 'int', 'TG retry backoff max, sec', 'Max retry sleep for Telegram API errors.', 'TG Poster'),
('tg_poster_send_delay_sec', '1'::jsonb, 'float', 'TG send delay, sec', 'Delay between Telegram sends.', 'TG Poster'),
('tg_poster_text_overflow_caption', '"⬇️ Описание под медиа"'::jsonb, 'str', 'TG overflow caption', 'Caption used when full text is longer than Telegram media caption limit.', 'TG Poster'),
('tg_poster_recent_window', '20'::jsonb, 'int', 'Ranking recent window', 'How many published posts are checked to reduce repeated categories and sources.', 'TG Poster'),
('tg_poster_category_repeat_penalty', '3'::jsonb, 'float', 'Category repeat penalty', 'Ranking penalty for recently published same category.', 'TG Poster'),
('tg_poster_source_repeat_penalty', '4'::jsonb, 'float', 'Source repeat penalty', 'Ranking penalty for recently published same source.', 'TG Poster')
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET description='Обрезает исходный raw-текст перед отправкой в LLM. Итоговый rewrite должен быть компактным: Telegram caption 1024 символа минус будущая строка хэштегов.'
WHERE key='ai_writer_max_text_chars';
UPDATE app_settings
SET value_json = to_jsonb($contract$
OUTPUT SCHEMA (return array matching input order):
{"rewrites":[{"id":123,"category_id":2,"text":"готовый текст без хэштегов","notes":"короткая заметка для редактора или null"}]}
Rules:
- Input is a JSON object with key "posts" containing accepted posts.
- Input has key "categories" with objects: id, name, tag.
- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.
- Pick exactly one category from the categories list in input and return its numeric id as category_id.
- Do not add links or hashtags to rewritten text.
- Keep text compact enough for Telegram media caption: target 850 characters, hard maximum 900 characters.
- Remember the system appends category and source hashtags after text, so leave space for them.
- Return JSON only. No markdown. No text outside JSON.
- Never return an empty object. Never omit the "rewrites" key.
- Return one rewrite for every input post id.
- The "text" field must be a ready-to-publish Russian post with normal paragraph line breaks encoded as JSON string newlines.
- The "notes" field must be a short editor note in Russian or null.
$contract$::text),
updated_at = NOW()
WHERE key='ai_writer_contract';
UPDATE app_settings
SET value_json = to_jsonb(
regexp_replace(
value_json #>> '{}',
'- 3-6 коротких строк или 2-5 компактных абзацев;',
'- 3-6 коротких строк или 2-4 компактных абзаца; итоговый text должен целиться в 850 символов и не превышать 900 символов, потому что система добавит хэштеги и Telegram caption ограничен 1024 символами;',
'g'
)
)
WHERE key='ai_writer_prompt'
AND (value_json #>> '{}') LIKE '%3-6 коротких строк или 2-5 компактных абзацев%';
@@ -0,0 +1,46 @@
UPDATE app_settings
SET value_json = '"⬇️ Описание"'::jsonb,
updated_at = NOW()
WHERE key='tg_poster_text_overflow_caption';
UPDATE app_settings
SET value_json = $schedule$[
{"id":"hour-00","time":"00:00","count":4,"enabled":true},
{"id":"hour-01","time":"01:00","count":4,"enabled":true},
{"id":"hour-02","time":"02:00","count":4,"enabled":true},
{"id":"hour-03","time":"03:00","count":4,"enabled":true},
{"id":"hour-04","time":"04:00","count":4,"enabled":true},
{"id":"hour-05","time":"05:00","count":4,"enabled":true},
{"id":"hour-06","time":"06:00","count":4,"enabled":true},
{"id":"hour-07","time":"07:00","count":4,"enabled":true},
{"id":"hour-08","time":"08:00","count":4,"enabled":true},
{"id":"hour-09","time":"09:00","count":4,"enabled":true},
{"id":"hour-10","time":"10:00","count":4,"enabled":true},
{"id":"hour-11","time":"11:00","count":4,"enabled":true},
{"id":"hour-12","time":"12:00","count":4,"enabled":true},
{"id":"hour-13","time":"13:00","count":4,"enabled":true},
{"id":"hour-14","time":"14:00","count":4,"enabled":true},
{"id":"hour-15","time":"15:00","count":4,"enabled":true},
{"id":"hour-16","time":"16:00","count":4,"enabled":true},
{"id":"hour-17","time":"17:00","count":4,"enabled":true},
{"id":"hour-18","time":"18:00","count":4,"enabled":true},
{"id":"hour-19","time":"19:00","count":4,"enabled":true},
{"id":"hour-20","time":"20:00","count":4,"enabled":true},
{"id":"hour-21","time":"21:00","count":4,"enabled":true},
{"id":"hour-22","time":"22:00","count":4,"enabled":true},
{"id":"hour-23","time":"23:00","count":4,"enabled":true}
]$schedule$::jsonb,
updated_at = NOW()
WHERE key='tg_poster_schedule_json';
UPDATE raw_posts
SET editorial_status='published',
updated_at=NOW()
WHERE publication_status='published'
AND editorial_status='accepted';
UPDATE raw_posts
SET editorial_status='publish_failed',
updated_at=NOW()
WHERE publication_status='publish_failed'
AND editorial_status='accepted';
@@ -0,0 +1,44 @@
-- Keep the AI-writer divider deterministic.
-- Target separator length: 17 characters.
UPDATE app_settings
SET value_json = to_jsonb(
regexp_replace(
value_json #>> '{}',
'Черта-разделитель:[^\n]*',
'Черта-разделитель: отдельная строка ровно из 17 символов. Используй строго эту строку без изменений: ━━━━━━━━━━━━━━━━━',
'g'
)
),
updated_at = NOW()
WHERE key = 'ai_writer_prompt'
AND (value_json #>> '{}') LIKE '%Черта-разделитель%';
UPDATE app_settings
SET value_json = to_jsonb(
CASE
WHEN (value_json #>> '{}') LIKE '%Separator line must be exactly%'
THEN regexp_replace(
value_json #>> '{}',
'Separator line must be exactly[^\n]*',
'Separator line must be exactly this 17-character line when used: ━━━━━━━━━━━━━━━━━',
'g'
)
ELSE (value_json #>> '{}') || E'\n- Separator line must be exactly this 17-character line when used: ━━━━━━━━━━━━━━━━━'
END
),
updated_at = NOW()
WHERE key = 'ai_writer_contract';
UPDATE raw_posts
SET rewritten_text = CASE
WHEN rewritten_text IS NULL THEN NULL
ELSE regexp_replace(rewritten_text, E'^[━─—-]{3,}\\s*$', '━━━━━━━━━━━━━━━━━', 'gm')
END,
final_text = CASE
WHEN final_text IS NULL THEN NULL
ELSE regexp_replace(final_text, E'^[━─—-]{3,}\\s*$', '━━━━━━━━━━━━━━━━━', 'gm')
END,
updated_at = NOW()
WHERE COALESCE(rewritten_text, '') ~ E'(?m)^[━─—-]{3,}\\s*$'
OR COALESCE(final_text, '') ~ E'(?m)^[━─—-]{3,}\\s*$';
@@ -0,0 +1,34 @@
-- Manual Raw-to-rewrite flow is implemented in admin.py.
-- Non-target category (sort_order/category_id 18, tag НЦК) must not enter editorial review.
UPDATE app_settings
SET value_json = to_jsonb(
CASE
WHEN (value_json #>> '{}') LIKE '%category_id 18%'
THEN value_json #>> '{}'
ELSE (value_json #>> '{}') || E'\n- If the post is non-target content, choose category_id 18 ("Не целевой контент", tag "НЦК"); the app will reject it automatically.'
END
),
updated_at = NOW()
WHERE key = 'ai_writer_contract';
UPDATE raw_posts
SET editorial_status='rejected',
qualification_status='rejected',
qualification_decision='rejected',
qualification_model_decision='rejected',
qualification_reason=COALESCE(NULLIF(qualification_reason, ''), 'AI writer selected non-target category'),
qualification_reject_tag=COALESCE(NULLIF(qualification_reject_tag, ''), 'non_target_content'),
editor_notes=COALESCE(NULLIF(editor_notes, ''), 'Отклонено AI: Не целевой контент'),
reviewed_at=COALESCE(reviewed_at, NOW()),
updated_at=NOW()
WHERE rewrite_status='ready'
AND COALESCE(editorial_status, 'review')='review'
AND (
rewrite_category_id=18
OR final_category_id=18
OR lower(COALESCE(rewrite_category_tag, '')) IN ('нцк', 'nck')
OR lower(COALESCE(final_category_tag, '')) IN ('нцк', 'nck')
OR lower(COALESCE(rewrite_category, '')) = lower('Не целевой контент')
OR lower(COALESCE(final_category, '')) = lower('Не целевой контент')
);
@@ -0,0 +1,67 @@
CREATE TABLE IF NOT EXISTS post_publications (
id BIGSERIAL PRIMARY KEY,
raw_post_id BIGINT NOT NULL REFERENCES raw_posts(id) ON DELETE CASCADE,
platform TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
target_id TEXT,
external_id TEXT,
url TEXT,
error TEXT,
attachments_json JSONB NOT NULL DEFAULT '[]'::jsonb,
published_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(raw_post_id, platform)
);
CREATE INDEX IF NOT EXISTS idx_post_publications_platform_status
ON post_publications(platform, status, target_id, published_at DESC);
CREATE INDEX IF NOT EXISTS idx_post_publications_raw_post
ON post_publications(raw_post_id);
INSERT INTO post_publications(raw_post_id, platform, status, target_id, external_id, url, error, attachments_json, published_at, updated_at)
SELECT
id,
'tg',
publication_status,
tg_publication_chat_id::text,
CASE WHEN tg_publication_message_ids IS NULL THEN NULL ELSE array_to_string(tg_publication_message_ids, ',') END,
tg_publication_url,
publication_error,
COALESCE(to_jsonb(tg_publication_message_ids), '[]'::jsonb),
published_at,
NOW()
FROM raw_posts
WHERE COALESCE(publication_status, 'pending') <> 'pending'
ON CONFLICT (raw_post_id, platform) DO UPDATE
SET status=EXCLUDED.status,
target_id=EXCLUDED.target_id,
external_id=EXCLUDED.external_id,
url=EXCLUDED.url,
error=EXCLUDED.error,
attachments_json=EXCLUDED.attachments_json,
published_at=EXCLUDED.published_at,
updated_at=NOW();
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('vk-poster', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('vk_poster_enabled', 'false'::jsonb, 'bool', 'VK poster enabled', 'Double safety flag for publishing accepted posts to VK.', 'VK Poster'),
('vk_poster_access_token', '""'::jsonb, 'secret', 'VK poster access token', 'Group/user token with wall.post and photos permissions. If empty, env VK_GROUP_ACCESS_TOKEN/VK_ACCESS_TOKEN is used.', 'VK Poster'),
('vk_poster_owner_id', '0'::jsonb, 'int', 'VK target owner_id', 'Target VK group owner_id, usually negative: -123456789.', 'VK Poster'),
('vk_poster_from_group', 'true'::jsonb, 'bool', 'Post from group', 'Use from_group=1 when publishing to a community wall.', 'VK Poster'),
('vk_poster_schedule_json', '[{"id":"vk-morning","time":"10:05","count":1,"enabled":true},{"id":"vk-evening","time":"18:05","count":1,"enabled":true}]'::jsonb, 'json', 'VK publication schedule', 'EKB time schedule rows: id, time HH:MM, count, enabled.', 'VK Poster'),
('vk_poster_interval_sec', '60'::jsonb, 'int', 'VK poster interval, sec', 'Pause between poster checks.', 'VK Poster'),
('vk_poster_timeout_sec', '90'::jsonb, 'int', 'VK API timeout, sec', 'Total timeout for VK API calls and media uploads.', 'VK Poster'),
('vk_poster_max_attempts', '3'::jsonb, 'int', 'VK post retry attempts', 'Retries for VK API/network errors.', 'VK Poster'),
('vk_poster_retry_backoff_max_sec', '30'::jsonb, 'int', 'VK retry backoff max, sec', 'Max retry sleep for VK API/network errors.', 'VK Poster'),
('vk_poster_send_delay_sec', '1'::jsonb, 'float', 'VK send delay, sec', 'Delay between VK posts.', 'VK Poster'),
('vk_poster_recent_window', '20'::jsonb, 'int', 'Ranking recent window', 'How many VK-published posts are checked to reduce repeated categories and sources.', 'VK Poster'),
('vk_poster_category_repeat_penalty', '3'::jsonb, 'float', 'Category repeat penalty', 'Ranking penalty for recently published same category.', 'VK Poster'),
('vk_poster_source_repeat_penalty', '4'::jsonb, 'float', 'Source repeat penalty', 'Ranking penalty for recently published same source.', 'VK Poster'),
('vk_poster_dry_run', 'false'::jsonb, 'bool', 'VK dry run', 'If true, prepares attachments but does not call wall.post. Keep false for production.', 'VK Poster')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,5 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('vk_poster_app_id', '""'::jsonb, 'str', 'VK OAuth app id', 'VK application id used for user OAuth code flow.', 'VK Poster'),
('vk_poster_client_secret', '""'::jsonb, 'secret', 'VK OAuth client secret', 'Protected key from VK application settings. Used server-side to exchange code for user access token.', 'VK Poster')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,6 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('vk_poster_refresh_token', '""'::jsonb, 'secret', 'VK OAuth refresh token', 'Refresh token from VK ID code flow. Used by VK poster to renew access token automatically.', 'VK Poster'),
('vk_poster_token_device_id', '""'::jsonb, 'str', 'VK OAuth device id', 'VK ID device_id returned with authorization code/token exchange.', 'VK Poster'),
('vk_poster_token_expires_at', '""'::jsonb, 'str', 'VK access token expires at', 'UTC ISO timestamp used by VK poster to refresh the access token before expiry.', 'VK Poster')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,18 @@
-- Make producer mention mandatory in every AI writer result.
UPDATE app_settings
SET value_json = to_jsonb(
CASE
WHEN (value_json #>> '{}') LIKE '%explicitly mention producer_name%'
OR (value_json #>> '{}') LIKE '%явно упоминает producer_name%'
THEN value_json #>> '{}'
ELSE replace(
value_json #>> '{}',
'- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.',
'- Each post includes producer_name and producer_tag. producer_name can be a manufacturer, shop, or publishing source.' || E'\n' ||
'- The "text" field for every rewrite must explicitly mention producer_name at least once. Do not satisfy this only through producer_tag, hashtags, notes, or metadata.'
)
END
),
updated_at = NOW()
WHERE key = 'ai_writer_contract';
+39
View File
@@ -0,0 +1,39 @@
CREATE TABLE IF NOT EXISTS post_reactions (
id BIGSERIAL PRIMARY KEY,
publication_id BIGINT NOT NULL REFERENCES post_publications(id) ON DELETE CASCADE,
raw_post_id BIGINT NOT NULL REFERENCES raw_posts(id) ON DELETE CASCADE,
platform TEXT NOT NULL,
reactor_key TEXT NOT NULL,
reaction TEXT NOT NULL,
target_id TEXT NOT NULL,
message_id TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
error TEXT,
attempts INTEGER NOT NULL DEFAULT 0,
reacted_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
UNIQUE(publication_id, reactor_key, message_id)
);
CREATE INDEX IF NOT EXISTS idx_post_reactions_status
ON post_reactions(platform, status, updated_at);
CREATE INDEX IF NOT EXISTS idx_post_reactions_raw_post
ON post_reactions(raw_post_id);
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('tg-reactor', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('tg_reactor_enabled', 'false'::jsonb, 'bool', 'TG reactor enabled', 'Double safety flag for Telegram reaction worker.', 'TG Reactor'),
('tg_reactor_bot_tokens', '""'::jsonb, 'secret', 'TG reactor bot tokens', 'One or more Telegram bot tokens separated by comma/newline. If empty, uses tg_poster_bot_token/TG_BOT_TOKEN.', 'TG Reactor'),
('tg_reactor_reactions_json', '["👍"]'::jsonb, 'json', 'TG reactions', 'Emoji reactions. One bot can set one reaction; multiple tokens can set multiple reactions.', 'TG Reactor'),
('tg_reactor_delay_sec', '60'::jsonb, 'int', 'Delay after publication, sec', 'How long to wait after TG publication before reacting.', 'TG Reactor'),
('tg_reactor_interval_sec', '60'::jsonb, 'int', 'TG reactor interval, sec', 'Pause between reaction worker checks.', 'TG Reactor'),
('tg_reactor_react_all_messages', 'false'::jsonb, 'bool', 'React to all media messages', 'If false, reacts only to the first message of each published post.', 'TG Reactor'),
('tg_reactor_max_attempts', '3'::jsonb, 'int', 'TG reaction retry attempts', 'Retries per publication/message/reactor.', 'TG Reactor'),
('tg_reactor_retry_backoff_max_sec', '30'::jsonb, 'int', 'TG reaction retry backoff max, sec', 'Max sleep for Telegram reaction retry errors.', 'TG Reactor')
ON CONFLICT (key) DO NOTHING;
+4
View File
@@ -0,0 +1,4 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('tg_reactor_since', '""'::jsonb, 'str', 'React only since', 'UTC ISO timestamp. If set, TG reactor ignores publications older than this moment.', 'TG Reactor')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,4 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('tg_reactor_reaction_pause_sec', '2'::jsonb, 'int', 'Pause between TG reactions, sec', 'Small pause after each successful reaction to avoid a burst of Telegram API calls.', 'TG Reactor')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,14 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('video_max_height', '720'::jsonb, 'int', 'Video max height', 'Maximum VK video height selected by yt-dlp before uploading to Telegram.', 'Uploader')
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET value_json='600'::jsonb,
updated_at=NOW()
WHERE key='uploader_yt_dlp_timeout_sec'
AND CASE
WHEN jsonb_typeof(value_json) = 'number' THEN (value_json #>> '{}')::int
WHEN jsonb_typeof(value_json) = 'string' AND (value_json #>> '{}') ~ '^\d+$' THEN (value_json #>> '{}')::int
ELSE 0
END < 600;
+13
View File
@@ -0,0 +1,13 @@
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('daily-report', TRUE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('daily_report_enabled', 'true'::jsonb, 'bool', 'Daily report enabled', 'Double safety flag for daily Telegram operational report.', 'Daily Report'),
('daily_report_bot_token', '""'::jsonb, 'secret', 'Daily report bot token', 'Telegram bot token for reports. If empty, uses tg_poster_bot_token or TG_BOT_TOKEN.', 'Daily Report'),
('daily_report_recipient_ids', '[442509142]'::jsonb, 'json', 'Daily report recipients', 'Telegram user/chat IDs that receive the daily report.', 'Daily Report'),
('daily_report_time', '"00:04"'::jsonb, 'str', 'Daily report time', 'EKB time HH:MM when yesterday report is sent.', 'Daily Report'),
('daily_report_interval_sec', '60'::jsonb, 'int', 'Daily report interval, sec', 'Pause between checks.', 'Daily Report'),
('daily_report_last_sent_date', '""'::jsonb, 'str', 'Last sent report date', 'Internal YYYY-MM-DD report date already sent automatically.', 'Daily Report')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,57 @@
ALTER TABLE content_categories
ADD COLUMN IF NOT EXISTS site_name TEXT,
ADD COLUMN IF NOT EXISTS site_slug TEXT,
ADD COLUMN IF NOT EXISTS site_enabled BOOLEAN NOT NULL DEFAULT TRUE;
UPDATE content_categories
SET site_name = CASE sort_order
WHEN 1 THEN 'Одежда'
WHEN 2 THEN 'Снаряжение'
WHEN 3 THEN 'Броня'
WHEN 4 THEN 'Аксессуары'
WHEN 5 THEN 'Связь'
WHEN 6 THEN 'Патчи'
WHEN 7 THEN 'Навигация'
WHEN 8 THEN 'Прицелы'
WHEN 9 THEN 'Ночные приборы'
WHEN 10 THEN 'Медицина'
WHEN 11 THEN 'Свет'
WHEN 12 THEN 'Электроника'
WHEN 13 THEN 'Ножи'
WHEN 14 THEN 'Оружейный обвес'
WHEN 15 THEN 'Практика'
WHEN 16 THEN 'Теория'
WHEN 17 THEN 'Пушки'
WHEN 18 THEN 'НЦК'
ELSE initcap(replace(tag, '_', ' '))
END,
site_slug = CASE sort_order
WHEN 1 THEN 'odezhda'
WHEN 2 THEN 'snaryazhenie'
WHEN 3 THEN 'bronya'
WHEN 4 THEN 'aksessuary'
WHEN 5 THEN 'svyaz'
WHEN 6 THEN 'patchi'
WHEN 7 THEN 'navigatsiya'
WHEN 8 THEN 'pricely'
WHEN 9 THEN 'nochnye-pribory'
WHEN 10 THEN 'medicina'
WHEN 11 THEN 'svet'
WHEN 12 THEN 'elektronika'
WHEN 13 THEN 'nozhi'
WHEN 14 THEN 'oruzheynyy-obves'
WHEN 15 THEN 'praktika'
WHEN 16 THEN 'teoriya'
WHEN 17 THEN 'pushki'
WHEN 18 THEN 'nck'
ELSE site_slug
END,
site_enabled = sort_order <> 18,
updated_at = NOW();
ALTER TABLE content_categories
ALTER COLUMN site_name SET NOT NULL,
ALTER COLUMN site_slug SET NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS idx_content_categories_site_slug
ON content_categories(site_slug);
+15
View File
@@ -0,0 +1,15 @@
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('site-poster', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('site_poster_enabled', 'false'::jsonb, 'bool', 'Site poster enabled', 'Double safety flag for automatic site publishing.', 'Site Poster'),
('site_poster_provider', '""'::jsonb, 'str', 'Site provider', 'Publish adapter: ghost now; wordpress can be added as a code adapter later.', 'Site Poster'),
('site_poster_base_url', '""'::jsonb, 'str', 'Site API base URL', 'Public/admin base URL for the selected site provider.', 'Site Poster'),
('site_poster_secret_file', '""'::jsonb, 'str', 'Site secret file', 'Server path to the selected provider API secret. Keep empty in git.', 'Site Poster'),
('site_poster_fallback_title', '"Новость"'::jsonb, 'str', 'Fallback article title', 'Title used when post text has no first line.', 'Site Poster'),
('site_poster_interval_sec', '300'::jsonb, 'int', 'Site poster interval, sec', 'Pause between automatic queue checks.', 'Site Poster'),
('site_poster_photo_batch_size', '10'::jsonb, 'int', 'Photo batch size', 'Maximum photo-ready posts per automatic run.', 'Site Poster'),
('site_poster_video_batch_size', '1'::jsonb, 'int', 'Video batch size', 'Maximum video-only posts per automatic run.', 'Site Poster')
ON CONFLICT (key) DO NOTHING;
+28
View File
@@ -0,0 +1,28 @@
INSERT INTO worker_controls(name, enabled, settings_json)
VALUES ('max-poster', FALSE, '{}'::jsonb)
ON CONFLICT (name) DO NOTHING;
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('max_poster_enabled', 'false'::jsonb, 'bool', 'MAX poster enabled', 'Double safety flag for publishing accepted posts to MAX.', 'MAX Poster'),
('max_poster_bot_token', '""'::jsonb, 'secret', 'MAX poster bot token', 'Bot access token for MAX API. Keep empty in git and set on server.', 'MAX Poster'),
('max_poster_chat_id', '""'::jsonb, 'str', 'MAX target chat id', 'Target MAX channel/group chat_id.', 'MAX Poster'),
('max_poster_api_base_url', '"https://platform-api2.max.ru"'::jsonb, 'str', 'MAX API base URL', 'MAX Bot API base URL. Since 2026 use platform-api2.max.ru.', 'MAX Poster'),
('max_poster_schedule_json', '[{"id":"hour-00","time":"00:00","count":4,"enabled":true},{"id":"hour-01","time":"01:00","count":4,"enabled":true},{"id":"hour-02","time":"02:00","count":4,"enabled":true},{"id":"hour-03","time":"03:00","count":4,"enabled":true},{"id":"hour-04","time":"04:00","count":4,"enabled":true},{"id":"hour-05","time":"05:00","count":4,"enabled":true},{"id":"hour-06","time":"06:00","count":4,"enabled":true},{"id":"hour-07","time":"07:00","count":4,"enabled":true},{"id":"hour-08","time":"08:00","count":4,"enabled":true},{"id":"hour-09","time":"09:00","count":4,"enabled":true},{"id":"hour-10","time":"10:00","count":4,"enabled":true},{"id":"hour-11","time":"11:00","count":4,"enabled":true},{"id":"hour-12","time":"12:00","count":4,"enabled":true},{"id":"hour-13","time":"13:00","count":4,"enabled":true},{"id":"hour-14","time":"14:00","count":4,"enabled":true},{"id":"hour-15","time":"15:00","count":4,"enabled":true},{"id":"hour-16","time":"16:00","count":4,"enabled":true},{"id":"hour-17","time":"17:00","count":4,"enabled":true},{"id":"hour-18","time":"18:00","count":4,"enabled":true},{"id":"hour-19","time":"19:00","count":4,"enabled":true},{"id":"hour-20","time":"20:00","count":4,"enabled":true},{"id":"hour-21","time":"21:00","count":4,"enabled":true},{"id":"hour-22","time":"22:00","count":4,"enabled":true},{"id":"hour-23","time":"23:00","count":4,"enabled":true}]'::jsonb, 'json', 'MAX publication schedule', 'EKB time schedule rows: id, time HH:MM, count, enabled. Defaults to the TG hourly schedule.', 'MAX Poster'),
('max_poster_interval_sec', '60'::jsonb, 'int', 'MAX poster interval, sec', 'Pause between poster checks.', 'MAX Poster'),
('max_poster_message_limit', '4000'::jsonb, 'int', 'MAX message limit', 'MAX text message limit.', 'MAX Poster'),
('max_poster_media_max_items', '10'::jsonb, 'int', 'MAX media max items', 'Maximum media attachments per MAX message.', 'MAX Poster'),
('max_poster_media_process_delay_sec', '3'::jsonb, 'float', 'MAX media process delay, sec', 'Pause after media upload before sending the message.', 'MAX Poster'),
('max_poster_video_ready_attempts', '5'::jsonb, 'int', 'MAX video ready attempts', 'Retries for sending a message while MAX is still processing uploaded video.', 'MAX Poster'),
('max_poster_video_ready_delay_sec', '10'::jsonb, 'float', 'MAX video ready delay, sec', 'Initial retry delay while MAX is processing uploaded video. Delay doubles after each attempt.', 'MAX Poster'),
('max_poster_max_attempts', '3'::jsonb, 'int', 'MAX post retry attempts', 'Retries for MAX API/network errors and attachment processing.', 'MAX Poster'),
('max_poster_retry_backoff_max_sec', '30'::jsonb, 'int', 'MAX retry backoff max, sec', 'Max retry sleep for MAX API/network errors.', 'MAX Poster'),
('max_poster_send_delay_sec', '1'::jsonb, 'float', 'MAX send delay, sec', 'Delay between MAX sends.', 'MAX Poster'),
('max_poster_recent_window', '20'::jsonb, 'int', 'Ranking recent window', 'How many MAX-published posts are checked to reduce repeated categories and sources.', 'MAX Poster'),
('max_poster_category_repeat_penalty', '3'::jsonb, 'float', 'Category repeat penalty', 'Ranking penalty for recently published same category.', 'MAX Poster'),
('max_poster_source_repeat_penalty', '4'::jsonb, 'float', 'Source repeat penalty', 'Ranking penalty for recently published same source.', 'MAX Poster'),
('max_poster_auto_reaction_enabled', 'true'::jsonb, 'bool', 'MAX auto reaction enabled', 'Try to set a reaction after publishing. If MAX API rejects it, publication remains successful.', 'MAX Poster'),
('max_poster_auto_reaction', '"👍"'::jsonb, 'str', 'MAX auto reaction', 'Emoji reaction to try after publishing.', 'MAX Poster'),
('max_poster_reaction_path_template', '"/messages/{message_id}/reactions"'::jsonb, 'str', 'MAX reaction path template', 'Relative API path for reaction call if supported by MAX API.', 'MAX Poster'),
('max_poster_dry_run', 'false'::jsonb, 'bool', 'MAX dry run', 'If true, prepares publication but does not call MAX send message.', 'MAX Poster')
ON CONFLICT (key) DO NOTHING;
@@ -0,0 +1,36 @@
INSERT INTO app_settings(key, value_json, value_type, title, description, category)
VALUES
('site_poster_provider', '""'::jsonb, 'str', 'Site provider', 'Publish adapter: ghost now; wordpress can be added as a code adapter later.', 'Site Poster'),
('site_poster_base_url', '""'::jsonb, 'str', 'Site API base URL', 'Public/admin base URL for the selected site provider.', 'Site Poster'),
('site_poster_secret_file', '""'::jsonb, 'str', 'Site secret file', 'Server path to the selected provider API secret. Keep empty in git.', 'Site Poster'),
('site_poster_fallback_title', '"Новость"'::jsonb, 'str', 'Fallback article title', 'Title used when post text has no first line.', 'Site Poster')
ON CONFLICT (key) DO NOTHING;
UPDATE app_settings
SET value_json='"ghost"'::jsonb,
updated_at=NOW()
WHERE key='site_poster_provider'
AND COALESCE(value_json #>> '{}', '') = ''
AND EXISTS (
SELECT 1 FROM app_settings legacy
WHERE legacy.key='site_poster_ghost_url'
AND COALESCE(legacy.value_json #>> '{}', '') <> ''
);
UPDATE app_settings target
SET value_json=to_jsonb(legacy.value_json #>> '{}'),
updated_at=NOW()
FROM app_settings legacy
WHERE target.key='site_poster_base_url'
AND legacy.key='site_poster_ghost_url'
AND COALESCE(target.value_json #>> '{}', '') = ''
AND COALESCE(legacy.value_json #>> '{}', '') <> '';
UPDATE app_settings target
SET value_json=to_jsonb(legacy.value_json #>> '{}'),
updated_at=NOW()
FROM app_settings legacy
WHERE target.key='site_poster_secret_file'
AND legacy.key='site_poster_key_file'
AND COALESCE(target.value_json #>> '{}', '') = ''
AND COALESCE(legacy.value_json #>> '{}', '') <> '';
+15
View File
@@ -0,0 +1,15 @@
[Unit]
Description=VK Parser MAX Poster
After=network.target postgresql.service
[Service]
Type=simple
WorkingDirectory=/opt/vk-parser
Environment=PYTHONPATH=/opt/vk-parser/src
ExecStart=/opt/vk-parser/.venv/bin/python -m vk_parser_app.workers.max_poster
Restart=always
RestartSec=10
User=root
[Install]
WantedBy=multi-user.target
+14
View File
@@ -0,0 +1,14 @@
[Unit]
Description=VK Parser Site Poster
After=network.target postgresql.service
[Service]
Type=simple
WorkingDirectory=/opt/vk-parser
Environment=PYTHONPATH=/opt/vk-parser/src
ExecStart=/opt/vk-parser/.venv/bin/python -m vk_parser_app.workers.site_poster
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
+32
View File
@@ -0,0 +1,32 @@
#!/usr/bin/env sh
set -eu
tg_pid=""
stop_children() {
if [ -n "$tg_pid" ] && kill -0 "$tg_pid" 2>/dev/null; then
kill "$tg_pid"
wait "$tg_pid" 2>/dev/null || true
fi
}
trap stop_children INT TERM EXIT
if [ -n "${TELEGRAM_API_ID:-}" ] && [ -n "${TELEGRAM_API_HASH:-}" ]; then
mkdir -p "${TELEGRAM_WORK_DIR:-/var/lib/telegram-bot-api}" "${TELEGRAM_TEMP_DIR:-/tmp/telegram-bot-api}"
telegram-bot-api \
--api-id="${TELEGRAM_API_ID}" \
--api-hash="${TELEGRAM_API_HASH}" \
--dir="${TELEGRAM_WORK_DIR:-/var/lib/telegram-bot-api}" \
--temp-dir="${TELEGRAM_TEMP_DIR:-/tmp/telegram-bot-api}" \
--http-port="${TELEGRAM_HTTP_PORT:-8081}" \
--local &
tg_pid="$!"
echo "Started local Telegram Bot API on 127.0.0.1:${TELEGRAM_HTTP_PORT:-8081}"
else
echo "TELEGRAM_API_ID/TELEGRAM_API_HASH are not set; local Telegram Bot API is disabled"
fi
python /app/scripts/apply_migrations.py
exec "$@"
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

+8
View File
@@ -0,0 +1,8 @@
DATABASE_URL=postgresql://forma_n8:change-me@127.0.0.1:5432/forma_n8
SITE_BASE_URL=http://127.0.0.1:8090
SITE_NAME=Forma N8
SITE_TAGLINE=Новости снаряжения
SITE_INDEXING_ENABLED=false
PUBLISH_API_TOKEN=change-me
UPLOAD_DIR=/var/lib/forma-n8/uploads
PAGE_SIZE=12
+55
View File
@@ -0,0 +1,55 @@
# Forma N8 Site
Отдельное SSR-приложение публичного сайта Forma N8. Контент и постоянные URL не зависят от будущего дизайна.
## Возможности
- лента новостей, страницы статей, категорий и производителей;
- PostgreSQL со стабильными `external_id` и `slug`;
- idempotent API публикации и отдельный upload API;
- RSS 2.0, sitemap, robots.txt;
- canonical, Open Graph и JSON-LD `NewsArticle`;
- системный деплой через nginx + systemd.
## Локальный запуск
```bash
python -m venv .venv
.venv/bin/pip install -r requirements.txt
export DATABASE_URL=postgresql://forma_n8:forma_n8@127.0.0.1:5432/forma_n8
PYTHONPATH=src .venv/bin/python scripts/migrate.py
PYTHONPATH=src .venv/bin/uvicorn forma_n8_site.app:app --port 8090
```
## Публикационный API
Сначала медиа загружается через `POST /api/v1/media` с заголовком `X-Forma-Token`, затем полученный относительный URL передаётся в статью.
`POST /api/v1/articles`:
```json
{
"external_id": "parser:518",
"title": "Заголовок новости",
"lead": "Краткое описание",
"body": "Основной текст без HTML",
"category_name": "Снаряжение",
"category_tag": "snaryazhenie",
"producer_name": "Производитель",
"producer_tag": "producer",
"source_url": "https://example.com/source",
"status": "published",
"published_at": "2026-06-21T10:00:00+05:00",
"media": [
{"type": "image", "url": "/media/ab/hash.jpg", "alt": "Описание изображения"}
]
}
```
Повторный запрос с тем же `external_id` обновляет материал, но сохраняет первоначальные slug и дату первой публикации. Это предотвращает дубли URL и потерю поисковой истории.
Публичное название категории берётся из таблицы `categories` по нормализованному `category_tag`. Описательные названия категорий из AI-контура не используются в URL, H1 и SEO-метаданных.
После подключения домена меняется только `SITE_BASE_URL` в `/etc/forma-n8.env`; затем настраивается TLS и отправляются sitemap в Google Search Console и Яндекс Вебмастер.
До подключения домена `SITE_INDEXING_ENABLED=false`: страницы получают `noindex`, а `robots.txt` запрещает обход IP-адреса. После настройки домена, HTTPS и 301-редиректа с IP значение меняется на `true`.
@@ -0,0 +1,46 @@
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS articles (
id BIGSERIAL PRIMARY KEY,
external_id TEXT NOT NULL UNIQUE,
slug TEXT NOT NULL UNIQUE,
title TEXT NOT NULL,
lead TEXT NOT NULL DEFAULT '',
body TEXT NOT NULL,
category_name TEXT NOT NULL,
category_tag TEXT NOT NULL,
producer_name TEXT NOT NULL,
producer_tag TEXT NOT NULL,
source_url TEXT,
seo_title TEXT NOT NULL DEFAULT '',
seo_description TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'draft' CHECK (status IN ('draft', 'published')),
published_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_articles_feed
ON articles(status, published_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS idx_articles_category
ON articles(category_tag, status, published_at DESC);
CREATE INDEX IF NOT EXISTS idx_articles_producer
ON articles(producer_tag, status, published_at DESC);
CREATE TABLE IF NOT EXISTS article_media (
id BIGSERIAL PRIMARY KEY,
article_id BIGINT NOT NULL REFERENCES articles(id) ON DELETE CASCADE,
media_type TEXT NOT NULL CHECK (media_type IN ('image', 'video')),
url TEXT NOT NULL,
alt_text TEXT NOT NULL DEFAULT '',
width INTEGER,
height INTEGER,
sort_order INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_article_media_order
ON article_media(article_id, sort_order, id);
@@ -0,0 +1,35 @@
CREATE TABLE IF NOT EXISTS categories (
tag TEXT PRIMARY KEY,
name TEXT NOT NULL,
sort_order INTEGER NOT NULL DEFAULT 100,
is_active BOOLEAN NOT NULL DEFAULT TRUE
);
INSERT INTO categories(tag, name, sort_order) VALUES
('odezhda', 'Одежда', 1),
('snaryazhenie', 'Снаряжение', 2),
('bronya', 'Броня', 3),
('aksessuary', 'Аксессуары', 4),
('svyaz', 'Связь', 5),
('patchi', 'Патчи', 6),
('navigaciya', 'Навигация', 7),
('pricely', 'Прицелы', 8),
('nochnye-pribory', 'Ночные приборы', 9),
('medicina', 'Медицина', 10),
('svet', 'Свет', 11),
('elektronika', 'Электроника', 12),
('nozhi', 'Ножи и мультитулы', 13),
('oruzheynyy-obves', 'Оружейный обвес', 14),
('praktika', 'Практика', 15),
('teoriya', 'Теория', 16),
('pushki', 'Оружие', 17)
ON CONFLICT(tag) DO UPDATE SET
name=EXCLUDED.name,
sort_order=EXCLUDED.sort_order;
UPDATE articles a
SET category_name=c.name,
updated_at=NOW()
FROM categories c
WHERE a.category_tag=c.tag
AND a.category_name<>c.name;
+23
View File
@@ -0,0 +1,23 @@
[Unit]
Description=Forma N8 public website
After=network.target postgresql.service
Wants=postgresql.service
[Service]
Type=simple
User=forma-n8
Group=forma-n8
WorkingDirectory=/opt/forma-n8
EnvironmentFile=/etc/forma-n8.env
Environment=PYTHONPATH=/opt/forma-n8/src
ExecStart=/opt/forma-n8/.venv/bin/uvicorn forma_n8_site.app:app --host 127.0.0.1 --port 8090 --proxy-headers
Restart=always
RestartSec=3
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/forma-n8
[Install]
WantedBy=multi-user.target
+31
View File
@@ -0,0 +1,31 @@
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
client_max_body_size 52m;
location /static/ {
alias /opt/forma-n8/static/;
expires 30d;
add_header Cache-Control "public, immutable";
access_log off;
}
location /media/ {
alias /var/lib/forma-n8/uploads/;
expires 30d;
add_header Cache-Control "public";
add_header X-Content-Type-Options "nosniff";
}
location / {
proxy_pass http://127.0.0.1:8090;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 60s;
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

@@ -0,0 +1,431 @@
@font-face {
font-family: "Roboto Condensed";
src: url("../fonts/roboto-condensed-cyrillic-400.woff2") format("woff2");
font-weight: 400;
font-display: swap;
}
@font-face {
font-family: "Roboto Condensed";
src: url("../fonts/roboto-condensed-cyrillic-700.woff2") format("woff2");
font-weight: 700;
font-display: swap;
}
:root {
color-scheme: dark;
--bg: #090c0d;
--surface: #101415;
--surface-strong: #151a1b;
--text: #eceee8;
--muted: #9ba19b;
--faint: #686f6b;
--border: #29302e;
--border-strong: #3a423f;
--olive: #a5ad62;
--red: #df4a40;
--max: 1240px;
--gutter: 24px;
--font: "Roboto Condensed", "Arial Narrow", Arial, sans-serif;
}
* { box-sizing: border-box; }
html { background: var(--bg); scroll-behavior: smooth; }
body {
margin: 0;
min-width: 320px;
background: var(--bg);
color: var(--text);
font-family: var(--font);
font-size: 18px;
line-height: 1.55;
letter-spacing: 0;
-webkit-font-smoothing: antialiased;
}
body::before {
content: "";
position: fixed;
inset: 0;
pointer-events: none;
opacity: .22;
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='120' height='120' viewBox='0 0 120 120'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.8' numOctaves='2' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.08'/%3E%3C/svg%3E");
}
a { color: inherit; text-decoration: none; }
a:hover { color: var(--olive); }
img, video { display: block; max-width: 100%; }
button, input { font: inherit; letter-spacing: 0; }
:focus-visible { outline: 2px solid var(--olive); outline-offset: 4px; }
.skip-link {
position: fixed;
z-index: 100;
top: 10px;
left: 10px;
transform: translateY(-160%);
padding: 10px 14px;
background: var(--text);
color: var(--bg);
}
.skip-link:focus { transform: none; }
.site-header {
position: sticky;
z-index: 20;
top: 0;
border-bottom: 1px solid var(--border);
background: rgba(9, 12, 13, .96);
backdrop-filter: blur(12px);
}
.header-inner {
width: min(var(--max), calc(100% - var(--gutter) * 2));
min-height: 76px;
margin: 0 auto;
display: grid;
grid-template-columns: 170px 1fr auto;
align-items: center;
gap: 32px;
}
.brand {
display: inline-flex;
width: fit-content;
align-items: center;
gap: 11px;
font-size: 15px;
font-weight: 700;
line-height: .95;
}
.brand img { width: 42px; height: 42px; border: 1px solid var(--border-strong); }
.main-nav { display: flex; justify-content: center; align-self: stretch; gap: 38px; }
.main-nav .nav { display: contents; margin: 0; padding: 0; list-style: none; }
.main-nav .nav li { display: contents; }
.nav-dropdown { position: relative; display: flex; align-items: center; }
.nav-dropdown-toggle { display: inline-flex; height: 100%; align-items: center; gap: 8px; padding: 0; border: 0; background: transparent; color: var(--muted); cursor: pointer; }
.nav-dropdown-toggle:hover, .nav-dropdown-toggle[aria-expanded="true"] { color: var(--text); }
.nav-dropdown-icon { width: 7px; height: 7px; border-right: 1.5px solid currentColor; border-bottom: 1.5px solid currentColor; transform: translateY(-2px) rotate(45deg); transition: transform .18s ease; }
.nav-dropdown-toggle[aria-expanded="true"] .nav-dropdown-icon { transform: translateY(2px) rotate(225deg); }
.category-menu { position: absolute; z-index: 30; top: calc(100% - 1px); left: 50%; display: none; grid-template-columns: repeat(2, minmax(170px, 1fr)); width: 430px; padding: 12px; border: 1px solid var(--border-strong); background: var(--bg); transform: translateX(-50%); box-shadow: 0 18px 44px rgba(0,0,0,.38); }
.category-menu.is-open { display: grid; }
.category-menu a { min-height: 40px; padding: 9px 11px; border-bottom: 1px solid var(--border); color: var(--muted); }
.category-menu a:hover { background: var(--surface-strong); color: var(--text); }
.main-nav a {
position: relative;
display: inline-flex;
align-items: center;
color: var(--muted);
font-size: 16px;
white-space: nowrap;
}
.main-nav a:hover, .main-nav a[aria-current="page"] { color: var(--text); }
.main-nav a[aria-current="page"]::after {
content: "";
position: absolute;
right: 0;
bottom: -1px;
left: 0;
height: 3px;
background: var(--olive);
}
.header-actions { display: flex; gap: 8px; }
.header-actions > .social-links { margin-right: 5px; padding-right: 10px; border-right: 1px solid var(--border); }
.icon-button {
display: grid;
width: 40px;
height: 40px;
padding: 0;
place-items: center;
border: 0;
background: transparent;
color: var(--text);
cursor: pointer;
}
.icon-button:hover { background: var(--surface-strong); color: var(--olive); }
.icon-button svg { width: 23px; height: 23px; fill: none; stroke: currentColor; stroke-width: 1.8; stroke-linecap: square; }
.menu-button { display: none; }
.search-form {
width: min(760px, calc(100% - var(--gutter) * 2));
margin: 0 auto;
padding: 16px 0 20px;
}
.search-form[hidden] { display: none; }
.search-form label { display: block; margin-bottom: 8px; color: var(--muted); font-size: 14px; }
.search-form > div { display: grid; grid-template-columns: 1fr auto; }
.search-form input {
min-width: 0;
height: 48px;
padding: 0 16px;
border: 1px solid var(--border-strong);
border-right: 0;
border-radius: 0;
background: var(--surface);
color: var(--text);
font-size: 17px;
}
.search-form button, .read-link, .empty-state a {
border: 1px solid var(--olive);
border-radius: 0;
background: transparent;
color: var(--text);
cursor: pointer;
}
.search-form button { padding: 0 24px; }
.search-form button:hover, .read-link:hover, .empty-state a:hover { background: var(--olive); color: var(--bg); }
.feed-shell, .article-page, .related-section, .site-footer {
position: relative;
width: min(var(--max), calc(100% - var(--gutter) * 2));
margin: 0 auto;
}
.page-heading { padding: 56px 0 46px; }
.page-heading h1 {
max-width: 900px;
margin: 0;
font-size: 68px;
font-weight: 700;
line-height: .98;
}
@media (min-width: 981px) {
.home-template .page-heading {
position: relative;
left: 50%;
width: min(1600px, calc(100vw - var(--gutter) * 2));
transform: translateX(-50%);
}
.home-template .page-heading h1 {
max-width: none;
font-size: clamp(58px, 4vw, 68px);
text-align: center;
}
}
.page-heading > p { margin: 20px 0 0; color: var(--muted); }
.lead-story {
display: grid;
grid-template-columns: minmax(0, 1.25fr) minmax(340px, .95fr);
min-height: 480px;
max-height: 680px;
border-bottom: 1px solid var(--border-strong);
}
.lead-media { min-height: 480px; max-height: 680px; overflow: hidden; background: var(--surface); }
.lead-media img, .lead-media video { width: 100%; height: 100%; object-fit: cover; transition: transform .45s ease; }
.lead-media:hover img { transform: scale(1.015); }
.lead-copy { display: flex; flex-direction: column; justify-content: center; padding: 34px 0 36px 44px; }
.lead-copy h2 { margin: 24px 0 22px; font-size: 40px; line-height: 1.14; }
.lead-copy h2 a:hover { color: var(--text); text-decoration: underline; text-decoration-color: var(--olive); text-underline-offset: 7px; }
.lead-copy > p { max-width: 580px; margin: 0; color: var(--muted); font-size: 20px; line-height: 1.5; }
.read-link { width: fit-content; margin-top: 30px; padding: 10px 15px; font-size: 15px; }
.read-link span { margin-left: 8px; }
.article-meta { display: flex; align-items: center; flex-wrap: wrap; gap: 0; color: var(--muted); font-size: 14px; }
.article-meta > * { display: inline-flex; align-items: center; }
.article-meta > * + *::before { content: ""; width: 1px; height: 18px; margin: 0 16px; background: var(--border-strong); }
.article-meta .category-link { color: var(--olive); }
.article-meta .producer-link { color: var(--red); }
.feed-list { width: 100%; }
.feed-list-top { border-top: 1px solid var(--border-strong); }
.feed-row {
display: grid;
grid-template-columns: 350px minmax(0, 1fr);
gap: 34px;
padding: 22px 0;
border-bottom: 1px solid var(--border);
}
.feed-row-media { aspect-ratio: 16 / 9; overflow: hidden; background: var(--surface); }
.feed-row-media img, .feed-row-media video { width: 100%; height: 100%; object-fit: cover; transition: transform .35s ease; }
.feed-row-media:hover img { transform: scale(1.02); }
.feed-row-copy { align-self: center; min-width: 0; }
.feed-row h2 { margin: 16px 0 8px; font-size: 25px; line-height: 1.2; }
.feed-row h2 a:hover { color: var(--text); text-decoration: underline; text-decoration-color: var(--olive); text-underline-offset: 5px; }
.feed-row p { max-width: 760px; margin: 0; color: var(--muted); font-size: 17px; line-height: 1.5; }
.preview-social { margin-top: 14px; }
.media-placeholder { display: grid; width: 100%; height: 100%; place-items: center; background: var(--surface); }
.media-placeholder img { width: 76px; height: 76px; opacity: .28; object-fit: contain; }
.feed-media { position: relative; }
.feed-media-feature, .feed-media-feature > img, .feed-media-feature > .media-placeholder { display: block; width: 100%; height: 100%; }
.feed-media-track { display: flex; width: 100%; height: 100%; overflow-x: auto; overflow-y: hidden; scroll-snap-type: x mandatory; scrollbar-width: none; }
.feed-media-track::-webkit-scrollbar { display: none; }
.feed-media-slide { flex: 0 0 100%; width: 100%; height: 100%; scroll-snap-align: start; }
.feed-media-slide > a, .feed-media-slide img, .feed-media-slide video { display: block; width: 100%; height: 100%; }
.feed-media-slide img, .feed-media-slide video { object-fit: cover; }
.feed-media-count { position: absolute; top: 10px; right: 10px; z-index: 2; padding: 4px 8px; border-radius: 999px; background: rgba(0,0,0,.68); color: #fff; font-size: 12px; line-height: 1; pointer-events: none; }
.pagination { display: grid; grid-template-columns: 1fr auto 1fr; align-items: center; min-height: 90px; font-size: 15px; }
.pagination > :last-child { justify-self: end; }
.pagination span { color: var(--faint); }
.pagination .older-posts { grid-column: 3; justify-self: end; }
.pagination .newer-posts { grid-column: 1; }
.pagination .page-number { grid-column: 2; grid-row: 1; }
.home-seo {
max-width: 960px;
margin: 54px auto 0;
padding: 40px 0 0;
border-top: 1px solid var(--border-strong);
}
.home-seo h2 { margin: 0 0 24px; font-size: 34px; line-height: 1.15; }
.home-seo p { margin: 0 0 18px; color: var(--muted); font-size: 17px; line-height: 1.7; }
.home-seo p:last-child { margin-bottom: 0; }
.taxonomy-heading { padding-bottom: 34px; }
.taxonomy-heading > p { margin: 0 0 10px; color: var(--olive); font-size: 15px; text-transform: uppercase; }
.taxonomy-heading h1 { font-size: 56px; }
.article-page { padding-top: 56px; }
.article-header { max-width: 960px; margin: 0 auto 42px; }
.article-breadcrumbs { display: flex; gap: 12px; margin-bottom: 28px; color: var(--muted); font-size: 14px; }
.article-breadcrumbs span { color: var(--faint); }
.article-header h1 { margin: 0; font-size: 58px; line-height: 1.06; }
.article-lead { max-width: 850px; margin: 26px 0 0; color: var(--muted); font-size: 23px; line-height: 1.45; }
.article-byline { display: flex; gap: 24px; margin-top: 28px; color: var(--muted); font-size: 14px; }
.article-byline a { color: var(--red); }
.article-media { display: grid; gap: 12px; margin-bottom: 46px; }
.article-media figure { margin: 0; background: var(--surface); }
.article-media img, .article-media video { width: 100%; max-height: 760px; object-fit: contain; }
.article-layout { display: grid; grid-template-columns: minmax(0, 760px) 260px; justify-content: center; gap: 78px; }
.article-body { font-size: 21px; line-height: 1.7; }
.article-body p { margin: 0 0 26px; }
.gh-content > * { max-width: 100%; }
.gh-content a { color: var(--olive); text-decoration: underline; text-underline-offset: 3px; }
.gh-content figure, .gh-content .kg-card { margin: 32px 0; }
.gh-content img, .gh-content video { width: 100%; height: auto; }
.gh-content blockquote { margin: 30px 0; padding-left: 24px; border-left: 3px solid var(--olive); color: var(--muted); }
.kg-width-wide, .kg-width-full { width: 100%; }
.kg-gallery-container { display: flex; flex-direction: column; gap: 10px; }
.kg-gallery-row { display: flex; gap: 10px; }
.kg-gallery-image { flex: 1 1 0; min-width: 0; margin: 0; overflow: hidden; background: var(--surface); }
.kg-gallery-image img { width: 100%; height: 100%; margin: 0; object-fit: cover; cursor: zoom-in; }
.media-lightbox { width: 100%; height: 100%; max-width: none; max-height: none; padding: 0; border: 0; background: rgba(0,0,0,.94); }
.media-lightbox::backdrop { background: rgba(0,0,0,.94); }
.media-lightbox img { width: 100%; height: 100%; object-fit: contain; }
.media-lightbox button { position: fixed; top: 18px; right: 18px; z-index: 2; width: 44px; height: 44px; border: 1px solid var(--border-strong); background: var(--bg); color: var(--text); font-size: 28px; cursor: pointer; }
.article-facts { align-self: start; border-top: 2px solid var(--olive); }
.article-facts > div { padding: 18px 0; border-bottom: 1px solid var(--border); }
.article-facts span { display: block; margin-bottom: 4px; color: var(--faint); font-size: 13px; text-transform: uppercase; }
.article-facts a { font-size: 17px; }
.article-facts .source-link { display: block; margin-top: 22px; color: var(--muted); font-size: 14px; }
.social-links { display: flex; align-items: center; flex-wrap: wrap; gap: 8px; margin-top: 13px; }
.social-link { display: inline-flex; min-height: 34px; align-items: center; gap: 7px; padding: 5px 10px; border: 1px solid var(--border); color: var(--muted); font-size: 14px; line-height: 1; transition: border-color .18s ease, color .18s ease, background .18s ease; }
.social-link:hover { border-color: var(--border-strong); background: var(--surface-strong); color: var(--text); }
.social-link svg { width: 18px; height: 18px; fill: currentColor; }
.social-link--vk:hover { color: #7ca8e8; }
.social-link--tg:hover { color: #58b9e8; }
.social-links--compact { gap: 4px; margin-top: 0; }
.social-links--compact .social-link { width: 34px; min-height: 34px; justify-content: center; padding: 0; }
.social-links--compact .social-link svg { width: 17px; height: 17px; }
.about-social { margin-top: 42px; padding-top: 24px; border-top: 1px solid var(--border-strong); }
.about-social > span { color: var(--faint); font-size: 13px; text-transform: uppercase; }
.related-section { margin-top: 90px; border-top: 1px solid var(--border-strong); }
.related-section > h2 { margin: 0; padding: 34px 0 18px; font-size: 32px; }
.empty-state, .not-found { min-height: 56vh; display: flex; flex-direction: column; justify-content: center; align-items: flex-start; }
.empty-state h2, .not-found h1 { margin: 0; font-size: 42px; }
.empty-state p { color: var(--muted); }
.empty-state a { margin-top: 16px; padding: 10px 15px; }
.not-found > p { margin: 0; color: var(--red); font-size: 18px; }
.not-found a { margin-top: 24px; color: var(--olive); }
.site-footer {
display: flex;
justify-content: space-between;
align-items: flex-end;
margin-top: 84px;
padding: 34px 0 46px;
border-top: 1px solid var(--border-strong);
color: var(--muted);
font-size: 14px;
}
.footer-brand { color: var(--text); font-size: 20px; font-weight: 700; }
.site-footer p { margin: 7px 0 0; }
.site-footer nav { display: flex; gap: 24px; }
.site-footer .social-links { margin-top: 18px; }
@media (max-width: 980px) {
.header-inner { grid-template-columns: 150px 1fr auto; gap: 16px; }
.main-nav { gap: 22px; }
.main-nav a { font-size: 14px; }
.page-heading h1 { font-size: 56px; }
.lead-story { grid-template-columns: 1.1fr .9fr; min-height: 420px; }
.lead-media { min-height: 420px; }
.lead-copy { padding-left: 30px; }
.lead-copy h2 { font-size: 33px; }
.feed-row { grid-template-columns: 290px minmax(0, 1fr); gap: 26px; }
.article-layout { grid-template-columns: minmax(0, 680px) 220px; gap: 44px; }
}
@media (max-width: 760px) {
:root { --gutter: 16px; }
body { font-size: 17px; }
.home-seo { margin-top: 38px; padding-top: 30px; }
.home-seo h2 { font-size: 28px; }
.home-seo p { font-size: 16px; line-height: 1.65; }
.header-inner { min-height: 64px; grid-template-columns: 1fr auto; }
.brand img { width: 38px; height: 38px; }
.main-nav {
position: absolute;
top: 64px;
right: 0;
left: 0;
display: none;
padding: 12px var(--gutter) 18px;
border-bottom: 1px solid var(--border);
background: var(--bg);
}
.main-nav.is-open { display: grid; }
.main-nav a { min-height: 44px; border-bottom: 1px solid var(--border); font-size: 16px; }
.nav-dropdown { display: block; }
.nav-dropdown-toggle { width: 100%; min-height: 44px; border-bottom: 1px solid var(--border); text-align: left; font-size: 16px; }
.category-menu { position: static; width: 100%; grid-template-columns: 1fr; padding: 0 0 8px 14px; border: 0; box-shadow: none; transform: none; }
.category-menu a { font-size: 15px; }
.main-nav a[aria-current="page"]::after { display: none; }
.menu-button { display: grid; }
.page-heading { padding: 38px 0 30px; }
.page-heading h1, .taxonomy-heading h1 { font-size: 44px; line-height: 1; }
.lead-story { display: block; min-height: 0; max-height: none; }
.lead-media { min-height: 0; max-height: min(72vh, 560px); aspect-ratio: 4 / 3; }
.lead-copy { padding: 25px 0 34px; }
.lead-copy h2 { margin: 20px 0 15px; font-size: 32px; }
.lead-copy > p { font-size: 18px; }
.feed-row { display: block; padding: 24px 0; }
.feed-row-media { width: 100%; max-height: min(72vh, 560px); aspect-ratio: 4 / 3; }
.feed-row-copy { padding-top: 16px; }
.feed-row h2 { margin: 10px 0 8px; font-size: 24px; }
.feed-row p { display: block; font-size: 16px; }
.preview-social { margin-top: 10px; }
.preview-social .social-link { width: 30px; min-height: 30px; }
.article-meta { font-size: 12px; }
.article-meta time { display: none; }
.article-meta > * + *::before { height: 14px; margin: 0 10px; }
.article-meta .category-link::before { display: none; }
.article-header { margin-bottom: 30px; }
.article-header h1 { font-size: 42px; }
.article-lead { font-size: 20px; }
.article-layout { display: block; }
.article-body { font-size: 19px; }
.article-facts { margin-top: 42px; }
.related-section { margin-top: 64px; }
}
@media (max-width: 460px) {
.feed-row h2 { font-size: 22px; line-height: 1.2; }
.article-meta .producer-link::before { display: none; }
.article-meta .category-link { display: none; }
.article-header h1 { font-size: 36px; }
.article-byline { display: grid; gap: 7px; }
.site-footer { display: grid; gap: 24px; }
}
@media (prefers-reduced-motion: reduce) {
html { scroll-behavior: auto; }
*, *::before, *::after { transition-duration: .01ms !important; }
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

@@ -0,0 +1,86 @@
(() => {
document.querySelectorAll('[data-feed-media]').forEach((container) => {
const source = container.querySelector('.feed-media-source');
const feature = container.querySelector('.feed-media-feature');
if (!source || !feature) return;
const videos = [...source.querySelectorAll('video')].map((item) => {
const video = item.cloneNode(true);
video.removeAttribute('poster');
video.controls = true;
video.playsInline = true;
video.preload = 'metadata';
return video;
});
const images = [...source.querySelectorAll('.kg-gallery-card img, .kg-image-card img')].map((item) => {
const link = document.createElement('a');
link.href = item.currentSrc || item.src;
link.setAttribute('aria-label', item.alt ? `Открыть: ${item.alt}` : 'Открыть изображение');
link.appendChild(item.cloneNode(true));
return link;
});
const items = videos.length ? [...videos, ...images] : [feature, ...images];
if (items.length === 1 && !videos.length) return;
const track = document.createElement('div');
track.className = 'feed-media-track';
items.forEach((item) => {
const slide = document.createElement('div');
slide.className = 'feed-media-slide';
slide.appendChild(item);
track.appendChild(slide);
});
container.replaceChildren(track, source);
if (items.length > 1) {
const count = document.createElement('span');
count.className = 'feed-media-count';
count.textContent = `1 / ${items.length}`;
container.appendChild(count);
track.addEventListener('scroll', () => {
const index = Math.round(track.scrollLeft / Math.max(track.clientWidth, 1));
count.textContent = `${Math.min(index + 1, items.length)} / ${items.length}`;
}, { passive: true });
}
});
})();
(() => {
const button = document.querySelector('[data-menu-toggle]');
const nav = document.querySelector('#main-nav');
if (!button || !nav) return;
button.addEventListener('click', () => {
const open = nav.classList.toggle('is-open');
button.setAttribute('aria-expanded', String(open));
});
})();
(() => {
const toggle = document.querySelector('[data-category-toggle]');
const menu = document.querySelector('[data-category-menu]');
if (!toggle || !menu) return;
const close = () => { menu.classList.remove('is-open'); toggle.setAttribute('aria-expanded', 'false'); };
toggle.addEventListener('click', (event) => {
event.stopPropagation();
const open = menu.classList.toggle('is-open');
toggle.setAttribute('aria-expanded', String(open));
});
document.addEventListener('click', (event) => { if (!menu.contains(event.target)) close(); });
document.addEventListener('keydown', (event) => { if (event.key === 'Escape') close(); });
})();
(() => {
const images = document.querySelectorAll('.gh-content .kg-gallery-image img');
if (!images.length || !('HTMLDialogElement' in window)) return;
const dialog = document.createElement('dialog');
dialog.className = 'media-lightbox';
dialog.innerHTML = '<button type="button" aria-label="Закрыть">×</button><img alt="">';
const preview = dialog.querySelector('img');
dialog.querySelector('button').addEventListener('click', () => dialog.close());
dialog.addEventListener('click', (event) => { if (event.target === dialog) dialog.close(); });
document.body.appendChild(dialog);
images.forEach((image) => image.addEventListener('click', () => {
preview.src = image.currentSrc || image.src;
preview.alt = image.alt || '';
dialog.showModal();
}));
})();
@@ -0,0 +1,81 @@
<!doctype html>
<html lang="ru">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- Google Tag Manager -->
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-MM4FR8LJ');</script>
<!-- End Google Tag Manager -->
{{#is "home"}}
<title>Forma N8 — все новости снаряжения в одном месте</title>
{{else}}
{{#is "post"}}
{{#post}}<title>Forma N8 — {{title}}</title>{{/post}}
{{else}}
{{#is "tag"}}
{{#tag}}<title>Forma N8 — {{name}}</title>{{/tag}}
{{else}}
<title>Forma N8 — {{meta_title}}</title>
{{/is}}
{{/is}}
{{/is}}
<link rel="icon" type="image/png" href="{{asset "images/favi.png"}}">
<link rel="stylesheet" href="{{asset "css/screen.css"}}">
{{ghost_head}}
</head>
<body class="{{body_class}}">
<!-- Google Tag Manager (noscript) -->
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-MM4FR8LJ"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<!-- End Google Tag Manager (noscript) -->
<a class="skip-link" href="#content">К содержанию</a>
<header class="site-header">
<div class="header-inner">
<a class="brand" href="{{@site.url}}" aria-label="Forma N8, главная">
<img src="{{asset "images/favi.png"}}" width="42" height="42" alt="">
<span>FORMA<br>N8</span>
</a>
<nav class="main-nav" id="main-nav" aria-label="Основная навигация">
<a href="{{@site.url}}">Лента</a>
<div class="nav-dropdown">
<button type="button" class="nav-dropdown-toggle" data-category-toggle aria-expanded="false">Категории <span class="nav-dropdown-icon" aria-hidden="true"></span></button>
<div class="category-menu" data-category-menu>
{{#get "tags" filter="visibility:public" limit="100" order="name asc"}}
{{#foreach tags}}<a href="{{url}}">{{name}}</a>{{/foreach}}
{{/get}}
</div>
</div>
<a href="{{@site.url}}/about/">О проекте</a>
</nav>
<div class="header-actions">
{{> "social-links" compact=true}}
<button class="icon-button" type="button" data-ghost-search aria-label="Открыть поиск" title="Поиск">
<svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="6.5"></circle><path d="m16 16 4 4"></path></svg>
</button>
<button class="icon-button menu-button" type="button" data-menu-toggle aria-expanded="false" aria-controls="main-nav" aria-label="Открыть меню" title="Меню">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M4 7h16M4 12h16M4 17h16"></path></svg>
</button>
</div>
</div>
</header>
<main id="content">{{{body}}}</main>
<footer class="site-footer">
<div><a class="footer-brand" href="{{@site.url}}">FORMA N8</a><p>Новости снаряжения и экипировки.</p>{{> "social-links"}}</div>
<nav aria-label="Служебная навигация"><a href="{{@site.url}}/rss/">RSS</a><a href="{{@site.url}}/sitemap.xml">Карта сайта</a></nav>
</footer>
<script src="{{asset "js/site.js"}}" defer></script>
<script>
(function(m,e,t,r,i,k,a){m[i]=m[i]||function(){(m[i].a=m[i].a||[]).push(arguments)};
m[i].l=1*new Date();for(var j=0;j<document.scripts.length;j++){if(document.scripts[j].src===r){return;}}
k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)})
(window,document,"script","https://mc.yandex.ru/metrika/tag.js","ym");
ym(110060335,"init",{clickmap:true,trackLinks:true,accurateTrackBounce:true,webvisor:true});
</script>
<noscript><div><img src="https://mc.yandex.ru/watch/110060335" style="position:absolute;left:-9999px" alt=""></div></noscript>
{{ghost_foot}}
</body>
</html>
@@ -0,0 +1,2 @@
{{!< default}}
<section class="feed-shell not-found"><p>{{statusCode}}</p><h1>{{message}}</h1><a href="{{@site.url}}">Вернуться в ленту</a></section>
@@ -0,0 +1,38 @@
{{!< default}}
<section class="feed-shell">
<header class="page-heading"><h1>Новости тактического снаряжения и экипировки</h1></header>
{{#foreach posts}}
{{#if @first}}
{{#is "home"}}
<article class="lead-story">
<div class="lead-media feed-media" data-feed-media>
<a class="feed-media-feature" href="{{url}}">{{#if feature_image}}<img src="{{img_url feature_image size="xl"}}" alt="{{#if feature_image_alt}}{{feature_image_alt}}{{else}}{{title}}{{/if}}">{{else}}<span class="media-placeholder"><img src="{{asset "images/favi.png"}}" alt=""></span>{{/if}}</a>
<div class="feed-media-source" hidden>{{content}}</div>
</div>
<div class="lead-copy">
<div class="article-meta">{{#if primary_tag}}<a class="category-link" href="{{primary_tag.url}}">{{primary_tag.name}}</a>{{/if}}<time datetime="{{date format="YYYY-MM-DD"}}">{{date format="DD.MM.YYYY"}}</time></div>
<h2><a href="{{url}}">{{title}}</a></h2>
<p>{{excerpt words="42"}}</p>
<a class="read-link" href="{{url}}">Читать новость <span aria-hidden="true">→</span></a>
<div class="preview-social">{{> "social-links" compact=true}}</div>
</div>
</article>
{{else}}
{{> "post-row"}}
{{/is}}
{{else}}
{{> "post-row"}}
{{/if}}
{{else}}
<div class="empty-state"><h2>Публикаций пока нет</h2><p>Лента скоро наполнится новостями.</p></div>
{{/foreach}}
{{> "pagination-ru"}}
{{#is "home"}}
<section class="home-seo" aria-labelledby="home-seo-title">
<h2 id="home-seo-title">О новостной ленте Forma N8</h2>
<p>Forma N8 — лента новостей о тактическом снаряжении, экипировке и EDC. Мы собираем новинки с десятков источников и публикуем в одном месте: плитоносцы и бронезащита, подсумки и разгрузочные системы, боевые брюки и рубашки, прицелы и оптика, тактические фонари, средства связи, медицина и ночные приборы.</p>
<p>В ленте — новинки российских и зарубежных производителей: Ars Arma, Wartech, SSO, Stich Profi, Giena Tactical, 5.45 Design, Red Cliff, VOIN, Sturmer, БАРС, SRVV, ANA Tactical, Mordor Tac, Centurion Gear, ENOTACTICAL, Black Buckle, MILGEAR, SKIF Armor, Atlant Armour, БронеЁж, R5 GEAR, VINDEX, JAEGER Equipment, Margul Gear, Gordeev Tactical, IGLA TAC, STRIX TAC, Колчуга, APEX, Пересвет, Гарсинг и многих других. Отслеживаем оптику Диполь, ARTELV и ARKON, гарнитуры VULCAN, Fidem и ОКБ Октава, фонари Nicron, патчи BAD GRINGO и RedWar Patch, ножи DAGGERR, Morpheus Knives и N.C.Custom.</p>
<p>Темы ленты охватывают весь комплект тактической экипировки: системы MOLLE и Molle-Minus, плиты стандартов Гранит, SAPI и ESAPI, ткани Cordura и NYCO, расцветки Multicam, Tiger Stripe и EMR. Новинки магазинов Академия Снаряжения, Semper Fi, ФИНИСТ и Империя ножей, обновления линеек, цены и отзывы пользователей — всё в одной новостной ленте о снаряжении.</p>
</section>
{{/is}}
</section>
@@ -0,0 +1,18 @@
{
"name": "forma-n8",
"description": "Strict editorial theme for Forma N8",
"version": "1.0.0",
"author": {
"name": "Forma N8",
"email": "admin@f-n8.ru"
},
"keywords": ["ghost-theme", "forma-n8", "news"],
"engines": {
"ghost": ">=6.0.0"
},
"license": "UNLICENSED",
"config": {
"posts_per_page": 25,
"card_assets": true
}
}
@@ -0,0 +1,2 @@
{{!< default}}
{{#post}}<article class="article-page">{{#if @page.show_title_and_feature_image}}<header class="article-header"><h1>{{title}}</h1>{{#if custom_excerpt}}<p class="article-lead">{{custom_excerpt}}</p>{{/if}}</header>{{#if feature_image}}<figure class="article-media"><img src="{{img_url feature_image size="xl"}}" alt="{{title}}"></figure>{{/if}}{{/if}}<div class="article-layout"><div class="article-body gh-content">{{content}}<aside class="about-social"><span>Forma N8 в социальных сетях</span>{{> "social-links"}}</aside></div></div></article>{{/post}}
@@ -0,0 +1,9 @@
{{#if pagination.pages}}
{{#match pagination.pages ">" 1}}
<nav class="pagination" role="navigation" aria-label="Пагинация">
{{#if pagination.prev}}<a class="newer-posts" href="{{page_url pagination.prev}}">← Новые публикации</a>{{/if}}
<span class="page-number">Страница {{pagination.page}} из {{pagination.pages}}</span>
{{#if pagination.next}}<a class="older-posts" href="{{page_url pagination.next}}">Старые публикации →</a>{{/if}}
</nav>
{{/match}}
{{/if}}
@@ -0,0 +1,12 @@
<article class="feed-row">
<div class="feed-row-media feed-media" data-feed-media>
<a class="feed-media-feature" href="{{url}}">{{#if feature_image}}<img src="{{img_url feature_image size="m"}}" alt="{{#if feature_image_alt}}{{feature_image_alt}}{{else}}{{title}}{{/if}}" loading="lazy">{{else}}<span class="media-placeholder"><img src="{{asset "images/favi.png"}}" alt=""></span>{{/if}}</a>
<div class="feed-media-source" hidden>{{content}}</div>
</div>
<div class="feed-row-copy">
<div class="article-meta">{{#if primary_tag}}<a class="category-link" href="{{primary_tag.url}}">{{primary_tag.name}}</a>{{/if}}<time datetime="{{date format="YYYY-MM-DD"}}">{{date format="DD.MM.YYYY"}}</time></div>
<h2><a href="{{url}}">{{title}}</a></h2>
<p>{{excerpt words="28"}}</p>
<div class="preview-social">{{> "social-links" compact=true}}</div>
</div>
</article>
@@ -0,0 +1,10 @@
<div class="social-links{{#if compact}} social-links--compact{{/if}}" aria-label="Forma N8 в социальных сетях">
<a class="social-link social-link--vk" href="https://vk.com/forma_n8" target="_blank" rel="noopener noreferrer" aria-label="Forma N8 во ВКонтакте" title="ВКонтакте">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12.8 17.6C5.4 17.6 1.2 12.5 1 4h3.7c.1 6.2 2.9 8.8 5.1 9.3V4h3.5v5.3c2.2-.2 4.5-2.6 5.3-5.3H22c-.6 3.3-3.1 5.7-4.9 6.7 1.8.8 4.7 2.9 5.8 6.9h-3.8c-.9-2.6-3.2-4.6-5.8-4.9v4.9h-.5Z"/></svg>
{{#unless compact}}<span>ВКонтакте</span>{{/unless}}
</a>
<a class="social-link social-link--tg" href="https://t.me/forma_n8" target="_blank" rel="noopener noreferrer" aria-label="Forma N8 в Telegram" title="Telegram">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M21.7 3.4 18.5 20.5c-.2 1.2-.9 1.5-1.9.9l-4.9-3.6-2.4 2.3c-.3.3-.5.5-1 .5l.3-5 9.1-8.2c.4-.4-.1-.6-.6-.2L5.9 14.3 1.1 12.8c-1-.3-1-1 .2-1.5L20 2.6c.9-.3 1.8.2 1.7.8Z"/></svg>
{{#unless compact}}<span>Telegram</span>{{/unless}}
</a>
</div>
@@ -0,0 +1,20 @@
{{!< default}}
{{#post}}
<article class="article-page">
<header class="article-header">
<div class="article-breadcrumbs"><a href="{{@site.url}}">Лента</a>{{#if primary_tag}}<span>/</span><a href="{{primary_tag.url}}">{{primary_tag.name}}</a>{{/if}}</div>
<h1>{{title}}</h1>
{{#if custom_excerpt}}<p class="article-lead">{{custom_excerpt}}</p>{{/if}}
<div class="article-byline"><time datetime="{{date format="YYYY-MM-DDTHH:mm:ssZ"}}">{{date format="DD.MM.YYYY · HH:mm"}} ЕКБ</time></div>
</header>
{{#if feature_image}}<figure class="article-media"><img src="{{img_url feature_image size="xl"}}" alt="{{#if feature_image_alt}}{{feature_image_alt}}{{else}}{{title}}{{/if}}"></figure>{{/if}}
<div class="article-layout">
<div class="article-body gh-content">{{content}}</div>
<aside class="article-facts">
{{#if primary_tag}}<div><span>Категория</span><a href="{{primary_tag.url}}">{{primary_tag.name}}</a></div>{{/if}}
<div><span>Опубликовано</span><time datetime="{{date format="YYYY-MM-DD"}}">{{date format="DD.MM.YYYY"}}</time></div>
<div><span>Наши каналы</span>{{> "social-links"}}</div>
</aside>
</div>
</article>
{{/post}}
@@ -0,0 +1,6 @@
{{!< default}}
<section class="feed-shell">
{{#tag}}<header class="page-heading taxonomy-heading"><p>Тема</p><h1>{{name}}</h1>{{#if description}}<p>{{description}}</p>{{/if}}</header>{{/tag}}
<div class="feed-list">{{#foreach posts}}{{> "post-row"}}{{/foreach}}</div>
{{> "pagination-ru"}}
</section>
+6
View File
@@ -0,0 +1,6 @@
fastapi==0.115.12
uvicorn[standard]==0.34.2
asyncpg==0.30.0
jinja2==3.1.6
python-multipart==0.0.20
pydantic-settings==2.9.1
+28
View File
@@ -0,0 +1,28 @@
import asyncio
import os
from pathlib import Path
import asyncpg
async def main() -> None:
root = Path(__file__).resolve().parents[1]
conn = await asyncpg.connect(os.environ["DATABASE_URL"])
try:
await conn.execute(
"CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW())"
)
for path in sorted((root / "db" / "migrations").glob("*.sql")):
exists = await conn.fetchval("SELECT 1 FROM schema_migrations WHERE version=$1", path.name)
if exists:
continue
async with conn.transaction():
await conn.execute(path.read_text(encoding="utf-8"))
await conn.execute("INSERT INTO schema_migrations(version) VALUES($1)", path.name)
print(f"applied {path.name}")
finally:
await conn.close()
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1 @@
"""Forma N8 public website."""
+448
View File
@@ -0,0 +1,448 @@
import hashlib
import hmac
import json
from contextlib import asynccontextmanager
from datetime import datetime, timezone
from pathlib import Path
from zoneinfo import ZoneInfo
from xml.etree import ElementTree as ET
from fastapi import Depends, FastAPI, File, Header, HTTPException, Query, Request, UploadFile
from fastapi.responses import FileResponse, HTMLResponse, PlainTextResponse, Response
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from .config import settings
from .db import connect, disconnect, get_pool
from .schemas import ArticleUpsert
from .text import excerpt, paragraphs, slugify
ROOT = Path(__file__).resolve().parents[2]
STATIC_DIR = ROOT / "static"
TEMPLATE_DIR = ROOT / "templates"
ALLOWED_UPLOADS = {
"image/jpeg": ".jpg",
"image/png": ".png",
"image/webp": ".webp",
"image/gif": ".gif",
"video/mp4": ".mp4",
"video/webm": ".webm",
}
@asynccontextmanager
async def lifespan(_: FastAPI):
settings.upload_dir.mkdir(parents=True, exist_ok=True)
await connect()
yield
await disconnect()
app = FastAPI(title=settings.site_name, docs_url=None, redoc_url=None, lifespan=lifespan)
app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
app.mount("/media", StaticFiles(directory=settings.upload_dir, check_dir=False), name="media")
templates = Jinja2Templates(directory=TEMPLATE_DIR)
templates.env.filters["paragraphs"] = paragraphs
templates.env.filters["excerpt"] = excerpt
LOCAL_TZ = ZoneInfo("Asia/Yekaterinburg")
templates.env.filters["ekb_date"] = lambda value: value.astimezone(LOCAL_TZ).strftime("%d.%m.%Y") if value else ""
templates.env.filters["ekb_time"] = lambda value: value.astimezone(LOCAL_TZ).strftime("%H:%M") if value else ""
def site_url(path: str = "") -> str:
return f"{settings.base_url}{path if path.startswith('/') else '/' + path if path else ''}"
def article_path(article: dict) -> str:
return f"/news/{article['slug']}"
templates.env.globals.update(
site_name=settings.site_name,
site_tagline=settings.site_tagline,
site_url=site_url,
article_path=article_path,
)
@app.middleware("http")
async def security_headers(request: Request, call_next):
response = await call_next(request)
response.headers.setdefault("X-Content-Type-Options", "nosniff")
response.headers.setdefault("X-Frame-Options", "DENY")
response.headers.setdefault("Referrer-Policy", "strict-origin-when-cross-origin")
response.headers.setdefault("Permissions-Policy", "camera=(), microphone=(), geolocation=()")
return response
def base_context(request: Request, **extra):
return {
"request": request,
"current_path": request.url.path,
"canonical_url": site_url(request.url.path),
"meta_title": f"{settings.site_tagline} — {settings.site_name}",
"meta_description": "Новости производителей тактического снаряжения, одежды, брони и аксессуаров.",
"noindex": not settings.site_indexing_enabled or bool(request.query_params.get("q")),
**extra,
}
ARTICLE_COLUMNS = """
a.*,
COALESCE(
jsonb_agg(
jsonb_build_object(
'type', m.media_type,
'url', m.url,
'alt', m.alt_text,
'width', m.width,
'height', m.height
) ORDER BY m.sort_order, m.id
) FILTER (WHERE m.id IS NOT NULL),
'[]'::jsonb
) AS media
"""
def normalize_article(row) -> dict:
article = dict(row)
media = article.get("media") or []
if isinstance(media, str):
try:
media = json.loads(media)
except json.JSONDecodeError:
media = []
article["media"] = media if isinstance(media, list) else []
return article
async def fetch_feed(*, page: int, q: str = "", category: str = "", producer: str = "", page_size: int | None = None):
pool = await get_pool()
where = ["a.status='published'", "a.published_at <= NOW()"]
args: list[object] = []
if q:
args.append(f"%{q}%")
where.append(f"(a.title ILIKE ${len(args)} OR a.lead ILIKE ${len(args)} OR a.body ILIKE ${len(args)} OR a.producer_name ILIKE ${len(args)})")
if category:
args.append(category)
where.append(f"a.category_tag=${len(args)}")
if producer:
args.append(producer)
where.append(f"a.producer_tag=${len(args)}")
where_sql = " AND ".join(where)
total = int(await pool.fetchval(f"SELECT COUNT(*) FROM articles a WHERE {where_sql}", *args) or 0)
limit = max(1, min(50, page_size or settings.page_size))
offset = (page - 1) * limit
rows = await pool.fetch(
f"""
SELECT {ARTICLE_COLUMNS}
FROM articles a
LEFT JOIN article_media m ON m.article_id=a.id
WHERE {where_sql}
GROUP BY a.id
ORDER BY a.published_at DESC, a.id DESC
LIMIT ${len(args) + 1} OFFSET ${len(args) + 2}
""",
*args,
limit,
offset,
)
return [normalize_article(row) for row in rows], total, limit
@app.get("/", response_class=HTMLResponse)
async def feed(request: Request, page: int = Query(1, ge=1), q: str = Query("", max_length=100)):
articles, total, limit = await fetch_feed(page=page, q=q.strip())
pages = max(1, (total + limit - 1) // limit)
canonical = site_url("/") if page == 1 else site_url(f"/?page={page}")
return templates.TemplateResponse(
"feed.html",
base_context(
request,
articles=articles,
page=page,
pages=pages,
total=total,
q=q.strip(),
canonical_url=canonical,
),
)
@app.get("/category/{tag}", response_class=HTMLResponse)
async def category_feed(request: Request, tag: str, page: int = Query(1, ge=1)):
pool = await get_pool()
category_name = await pool.fetchval("SELECT name FROM categories WHERE tag=$1 AND is_active=TRUE", tag)
if not category_name:
raise HTTPException(status_code=404)
articles, total, limit = await fetch_feed(page=page, category=tag)
name = str(category_name)
pages = max(1, (total + limit - 1) // limit)
return templates.TemplateResponse(
"taxonomy.html",
base_context(
request,
articles=articles,
page=page,
pages=pages,
heading=name,
taxonomy_label="Категория",
canonical_url=site_url(f"/category/{tag}" + (f"?page={page}" if page > 1 else "")),
meta_title=f"{name}: новости — {settings.site_name}",
meta_description=f"Последние новости в категории «{name}»: производители, новинки и обзоры снаряжения.",
noindex=total == 0,
),
)
@app.get("/brand/{tag}", response_class=HTMLResponse)
async def producer_feed(request: Request, tag: str, page: int = Query(1, ge=1)):
articles, total, limit = await fetch_feed(page=page, producer=tag)
if not articles and page == 1:
raise HTTPException(status_code=404)
name = articles[0]["producer_name"] if articles else tag
pages = max(1, (total + limit - 1) // limit)
return templates.TemplateResponse(
"taxonomy.html",
base_context(
request,
articles=articles,
page=page,
pages=pages,
heading=name,
taxonomy_label="Производитель",
canonical_url=site_url(f"/brand/{tag}" + (f"?page={page}" if page > 1 else "")),
meta_title=f"{name}: новости и новинки — {settings.site_name}",
meta_description=f"Новости и новинки производителя {name}: снаряжение, одежда и экипировка.",
),
)
@app.get("/news/{slug}", response_class=HTMLResponse)
async def article(request: Request, slug: str):
pool = await get_pool()
row = await pool.fetchrow(
f"""
SELECT {ARTICLE_COLUMNS}
FROM articles a
LEFT JOIN article_media m ON m.article_id=a.id
WHERE a.slug=$1 AND a.status='published' AND a.published_at <= NOW()
GROUP BY a.id
""",
slug,
)
if not row:
raise HTTPException(status_code=404)
item = normalize_article(row)
related = await pool.fetch(
f"""
SELECT {ARTICLE_COLUMNS}
FROM articles a
LEFT JOIN article_media m ON m.article_id=a.id
WHERE a.status='published' AND a.id<>$1
AND (a.category_tag=$2 OR a.producer_tag=$3)
GROUP BY a.id
ORDER BY (a.producer_tag=$3) DESC, a.published_at DESC
LIMIT 3
""",
item["id"],
item["category_tag"],
item["producer_tag"],
)
canonical = site_url(article_path(item))
image = item["media"][0]["url"] if item["media"] else site_url("/static/favi.png")
if image.startswith("/"):
image = site_url(image)
json_ld = {
"@context": "https://schema.org",
"@type": "NewsArticle",
"headline": item["title"],
"description": item["seo_description"] or item["lead"] or excerpt(item["body"]),
"datePublished": item["published_at"].isoformat(),
"dateModified": item["updated_at"].isoformat(),
"mainEntityOfPage": canonical,
"image": [image],
"author": {"@type": "Organization", "name": settings.site_name},
"publisher": {
"@type": "Organization",
"name": settings.site_name,
"logo": {"@type": "ImageObject", "url": site_url("/static/favi.png")},
},
}
return templates.TemplateResponse(
"article.html",
base_context(
request,
article=item,
related=[normalize_article(x) for x in related],
canonical_url=canonical,
meta_title=item["seo_title"] or f"{item['title']} — {settings.site_name}",
meta_description=item["seo_description"] or item["lead"] or excerpt(item["body"]),
og_image=image,
json_ld=json.dumps(json_ld, ensure_ascii=False),
),
)
def require_api_token(x_forma_token: str = Header(default="")) -> None:
if not settings.publish_api_token or not hmac.compare_digest(x_forma_token, settings.publish_api_token):
raise HTTPException(status_code=401, detail="invalid token")
@app.post("/api/v1/media", dependencies=[Depends(require_api_token)])
async def upload_media(file: UploadFile = File(...)):
content_type = (file.content_type or "").split(";", 1)[0].lower()
if content_type not in ALLOWED_UPLOADS:
raise HTTPException(status_code=415, detail="unsupported media type")
data = await file.read(50 * 1024 * 1024 + 1)
if len(data) > 50 * 1024 * 1024:
raise HTTPException(status_code=413, detail="file too large")
digest = hashlib.sha256(data).hexdigest()
suffix = ALLOWED_UPLOADS[content_type]
target = settings.upload_dir / digest[:2] / f"{digest}{suffix}"
target.parent.mkdir(parents=True, exist_ok=True)
if not target.exists():
target.write_bytes(data)
return {"url": f"/media/{digest[:2]}/{target.name}", "sha256": digest, "content_type": content_type}
@app.post("/api/v1/articles", dependencies=[Depends(require_api_token)])
async def upsert_article(payload: ArticleUpsert):
pool = await get_pool()
existing = await pool.fetchrow("SELECT id, slug, published_at FROM articles WHERE external_id=$1", payload.external_id)
requested_slug = slugify(payload.slug or payload.title)
stable_slug = existing["slug"] if existing else f"{requested_slug}-{slugify(payload.external_id)[-24:]}"
published_at = payload.published_at or datetime.now(timezone.utc)
category_tag = slugify(payload.category_tag)
category_name = await pool.fetchval("SELECT name FROM categories WHERE tag=$1 AND is_active=TRUE", category_tag)
category_name = str(category_name or payload.category_name).strip()
async with pool.acquire() as conn:
async with conn.transaction():
row = await conn.fetchrow(
"""
INSERT INTO articles(
external_id, slug, title, lead, body, category_name, category_tag,
producer_name, producer_tag, source_url, seo_title, seo_description,
status, published_at
)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14)
ON CONFLICT(external_id) DO UPDATE SET
title=EXCLUDED.title,
lead=EXCLUDED.lead,
body=EXCLUDED.body,
category_name=EXCLUDED.category_name,
category_tag=EXCLUDED.category_tag,
producer_name=EXCLUDED.producer_name,
producer_tag=EXCLUDED.producer_tag,
source_url=EXCLUDED.source_url,
seo_title=EXCLUDED.seo_title,
seo_description=EXCLUDED.seo_description,
status=EXCLUDED.status,
published_at=COALESCE(articles.published_at, EXCLUDED.published_at),
updated_at=NOW()
RETURNING id, slug, status, published_at, updated_at
""",
payload.external_id,
stable_slug,
payload.title.strip(),
payload.lead.strip(),
payload.body.strip(),
category_name,
category_tag,
payload.producer_name.strip(),
slugify(payload.producer_tag),
str(payload.source_url) if payload.source_url else None,
(payload.seo_title or payload.title).strip(),
(payload.seo_description or payload.lead or excerpt(payload.body)).strip(),
payload.status,
published_at,
)
await conn.execute("DELETE FROM article_media WHERE article_id=$1", row["id"])
for index, media in enumerate(payload.media):
await conn.execute(
"""
INSERT INTO article_media(article_id, media_type, url, alt_text, width, height, sort_order)
VALUES($1,$2,$3,$4,$5,$6,$7)
""",
row["id"], media.type, media.url, media.alt or payload.title,
media.width, media.height, media.sort_order or index,
)
return {
"id": row["id"],
"external_id": payload.external_id,
"status": row["status"],
"url": site_url(f"/news/{row['slug']}") if row["status"] == "published" else None,
"published_at": row["published_at"],
"updated_at": row["updated_at"],
}
@app.get("/rss.xml")
async def rss():
articles, _, _ = await fetch_feed(page=1, page_size=50)
root = ET.Element("rss", {"version": "2.0", "xmlns:media": "http://search.yahoo.com/mrss/"})
channel = ET.SubElement(root, "channel")
ET.SubElement(channel, "title").text = f"{settings.site_name} — {settings.site_tagline}"
ET.SubElement(channel, "link").text = settings.base_url
ET.SubElement(channel, "description").text = "Новости производителей тактического снаряжения и экипировки."
ET.SubElement(channel, "language").text = "ru"
for item in articles[:50]:
node = ET.SubElement(channel, "item")
url = site_url(article_path(item))
ET.SubElement(node, "title").text = item["title"]
ET.SubElement(node, "link").text = url
ET.SubElement(node, "guid", {"isPermaLink": "true"}).text = url
ET.SubElement(node, "description").text = item["lead"] or excerpt(item["body"])
ET.SubElement(node, "pubDate").text = item["published_at"].strftime("%a, %d %b %Y %H:%M:%S +0000")
ET.SubElement(node, "category").text = item["category_name"]
if item["media"]:
ET.SubElement(node, "media:content", {"url": site_url(item["media"][0]["url"]) if item["media"][0]["url"].startswith("/") else item["media"][0]["url"], "medium": "image"})
return Response(ET.tostring(root, encoding="utf-8", xml_declaration=True), media_type="application/rss+xml")
@app.get("/sitemap.xml")
async def sitemap():
pool = await get_pool()
rows = await pool.fetch("SELECT slug, updated_at FROM articles WHERE status='published' ORDER BY id")
categories = await pool.fetch("SELECT category_tag AS tag, MAX(updated_at) AS updated_at FROM articles WHERE status='published' GROUP BY category_tag")
producers = await pool.fetch("SELECT producer_tag AS tag, MAX(updated_at) AS updated_at FROM articles WHERE status='published' GROUP BY producer_tag")
root = ET.Element("urlset", {"xmlns": "http://www.sitemaps.org/schemas/sitemap/0.9"})
entries = [(site_url("/"), None)]
entries += [(site_url(f"/category/{row['tag']}"), row["updated_at"]) for row in categories]
entries += [(site_url(f"/brand/{row['tag']}"), row["updated_at"]) for row in producers]
entries += [(site_url(f"/news/{row['slug']}"), row["updated_at"]) for row in rows]
for loc, updated in entries:
node = ET.SubElement(root, "url")
ET.SubElement(node, "loc").text = loc
if updated:
ET.SubElement(node, "lastmod").text = updated.date().isoformat()
return Response(ET.tostring(root, encoding="utf-8", xml_declaration=True), media_type="application/xml")
@app.get("/robots.txt", response_class=PlainTextResponse)
async def robots():
if not settings.site_indexing_enabled:
return "User-agent: *\nDisallow: /\n"
return f"User-agent: *\nAllow: /\nDisallow: /api/\nSitemap: {site_url('/sitemap.xml')}\n"
@app.get("/favicon.ico", include_in_schema=False)
async def favicon():
return FileResponse(STATIC_DIR / "favi.png", media_type="image/png")
@app.get("/health")
async def health():
pool = await get_pool()
await pool.fetchval("SELECT 1")
return {"status": "ok"}
@app.exception_handler(404)
async def not_found(request: Request, _):
return templates.TemplateResponse(
"404.html",
base_context(request, meta_title=f"Страница не найдена — {settings.site_name}", noindex=True),
status_code=404,
)
+23
View File
@@ -0,0 +1,23 @@
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
database_url: str = "postgresql://forma_n8:forma_n8@127.0.0.1:5432/forma_n8"
site_base_url: str = "http://127.0.0.1:8090"
site_name: str = "Forma N8"
site_tagline: str = "Новости снаряжения"
site_indexing_enabled: bool = False
publish_api_token: str = ""
upload_dir: Path = Path("/var/lib/forma-n8/uploads")
page_size: int = 12
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
@property
def base_url(self) -> str:
return self.site_base_url.rstrip("/")
settings = Settings()
+24
View File
@@ -0,0 +1,24 @@
import asyncpg
from .config import settings
pool: asyncpg.Pool | None = None
async def connect() -> asyncpg.Pool:
global pool
if pool is None:
pool = await asyncpg.create_pool(settings.database_url, min_size=1, max_size=10, command_timeout=30)
return pool
async def disconnect() -> None:
global pool
if pool is not None:
await pool.close()
pool = None
async def get_pool() -> asyncpg.Pool:
return pool or await connect()
@@ -0,0 +1,31 @@
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, Field, HttpUrl
class MediaInput(BaseModel):
type: Literal["image", "video"] = "image"
url: str = Field(min_length=1, max_length=2000)
alt: str = Field(default="", max_length=500)
width: int | None = Field(default=None, ge=1)
height: int | None = Field(default=None, ge=1)
sort_order: int = Field(default=0, ge=0)
class ArticleUpsert(BaseModel):
external_id: str = Field(min_length=1, max_length=160)
slug: str | None = Field(default=None, max_length=220)
title: str = Field(min_length=5, max_length=300)
lead: str = Field(default="", max_length=1000)
body: str = Field(min_length=20, max_length=50000)
category_name: str = Field(min_length=1, max_length=120)
category_tag: str = Field(min_length=1, max_length=120)
producer_name: str = Field(min_length=1, max_length=200)
producer_tag: str = Field(min_length=1, max_length=160)
source_url: HttpUrl | None = None
seo_title: str | None = Field(default=None, max_length=300)
seo_description: str | None = Field(default=None, max_length=500)
status: Literal["draft", "published"] = "published"
published_at: datetime | None = None
media: list[MediaInput] = Field(default_factory=list, max_length=20)
+37
View File
@@ -0,0 +1,37 @@
import re
from html import escape
from markupsafe import Markup
TRANSLIT = str.maketrans(
{
"а": "a", "б": "b", "в": "v", "г": "g", "д": "d", "е": "e", "ё": "e",
"ж": "zh", "з": "z", "и": "i", "й": "y", "к": "k", "л": "l", "м": "m",
"н": "n", "о": "o", "п": "p", "р": "r", "с": "s", "т": "t", "у": "u",
"ф": "f", "х": "h", "ц": "c", "ч": "ch", "ш": "sh", "щ": "sch",
"ъ": "", "ы": "y", "ь": "", "э": "e", "ю": "yu", "я": "ya",
}
)
def slugify(value: str) -> str:
value = value.strip().lower().translate(TRANSLIT)
value = re.sub(r"[^a-z0-9]+", "-", value).strip("-")
return value[:180] or "article"
def paragraphs(value: str) -> Markup:
blocks = []
for part in re.split(r"\n\s*\n", value.strip()):
lines = "<br>".join(escape(line.strip()) for line in part.splitlines() if line.strip())
if lines:
blocks.append(f"<p>{lines}</p>")
return Markup("".join(blocks))
def excerpt(value: str, limit: int = 180) -> str:
compact = re.sub(r"\s+", " ", value or "").strip()
if len(compact) <= limit:
return compact
return compact[: limit - 1].rsplit(" ", 1)[0] + "…"
Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

+348
View File
@@ -0,0 +1,348 @@
@font-face {
font-family: "Roboto Condensed";
src: url("/static/fonts/roboto-condensed-cyrillic-400.woff2") format("woff2");
font-weight: 400;
font-display: swap;
}
@font-face {
font-family: "Roboto Condensed";
src: url("/static/fonts/roboto-condensed-cyrillic-700.woff2") format("woff2");
font-weight: 700;
font-display: swap;
}
:root {
color-scheme: dark;
--bg: #090c0d;
--surface: #101415;
--surface-strong: #151a1b;
--text: #eceee8;
--muted: #9ba19b;
--faint: #686f6b;
--border: #29302e;
--border-strong: #3a423f;
--olive: #a5ad62;
--red: #df4a40;
--max: 1240px;
--gutter: 24px;
--font: "Roboto Condensed", "Arial Narrow", Arial, sans-serif;
}
* { box-sizing: border-box; }
html { background: var(--bg); scroll-behavior: smooth; }
body {
margin: 0;
min-width: 320px;
background: var(--bg);
color: var(--text);
font-family: var(--font);
font-size: 18px;
line-height: 1.55;
letter-spacing: 0;
-webkit-font-smoothing: antialiased;
}
body::before {
content: "";
position: fixed;
inset: 0;
pointer-events: none;
opacity: .22;
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='120' height='120' viewBox='0 0 120 120'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='.8' numOctaves='2' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='.08'/%3E%3C/svg%3E");
}
a { color: inherit; text-decoration: none; }
a:hover { color: var(--olive); }
img, video { display: block; max-width: 100%; }
button, input { font: inherit; letter-spacing: 0; }
:focus-visible { outline: 2px solid var(--olive); outline-offset: 4px; }
.skip-link {
position: fixed;
z-index: 100;
top: 10px;
left: 10px;
transform: translateY(-160%);
padding: 10px 14px;
background: var(--text);
color: var(--bg);
}
.skip-link:focus { transform: none; }
.site-header {
position: sticky;
z-index: 20;
top: 0;
border-bottom: 1px solid var(--border);
background: rgba(9, 12, 13, .96);
backdrop-filter: blur(12px);
}
.header-inner {
width: min(var(--max), calc(100% - var(--gutter) * 2));
min-height: 76px;
margin: 0 auto;
display: grid;
grid-template-columns: 170px 1fr auto;
align-items: center;
gap: 32px;
}
.brand {
display: inline-flex;
width: fit-content;
align-items: center;
gap: 11px;
font-size: 15px;
font-weight: 700;
line-height: .95;
}
.brand img { width: 42px; height: 42px; border: 1px solid var(--border-strong); }
.main-nav { display: flex; justify-content: center; align-self: stretch; gap: 38px; }
.main-nav a {
position: relative;
display: inline-flex;
align-items: center;
color: var(--muted);
font-size: 16px;
white-space: nowrap;
}
.main-nav a:hover, .main-nav a[aria-current="page"] { color: var(--text); }
.main-nav a[aria-current="page"]::after {
content: "";
position: absolute;
right: 0;
bottom: -1px;
left: 0;
height: 3px;
background: var(--olive);
}
.header-actions { display: flex; gap: 8px; }
.icon-button {
display: grid;
width: 40px;
height: 40px;
padding: 0;
place-items: center;
border: 0;
background: transparent;
color: var(--text);
cursor: pointer;
}
.icon-button:hover { background: var(--surface-strong); color: var(--olive); }
.icon-button svg { width: 23px; height: 23px; fill: none; stroke: currentColor; stroke-width: 1.8; stroke-linecap: square; }
.menu-button { display: none; }
.search-form {
width: min(760px, calc(100% - var(--gutter) * 2));
margin: 0 auto;
padding: 16px 0 20px;
}
.search-form[hidden] { display: none; }
.search-form label { display: block; margin-bottom: 8px; color: var(--muted); font-size: 14px; }
.search-form > div { display: grid; grid-template-columns: 1fr auto; }
.search-form input {
min-width: 0;
height: 48px;
padding: 0 16px;
border: 1px solid var(--border-strong);
border-right: 0;
border-radius: 0;
background: var(--surface);
color: var(--text);
font-size: 17px;
}
.search-form button, .read-link, .empty-state a {
border: 1px solid var(--olive);
border-radius: 0;
background: transparent;
color: var(--text);
cursor: pointer;
}
.search-form button { padding: 0 24px; }
.search-form button:hover, .read-link:hover, .empty-state a:hover { background: var(--olive); color: var(--bg); }
.feed-shell, .article-page, .related-section, .site-footer {
position: relative;
width: min(var(--max), calc(100% - var(--gutter) * 2));
margin: 0 auto;
}
.page-heading { padding: 56px 0 46px; }
.page-heading h1 {
max-width: 900px;
margin: 0;
font-size: 68px;
font-weight: 700;
line-height: .98;
}
.page-heading > p { margin: 20px 0 0; color: var(--muted); }
.lead-story {
display: grid;
grid-template-columns: minmax(0, 1.25fr) minmax(340px, .95fr);
min-height: 480px;
border-bottom: 1px solid var(--border-strong);
}
.lead-media { min-height: 480px; overflow: hidden; background: var(--surface); }
.lead-media img, .lead-media video { width: 100%; height: 100%; object-fit: cover; transition: transform .45s ease; }
.lead-media:hover img { transform: scale(1.015); }
.lead-copy { display: flex; flex-direction: column; justify-content: center; padding: 34px 0 36px 44px; }
.lead-copy h2 { margin: 24px 0 22px; font-size: 40px; line-height: 1.14; }
.lead-copy h2 a:hover { color: var(--text); text-decoration: underline; text-decoration-color: var(--olive); text-underline-offset: 7px; }
.lead-copy > p { max-width: 580px; margin: 0; color: var(--muted); font-size: 20px; line-height: 1.5; }
.read-link { width: fit-content; margin-top: 30px; padding: 10px 15px; font-size: 15px; }
.read-link span { margin-left: 8px; }
.article-meta { display: flex; align-items: center; flex-wrap: wrap; gap: 0; color: var(--muted); font-size: 14px; }
.article-meta > * { display: inline-flex; align-items: center; }
.article-meta > * + *::before { content: ""; width: 1px; height: 18px; margin: 0 16px; background: var(--border-strong); }
.article-meta .category-link { color: var(--olive); }
.article-meta .producer-link { color: var(--red); }
.feed-list { width: 100%; }
.feed-list-top { border-top: 1px solid var(--border-strong); }
.feed-row {
display: grid;
grid-template-columns: 350px minmax(0, 1fr);
gap: 34px;
padding: 22px 0;
border-bottom: 1px solid var(--border);
}
.feed-row-media { aspect-ratio: 16 / 9; overflow: hidden; background: var(--surface); }
.feed-row-media img, .feed-row-media video { width: 100%; height: 100%; object-fit: cover; transition: transform .35s ease; }
.feed-row-media:hover img { transform: scale(1.02); }
.feed-row-copy { align-self: center; min-width: 0; }
.feed-row h2 { margin: 16px 0 8px; font-size: 25px; line-height: 1.2; }
.feed-row h2 a:hover { color: var(--text); text-decoration: underline; text-decoration-color: var(--olive); text-underline-offset: 5px; }
.feed-row p { max-width: 760px; margin: 0; color: var(--muted); font-size: 17px; line-height: 1.5; }
.media-placeholder { display: grid; width: 100%; height: 100%; place-items: center; background: var(--surface); }
.media-placeholder img { width: 76px; height: 76px; opacity: .28; object-fit: contain; }
.pagination { display: grid; grid-template-columns: 1fr auto 1fr; align-items: center; min-height: 90px; font-size: 15px; }
.pagination > :last-child { justify-self: end; }
.pagination span { color: var(--faint); }
.taxonomy-heading { padding-bottom: 34px; }
.taxonomy-heading > p { margin: 0 0 10px; color: var(--olive); font-size: 15px; text-transform: uppercase; }
.taxonomy-heading h1 { font-size: 56px; }
.article-page { padding-top: 56px; }
.article-header { max-width: 960px; margin: 0 auto 42px; }
.article-breadcrumbs { display: flex; gap: 12px; margin-bottom: 28px; color: var(--muted); font-size: 14px; }
.article-breadcrumbs span { color: var(--faint); }
.article-header h1 { margin: 0; font-size: 58px; line-height: 1.06; }
.article-lead { max-width: 850px; margin: 26px 0 0; color: var(--muted); font-size: 23px; line-height: 1.45; }
.article-byline { display: flex; gap: 24px; margin-top: 28px; color: var(--muted); font-size: 14px; }
.article-byline a { color: var(--red); }
.article-media { display: grid; gap: 12px; margin-bottom: 46px; }
.article-media figure { margin: 0; background: var(--surface); }
.article-media img, .article-media video { width: 100%; max-height: 760px; object-fit: contain; }
.article-layout { display: grid; grid-template-columns: minmax(0, 760px) 260px; justify-content: center; gap: 78px; }
.article-body { font-size: 21px; line-height: 1.7; }
.article-body p { margin: 0 0 26px; }
.article-facts { align-self: start; border-top: 2px solid var(--olive); }
.article-facts > div { padding: 18px 0; border-bottom: 1px solid var(--border); }
.article-facts span { display: block; margin-bottom: 4px; color: var(--faint); font-size: 13px; text-transform: uppercase; }
.article-facts a { font-size: 17px; }
.article-facts .source-link { display: block; margin-top: 22px; color: var(--muted); font-size: 14px; }
.related-section { margin-top: 90px; border-top: 1px solid var(--border-strong); }
.related-section > h2 { margin: 0; padding: 34px 0 18px; font-size: 32px; }
.empty-state, .not-found { min-height: 56vh; display: flex; flex-direction: column; justify-content: center; align-items: flex-start; }
.empty-state h2, .not-found h1 { margin: 0; font-size: 42px; }
.empty-state p { color: var(--muted); }
.empty-state a { margin-top: 16px; padding: 10px 15px; }
.not-found > p { margin: 0; color: var(--red); font-size: 18px; }
.not-found a { margin-top: 24px; color: var(--olive); }
.site-footer {
display: flex;
justify-content: space-between;
align-items: flex-end;
margin-top: 84px;
padding: 34px 0 46px;
border-top: 1px solid var(--border-strong);
color: var(--muted);
font-size: 14px;
}
.footer-brand { color: var(--text); font-size: 20px; font-weight: 700; }
.site-footer p { margin: 7px 0 0; }
.site-footer nav { display: flex; gap: 24px; }
@media (max-width: 980px) {
.header-inner { grid-template-columns: 150px 1fr auto; gap: 16px; }
.main-nav { gap: 22px; }
.main-nav a { font-size: 14px; }
.page-heading h1 { font-size: 56px; }
.lead-story { grid-template-columns: 1.1fr .9fr; min-height: 420px; }
.lead-media { min-height: 420px; }
.lead-copy { padding-left: 30px; }
.lead-copy h2 { font-size: 33px; }
.feed-row { grid-template-columns: 290px minmax(0, 1fr); gap: 26px; }
.article-layout { grid-template-columns: minmax(0, 680px) 220px; gap: 44px; }
}
@media (max-width: 760px) {
:root { --gutter: 16px; }
body { font-size: 17px; }
.header-inner { min-height: 64px; grid-template-columns: 1fr auto; }
.brand img { width: 38px; height: 38px; }
.main-nav {
position: absolute;
top: 64px;
right: 0;
left: 0;
display: none;
padding: 12px var(--gutter) 18px;
border-bottom: 1px solid var(--border);
background: var(--bg);
}
.main-nav.is-open { display: grid; }
.main-nav a { min-height: 44px; border-bottom: 1px solid var(--border); font-size: 16px; }
.main-nav a[aria-current="page"]::after { display: none; }
.menu-button { display: grid; }
.page-heading { padding: 38px 0 30px; }
.page-heading h1, .taxonomy-heading h1 { font-size: 44px; line-height: 1; }
.lead-story { display: block; min-height: 0; }
.lead-media { min-height: 0; aspect-ratio: 4 / 3; }
.lead-copy { padding: 25px 0 34px; }
.lead-copy h2 { margin: 20px 0 15px; font-size: 32px; }
.lead-copy > p { font-size: 18px; }
.feed-row { grid-template-columns: 132px minmax(0, 1fr); gap: 16px; padding: 18px 0; }
.feed-row-media { aspect-ratio: 4 / 3; }
.feed-row h2 { margin: 10px 0 0; font-size: 20px; }
.feed-row p { display: none; }
.article-meta { font-size: 12px; }
.article-meta time { display: none; }
.article-meta > * + *::before { height: 14px; margin: 0 10px; }
.article-meta .category-link::before { display: none; }
.article-header { margin-bottom: 30px; }
.article-header h1 { font-size: 42px; }
.article-lead { font-size: 20px; }
.article-layout { display: block; }
.article-body { font-size: 19px; }
.article-facts { margin-top: 42px; }
.related-section { margin-top: 64px; }
}
@media (max-width: 460px) {
.feed-row { grid-template-columns: 112px minmax(0, 1fr); gap: 13px; }
.feed-row h2 { font-size: 18px; line-height: 1.18; }
.article-meta .producer-link::before { display: none; }
.article-meta .category-link { display: none; }
.article-header h1 { font-size: 36px; }
.article-byline { display: grid; gap: 7px; }
.site-footer { display: grid; gap: 24px; }
}
@media (prefers-reduced-motion: reduce) {
html { scroll-behavior: auto; }
*, *::before, *::after { transition-duration: .01ms !important; }
}
+16
View File
@@ -0,0 +1,16 @@
const menuButton = document.querySelector('[data-menu-toggle]');
const nav = document.querySelector('#main-nav');
const searchButton = document.querySelector('[data-search-toggle]');
const searchForm = document.querySelector('[data-search-form]');
menuButton?.addEventListener('click', () => {
const open = menuButton.getAttribute('aria-expanded') === 'true';
menuButton.setAttribute('aria-expanded', String(!open));
nav?.classList.toggle('is-open', !open);
});
searchButton?.addEventListener('click', () => {
const opening = searchForm?.hasAttribute('hidden');
searchForm?.toggleAttribute('hidden', !opening);
if (opening) searchForm?.querySelector('input')?.focus();
});
+4
View File
@@ -0,0 +1,4 @@
{% extends "base.html" %}
{% block content %}
<section class="not-found"><p>404</p><h1>Страница не найдена</h1><a href="/">Вернуться в ленту</a></section>
{% endblock %}
@@ -0,0 +1,29 @@
{% macro media(article, eager=false) %}
{% set item = article.media[0] if article.media else none %}
{% if item and item.type == 'image' %}
<img src="{{ item.url }}" alt="{{ item.alt or article.title }}" {% if item.width %}width="{{ item.width }}"{% endif %} {% if item.height %}height="{{ item.height }}"{% endif %} loading="{{ 'eager' if eager else 'lazy' }}" decoding="async">
{% elif item and item.type == 'video' %}
<video src="{{ item.url }}" muted playsinline preload="metadata" aria-label="{{ item.alt or article.title }}"></video>
{% else %}
<span class="media-placeholder"><img src="/static/favi.png" alt="" width="96" height="96"></span>
{% endif %}
{% endmacro %}
{% macro meta(article) %}
<div class="article-meta">
<time datetime="{{ article.published_at.isoformat() }}">{{ article.published_at|ekb_date }} · {{ article.published_at|ekb_time }}</time>
<a class="category-link" href="/category/{{ article.category_tag }}">{{ article.category_name }}</a>
<a class="producer-link" href="/brand/{{ article.producer_tag }}">{{ article.producer_name }}</a>
</div>
{% endmacro %}
{% macro row(article) %}
<article class="feed-row">
<a class="feed-row-media" href="{{ article_path(article) }}" tabindex="-1">{{ media(article) }}</a>
<div class="feed-row-copy">
{{ meta(article) }}
<h2><a href="{{ article_path(article) }}">{{ article.title }}</a></h2>
<p>{{ article.lead or article.body|excerpt(210) }}</p>
</div>
</article>
{% endmacro %}
+7
View File
@@ -0,0 +1,7 @@
{% if pages > 1 %}
<nav class="pagination" aria-label="Страницы ленты">
{% if page > 1 %}<a rel="prev" href="?page={{ page - 1 }}{% if q %}&q={{ q|urlencode }}{% endif %}">← Новее</a>{% else %}<span></span>{% endif %}
<span>{{ page }} / {{ pages }}</span>
{% if page < pages %}<a rel="next" href="?page={{ page + 1 }}{% if q %}&q={{ q|urlencode }}{% endif %}">Старее →</a>{% endif %}
</nav>
{% endif %}
+38
View File
@@ -0,0 +1,38 @@
{% extends "base.html" %}
{% from "_article_rows.html" import media, row %}
{% block og_type %}article{% endblock %}
{% block content %}
<article class="article-page">
<header class="article-header">
<div class="article-breadcrumbs"><a href="/">Лента</a><span>/</span><a href="/category/{{ article.category_tag }}">{{ article.category_name }}</a></div>
<h1>{{ article.title }}</h1>
{% if article.lead %}<p class="article-lead">{{ article.lead }}</p>{% endif %}
<div class="article-byline">
<time datetime="{{ article.published_at.isoformat() }}">{{ article.published_at|ekb_date }} · {{ article.published_at|ekb_time }} ЕКБ</time>
<a href="/brand/{{ article.producer_tag }}">{{ article.producer_name }}</a>
</div>
</header>
{% if article.media %}
<div class="article-media">
{% for item in article.media %}
{% if item.type == 'image' %}<figure><img src="{{ item.url }}" alt="{{ item.alt or article.title }}" loading="{{ 'eager' if loop.first else 'lazy' }}" decoding="async"></figure>{% endif %}
{% if item.type == 'video' %}<figure><video src="{{ item.url }}" controls playsinline preload="metadata"></video></figure>{% endif %}
{% endfor %}
</div>
{% endif %}
<div class="article-layout">
<div class="article-body">{{ article.body|paragraphs }}</div>
<aside class="article-facts">
<div><span>Категория</span><a href="/category/{{ article.category_tag }}">{{ article.category_name }}</a></div>
<div><span>Производитель</span><a href="/brand/{{ article.producer_tag }}">{{ article.producer_name }}</a></div>
{% if article.source_url %}<a class="source-link" href="{{ article.source_url }}" rel="nofollow noopener" target="_blank">Первоисточник ↗</a>{% endif %}
</aside>
</div>
</article>
{% if related %}
<section class="related-section">
<h2>По теме</h2>
<div class="feed-list">{% for item in related %}{{ row(item) }}{% endfor %}</div>
</section>
{% endif %}
{% endblock %}
+67
View File
@@ -0,0 +1,67 @@
<!doctype html>
<html lang="ru">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{ meta_title }}</title>
<meta name="description" content="{{ meta_description }}">
{% if noindex %}<meta name="robots" content="noindex,follow">{% else %}<meta name="robots" content="index,follow,max-image-preview:large">{% endif %}
<link rel="canonical" href="{{ canonical_url }}">
<link rel="alternate" type="application/rss+xml" title="Forma N8 RSS" href="{{ site_url('/rss.xml') }}">
<link rel="icon" type="image/png" href="/static/favi.png">
<meta property="og:site_name" content="Forma N8">
<meta property="og:locale" content="ru_RU">
<meta property="og:type" content="{% block og_type %}website{% endblock %}">
<meta property="og:title" content="{{ meta_title }}">
<meta property="og:description" content="{{ meta_description }}">
<meta property="og:url" content="{{ canonical_url }}">
{% if og_image %}<meta property="og:image" content="{{ og_image }}">{% endif %}
<link rel="stylesheet" href="/static/site.css?v=1">
{% if json_ld %}<script type="application/ld+json">{{ json_ld|safe }}</script>{% endif %}
</head>
<body>
<a class="skip-link" href="#content">К содержанию</a>
<header class="site-header">
<div class="header-inner">
<a class="brand" href="/" aria-label="Forma N8, главная">
<img src="/static/favi.png" width="44" height="44" alt="">
<span>FORMA<br>N8</span>
</a>
<nav class="main-nav" id="main-nav" aria-label="Основная навигация">
<a href="/" {% if current_path == '/' %}aria-current="page"{% endif %}>Лента</a>
<a href="/category/snaryazhenie">Снаряжение</a>
<a href="/category/odezhda">Одежда</a>
<a href="/category/bronya">Броня</a>
<a href="/category/aksessuary">Аксессуары</a>
</nav>
<div class="header-actions">
<button class="icon-button" type="button" data-search-toggle aria-label="Открыть поиск" title="Поиск">
<svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="6.5"></circle><path d="m16 16 4 4"></path></svg>
</button>
<button class="icon-button menu-button" type="button" data-menu-toggle aria-expanded="false" aria-controls="main-nav" aria-label="Открыть меню" title="Меню">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M4 7h16M4 12h16M4 17h16"></path></svg>
</button>
</div>
</div>
<form class="search-form" action="/" method="get" data-search-form {% if not q %}hidden{% endif %}>
<label for="site-search">Поиск по новостям</label>
<div>
<input id="site-search" name="q" value="{{ q or '' }}" type="search" maxlength="100" placeholder="Название, производитель, снаряжение">
<button type="submit">Найти</button>
</div>
</form>
</header>
<main id="content">{% block content %}{% endblock %}</main>
<footer class="site-footer">
<div>
<a class="footer-brand" href="/">FORMA N8</a>
<p>Новости снаряжения и экипировки.</p>
</div>
<nav aria-label="Служебная навигация">
<a href="/rss.xml">RSS</a>
<a href="/sitemap.xml">Sitemap</a>
</nav>
</footer>
<script src="/static/site.js?v=1" defer></script>
</body>
</html>
+34
View File
@@ -0,0 +1,34 @@
{% extends "base.html" %}
{% from "_article_rows.html" import media, meta, row %}
{% block content %}
<section class="feed-shell">
<header class="page-heading">
<h1>Новости снаряжения</h1>
{% if q %}<p>Результаты поиска: «{{ q }}»</p>{% endif %}
</header>
{% if articles %}
{% if page == 1 and not q %}
{% set lead = articles[0] %}
<article class="lead-story">
<a class="lead-media" href="{{ article_path(lead) }}">{{ media(lead, true) }}</a>
<div class="lead-copy">
{{ meta(lead) }}
<h2><a href="{{ article_path(lead) }}">{{ lead.title }}</a></h2>
<p>{{ lead.lead or lead.body|excerpt(340) }}</p>
<a class="read-link" href="{{ article_path(lead) }}">Читать новость <span aria-hidden="true">→</span></a>
</div>
</article>
<div class="feed-list">
{% for article in articles[1:] %}{{ row(article) }}{% endfor %}
</div>
{% else %}
<div class="feed-list feed-list-top">
{% for article in articles %}{{ row(article) }}{% endfor %}
</div>
{% endif %}
{% include "_pagination.html" %}
{% else %}
<div class="empty-state"><h2>Ничего не найдено</h2><p>Попробуйте изменить запрос или вернуться к ленте.</p><a href="/">Вернуться в ленту</a></div>
{% endif %}
</section>
{% endblock %}
+18
View File
@@ -0,0 +1,18 @@
{% extends "base.html" %}
{% from "_article_rows.html" import row %}
{% block content %}
<section class="feed-shell taxonomy-shell">
<header class="page-heading taxonomy-heading">
<p>{{ taxonomy_label }}</p>
<h1>{{ heading }}</h1>
</header>
{% if articles %}
<div class="feed-list feed-list-top">
{% for article in articles %}{{ row(article) }}{% endfor %}
</div>
{% include "_pagination.html" %}
{% else %}
<div class="empty-state"><h2>Публикаций пока нет</h2><p>Новые материалы появятся здесь после публикации.</p><a href="/">Вернуться в ленту</a></div>
{% endif %}
</section>
{% endblock %}
+12
View File
@@ -0,0 +1,12 @@
aiohttp==3.12.13
aiogram==3.21.0
asyncpg==0.30.0
fastapi==0.115.14
jinja2==3.1.6
loguru==0.7.3
litellm==1.76.0
pydantic-settings==2.10.1
Pillow==11.3.0
python-multipart==0.0.20
uvicorn[standard]==0.35.0
yt-dlp==2026.6.9
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
import asyncio
from pathlib import Path
from vk_parser_app.db import apply_migrations, close_pool
async def main() -> None:
root = Path(__file__).resolve().parents[1]
await apply_migrations(root / "db" / "migrations")
await close_pool()
if __name__ == "__main__":
asyncio.run(main())
+76
View File
@@ -0,0 +1,76 @@
from __future__ import annotations
import asyncio
from loguru import logger
from vk_parser_app.constants import PUBLICATION_STATUS_PUBLISHED
from vk_parser_app.db import close_pool, fetch_int_setting, fetch_setting, get_pool
from vk_parser_app.workers.max_poster import MAXAPIClient, MAXAPIError, MAXPoster
async def main() -> None:
pool = await get_pool()
token = str(await fetch_setting("max_poster_bot_token", "") or "").strip()
base_url = str(await fetch_setting("max_poster_api_base_url", "https://platform-api2.max.ru") or "").strip()
if not token:
raise RuntimeError("max_poster_bot_token is empty")
rows = await pool.fetch(
"""
SELECT id, raw_post_id, external_id
FROM post_publications
WHERE platform='max'
AND status=$1
AND COALESCE(url, '')=''
AND COALESCE(external_id, '')<>''
ORDER BY published_at DESC NULLS LAST, id DESC
""",
PUBLICATION_STATUS_PUBLISHED,
)
poster = MAXPoster()
updated = 0
missing = 0
async with MAXAPIClient(
token,
base_url,
timeout_sec=max(30, await fetch_int_setting("max_poster_timeout_sec", 120)),
max_attempts=max(1, await fetch_int_setting("max_poster_max_attempts", 3)),
retry_backoff_max_sec=max(1, await fetch_int_setting("max_poster_retry_backoff_max_sec", 30)),
) as client:
for row in rows:
try:
data = await client.get_message(str(row["external_id"]))
except MAXAPIError as exc:
missing += 1
logger.warning(
"MAX URL backfill skipped raw_post={} message={} error={}",
row["raw_post_id"],
row["external_id"],
exc,
)
continue
url = poster.message_url_from_response(data)
if not url:
missing += 1
continue
await pool.execute(
"""
UPDATE post_publications
SET url=$2,
updated_at=NOW()
WHERE id=$1
""",
int(row["id"]),
url,
)
updated += 1
logger.info("MAX URL backfilled raw_post={} url={}", row["raw_post_id"], url)
logger.info("MAX URL backfill done: scanned={} updated={} missing={}", len(rows), updated, missing)
await close_pool()
if __name__ == "__main__":
asyncio.run(main())
+114
View File
@@ -0,0 +1,114 @@
from __future__ import annotations
import asyncio
import html
import json
import os
from pathlib import Path
from typing import Any
from aiogram.types import FSInputFile
from vk_parser_app.constants import MEDIA_STATUS_FAILED, MEDIA_STATUS_LINK_ONLY
from vk_parser_app.db import get_pool
from vk_parser_app.workers.vk_storage_uploader import TMP_DIR, TMP_PREFIX, TelegramStorageUploader
def jlog(event: str, **payload: Any) -> None:
print(json.dumps({"event": event, **payload}, ensure_ascii=False, default=str), flush=True)
async def load_pending_videos(worker: TelegramStorageUploader, limit: int) -> list[dict[str, Any]]:
rows = await worker.pool.fetch(
"""
SELECT m.*, rp.original_url AS post_url
FROM raw_post_media m
JOIN raw_posts rp ON rp.id=m.raw_post_id
WHERE m.media_type='video'
AND m.status='pending'
AND COALESCE(m.tg_file_id, m.storage_attachment_id, '') = ''
AND COALESCE(m.error, '') ILIKE '%timeout%'
AND COALESCE(m.duration_sec, 0) <= $1
ORDER BY COALESCE(rp.publication_status, 'pending')='published', m.id DESC
LIMIT $2
""",
worker.video_max_duration_sec,
limit,
)
return [dict(row) for row in rows]
async def send_video_to_storage(worker: TelegramStorageUploader, media: dict[str, Any], path: str) -> None:
caption = (
f"Backfill video media #{int(media['id'])} for raw_post #{int(media['raw_post_id'])}\n"
f'<a href="{html.escape(str(media.get("post_url") or ""), quote=True)}">source post</a>'
)
message = await worker.tg_retry(
lambda: worker.bot.send_video(
video=FSInputFile(path, filename=f"video_{int(media['id'])}.mp4"),
caption=caption[: worker.caption_limit],
parse_mode="HTML",
**worker.chat_kwargs(),
)
)
if not message.video:
raise RuntimeError("Telegram accepted message without video payload")
await worker.mark_media_uploaded(int(media["id"]), message.video.file_id, message.video.file_unique_id)
async def main() -> None:
limit = int(os.getenv("BACKFILL_LIMIT", "200"))
worker = TelegramStorageUploader()
await worker.init()
videos = await load_pending_videos(worker, limit)
jlog("start", count=len(videos), limit=limit)
ok = 0
failed = 0
link_only = 0
try:
for media in videos:
media_id = int(media["id"])
raw_post_id = int(media["raw_post_id"])
path = str(TMP_DIR / f"{TMP_PREFIX}backfill_{raw_post_id}_{media_id}.mp4")
jlog(
"video_start",
media_id=media_id,
raw_post_id=raw_post_id,
url=media.get("original_url"),
duration_sec=media.get("duration_sec"),
)
try:
info = await worker.download_video(str(media.get("original_url") or ""), path)
if not info:
await worker.mark_media_failed_attempt(media_id, "video download failed")
failed += 1
jlog("video_failed", media_id=media_id, raw_post_id=raw_post_id, error="video download failed")
continue
if info.get("error"):
if info.get("permanent"):
await worker.mark_media_link_only(media_id, str(info["error"]))
link_only += 1
else:
await worker.mark_media_failed_attempt(media_id, str(info["error"]))
failed += 1
jlog("video_failed", media_id=media_id, raw_post_id=raw_post_id, error=info.get("error"))
continue
await send_video_to_storage(worker, media, path)
ok += 1
jlog("video_uploaded", media_id=media_id, raw_post_id=raw_post_id, size_bytes=info.get("size_bytes"))
await asyncio.sleep(worker.media_upload_delay_sec)
except Exception as exc:
await worker.mark_media_failed_attempt(media_id, str(exc))
failed += 1
jlog("video_failed", media_id=media_id, raw_post_id=raw_post_id, error=str(exc))
finally:
Path(path).unlink(missing_ok=True)
finally:
await worker.close()
if worker.pool:
await worker.pool.close()
jlog("done", ok=ok, failed=failed, link_only=link_only, total=len(videos))
if __name__ == "__main__":
asyncio.run(main())

Some files were not shown because too many files have changed in this diff Show More