diff --git a/docs/charter.md b/docs/charter.md index 4e4cc3d..13ff2b3 100644 --- a/docs/charter.md +++ b/docs/charter.md @@ -165,7 +165,7 @@ Connected. Real environment integration tests must be marked with `#[ignore]` an ## 7. Document boundaries -Answer only one question per document; **Tables, lists, or values ​​may not be copied across documents. ** +Answer only one question per document; **tables, lists and values must not be duplicated across documents.** | Documentation | What to answer | Scope of authority | |---|---|---| diff --git a/docs/deployment.md b/docs/deployment.md index 0a29e01..5077f6b 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -267,11 +267,16 @@ business authentication. ### 10.8 Configuration convergence across origins -One SPA edit writes once to Redis. The origins converge **lazily, at the request boundary**: every -request-bearing entry point (`/worker`, `/reconcile`, `/healthz`, `/push/jmap`, `/api/*`) calls -`refresh_business_config`, which reads the global revision from Redis and returns immediately when -`remote_revision <= local_revision`. There is no polling loop and no watch, so an origin that -receives no traffic keeps the configuration it already had. +One SPA edit writes once to Redis. The origins converge **lazily, at the request boundary**: each +entry point that calls `refresh_business_config` re-reads the global revision from Redis and +returns immediately when `remote_revision <= local_revision`. There is no polling loop and no +watch, so an origin that receives no traffic keeps the configuration it already had. + +The entry points are `/ready`, `/worker`, `/reconcile`, `/push/jmap`, `/webhook/tg`, +`/api/config`, `/api/business-config` and `/api/telegram/register-webhook`. `GET /healthz` is +**not** among them — it is a static `200 "ok"` and never refreshes — and neither does +`GET /api/status`, `/api/enabled`, `/api/push/register`, `/api/push/disable` or the +`/api/admin/session` pair. **Known behavior: a failed rebuild is not retried until the next revision.** The revision check above is an early exit, not a retry guard. When `build_worker` fails for one origin — a JMAP diff --git a/docs/deployment.zh-CN.md b/docs/deployment.zh-CN.md index 7da9aab..489557d 100644 --- a/docs/deployment.zh-CN.md +++ b/docs/deployment.zh-CN.md @@ -221,17 +221,18 @@ HTTPS URL，之后才进行路由校验或转发。Worker 使用“404”拒绝 ### 10.5 回调地址 -使用网关时，将Telegram的webhook和Stalwart的push回调设置给Worker -域。外部调度程序可以对转发路径使用相同的 URL。引导程序和 -诊断端点仍然需要后端源。 +使用网关时，将 Telegram 的 webhook 和 Stalwart 的 push 回调都设置为 Worker URL。 +外部调度程序（cron/`systemd`/K8s CronJob）对已转发的路径使用相同的 URL。 +引导程序必须在后端源上执行；诊断端点需要后端源**和** debug token——Worker 都不转发它们。 ### 10.6 网关健康状态 -`GET /healthz-worker` 报告 Worker 版本和后端可用性。两者都使用 -用于验证部署的“可用”和“版本”； `status: no-backends` 可以返回 -当未配置有效源时，HTTP 200。 +`GET /healthz-worker` 报告 Worker 的版本（`version`）和健康的 origin 数量 +（`available`）。两者都用于验证部署：`version` 告诉你当前是哪次部署在应答， +`available` 告诉你有多少 origin 健康。`status: no-backends` 在返回 HTTP 200 时 +是**正确**的。 + -### 10.7 功能边界 Worker 不是 Redis 代理，也不会使后端无状态；共享外部 Redis 仍然需要。它不添加数据库、长期连接或第二层 @@ -239,10 +240,14 @@ Redis 仍然需要。它不添加数据库、长期连接或第二层 ### 10.8 多源站之间的配置收敛 -SPA 的一次编辑只写入 Redis 一次。各源站**惰性收敛，发生在请求边界上**：每个带请求的入口 -（`/worker`、`/reconcile`、`/healthz`、`/push/jmap`、`/api/*`）都会调用 -`refresh_business_config`，它先从 Redis 读全局版本，当 `remote_revision <= local_revision` 时 -立即返回。没有轮询循环，也没有 watch，所以一台完全收不到流量的源站会一直保留它原有的配置。 +SPA 的一次编辑只写入 Redis 一次。各源站**惰性收敛，发生在请求边界上**：每个调用 +`refresh_business_config` 的入口都会先从 Redis 读全局版本，当 `remote_revision <= local_revision` +时立即返回。没有轮询循环，也没有 watch，所以一台完全收不到流量的源站会一直保留它原有的配置。 + +会调用的入口是 `/ready`、`/worker`、`/reconcile`、`/push/jmap`、`/webhook/tg`、 +`/api/config`、`/api/business-config`、`/api/telegram/register-webhook`。`GET /healthz` +**不**在其中——它只是一个静态的 `200 "ok"`，从不刷新；`GET /api/status`、`/api/enabled`、 +`/api/push/register`、`/api/push/disable` 以及 `/api/admin/session` 那一对也不在其中。 **已知行为：重建失败后不会重试，直到下一次版本更新。** 上面的版本判断只是一个提前返回，它不是 重试保护。当某台源站的 `build_worker` 失败——JMAP session URL 不可达，或 `llm_api_key` / diff --git a/docs/design.md b/docs/design.md index 0584e26..a496f93 100644 --- a/docs/design.md +++ b/docs/design.md @@ -71,21 +71,21 @@ Source: Read directly from the `stalwartlabs/jmap-client` warehouse `stalwartlab ### 2.2 Module structure (`src/`) ``` -lib.rs 顶层：URI / Method / DataType / PushObject / Error 枚举 -client.rs Client：认证、connect、session、send_request、event_source、ws -core/ request / response / get / set / query / query_changes / changes / error / session -email/ mod + helpers（email_get/email_query/email_import/email_copy/email_set/email_parse…） +lib.rs 顶层：URI / Method / DataType / PushObject / Error 枚举 +client.rs Client：认证、connect、session、send_request、event_source、ws +core/ request / response / get / set / query / query_changes / changes / error / session +email/ mod + helpers（email_get/email_query/email_import/email_copy/email_set/email_parse…） email_submission/ helpers（email_submission_get/query/set → 发送邮件 + 状态查询） -mailbox/ helpers（mailbox_create/query/get/set/destroy…） -thread/ thread/get -identity/ Identity（发件身份）get/set -blob/ blob 上传/复制（附件） -sieve/ SieveScript（服务端过滤脚本） +mailbox/ helpers（mailbox_create/query/get/set/destroy…） +thread/ thread/get +identity/ Identity（发件身份）get/set +blob/ blob 上传/复制（附件） +sieve/ SieveScript（服务端过滤脚本） vacation_response/ -principal/ Principal（账户/共享） +principal/ Principal（账户/共享） push_subscription/ PushSubscription（HTTP push 回调注册） -event_source/ SSE 流：mod / parser / stream -client_ws/ WebSocket 客户端（feature = "websockets"） +event_source/ SSE 流：mod / parser / stream +client_ws/ WebSocket 客户端（feature = "websockets"） ``` ### 2.3 Key capabilities mapped to this Bot @@ -111,14 +111,14 @@ client_ws/ WebSocket 客户端（feature = "websockets"） ### 2.5 Error model (`Error` enumeration) ``` -Transport(reqwest::Error) 网络/TLS/超时 -Parse(serde_json::Error) 序列化 -Internal(String) 客户端内部 +Transport(reqwest::Error) 网络/TLS/超时 +Parse(serde_json::Error) 序列化 +Internal(String) 客户端内部 Problem(Box) JMAP problem-details Server(String) -Method(MethodError) JMAP method-level 错误（NotJSON/Forbidden/RateLimit/StateMismatch…） -Set(SetError) /set 级别错误（每条记录的 creation/update/destroy 失败） -WebSocket(...) ws 错误（feature 开启时） +Method(MethodError) JMAP method-level 错误（NotJSON/Forbidden/RateLimit/StateMismatch…） +Set(SetError) /set 级别错误（每条记录的 creation/update/destroy 失败） +WebSocket(...) ws 错误（feature 开启时） ``` - `MethodErrorType` fine-grained: `ServerUnavailable`/`ServerFail`/`RateLimit`/`InvalidArguments`/`Forbidden`/`StateMismatch`/`TooManyChanges`… → Can directly drive Bot’s retry/current limiting/state reset strategy. @@ -175,37 +175,37 @@ Candidate comparison (`teloxide` / `grammers` / old `telegram-bot`), 6 reasons w ### 5.1 High-level topology ``` - ┌───────────────────────────────────────────────────┐ - │ Telegram │ - └───────────────────────┬───────────────────────────┘ - Webhook(HTTPS) │ - ┌───────────────────────▼───────────────────────────┐ - │ HTTP 入口 (axum，单端口 PORT) │ - │ POST /webhook/tg · POST /push/jmap · │ - │ POST /reconcile · GET /healthz /ready │ - │ GET / + admin session + config APIs (管理 SPA) │ - └───────────────────────┬───────────────────────────┘ - 命令 │ 通知 ↓(Redis Streams worker) - ┌───────────────────────▼───────────────────────────┐ - │ bot 层 (handlers) │ - │ /list /read /send /search /folders /flag ... │ - └───────────────────────┬───────────────────────────┘ - 领域 │ - ┌───────────────────────▼───────────────────────────┐ - │ jmap 适配层 (JmapService) │ - │ list_folders/list_emails/read_email/send_email… │ - │ + sinceState 缓存 → 外部 Redis (MOD-SINCESTATE) │ - └───────────────────────┬───────────────────────────┘ - JMAP │ HTTPS（短请求） - ┌───────────────────────▼───────────────────────────┐ - │ Stalwart JMAP Server │ - │ session_url / email/* / mailbox/* / submission/* │ - └───────────────────────▲───────────────────────────┘ - Push 回调(HTTPS)│ + 外部 Cron 对账(FLOW-RECONCILE) - ┌───────────────────────┴───────────────────────────┐ - │ push handler → Redis Streams(MOD-STREAMS) │ - │ worker: Email/changes → 通知 → 发往 TG → 推进游标│ - └───────────────────────────────────────────────────┘ + ┌───────────────────────────────────────────────────┐ + │ Telegram │ + └───────────────────────┬───────────────────────────┘ + Webhook(HTTPS) │ + ┌───────────────────────▼───────────────────────────┐ + │ HTTP 入口 (axum，单端口 PORT) │ + │ POST /webhook/tg · POST /push/jmap · │ + │ POST /reconcile · GET /healthz /ready │ + │ GET / + admin session + config APIs (管理 SPA) │ + └───────────────────────┬───────────────────────────┘ + 命令 │ 通知 ↓(Redis Streams worker) + ┌───────────────────────▼───────────────────────────┐ + │ bot 层 (handlers) │ + │ /list /read /send /search /folders /flag ... │ + └───────────────────────┬───────────────────────────┘ + 领域 │ + ┌───────────────────────▼───────────────────────────┐ + │ jmap 适配层 (JmapService) │ + │ list_folders/list_emails/read_email/send_email… │ + │ + sinceState 缓存 → 外部 Redis (MOD-SINCESTATE) │ + └───────────────────────┬───────────────────────────┘ + JMAP │ HTTPS（短请求） + ┌───────────────────────▼───────────────────────────┐ + │ Stalwart JMAP Server │ + │ session_url / email/* / mailbox/* / submission/* │ + └───────────────────────▲───────────────────────────┘ + Push 回调(HTTPS)│ + 外部 Cron 对账(FLOW-RECONCILE) + ┌───────────────────────┴───────────────────────────┐ + │ push handler → Redis Streams(MOD-STREAMS) │ + │ worker: Email/changes → 通知 → 发往 TG → 推进游标│ + └───────────────────────────────────────────────────┘ ``` > Deployment form = **General HTTPS-only Docker, no long connection** (deployment.md `C-NO-LONG-CONN`/`C-HTTPS-INBOUND`): > Telegram is Webhook only; JMAP real-time is Push callback only; slow tasks are asynchronously stepped into Redis Streams; reconciliation is triggered by **External HTTPS Cron**. @@ -216,47 +216,47 @@ Candidate comparison (`teloxide` / `grammers` / old `telegram-bot`), 6 reasons w - **Status quo**: The first (and currently only) channel is Telegram. The goal is to retain the ability to expand DingTalk/Feishu in parallel, but not to achieve it ahead of time. - **Principle**: Decoupling the domain layer and channel layer - Mail/JMAP/AI/intention state machine ** must not rely on Telegram type**. - **Abstract (not over-engineered)**: No empty traits are reserved. `src/channel.rs` had three zero implementations - `#[expect(dead_code)]` Placeholder trait (`Channel` / `Notifier` / `MessageAdapter`, comment readme - "Stable ID + Phase 0 placeholder"), **Deleted** (see `docs/retired.md` for registration) - Actual Telegram outbound - `channel::telegram::TelegramClient::send_text` (self-developed by `reqwest`), by `worker.rs` - `MetadataWorker` and `notify.rs` are held directly and are never passed through these three traits. The command is parsed in - `parse_intent` (`src/worker.rs`), rendering logic is scattered in `src/worker.rs` / `src/notify.rs`. - What is passed between the domain and the channel is the domain `Notification` (`src/domain.rs`), and the channel only adapts at the edge. + `#[expect(dead_code)]` Placeholder trait (`Channel` / `Notifier` / `MessageAdapter`, comment readme + "Stable ID + Phase 0 placeholder"), **Deleted** (see `docs/retired.md` for registration) - Actual Telegram outbound + `channel::telegram::TelegramClient::send_text` (self-developed by `reqwest`), by `worker.rs` + `MetadataWorker` and `notify.rs` are held directly and are never passed through these three traits. The command is parsed in + `parse_intent` (`src/worker.rs`), rendering logic is scattered in `src/worker.rs` / `src/notify.rs`. + What is passed between the domain and the channel is the domain `Notification` (`src/domain.rs`), and the channel only adapts at the edge. - **Constraints**: - - The public interface of `jmap` / `llm` / intent state machine / `notify::core` only accepts/returns domain types; - - Specific channel SDK types must not appear in the domain layer (the channel layer does not currently introduce third-party Bot frameworks, see §4); - - Add new channel = new adapter + assembly, do not change the domain layer. When abstraction is really needed, add it with the implementation and leave no empty traits. + - The public interface of `jmap` / `llm` / intent state machine / `notify::core` only accepts/returns domain types; + - Specific channel SDK types must not appear in the domain layer (the channel layer does not currently introduce third-party Bot frameworks, see §4); + - Add new channel = new adapter + assembly, do not change the domain layer. When abstraction is really needed, add it with the implementation and leave no empty traits. - **What not to do**: Do not define a polymorphic configuration registry, do not abstract the "channel capability matrix" in advance, do not build a plugins mechanism; evolve on demand (YAGNI). ### 5.3 Running model (short request, no long connection) - Single binary `message-weave`, `#[tokio::main]`. - On startup: - 1. Load configuration (`config::Config`). - 2. Construct `JmapService` (`Client::connect` completes session parsing, account_id caching, mailbox role→id mapping warm-up); sinceState is restored from external Redis (`MOD-SINCESTATE`). - 3. Start the **HTTP portal** (axum, single port `PORT`): `/webhook/tg`, `/push/jmap`, `/reconcile`, `/healthz`, `/ready`; three of the write paths first pass the `SAF-AUTH-*` portal authentication (fail-closed, §7.3), `/healthz`, `/ready` For the public probe (`SAF-PROBE-PUBLIC`); `/ready` has done end-to-end detection (configuration + Redis + outbound read-only detection JMAP session `GET` and TG `getMe`, each 3s, parallel, about 3s at worst). - 4. Start **Redis Streams worker** (background task) to consume Push events → `Email/changes` → Notification → Send to TG → Push sinceState → XACK. - 5. **Does not hold any long connections and does not build self-timers** (`C-NO-LONG-CONN`); reconciliation is triggered by **external HTTPS Cron** `/reconcile` (`FLOW-RECONCILE`). + 1. Load configuration (`config::Config`). + 2. Construct `JmapService` (`Client::connect` completes session parsing, account_id caching, mailbox role→id mapping warm-up); sinceState is restored from external Redis (`MOD-SINCESTATE`). + 3. Start the **HTTP portal** (axum, single port `PORT`): `/webhook/tg`, `/push/jmap`, `/reconcile`, `/healthz`, `/ready`; three of the write paths first pass the `SAF-AUTH-*` portal authentication (fail-closed, §7.3), `/healthz`, `/ready` For the public probe (`SAF-PROBE-PUBLIC`); `/ready` has done end-to-end detection (configuration + Redis + outbound read-only detection JMAP session `GET` and TG `getMe`, each 3s, parallel, about 3s at worst). + 4. Start **Redis Streams worker** (background task) to consume Push events → `Email/changes` → Notification → Send to TG → Push sinceState → XACK. + 5. **Does not hold any long connections and does not build self-timers** (`C-NO-LONG-CONN`); reconciliation is triggered by **external HTTPS Cron** `/reconcile` (`FLOW-RECONCILE`). - Notification sending and command processing share `Arc`, and internal `tokio::sync::RwLock` protects the variable cache; cross-request status always falls to external Redis (`C-REDIS-ONLY-STATE`). - **Multiple instances and load balancing (deployment form, `ARCH-LB-WORKER`)**: Since the state is all external Redis (`C-REDIS-ONLY-STATE`) and the delivery is idempotent (`MOD-DEDUP`), **the same image can be instantiated across multiple serverless platforms**, free Cloudflare Worker is used as the only entrance and failover (`C-LB-SINGLE-REG-URL`); `/reconcile` Use Redis to lock a single instance (`SAF-RECONCILE-LOCK`), and Streams to automatically amortize using the same consumer group (`MOD-STREAMS-GROUP`). See deployment.md §10 for details. **Field/channel logic does not need to be changed. ** - **Production redline (`C-NO-DB` / `C-NO-LOCAL-WRITE` / `C-LOG-STDOUT-ONLY` / `SAF-LOG-PURITY` / `C-NO-STATEFUL-RECOVERY`, see deployment.md §0 / §8.2)**: - - **Production uses no database** (`C-NO-DB`): No SQLite/Postgres/MySQL/embedded database; Redis is the only production state store. - - **Disable local file/directory writing** (`C-NO-LOCAL-WRITE`): no log files, no data files, no temporary cache, no local volumes mounted. - - **Log only writes to stdout/stderr** (`C-LOG-STDOUT-ONLY`): The container/platform is responsible for collecting and placing disk; disable the file log backend. - - **Log and Redis write content constraints** (`SAF-LOG-PURITY`): Only structured events, counts, timestamps, desensitized summaries; **Prohibited** key original text, JMAP email body, AI request/response, attachment content. - - **It is prohibited to rely on in-process status for production recovery** (`C-NO-STATEFUL-RECOVERY`): Restart and continuation (deduplication, sinceState, Streams breakpoints, circuit breaker counts, sessions) are all implemented by external Redis + JMAP reconciliation; the in-process cache is only for performance optimization, and the loss must be safe and reentrant. + - **Production uses no database** (`C-NO-DB`): No SQLite/Postgres/MySQL/embedded database; Redis is the only production state store. + - **Disable local file/directory writing** (`C-NO-LOCAL-WRITE`): no log files, no data files, no temporary cache, no local volumes mounted. + - **Log only writes to stdout/stderr** (`C-LOG-STDOUT-ONLY`): The container/platform is responsible for collecting and placing disk; disable the file log backend. + - **Log and Redis write content constraints** (`SAF-LOG-PURITY`): Only structured events, counts, timestamps, desensitized summaries; **Prohibited** key original text, JMAP email body, AI request/response, attachment content. + - **It is prohibited to rely on in-process status for production recovery** (`C-NO-STATEFUL-RECOVERY`): Restart and continuation (deduplication, sinceState, Streams breakpoints, circuit breaker counts, sessions) are all implemented by external Redis + JMAP reconciliation; the in-process cache is only for performance optimization, and the loss must be safe and reentrant. - Graceful exit: signal + `CancellationToken`, flush Redis Streams pending entries before exiting (**Do not write local files**, `C-NO-LOCAL-WRITE`). - **Current status**: Configuration guidance, HTTP entry and write entry authentication completed (`R1`/`SAF-AUTH-*`, fail-closed); JMAP session, Email/changes, PushSubscription create/update, Redis Streams worker and `/reconcile` have been connected. `/reconcile` and the Push closed loop still need to complete end-to-end acceptance in the real Stalwart, Redis, and Telegram environments. You cannot claim that the production link has been verified based on local access control alone. ### 5.4 Data flow: new email push (critical path, FLOW-NEW-MAIL) ``` Stalwart JMAP Push → POST /push/jmap (StateChange{Email/EmailDelivery: new_state}) - → 校验 pushSubscriptionId + verificationCode → 幂等去重(MOD-DEDUP) → 入 Redis Streams(MOD-STREAMS) → 立即 2xx ACK - → worker: XREADGROUP → 用 sinceState(存 Redis, MOD-SINCESTATE) 调 Email/changes → 取 created[] 的 id - → read_email 封装 email_get（Subject, Preview, From, TextBody, BodyValues, Size, ReceivedAt, HasAttachment）→ EmailMetadata - → notify::core 组装领域 Notification（仅元数据 + 行内按钮意图，绝不含正文） - → channel::telegram::TelegramClient::send_notification(chat, notification)（内部渲染文本后经 send_text 发出） - → 更新 sinceState（写外部 Redis）→ XACK - → 若 Redis 丢失：由外部 Cron 对账(FLOW-RECONCILE) 重建游标并补发 + → 校验 pushSubscriptionId + verificationCode → 幂等去重(MOD-DEDUP) → 入 Redis Streams(MOD-STREAMS) → 立即 2xx ACK + → worker: XREADGROUP → 用 sinceState(存 Redis, MOD-SINCESTATE) 调 Email/changes → 取 created[] 的 id + → read_email 封装 email_get（Subject, Preview, From, TextBody, BodyValues, Size, ReceivedAt, HasAttachment）→ EmailMetadata + → notify::core 组装领域 Notification（仅元数据 + 行内按钮意图，绝不含正文） + → channel::telegram::TelegramClient::send_notification(chat, notification)（内部渲染文本后经 send_text 发出） + → 更新 sinceState（写外部 Redis）→ XACK + → 若 Redis 丢失：由外部 Cron 对账(FLOW-RECONCILE) 重建游标并补发 ``` > Key: `Email/changes` lists the new email id in `created` to avoid full `email_query`; `sinceState` is stored in external Redis (`C-REDIS-ONLY-STATE`). When Redis is lost, it will be reconstructed from JMAP by reconciliation, and the **fact source is in JMAP**. > Notifications only contain sender/subject/time (+number of attachments), the text never appears in the notification (see `docs/charter.md §3` Security Boundary `SAF-NOTIFY-META`). @@ -265,36 +265,36 @@ Stalwart JMAP Push → POST /push/jmap (StateChange{Email/EmailDelivery: new_sta ### 5.5 Data flow: command `/read ` (long mail branch) ``` TG /read 3 → handler 取会话里的 folder+page 游标 - → JmapService.list_emails(folder, page=3) - → 取第 3 封 id → read_email(id, body=true) # JMAP 直取，绝不经 LLM - → 格式化： - 正文 ≤ long_email_char_limit(默认 4000) → 正常发送（HTML escape） - 正文 > 阈值 → 不发全文，发 Preview(120) + 发件人/主题/附件概览 + 选项： - "[AI 总结](约300字，需确认) / [继续查看原文(截断，标注‘完整请电脑查看’)]" - → reply + → JmapService.list_emails(folder, page=3) + → 取第 3 封 id → read_email(id, body=true) # JMAP 直取，绝不经 LLM + → 格式化： + 正文 ≤ long_email_char_limit(默认 4000) → 正常发送（HTML escape） + 正文 > 阈值 → 不发全文，发 Preview(120) + 发件人/主题/附件概览 + 选项： + "[AI 总结](约300字，需确认) / [继续查看原文(截断，标注‘完整请电脑查看’)]" + → reply （"AI 总结"分支必须用户确认后才把正文交 LLM 生成 ~300 字摘要；见 §12.3） ``` -### 5.6 Session State Machine (Conversation FSM) +### 5.6 Conversation FSM -There is currently no FSM. The command route is parsed into `Intent` by `parse_intent` in `src/worker.rs`; the AI ​​authorization state is a Boolean expiration time in Redis (see the AI ​​authorization state section of `docs/reference.md` for the key and TTL), not a multi-step confirmation state. +There is currently **no FSM**. `parse_intent` in `src/worker.rs` parses the command line into an `Intent`; the AI consent state is a single boolean plus an expiry in Redis (key and TTL in the AI consent section of `docs/reference.md`), not a multi-step confirmation state. -Invariants that still hold true: AI digests must first be explicitly authorized by the user, and LLM must not be called without authorization. Once the authorization takes effect, the digest will pull the full text of the email body and send it to LLM (`read_email` requests `TextBody` + `BodyValues` and `fetch_text_body_values(true)`, see `src/domain/jmap/client.rs`), but the body will not be sent back to Telegram** - only the summary will be sent outbound, or it will fall back to the first 300 if it fails. characters (`worker.rs:227` Note: body text never reaches Telegram). +The invariant that still holds: an AI summary requires the user's explicit consent first — **without a consent state the LLM is never called**. Once consent is live the summary pulls the full email body and sends it to the LLM (`read_email` requests `TextBody` + `BodyValues` via `fetch_text_body_values(true)`, see `src/domain/jmap/client.rs`), but the body is not sent back to Telegram — outbound sends only the summary, or falls back to the first 300 characters of the body if the LLM call fails (`fallback` in `src/worker.rs`). See also the doc comment on `MetadataWorker`: "body text never reaches Telegram". -The designed 5-state FSM (`Idle` / `AwaitClarify` / `AwaitConfirm` / `Analyzing` / `AwaitFallback`), together with the state transition table and channel neutrality description, can be found in `docs/retired.md`. +The 5-state FSM that was designed (`Idle` / `AwaitClarify` / `AwaitConfirm` / `Analyzing` / `AwaitFallback`), with its state-transition table and channel-neutrality notes, is in `docs/retired.md`. ### 5.7 Email search (`/search`, `bfe0fd8` implemented) -`/search ` and the Chinese prefix `search`/`find`/`retrieval` (**only prefix matching**, to avoid "Help me search..." being hijacked into a search) triggers `Intent::Search`, through `JmapService::search_emails(account_id, query, limit)` and `email_query`(`Filter::text`) to get the ID + `SearchSnippet/get` Highlight `subject`/`preview`. +`/search `, plus the Chinese prefixes `搜索` / `查找` / `检索` (**prefix match only**, so "help me search…" is not hijacked into a search), route to `Intent::Search`, which goes through `JmapService::search_emails(account_id, query, limit)` with `email_query` (`Filter::text`) for the IDs and `SearchSnippet/get` for highlighted `subject` / `preview`. Key boundaries: -- **Highlight downgrade**: `SearchSnippet/get` does not support (`unknownMethod`) or returns empty snippets instead of an error when it times out; rendering is downgraded to a pure ID list (`highlighted snippets are temporarily unavailable, the following are matching email IDs`), and snippets are not created. -- **Plain text outbound**: `` highlighted tags are stripped by `strip_mark_tags` (case-insensitive, unclosed tags are discarded), and then decoded in a single pass by `unescape_html_entities`; tags are stripped first and then entities are decoded to prevent the literal `` (escaped by the server as ``) in the mailbox body from being restored to a real tag and accidentally deleted. `subject`/`preview` have 120/160 character truncation upper limit respectively (`truncate_chars`, the same tool as text truncation). +- **Snippet degradation**: `SearchSnippet/get` being unsupported (`unknownMethod`) or timing out with empty snippets instead of an error degrades rendering to a plain ID list (`（高亮片段暂不可用，以下为匹配的邮件 ID）`); no snippets are fabricated. +- **Plain text outbound**: `` tags are stripped by `strip_mark_tags` (case-insensitive, unclosed tags discarded) and then decoded in one pass by `unescape_html_entities`; tags are stripped **before** entities are decoded so that a literal `` in the mailbox body (escaped by the server as ``) is not restored to a real tag and then accidentally deleted. `subject` and `preview` are capped at 120 and 160 characters (`truncate_chars`, the same helper as text truncation). - **Failed mapping**: JMAP side failed → `SearchReply::Retry` → `process_telegram` returns `Err(())` → coordinator 503 + `Retry-After` retry (consistent with `/reconcile`, other JMAP paths), **no panic, no error stack returned to user**. -- **Empty query**: `/search` (no keywords) → The guidance prompt "Please provide search keywords, for example: /search invoice", no request is sent. No match → `No messages matching "..." were found. `. -- **Each hit contains `email_id`**, you can directly access `/summary ` to enter the AI ​​summary process. -- **Body-level highlighting is not possible in locked versions**: jmap-client `0.4.2`’s `SearchSnippet` only models `emailId`/`subject`/`preview`, without `bodyProperties`/`parts`. According to RFC 8621 §5, the body-level `body: String[Id]` is not modeled by this crate and (none) `deny_unknown_fields`) are silently discarded by serde. The text highlighting of each part needs to bypass the crate and be sent directly to the original JMAP, which is not currently implemented. -- **No new environment variable**; `/search` will give a friendly error when the configuration is missing. +- **Empty query**: `/search` with no keywords emits a guidance prompt (`请提供搜索关键词，例如：/search 发票`) and makes no request. No matches returns `没有找到匹配「」的邮件。`. +- **Each hit carries `email_id`**, so `/summary ` jumps straight into the AI summary flow. +- **Body-level highlighting is not possible in locked versions**: jmap-client `0.4.2`’s `SearchSnippet` only models `emailId`/`subject`/`preview`, without `bodyProperties`/`parts`. Per RFC 8621 §5 the body-level `body: String[Id]` is not modeled by the crate and, with no `deny_unknown_fields`, is silently dropped by serde. Per-part body highlighting would require bypassing the crate and calling JMAP directly, which is not implemented. +- **No new environment variable**; when the configuration is missing, `/search` reports a friendly error. --- @@ -303,30 +303,30 @@ Key boundaries: Actual project structure (all code anchors in this document refer to this structure; see `docs/retired.md` for the historical target module splitting plan): ``` message-weave/ -├── Cargo.toml # jmap-client 0.4.2 / redis 0.27 / reqwest 0.13；不含 teloxide -├── .env.example # 仅 2 个必填项的占位样例（REDIS_URL / CONFIG_ENCRYPTION_KEY），不含业务配置 +├── Cargo.toml # jmap-client 0.4.2 / redis 0.27 / reqwest 0.13；不含 teloxide +├── .env.example # 仅 2 个必填项的占位样例（REDIS_URL / CONFIG_ENCRYPTION_KEY），不含业务配置 ├── .gitignore -├── Dockerfile # 仅本地开发用；生产不执行（C-DEBIAN-SLIM，边界见 deployment.md §3） -├── hoststack.yaml # 生产部署真源（runtime/build/start/healthCheck），非网关清单，见 §5.1 +├── Dockerfile # 仅本地开发用；生产不执行（C-DEBIAN-SLIM，边界见 deployment.md §3） +├── hoststack.yaml # 生产部署真源（runtime/build/start/healthCheck），非网关清单，见 §5.1 ├── src/ -│ ├── main.rs # tokio main：读 PORT/REDIS_URL/CONFIG_ENCRYPTION_KEY 后启动 webhook HTTP 入口（单入口，无 CLI 子命令） -│ ├── config.rs # Redis 业务配置反序列化（serde → BusinessConfigWire → Config）；CONFIG_ENCRYPTION_KEY 解析 -│ ├── error.rs # BotError 统一错误（见 §8.1） -│ ├── domain.rs # 领域层根（渠道中立，见 §5.2） -│ ├── domain/jmap.rs # JmapBackend trait + MockBackend（契约测试不联网） -│ ├── domain/jmap/client.rs # 包装 jmap_client::Client（真实只读 adapter，MOD-JMAP-CLIENT） -│ ├── state.rs # Redis 读写封装：配置/开关/会话/TTL 键（C-REDIS-ONLY-STATE） -│ ├── channel.rs # 渠道层：TelegramClient（reqwest 自研出站）；领域类型在 src/domain.rs -│ ├── notify.rs # /push/jmap 校验入队 + Redis Streams worker + /reconcile 对账 -│ ├── worker.rs # parse_intent → Intent，命令路由与 AI 授权判定 -│ ├── ai.rs # LlmClient / summarize：LLM 摘要调用 -│ └── web.rs # /config 静态前端路由 + include_str! 嵌入 + CSP -├── web/ # 前端资源，构建期 include_str! 嵌入二进制（C-NO-LOCAL-WRITE） -│ ├── index.html -│ ├── styles.css -│ ├── config.js -│ └── config.test.mjs # Node 原生断言测试（非 cargo 测试） -└── cloudflare-worker/ # 边缘反向代理，JS，不在 Rust 工程中 +│ ├── main.rs # tokio main：读 PORT/REDIS_URL/CONFIG_ENCRYPTION_KEY 后启动 webhook HTTP 入口（单入口，无 CLI 子命令） +│ ├── config.rs # Redis 业务配置反序列化（serde → BusinessConfigWire → Config）；CONFIG_ENCRYPTION_KEY 解析 +│ ├── error.rs # BotError 统一错误（见 §8.1） +│ ├── domain.rs # 领域层根（渠道中立，见 §5.2） +│ ├── domain/jmap.rs # JmapBackend trait + MockBackend（契约测试不联网） +│ ├── domain/jmap/client.rs # 包装 jmap_client::Client（真实只读 adapter，MOD-JMAP-CLIENT） +│ ├── state.rs # Redis 读写封装：配置/开关/会话/TTL 键（C-REDIS-ONLY-STATE） +│ ├── channel.rs # 渠道层：TelegramClient（reqwest 自研出站）；领域类型在 src/domain.rs +│ ├── notify.rs # /push/jmap 校验入队 + Redis Streams worker + /reconcile 对账 +│ ├── worker.rs # parse_intent → Intent，命令路由与 AI 授权判定 +│ ├── ai.rs # LlmClient / summarize：LLM 摘要调用 +│ └── web.rs # /config 静态前端路由 + include_str! 嵌入 + CSP +├── web/ # 前端资源，构建期 include_str! 嵌入二进制（C-NO-LOCAL-WRITE） +│ ├── index.html +│ ├── styles.css +│ ├── config.js +│ └── config.test.mjs # Node 原生断言测试（非 cargo 测试） +└── cloudflare-worker/ # 边缘反向代理，JS，不在 Rust 工程中 ``` Testing: **No integrated test directory**, and no stand-alone util tool module. All Rust tests are `#[cfg(test)]` unit tests within each module; the only test on the front end is `web/config.test.mjs`. @@ -407,16 +407,16 @@ Testing: **No integrated test directory**, and no stand-alone util tool module. ### 7.3 Access control - **Entry authentication (hard constraints `SAF-AUTH-RECONCILE`/`SAF-AUTH-TG-WEBHOOK`/`SAF-AUTH-JMAP-PUSH`)**: The three write paths must be authenticated first, **fail-closed**—— - - `/reconcile`: Request header `Authorization: Bearer `; - - `/webhook/tg`: Request header `X-Telegram-Bot-Api-Secret-Token == TG_WEBHOOK_SECRET`; - - `/push/jmap`: Verify Redis short-term status according to `pushSubscriptionId`; StateChange is released only after the first verification is successfully written back by JMAP `PushSubscription/set`. - The comparison uses the **constant time** algorithm (`subtle`, anti-timing side channel); verification failure will always be `401` and **will not produce any side effects/status changes** before the authentication is passed. After the business configuration is completed, the credentials for Webhook, Push, Reconcile and Push registration interfaces must be valid; when the startup boot variable is missing, the configuration boot mode will be entered and authentication will not be bypassed (§7.1). + - `/reconcile`: Request header `Authorization: Bearer `; + - `/webhook/tg`: Request header `X-Telegram-Bot-Api-Secret-Token == TG_WEBHOOK_SECRET`; + - `/push/jmap`: Verify Redis short-term status according to `pushSubscriptionId`; StateChange is released only after the first verification is successfully written back by JMAP `PushSubscription/set`. + The comparison uses the **constant time** algorithm (`subtle`, anti-timing side channel); verification failure will always be `401` and **will not produce any side effects/status changes** before the authentication is passed. After the business configuration is completed, the credentials for Webhook, Push, Reconcile and Push registration interfaces must be valid; when the startup boot variable is missing, the configuration boot mode will be entered and authentication will not be bypassed (§7.1). - **Health probe (`SAF-PROBE-PUBLIC`)**: `/healthz` (`ARCH-HEALTHZ`) and `/ready` are **public probes** - no authentication, only returns health status, **does not contain any sensitive information** (does not echo configuration/key/internal error details). - - `/healthz` = liveness (process survival), long-term stable semantics. - - `/ready` does end-to-end probing: configuration integrity + Redis reachability + outbound read-only probing (`GET {jmap_origin}/.well-known/jmap`, Basic authentication with configuration; `GET https://api.telegram.org/bot/getMe`; each `PROBE_TIMEOUT` = 3000ms, **Parallel** (`tokio::join!`), worst-case scenario 3s); all four pass `200` and readiness report (`{"status":"ready","configured":...,"jmap":...,"telegram":...}`, where `jmap`/`telegram` is the real probe result), any one fails `503` (standard error envelope `{"error":"service_unavailable","request_id":}` + `Retry-After: 30`). The probe is read-only, read-only configuration status, and will not trigger business side effects such as email synchronization, nor will it echo token or third-party response content; `refresh_business_config` only reads Redis, and has no state to write. Therefore `/ready` requires the outbound egress to the JMAP host and `api.telegram.org:443` to be reachable (if the egress requires a proxy, `/ready` is not available, see deployment.md). + - `/healthz` = liveness (process survival), long-term stable semantics. + - `/ready` does end-to-end probing: configuration integrity + Redis reachability + outbound read-only probing (`GET {jmap_origin}/.well-known/jmap`, Basic authentication with configuration; `GET https://api.telegram.org/bot/getMe`; each `PROBE_TIMEOUT` = 3000ms, **Parallel** (`tokio::join!`), worst-case scenario 3s); all four pass `200` and readiness report (`{"status":"ready","configured":...,"jmap":...,"telegram":...}`, where `jmap`/`telegram` is the real probe result), any one fails `503` (standard error envelope `{"error":"service_unavailable","request_id":}` + `Retry-After: 30`). The probe is read-only, read-only configuration status, and will not trigger business side effects such as email synchronization, nor will it echo token or third-party response content; `refresh_business_config` only reads Redis, and has no state to write. Therefore `/ready` requires the outbound egress to the JMAP host and `api.telegram.org:443` to be reachable (if the egress requires a proxy, `/ready` is not available, see deployment.md). - **chat whitelist (hard constraint `SAF-CHAT-ALLOWLIST`)**: `CHAT_ALLOWLIST` is a **required** configuration; any inbound event (action triggered by TG command/callback) must verify `chat.id ∈ CHAT_ALLOWLIST` before **making any JMAP call, AI call or status change**. If it is not in the whitelist, **directly reject and terminate** (prevent token can be called by anyone after being leaked). Phase 0 has completed the `CHAT_ALLOWLIST` parsing skeleton; the forced rejection logic has been implemented with Telegram channel access (`telegram_webhook` of `src/notify.rs` verifies the whitelist before any JMAP/AI/status operation, and terminates if rejected). - **Command Minimization**: Only necessary commands are exposed; writing operations such as sending emails must be confirmed twice (sending emails is not currently implemented, see §10.3). -- **Rate**: No local token bucket is built on the outbound side; Telegram's outbound sending is retried according to the Redis operating parameter `max_retries` (see `docs/reference.md` §6.1 for default values ​​and hard upper limits); currently only Redis current limiting is done for Push verification code writing (`ratelimit:push-verify:*`). 429 under the 30 msg/s limit of the Telegram server **Automatic backoff in `parameters.retry_after` seconds** (`channel.rs`, `f4cae00`): `retry_after_ms` is truncated to the 60s budget limit after parsing, and the exponential backoff is used when this field is missing or non-numeric `backoff_delay_ms` (starting from 250ms, capped 4s), the overall retry budget is 60s. There is still no local token bucket current limit - if the budget is exceeded, a failure will be returned directly and the upstream will try again. +- **Rate**: No local token bucket is built on the outbound side; Telegram's outbound sending is retried according to the Redis operating parameter `max_retries` (see `docs/reference.md` §6.1 for default values and hard upper limits); currently only Redis current limiting is done for Push verification code writing (`ratelimit:push-verify:*`). 429 under the 30 msg/s limit of the Telegram server **Automatic backoff in `parameters.retry_after` seconds** (`channel.rs`, `f4cae00`): `retry_after_ms` is truncated to the 60s budget limit after parsing, and the exponential backoff is used when this field is missing or non-numeric `backoff_delay_ms` (starting from 250ms, capped 4s), the overall retry budget is 60s. There is still no local token bucket current limit - if the budget is exceeded, a failure will be returned directly and the upstream will try again. ### 7.3.1 Configuration Management API - `GET /` provides a SPA embedded in the Rust binary; `/assets/config.js` and `/assets/styles.css` provide page resources. The service does not read or write local files while running (`C-NO-LOCAL-WRITE`). @@ -424,15 +424,15 @@ Testing: **No integrated test directory**, and no stand-alone util tool module. - `POST /api/admin/session` accepts `Authorization: Bearer ` and returns `{ "session": "", "expires_in": 1800 }`; the admin session only saves the digest in Redis and expires after 1,800 seconds. `POST /api/admin/session/revoke` revokes the current session and returns `204` successfully. - The admin page only saves the session in JavaScript memory. Request settings `credentials: omit`, `cache: no-store`, without using cookies, localStorage or sessionStorage. The admin API accepts a valid admin session; the compatibility path also accepts `WORKER_TOKEN`. Worker transparently transmits the authentication header (`SAF-LB-PASSTHRU`) as it is. - `GET /api/config` and `PUT /api/config` only read and write non-sensitive running parameters: - ```json - { - "jmap_timeout_ms": 15000, - "telegram_timeout_ms": 10000, - "llm_timeout_ms": 30000, - "max_retries": 3 - } - ``` -- The three timeout units are all milliseconds; the default value, value range and hard upper limit of `max_retries` are only authoritative with `docs/reference.md` §6.1 (the table below and the example response body above are only for illustration, and the values ​​​​are not repeated). `max_retries` represents the maximum number of retries beyond the first request. When there is no configuration in Redis, GET returns the default value object; PUT successfully returns the same object after saving and persists to Redis (`C-REDIS-ONLY-STATE`). + ```json + { + "jmap_timeout_ms": 15000, + "telegram_timeout_ms": 10000, + "llm_timeout_ms": 30000, + "max_retries": 3 + } + ``` +- The three timeout units are all milliseconds; the default value, value range and hard upper limit of `max_retries` are only authoritative with `docs/reference.md` §6.1 (the table below and the example response body above are only for illustration, and the values are not repeated). `max_retries` represents the maximum number of retries beyond the first request. When there is no configuration in Redis, GET returns the default value object; PUT successfully returns the same object after saving and persists to Redis (`C-REDIS-ONLY-STATE`). - `GET /api/business-config` reads back saved configuration for management SPA pre-populated form: returns `{ "configured", "revision", "values", "secrets_present" }`, where `values` is the value of 10 non-key fields and `secrets_present` is 6 Boolean existence of a key (`bot_token`, `jmap_password`, `telegram_webhook_secret`, `reconcile_token`, `worker_token`, `llm_api_key`) - **Never return the key value itself** (`SAF-NO-SECRET-ECHO`). Not yet saved, staleness returns `200` + `configured: false`, `revision: 0`, empty `values` and an existence boolean of all false. SPA therefore has only one code path and does not need to distinguish between first-time configuration and modified configuration. - `PUT /api/business-config` accepts `BusinessConfigPatch` (complete or partial), **only replaces the fields that appear in the submission body, and the rest inherits the stored values** (`BusinessConfigPatch::apply`); Push verification does not belong to Wire, is dynamically generated by Stalwart and written back by the backend. Verification is the only write gate, and the verification object is the merged configuration rather than the commit itself: patches are never verified individually, and existing complete wires are never discarded. `validate_business_wire` runs first, and returns the real `422` when rejected; if the merge result is legal, the configuration **must** be persisted (the merged Wire is persisted, not the request body), and then the client is built and the worker is switched to run. Therefore, when the dependency is not connected, `200` + `{ "persisted": true, "runtime_applied": false, "warnings": [{"component", "step", "detail"}] }` is returned instead of `503` - the configuration has been saved and the fault is reported truthfully instead of being silently swallowed by an unreachable JMAP. `runtime_applied` is only `true` when the reload submission is successful; in both cases, the `x-business-config-revision` header is returned; the request body can carry `revision` (that is, the `u64` returned by `GET`) as a control field - it does not participate in `apply`, does not enter Wire, and is not used as a saved setting. It is only used to determine whether this submission is based on the latest version. See the next article for the semantics of `409`. The response never echoes configuration or keys. The key is "replaced only after submission": the **omitted** key inherits the existing value, so the form backfilled by `GET` only needs to submit the fields to be changed; but the backend does not distinguish between "unsubmitted" and "submitted empty string", and explicitly submitting `""` will actually clear the key - "leave blank and keep" is a client contract implemented by SPA by discarding blank key fields before submission, not a server-side guarantee. `chat_allowlist` must be a non-empty list, empty arrays will be rejected by `validate_business_wire` (422). Incremental semantics only hold after **existing configuration**: if there is no existing configuration, there is no fallback value, and the patch must come with all required fields, otherwise `422` will be returned (the entry created from scratch is `POST /api/bootstrap`). - `POST /api/business-config/preflight` performs exactly the same validation and client build on the submission body, returning component-by-component conclusions, **without writing, without touching the running worker**: `{ "persisted": false, "validation": { "ok", "errors" }, "components": { "jmap": {...}, "llm": ... } }`. Different from PUT, its submission body must be **complete** `BusinessConfigWire` (no patch merging), so SPA must first merge the value read back by `GET` into the local form before pre-verification. `components` is `null` when the verification has failed (rejected Wire does not need to probe the client again); when `llm_enabled`/`llm_allow_net` is false, `llm` is `null` instead of false `ok`. Authentication is consistent with the management session gate (`config_authorized`) and is deliberately not hung after `DEBUG_TOKEN`. @@ -475,14 +475,14 @@ use thiserror::Error; #[derive(Debug, Error)] pub enum BotError { - #[error("configuration error: {0}")] - Config(String), - #[error("I/O error: {0}")] - Io(#[from] std::io::Error), - #[error("JSON error: {0}")] - Json(#[from] serde_json::Error), - #[error("state error")] - State(#[from] crate::state::StateError), + #[error("configuration error: {0}")] + Config(String), + #[error("I/O error: {0}")] + Io(#[from] std::io::Error), + #[error("JSON error: {0}")] + Json(#[from] serde_json::Error), + #[error("state error")] + State(#[from] crate::state::StateError), } ``` @@ -531,17 +531,17 @@ The design target form once included six additional variants: `Jmap(#[from] jmap - Notification deduplication: the same `email_id` will not be pushed repeatedly. - Text escaping: Message text containing `