Compare commits
7 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
6b0ac99d19 | ||
|
|
44d1c79bef | ||
|
|
ac9f249961 | ||
|
|
e26b5bc6ca | ||
|
|
5c9157f5d7 | ||
|
|
1de59971f6 | ||
|
|
6af676421b |
10
AGENTS.md
10
AGENTS.md
@ -1,6 +1,6 @@
|
|||||||
# dpl 项目维护
|
# dpl 项目维护
|
||||||
|
|
||||||
开始维护前阅读 [维护入口](docs/maintenance/README.md)，再按任务范围阅读架构、业务约束、发布恢复与验收状态。用户最新明确要求优先；文档中的历史状态必须按当前代码和运行环境复核。
|
先读 [CoDocs 项目总览](/Users/hellcat/Documents/CoDocs/项目总览.md) 与首次会话的 [持续维护协议](/Users/hellcat/Documents/CoDocs/00-协作规范/维护协议.md)，再阅读 [维护入口](docs/maintenance/README.md)，再按任务范围阅读架构、业务约束、发布恢复与验收状态。用户最新明确要求优先；文档中的历史状态必须按当前代码和运行环境复核。
|
||||||
|
|
||||||
- 保留用户及其他任务既有修改，提交只包含本次范围；不要重置或清理无关文件。
|
- 保留用户及其他任务既有修改，提交只包含本次范围；不要重置或清理无关文件。
|
||||||
- 默认 Git 发布：提交、推送，再让服务器更新到明确版本并验证；未经明确要求不得打包覆盖。
|
- 默认 Git 发布：提交、推送，再让服务器更新到明确版本并验证；未经明确要求不得打包覆盖。
|
||||||
@ -10,4 +10,12 @@
|
|||||||
- 本地开发优先 dev/热更新；构建用于发布或构建验证。相关检查与命令见 docs/maintenance/ARCHITECTURE.md。
|
- 本地开发优先 dev/热更新；构建用于发布或构建验证。相关检查与命令见 docs/maintenance/ARCHITECTURE.md。
|
||||||
- 后台变更实际验证显示、切换、保存与权限；HTTP 200 不代表界面可用。测试、提交、发布和真实验收分别报告，未执行的步骤不能写成完成。
|
- 后台变更实际验证显示、切换、保存与权限；HTTP 200 不代表界面可用。测试、提交、发布和真实验收分别报告，未执行的步骤不能写成完成。
|
||||||
|
|
||||||
|
## 批量节点接入：默认启动一个子助手
|
||||||
|
|
||||||
|
后续新批次默认使用 [正式适配器入口](docs/maintenance/NODE_BATCH_RUNTIME.md)，由 `configure-site.py` 固定计划，再由 `batch.py` 的 `ssh-site-v1` 运行；动态读取验收账号应得节点集合，不固定节点数量。没有有效验收账号时自动创建本任务受限测试用户并负责禁用收尾；SSH始终经已登记白色机。具体实现、证据续跑和原服务调查边界见该专题；旧冻结计划及其处理脚本不改摘要迁就新实现。仅维护脚本/文档不触发真实接入或线上创建测试用户。
|
||||||
|
|
||||||
|
用户要求执行批量节点接入（明确说批量，或同批接入两个及以上节点）时，主协调者必须默认启动一个子助手，无需再次询问是否使用助手：核查验收助手负责候选服务器、端口、中转与已有资源冲突核查，并独立审查脚本、测试并发/查重/恢复及执行接入验收。主协调者负责统一参数、展示编号、端口、唯一账本及生产写入。完整分工、执行关卡与消耗控制见 [批量接入协作流程](docs/maintenance/NODE_BATCH.md#多智能体协作流程)。
|
||||||
|
|
||||||
|
这项授权针对当前任务的子助手，不自动新建或分叉普通聊天，也不授权向已有聊天发消息。仅讨论方案、维护脚本/文档或单节点任务不自动触发。若子助手工具不可用、并发名额不足或助手失败，说明具体缺口并恢复/等待；未经用户明确允许，不静默省略角色或跳过验收。当前批次已有对应助手时复用，不重复启动。具体线上变更仍以用户本次接入授权为限；项目规则不启动无人值守接入。
|
||||||
|
|
||||||
本文件提供维护入口，不扩大当前任务授权，也不启动无人值守自动维护。
|
本文件提供维护入口，不扩大当前任务授权，也不启动无人值守自动维护。
|
||||||
|
|||||||
@ -2,11 +2,11 @@
|
|||||||
|
|
||||||
## 目标与连接
|
## 目标与连接
|
||||||
|
|
||||||
本项目生产站点为 https://dp-link.net，后台 /dove；后端目录 /www/wwwroot/dp-link.net，服务器逻辑名 site-jp02。通过既定白色跳板连接；具体连接资料参见本机 /Users/hellcat/Documents/CoDocs/项目关系.md 与 vps连接资料.md，不把它们的内容提交到公开仓库。
|
本项目生产站点为 https://dp-link.net，后台 /dove；后端目录 /www/wwwroot/dp-link.net，服务器逻辑名 site-jp02。通过既定白色跳板连接；项目与连接入口参见 [dpl 档案](/Users/hellcat/Documents/CoDocs/10-项目/dpl.md) 和 [私有资源索引](/Users/hellcat/Documents/CoDocs/30-私有资源/README.md)，不把它们的内容提交到公开仓库。
|
||||||
|
|
||||||
本地 http://dpl.loc/dove 已配置到当前 Laravel public/。本机 Nginx 配置 /usr/local/etc/nginx/servers/dpl.loc.conf，PHP-FPM 使用现有 9000 端口。系统级服务重载可能需要 Mac 管理员授权；不持久保存系统密码。
|
本地 http://dpl.loc/dove 已配置到当前 Laravel public/。本机 Nginx 配置 /usr/local/etc/nginx/servers/dpl.loc.conf，PHP-FPM 使用现有 9000 端口。系统级服务重载可能需要 Mac 管理员授权；不持久保存系统密码。
|
||||||
|
|
||||||
本地开发与联调访问规则（用户于 2026-10-11 指定）：本地不访问 tp-link.net，需要访问远端站点时使用 https://dpl.tomshell.com.de 代替；本地站点仍使用 http://dpl.loc。该访问约定不自动修改生产支付回调域名或生产配置。
|
本地开发与联调访问规则：本地站点仍使用 `http://dpl.loc`；本地访问线上项目域名必须查询 [CoDocs 反代表](/Users/hellcat/Documents/CoDocs/20-基础设施/访问与反代.md) 并走明确登记的白色机反代，dpl 主站入口为 `https://dpl.tomshell.com.de`。缺映射/用途不符时询问用户，不直接或变相访问源站。该访问约定不自动修改生产支付回调域名或生产配置。
|
||||||
|
|
||||||
## Git 发布顺序
|
## Git 发布顺序
|
||||||
|
|
||||||
|
|||||||
157
docs/maintenance/NODE_BATCH.md
Normal file
157
docs/maintenance/NODE_BATCH.md
Normal file
@ -0,0 +1,157 @@
|
|||||||
|
# 节点批量接入
|
||||||
|
|
||||||
|
本专题说明批量调度；单节点操作、append 的保护范围与验收仍以 [执行手册](NODE_RUNBOOK.md) 为准，展示编号以 [命名规则](NODE_ONBOARDING.md#节点命名与编号) 为准。
|
||||||
|
|
||||||
|
2026-10-16：新增标准库 Python 调度器、命令适配器和离线进程测试。没有部署、连接服务器或增加生产节点。调度器可以运行经过审阅的任务适配器；**没有内置 Cloudflare/中转后台 API，也没有替使用者实现各站点的 SSH 登录与现场检查。**生产任务必须先按下述协议配置命令，不能将离线测试通过视为生产全链路可用。
|
||||||
|
|
||||||
|
## 文件与运行方式
|
||||||
|
|
||||||
|
2026-10-17新增[正式站点适配器](NODE_BATCH_RUNTIME.md)：后续新批次默认用`configure-site.py`生成`ssh-site-v1`计划，具备动态完整节点验收、受限测试账号、白色机SSH会话复用、证据续跑及阶段事件。已通过本地隔离测试，尚未部署或线上演练。本页下述外部命令适配器协议保留兼容，旧任务脚本和既有冻结摘要不修改；旧命令适配器不透传缓存证据，继续逐阶段现场probe。
|
||||||
|
|
||||||
|
| 文件 | 用途 |
|
||||||
|
| --- | --- |
|
||||||
|
| `scripts/node-onboarding/batch.py` | 参数校验、资源预留、并发、进程间锁、逐阶段账本与恢复 |
|
||||||
|
| `scripts/node-onboarding/command-adapter.py` | 调用经过摘要固定的任务命令；不经过 shell，缺命令返回等待 |
|
||||||
|
| `tests/node-onboarding/test_batch.py` | 临时目录模拟远端，验证并发、重复运行和回包丢失；不加载 Laravel/.env |
|
||||||
|
|
||||||
|
在仓库根目录运行，计划文件与状态目录用绝对路径，放在受保护任务目录。计划只含非秘密参数和引用；不要加入密码、令牌、私钥内容、含订阅令牌的完整 URL 或命令行凭据。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 scripts/node-onboarding/batch.py plan --plan /private/task/batch.json --state-root /private/task/ledger
|
||||||
|
python3 scripts/node-onboarding/batch.py run --plan /private/task/batch.json --state-root /private/task/ledger --jobs 4 --timeout 900 --execute
|
||||||
|
python3 scripts/node-onboarding/batch.py status --plan /private/task/batch.json --state-root /private/task/ledger
|
||||||
|
```
|
||||||
|
|
||||||
|
`plan` 仅做本地参数校验，不预留资源、不访问网络、不生成状态目录。`status` 只读账本，不触发命令或创建账本。`run --execute` 才可能执行写入。退出码 0 表示本次要求完成；2 表示校验失败、等待、冲突、未知或未完成，不应自动当成临时网络错误无限重试。默认并发 4，上限 16；timeout 是每次适配器调用的超时秒数，不是整个批次的时限。
|
||||||
|
|
||||||
|
## 参数格式
|
||||||
|
|
||||||
|
以下使用保留示例地址，不能直接运行生产任务。`provision_commit` 填已审阅、已推送的完整 Git 提交，bundle 与客户端二进制路径必须由前检核验。VPS ID 与实际地址从 CoDocs 对应资源读取，不能从展示编号推断。所有节点放在同一个 `nodes` 数组，SOCKS 端口各不相同。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"mode": "append",
|
||||||
|
"batch_id": "jp-batch-20261016",
|
||||||
|
"context": {
|
||||||
|
"panel_vps": "VPS-001",
|
||||||
|
"gateway_vps": "VPS-002",
|
||||||
|
"jump_vps": "VPS-003",
|
||||||
|
"app_root": "/absolute/panel/app",
|
||||||
|
"panel_url": "https://panel.example.invalid",
|
||||||
|
"subscription_url": "https://subscribe.example.invalid",
|
||||||
|
"provision_commit": "0000000000000000000000000000000000000000",
|
||||||
|
"bundle": "/private/task/reviewed.bundle",
|
||||||
|
"client_binary": "/absolute/gateway/sing-box"
|
||||||
|
},
|
||||||
|
"adapter": ["/absolute/python3", "/absolute/repo/scripts/node-onboarding/command-adapter.py"],
|
||||||
|
"command_profile": {
|
||||||
|
"files": {"/private/task/handler.py": "填写该文件的SHA256"},
|
||||||
|
"phases": {
|
||||||
|
"preflight": {
|
||||||
|
"probe": ["/absolute/python3", "/private/task/handler.py", "--plan", "{plan}", "--node", "{node}", "--phase", "{phase}", "--action", "{action}", "--state-dir", "{state_dir}"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nodes": [{
|
||||||
|
"key": "jp-new-a",
|
||||||
|
"country": "JP", "sequence": 2, "flag": "🇯🇵", "country_name": "日本",
|
||||||
|
"name": "🇯🇵 日本 JP - 02",
|
||||||
|
"vps_id": "VPS-011", "node_address": "192.0.2.11", "listen_port": 24443,
|
||||||
|
"relay_host": "relay.example.invalid", "relay_port": 47011,
|
||||||
|
"tls_domain": "jp-new-a.example.invalid", "formal_group_id": 1,
|
||||||
|
"expected_exit": "192.0.2.11", "socks_port": 19011
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
示例只声明 preflight 命令，其他阶段必然等待。实际任务可为八个阶段逐一登记 `probe`，并在 stage/dns/provision/relay/publish 登记有权限执行的 `apply`。缺少 API 的阶段可以只提供读取实际状态的 probe，由操作者完成外部操作后续跑。不要使用“手工勾选成功”或静态回执替代现场 probe。
|
||||||
|
|
||||||
|
`files` 必须覆盖本地适配器、handler 及其加载的操作配置/脚本；不要只固定 Python 解释器。任一摘要变化停止，不接受在原批次中改命令后直接续跑。远端代码、安装包、二进制也由前检按明确 Git 提交/摘要核验。通用调度器不会自动检查命令是否遵守这些语义，也不会替命令构造 SSH 配置。`adapter` 亦可指向符合协议的专用程序，不必使用命令适配器。
|
||||||
|
|
||||||
|
## 调度与操作协议
|
||||||
|
|
||||||
|
所有候选节点先通过 preflight，才能开始任何 stage。每个节点依次执行以下流程；独立节点同时推进，共享写入由锁串行执行。
|
||||||
|
|
||||||
|
| 阶段 | 并发边界 | 任务命令必须核验的事实 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| preflight | 按 VPS 并发，全批屏障 | 当前代码/工具/系统依赖；VPS 与地址；空闲端口；现有机器/节点/组/名称/域名/中转冲突；目标正式组；保留资源基线 |
|
||||||
|
| stage | 面板串行 | 参数与 manifest 完整匹配；隔离 QA 和新节点确属本任务；创建后回读对象及私有传输文件 |
|
||||||
|
| dns | 全局串行 | 本次域名实际 DNS 指向；有权限的 API 才能 apply；无 API 则等待 |
|
||||||
|
| provision | 按 VPS 并发 | 已推送提交、安装包摘要、独立服务/配置归属、证书/SNI、旧服务保持；不能仅按文件存在判定完成 |
|
||||||
|
| relay | 全局串行 | 独立规则的入口和目标完全匹配；网关侧严格 TLS/SNI 连通；不得占用旧规则 |
|
||||||
|
| verify | 按 VPS 并发 | 从隔离网关获取真实完整 QA 订阅，用 `verify-node.py --run` 验证预期出口/国家并核对实际流量上报；报告绑定本请求 |
|
||||||
|
| publish | 面板串行 | `promote.php --mode=append` 先预览后应用；只开放本次新节点到指定已有组并禁用 QA；不能重写套餐、用户组或订阅地址 |
|
||||||
|
| audit | 按 VPS 并发 | 正式多节点订阅完整、旧节点/网关/套餐/用户关联/订阅地址符合基线、QA 禁用且订阅拒绝/不下发；新节点仍可用 |
|
||||||
|
|
||||||
|
preflight/verify/audit 仅调用 `probe`，没有 apply。verify 的 probe 会产生隔离测试流量，可以启动/停止本次临时客户端；不能修改正式配置。已经完成 audit 的节点，重跑只做 preflight 与完整 audit，避免重新激活 QA。尚未完成的节点会从第一阶段重新逐项 probe，确认后才越过。
|
||||||
|
|
||||||
|
适配器接受 `--plan FILE --node FILE --phase PHASE --action probe|apply --state-dir DIR`。node 文件包含 `request_sha256`，它绑定 context 与完整节点参数。标准输出必须只含一条 JSON，字段如下：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"key":"jp-new-a","phase":"stage","request_sha256":"从node文件原样读取","status":"absent","retry_safe":true}
|
||||||
|
```
|
||||||
|
|
||||||
|
| status | 含义与调度行为 |
|
||||||
|
| --- | --- |
|
||||||
|
| ready | 当前现场满足本阶段所有要求，可以继续；不能只依据上次成功日志 |
|
||||||
|
| absent | 已权威确认本阶段尚未执行；只有同时明确 `retry_safe: true` 才能调用 apply |
|
||||||
|
| waiting | 外部依赖未就绪或需人工处理，本节点停下；不占工作线程反复轮询 |
|
||||||
|
| unknown | 现场无法确认，不写入，不自动重试 apply |
|
||||||
|
| conflict | 资源占用、参数漂移或归属冲突；阻止该节点，通知其余任务停止继续写入 |
|
||||||
|
|
||||||
|
apply 返回成功后仍必须 probe 确认。退出码非零、输出损坏、身份不符、超时或 Ctrl-C 后，远端结果一律按未知处理；重跑仍从现场读取。apply 命令必须支持安全处理部分完成的现场；不能把“有进程/有目录但未完成”随意返回 absent。stage 的现有对象没有匹配 manifest、或事务结果与 manifest 不一致时应 conflict/unknown，人工核实，不能自动创建第二套对象。
|
||||||
|
|
||||||
|
命令适配器的占位符仅支持 plan/node/state_dir/key/phase/action；每个元素作为独立 argv 传递，不解释 shell。操作脚本需要参数时自行读取受保护的 JSON。禁止在 argv/标准输出加入凭据；密码与令牌由 handler 按 CoDocs 私有引用按需读取，SSH 保持经白色机跳转；本地 HTTP 继续使用明确登记的反代。不要为了批量化改变既有访问规则。
|
||||||
|
|
||||||
|
## 查重、锁与断点恢复
|
||||||
|
|
||||||
|
计划内拒绝重复的部署 key、展示名、国家编号、VPS ID、实际落地 IP、TLS 域名、中转入口域名+端口与网关 SOCKS 端口。展示名格式与旗帜/国家/两位编号必须匹配。**这些是本地候选查重，已在线资源的查重必须由 preflight/stage 实际查询。**同一物理地址使用不同 VPS 别名也会被拦下。
|
||||||
|
|
||||||
|
状态目录是唯一协调入口：SQLite 事务原子预留身份，同目录 fcntl 锁协调进程。VPS/IP/TLS 与中转入口预留跨面板；名称/编号/key 按面板应用划分；SOCKS 按网关划分。相同节点参数可重跑；不同参数不能借用同一批次或占用已有预留。预留不自动释放，不提供强制接管命令。
|
||||||
|
|
||||||
|
所有参与聊天/进程必须使用**同一台协调机、同一个状态目录**。另建账本、复制状态目录、在另一台主机启动独立协调器都不能获得跨机锁保证。对共享面板的非批量手工操作也必须协调暂停，不能指望本地锁约束后台人工修改。每次写入前的远端归属/参数检查仍必要。
|
||||||
|
|
||||||
|
中断后保留原计划、账本、远端 manifest 和操作日志。先用 status 查看，再以同一计划、同一状态目录运行相同 run 命令；不要删除账本、改 batch_id 或清除 manifest 来“解除报错”。冲突需先核对原始请求与现场，不能通过修改参数静默接管。未知结果宁可停住；恢复不保证所有断点都能无人干预推进。
|
||||||
|
|
||||||
|
SIGINT/SIGTERM 会停止派发后续写入，终止当前本地适配器进程组；**这不保证已发出的远端命令停止，也不自动回滚**。其他已开始的节点可能已完成步骤。已完成节点保留成果；未完成节点留在其最后安全阶段。需要回退时按执行手册逐字段预览与回退，不做全批删除。
|
||||||
|
|
||||||
|
### 原服务自行重启时的恢复
|
||||||
|
|
||||||
|
PID/启动时间漂移应先停止写入，不直接刷新基线。读取服务生命周期、退出原因、重启策略、内核OOM与本任务命令记录，区分本次变更影响和原服务自身故障。确认已恢复时，保留原基线只读副本及SHA256、事故日志、两次稳定采样、443监听归属与当前配置/二进制/unit哈希；没有历史哈希就明确不能证明前后相同。独立助手复核后，只更新本任务进程比对所需字段，保存diff、原因与证据路径；不修改原服务、不删除账本、不改变冻结代码/计划。再次漂移必须重新调查，不能滚动更新绕过保护。2026-10-17 jp03 QUIC sniffer自身panic的实际处理见 [STATUS](STATUS.md)。
|
||||||
|
|
||||||
|
## 多智能体协作流程
|
||||||
|
|
||||||
|
用户于 2026-10-16 明确授权：以后执行批量节点接入，默认启动一个子助手，按本节分工执行，不再重复询问是否需要助手。触发入口见项目根 [AGENTS.md](../../AGENTS.md)。这里的“启动”由主协调者使用子助手工具完成；`batch.py` 本身不调用模型或创建助手。明确要求批量，或同批接入两个及以上节点时触发；仅讨论、修改脚本/文档、单节点接入不自动触发。用户当次明确调整角色或数量时，以最新要求为准。
|
||||||
|
|
||||||
|
| 角色 | 固定职责 | 写入与协作边界 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 主协调者 | 明确本批范围，统一参数、展示编号与端口，固定计划，持有唯一账本，安排生产写入并汇总结果 | 只有主协调者启动本批正式调度和共享资源写入；不以助手已派发代替工作完成 |
|
||||||
|
| 核查验收助手 | 接入前核查候选服务器身份、既有节点/机器、端口、中转、DNS 和归属冲突；独立审查脚本与参数、测试并发/查重/恢复；开放前后核对 QA、正式订阅和旧资源基线 | 分阶段完成两类职责，返回每节点结论、证据及缺口；不自行分配最终编号、创建节点、改后台或另起调度。真实联网验收的对象、端口和时机由主协调者统一安排 |
|
||||||
|
|
||||||
|
执行顺序：
|
||||||
|
|
||||||
|
1. 主协调者在生产写入前启动/复用一个核查验收助手，给出当前任务范围、必要资料入口、受保护资源、分阶段输出和完成标准。当前批次已存在对应子助手时继续使用，不在每个节点或每次续跑时重新创建。
|
||||||
|
2. 助手分阶段核查资源与审查方案；主协调者并行准备参数与调度。助手完成冲突核查和相关隔离测试/审查后，主协调者解决影响写入的缺口，固定同一参数单，再运行全批 preflight。检查发现变更时重新核对受影响项，不把旧回执沿用到不同参数。
|
||||||
|
3. 独立节点由脚本按 `--jobs` 并发，面板/DNS/中转按共享锁执行。正式开放前，核查验收助手核对本节点隔离 QA、严格 TLS、实际出口与上报证据；主协调者确认后执行 publish。助手不得另起同批调度、重复 stage、共用浏览器会话切换账户或绕开账本。
|
||||||
|
4. 正式开放后，核查验收助手检查正式订阅、QA 禁用/拒绝、旧资源基线并报告通过或具体缺口。主协调者核实后逐节点报告完成、等待或需用户操作；批次全部完成必须有对应实际验收，不能以子助手结束作为完成证明。
|
||||||
|
5. 子助手不可用、并发名额不足或失败时，主协调者明确报告，复用/恢复角色或等待名额；不自动追加更多助手、跳过必需角色或把未知写成通过。暂停和停止指令必须传达给子助手，并停止派发后续写入；已发出的远端操作仍按断点恢复规则回读。
|
||||||
|
|
||||||
|
主协调者与一个子助手分别消耗处理其输入和输出所需的 token，总量不是固定倍数。主协调者默认仅传必要背景、资料路径、节点参数与任务边界，避免复制整段历史；每项检查明确归属，汇报用结论、证据位置与缺口，不反复粘贴完整日志。独立验收保留必要的交叉核对，不为节省 token 省略验收。没有固定 token 配额或价格承诺。
|
||||||
|
|
||||||
|
该常驻授权仅允许启动当前任务的子助手；不自动新建/分叉侧边栏普通聊天，向已有聊天派消息仍需用户明确授权。节点并发数由脚本控制，助手数保持一个；不会为十个节点启动十个助手。项目规则不扩大实际生产变更、凭据、访问路线或后台自动化授权。
|
||||||
|
|
||||||
|
## 本地验证
|
||||||
|
|
||||||
|
本地测试：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 -m unittest discover -s tests/node-onboarding -p 'test_batch.py' -v
|
||||||
|
```
|
||||||
|
|
||||||
|
本次 11 项测试通过，覆盖：plan/status 无副作用，计划内重复与命名约束，全批前检屏障，独立节点重叠执行/面板串行，完整重跑不重复写，模拟写成后回包丢失，SIGTERM 中断后续跑，跨批预留/参数漂移，两进程竞争、错误身份回执、命令摘要变更及配置完整的适配器流程。测试没有验证生产命令的正确性、DNS/中转 API、真实 SSH/网络、现场负载或十节点耗时。生产批次的候选参数、命令 profile 与现场验收须在具体获授权的接入任务中另行核实。
|
||||||
|
|
||||||
|
|
||||||
|
2026-10-17追加面板隔离测试：`php vendor/bin/phpunit --no-configuration tests/node-onboarding/PanelChangeIsolatedTest.php`，9 tests / 65 assertions通过。真实Eloquent与SQLite内存库，无.env/Redis/队列/网络；覆盖append保护范围、QA禁用、幂等、事务失败/通知失败恢复和冲突拒绝。NodeSync为测试替身，生产通知与用户下发仍需现场验收。
|
||||||
|
|
||||||
|
2026-10-17：已在用户授权的三日本节点批次使用同一冻结计划/账本及一个核查验收助手完成生产接入与分阶段恢复；DNS/中转仍由注册后台界面保存后现场probe，无通用服务商API写入。精确版本、五节点订阅/QA/原资源验收及进程基线事故见[NODE_ONBOARDING](NODE_ONBOARDING.md#2026-10-17-三个日本节点批量接入)。本轮三节点结果不推导十节点耗时或无人值守能力。
|
||||||
121
docs/maintenance/NODE_BATCH_RUNTIME.md
Normal file
121
docs/maintenance/NODE_BATCH_RUNTIME.md
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
# 批量接入正式适配器
|
||||||
|
|
||||||
|
本专题是后续批量接入的默认技术入口。适配器实现、隔离测试及文档于2026-10-17完成；**本轮没有新增线上节点、部署新运维版本或运行真实SSH/客户端验收**。上一轮三节点实际验收使用的是已冻结旧任务适配器，不能套用为新实现的生产验收。
|
||||||
|
|
||||||
|
`batch.py`保留原命令适配器兼容，新计划默认用`configure-site.py`生成`runtime.kind=ssh-site-v1`。旧计划、账本、任务handler一律保留，不修改源码摘要后直接续跑旧任务。生产操作仍须具体接入授权、独立助手审核、Git运维部署及现场验收。
|
||||||
|
|
||||||
|
## 文件与职责
|
||||||
|
|
||||||
|
| 文件 | 职责 |
|
||||||
|
| --- | --- |
|
||||||
|
| `configure-site.py` | 本地合并明确站点参数，固定解释器、Paramiko版本、源码与非秘密配置摘要；不联网、不写账本、不覆盖旧计划 |
|
||||||
|
| `site_adapter.py` | 复用正式stage/provision/promote流程；按国家、组、批次和路径参数执行，按需连接服务器，进行动态正式订阅验收 |
|
||||||
|
| `ssh_pool.py` | 线程内复用白色机及下游SSH会话；下游强制通过白色机direct-tcpip，不提供直连回退 |
|
||||||
|
| `panel-acceptance.php` | 开工前选择有效正式组账号；缺少时创建本批受限非管理员测试用户，记录归属并收尾禁用 |
|
||||||
|
| `acceptance.py` | 动态集合、逐节点结果、原始证据摘要、QA身份/禁用/拒绝/下发排除和时效的完整自动审计 |
|
||||||
|
|
||||||
|
## 准备与Git部署
|
||||||
|
|
||||||
|
先按CoDocs确认VPS、白色机访问路线、正式组、展示编号及资源冲突。Python执行环境使用`requirements.txt`的Paramiko 5.0.0；配置与执行必须使用同一解释器。安装来源限官方包源/已审核环境，不临时导入任务目录中的未知transport模块。
|
||||||
|
|
||||||
|
生产前先提交并推送源码，再生成、推送和部署独立运维Git版本：保留原安装脚本，加入`panel-acceptance.php`及其`operations.php`，还有stage/promote/rollback/readiness/verify-node入口；面板、网关和目标节点的对应运维目录必须为计划中完整`provision_commit`且无工作区差异。配置文件和凭据不入Git。独立快照由正式`bundle-ops.py`生成，保持基础运维树、工作目录和真实Git索引不变，仅复制10个明确的已提交运维文件，默认拒绝已有分支或bundle，不联网：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 scripts/node-onboarding/bundle-ops.py --source FULL_SOURCE_COMMIT --base FULL_OPS_BASE_COMMIT --branch codex/node-tools-BATCH --bundle /private/task/provision.bundle
|
||||||
|
```
|
||||||
|
|
||||||
|
同一运维分支的后续修订只有显式`--advance`才允许，且`--base`必须精确等于当前分支头；新提交以该base为父并通过旧值比较更新，仅快进，不强推或覆盖并行修改。bundle仍要求新路径。随后按已授权任务推送返回的运维分支、审核commit/bundle摘要，再经白色机部署到明确版本。当前源码/运维快照的提交推送与远端部署分别报告；本轮不自动切换任何生产应用或旧任务的运维目录。
|
||||||
|
|
||||||
|
适配器在stage前检查本地依赖、远端Git版本及工具、正式组和账号、节点既有配置、独立端口、原服务、证书归属与钩子。既有证书无本任务owner、全局证书钩子或目标证书的额外pre/post钩子均停止。缺依赖须在准备阶段解决，不先创建一批面板对象后再临时安装工具。
|
||||||
|
|
||||||
|
Certbot准备不能只检查`command -v`：还须实际运行`certbot --version`。2026-10-17后续批次两台Debian10虽有Snap Certbot、renew timer和生成的profiles，但既有AppArmor加载服务缺失，证书命令报`missing profile`。项目提供`load-certbot-apparmor.sh`、`dpl-certbot-apparmor.service`及renew drop-in作为该情形的限定修复：只校验并加载已安装Certbot/core生成的profiles，保留其原有模式；不启用全局AppArmor服务、不关闭安全校验、不重启XrayR。仅在现场确认符合该缺口的主机部署到独立Git目录`/opt/dpl-certbot-security/provision`，先`--check`、再加载并验证工具；原续期命令前重检profiles，启动时也加载。安装前保存原单位/drop-in存在性及已加载profile清单；失败不签发，回退先确认无证书进程，再撤销本任务unit/drop-in并按原清单处理本次新加载项，不卸载其他任务profile。本段修复的实际部署及验收状态见[STATUS](STATUS.md)。
|
||||||
|
|
||||||
|
## 本地参数文件
|
||||||
|
|
||||||
|
全部放在权限700的任务目录，文件600。基础计划沿用[NODE_BATCH](NODE_BATCH.md#参数格式)，包括本批任意数量、任意已支持国家的候选节点；不固定五节点、日本、组1或任务日期。新增站点配置JSON如下，示例路径必须换为实际已核实值：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ops_root": "/opt/dpl-batch/provision",
|
||||||
|
"node_root": "/opt/dpl-node/provision",
|
||||||
|
"batch_backup_root": "/root/codex-backups/dpl/BATCH",
|
||||||
|
"node_backup_root": "/root/codex-backups/dpl/BATCH-nodes",
|
||||||
|
"qa_root": "/var/lib/dpl-sgw/BATCH",
|
||||||
|
"provider_snapshot": "/private/task/providers.json",
|
||||||
|
"exit_registry": "/private/task/exits.json",
|
||||||
|
"ssh_config": "/private/task/ssh.json",
|
||||||
|
"preserved_service": {
|
||||||
|
"unit": "XrayR", "binary": "/usr/local/XrayR/XrayR",
|
||||||
|
"config": "/etc/XrayR/config.yml", "port": 443
|
||||||
|
},
|
||||||
|
"formal_socks_port": 19203,
|
||||||
|
"client_sha256": "填已核验官方客户端文件的64位SHA256",
|
||||||
|
"gateway_containers": ["dpl-subscribe-gateway", "dpl-subscribe-admin", "subscribe-gateway", "subscribe-admin"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
批次备份目录必须独立且固定，不跨实际批次复用旧资源基线。新节点端口、正式客户端端口均查重预留；共享网关客户端还使用同一账本内的进程锁。当前适配器用于独立AnyTLS节点及明确列出的既有服务；不自动把白色机、面板或网关当节点宿主。
|
||||||
|
|
||||||
|
`ssh.json`含`known_hosts`绝对路径及`hosts`映射，key为准确CoDocs VPS ID。每台登记`host`、`port`、`username`、用户已批准的`host_key_sha256`（OpenSSH SHA256格式）、可选`credential_ref`；白色机额外登记`role=white_jump`，ID须与`context.jump_vps`一致。候选IP必须与冻结计划相同。`credential_ref`只引用既定安全存储中的600文件。默认JSON支持password/key_filename/passphrase；既有CoDocs Markdown可设`credential_format=codocs-password`，仅按原文件中唯一的“密码：”字段在内存读取，不能为了适配而复制或改写原凭据正文。内容不得进入计划、日志、argv或Git。配置和出口登记摘要固定，凭据只按需读取，不默认自动轮换。
|
||||||
|
|
||||||
|
只从known-hosts文件选择与冻结配置中已批准指纹完全相符的公钥；找不到匹配时在读取/发送凭据前停止。全局known-hosts增加无关记录无需改变本批受信任指纹。不关闭主机密钥检查，不自动TOFU；首次信任先按访问规则核验/批准精确指纹，保存公开known-host条目。会话丢失只标记未知，不自动重发安装/开放命令；下次仍从现场probe恢复。
|
||||||
|
|
||||||
|
`exits.json`按SNI登记独立已知预期，例如：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"node.example.invalid":{"expected_exit":"192.0.2.11","country":"JP"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
覆盖该账号当前应获得的全部旧节点与新候选，不能从本次实测结果反填预期来让检查通过。增加或改变节点时，按CoDocs/授权现场证据更新新任务的登记；遇到未知预期或不支持协议停止，不能静默省略。当前站点适配器针对IPv4 A记录、固定端口AnyTLS；端口范围或其他协议停止而非静默省略，扩展需另补验证能力。
|
||||||
|
|
||||||
|
`providers.json`是实际已登记后台UI或授权官方API的近期观察，最多900秒有效，不使用语言相关DOM字符串：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"captured_at": 0,
|
||||||
|
"dns": [{"type":"A","name":"node.example.invalid","address":"192.0.2.11","proxy":false}],
|
||||||
|
"relays": [{"name":"dpl-new-anytls","host":"relay.example.invalid","port":47011,"target":"192.0.2.11","listen_port":24443,"proxy_protocol_in":false,"proxy_protocol_out":false}]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`captured_at`填实际采集Unix时间。观察文件可以刷新，不能把静态勾选当现场证据。资源存在但参数/归属不符停止；尚未创建的DNS/中转仍按注册后台批量操作、更新观察后续跑。本实现没有新增Cloudflare/中转官方API写入权限，也不默认直连其源站。
|
||||||
|
|
||||||
|
## 执行与等待
|
||||||
|
|
||||||
|
```sh
|
||||||
|
/path/to/reviewed/python scripts/node-onboarding/configure-site.py --plan /private/task/base.json --site /private/task/site.json --output /private/task/frozen.json
|
||||||
|
/path/to/reviewed/python scripts/node-onboarding/batch.py plan --plan /private/task/frozen.json --state-root /private/task/ledger
|
||||||
|
/path/to/reviewed/python scripts/node-onboarding/batch.py run --plan /private/task/frozen.json --state-root /private/task/ledger --jobs 3 --timeout 900 --execute --events
|
||||||
|
```
|
||||||
|
|
||||||
|
配置/plan只做本地工作，不创建线上测试用户。`run --execute`进入准备前检时才允许准备验收账号，随后全批前检屏障通过才stage。`--events`输出无秘密的阶段状态JSON行，最后一行为批次结果；不加此参数保持原单JSON输出协议。账本`events.jsonl`仅追加状态变化，等待时使用进程输出/事件，不每几秒重复查全部状态。只有有新依赖证据、外部DNS/中转变化或用户恢复指令时续跑，不能无限重试。
|
||||||
|
|
||||||
|
SSH会话仅在本次进程、同一工作线程内复用，线程不共用可变SSH客户端；进程退出关闭连接，不建立后台守护进程。每个阶段通过lazy连接只访问必要主机。Paramiko channel有远端命令超时；进程内模式不承诺`--timeout`能杀死已发送远端命令，停止后仍须核实实际结果。
|
||||||
|
|
||||||
|
## 动态验收和测试用户
|
||||||
|
|
||||||
|
准备阶段先按正式组、banned、额度和有效期选既有账号，优先管理员且最多检查20个候选，避免全表巡检客户凭据；检查真实`ServerService::getAvailableServers`及`getAvailableUsers`，包括插件过滤；候选被实际节点下发过滤时不算可用。未找到可用账号时自动创建本任务测试用户：非管理员、100MiB、1小时、1设备、无订单/套餐改写及通知消息。私有journal在DB提交前记录身份，丢失回包时恢复同一用户，不重复创建或接管同名用户。
|
||||||
|
|
||||||
|
新节点仍沿用各自隔离QA→严格TLS/出口/上报→独立助手按精确报告审核→append开放并禁用QA。适配器直接生成`key-evidence.json`，包含原始报告字节SHA256、请求摘要、QA身份、报告时间及最新在线/双向上报状态，600原子保存，不再需要任务专用采集脚本。助手完成独立审核后，主协调者在同目录记录`key-reviewed.json`：`request_sha256`、`report_sha256`、`qa_user_id`、`verified_at`必须与该报告完全相同；未审核不得自动填批准文件。全部本组候选开放后，正式验收按账号**当前实际应获得的集合**生成完整快照，与客户端完整订阅逐条校验，并逐条实际联网。集合绑定节点ID、标签、端点、SNI、独立预期出口/国家；验收后再取一次集合，增加、删除或改变任一项均阻止旧报告通过，不固定节点数。旧节点同样必须覆盖。
|
||||||
|
|
||||||
|
每组正式客户端串行运行，保留完整订阅DNS/路由，只给诊断目的地固定出站。新建测试用户可能等待节点拉取用户列表；只对通用客户端运行失败最多做3轮诊断读测试，间隔15秒且共用900秒预算。结构、TLS或出口断言错误不重试；不重放生产写入、不另建用户来绕过失败。证据绑定完整计划摘要、节点集合摘要、原始报告及QA拒绝文件SHA256、QA实际ID、403、banned、实际下发排除及900秒时效。任一缺项或摘要/集合变化失败。旧节点关键配置、全部套餐、原用户关联和订阅地址按批次原始基线核对，网关容器ID/启动时间/重启数再次核对。
|
||||||
|
|
||||||
|
正式验收成功或失败都禁用本任务新建的正式测试用户；既有账号不被禁用或改密码/权限。进程正常退出在finally收尾，即使等待或部分前检失败也尝试清理；遇到断网/SIGKILL无法保证立即清理，记录的归属、1小时有效期与额度是兜底，恢复后先核对旧journal。已关闭用户绝不重新启用；下一次需要用户时创建新受限租约，保留历史并在最终审计核对其403/下发排除。完成要求由服务端实际集合/订阅拒绝证明，不承诺立即踢断已经建立的客户端会话。
|
||||||
|
|
||||||
|
## 续跑与原服务调查
|
||||||
|
|
||||||
|
preflight/publish/audit始终现场执行。只有先前状态为done、收到真实前检依赖摘要、请求/代码/config未变、证据未过期，才可复用stage/dns/provision/relay/verify。前驱依赖变化使后续缓存失效；running/unknown/conflict绝不跳过。旧适配器不提供这些证据时仍按原逐阶段probe逻辑执行。生产写入仍先probe、持久化意图、apply后probe，不为速度跳过归属检查。
|
||||||
|
|
||||||
|
原服务保存最初unit/程序/配置SHA256、监听、active及PID/启动时间/重启数。配置/程序/监听改变或不可用停止；仅进程变化分类为“需调查”，不宣称是DPL导致，也不自动刷新基线。记录观察后由操作者及助手核查生命周期、OOM、操作时间线和稳定采样；认可时在本节点私有备份登记`preserved-service-review.json`，绑定最初/本次观察摘要、原因及证据SHA256。原基线保持；再次重启不匹配旧审核，须重新调查。
|
||||||
|
|
||||||
|
## 验证范围
|
||||||
|
|
||||||
|
新增Python测试覆盖动态1/2/7/23节点、集合变化/漏旧节点、出口/TLS/QA身份/摘要/时效、白色机跳转及严格密钥策略、线程会话复用/断线、依赖续跑及未知/过期状态、原服务精确调查回执、远端脚本与PHP构造。PHP隔离测试使用真实Eloquent/SQLite和ServerService，覆盖自动受限用户、既有管理员复用、真实下发、无效账号、journal恢复、关闭后新租约、归属漂移、插件排除及失败收尾；不加载.env/应用provider/Redis/HTTP。生产停机、失败回退、远端真实会话复用性能及十节点耗时尚未演练，不承诺固定提速倍数。
|
||||||
|
|
||||||
|
隔离回归入口（不使用项目默认PHPUnit配置）：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 -B -m unittest discover -s tests/node-onboarding -p 'test_*.py' -q
|
||||||
|
php vendor/bin/phpunit --no-configuration tests/node-onboarding/PanelChangeIsolatedTest.php
|
||||||
|
php vendor/bin/phpunit --no-configuration tests/node-onboarding/AcceptanceUserIsolatedTest.php
|
||||||
|
```
|
||||||
@ -1,7 +1,25 @@
|
|||||||
# 节点与订阅网关接入
|
# 节点与订阅网关接入
|
||||||
|
|
||||||
|
后续接入的参数与命令唯一入口：[节点接入执行手册](NODE_RUNBOOK.md)。首次初始化与追加节点分别执行，不自动改订阅地址、套餐或用户关联。
|
||||||
|
|
||||||
首节点历史核对日期：2026-10-11。本文是后续新增 dpl 节点的执行入口。首节点部署、服务器侧完整订阅联网与正式套餐开放已完成；用户明确要求跳过本地连通测试。Cloudflare 本聊天已正常访问，DNS/严格 TLS、有效订阅响应、HTTP/2 与 gzip 解码已复核；两个专用订阅区域的全域缓存绕过、两机场订阅路径 BIC 例外及原机场严格回源已部署并复验。已实施范围与可选未实施调优见下文。凭据、机器令牌、用户 UUID 与订阅令牌只存受保护文件，不写入本文或 Git。
|
首节点历史核对日期：2026-10-11。本文是后续新增 dpl 节点的执行入口。首节点部署、服务器侧完整订阅联网与正式套餐开放已完成；用户明确要求跳过本地连通测试。Cloudflare 本聊天已正常访问，DNS/严格 TLS、有效订阅响应、HTTP/2 与 gzip 解码已复核；两个专用订阅区域的全域缓存绕过、两机场订阅路径 BIC 例外及原机场严格回源已部署并复验。已实施范围与可选未实施调优见下文。凭据、机器令牌、用户 UUID 与订阅令牌只存受保护文件，不写入本文或 Git。
|
||||||
|
|
||||||
|
## 2026-10-17 三个日本节点批量接入
|
||||||
|
|
||||||
|
按用户授权新增日本 JP - 02/03/04，分别部署于 node-jp03/04/05；展示编号、VPS、部署key和数据库ID独立。三台均为本次登录核实的 Debian11/x86_64、JP IPv4出口；新建独立 AnyTLS 24443、DNS only A记录及中转规则 dpl-jp03/04/05-anytls，入口47203/47204/47205，总规则25→28。收发Proxy Protocol均关闭，未改原443规则。
|
||||||
|
|
||||||
|
三个隔离QA分别实测严格TLS/SNI、实际IPv4出口与国家、双向流量及在线心跳，经一个核查验收助手逐报告摘要复核后，主协调者以append预览/应用，只将新节点加入基础节点组1并禁用本次QA。QA账号19/20/18分别属于jp03/04/05；节点/机器ID分别4/5/3，不能据数据库ID推断展示顺序。
|
||||||
|
|
||||||
|
本次使用既有正式组有效管理员账号ID1，确认其在五节点实际下发集合中；未为正式验收另建账号或更改该账号权限/密码，不代表普通付费用户、购买或套餐流程验收。正式有效订阅实取后包含美国01和日本01至04共五个AnyTLS出站，官方sing-box 1.14.3逐一实际联网；五个端点均严格TLS，出口/国家分别与节点登记相符，完整订阅DNS/路由校验通过。三个QA订阅均403且不在节点实际用户下发集合；既有全部套餐、正式用户组/套餐关联、订阅地址和旧两节点关键配置逐字段符合基线。四个新旧网关容器ID、启动时间及重启计数保持原值；未部署生产应用代码、改价格/套餐或重启共享网关。
|
||||||
|
|
||||||
|
三张独立证书有效期至2027-01-14 UTC，沿用既有Snap Certbot调度，独立renewal与仅重启dpl-node的deploy hook；逐台续期dry-run成功，原XrayR 443保留。例外：jp03旧XrayR在DPL尚未安装时因既有QUIC sniffer越界panic，经systemd自动恢复（198435→215765）；前检自动停止后保留原基线/哈希、事故日志、两次稳定采样和diff，经独立复核仅更新任务进程检查字段。其余两台旧PID保持。未修改旧XrayR配置、程序或服务；无原配置哈希，不宣称前后哈希一致。不能把此次结果写成三台原PID全程不变。
|
||||||
|
|
||||||
|
源码6af6764已提交并推送；独立运维提交b8d10fac963d63825c97d36d41ff7f525116bbfd已推送codex/node-batch-20261017并部署到面板/网关/三台新节点。PHP语法检查、9项/65断言面板隔离测试与11项批量进程测试通过；真实通知/用户下发由本次在线和实际用户集合补充验证，未演练生产故障回退。
|
||||||
|
|
||||||
|
唯一任务计划与账本：codex/output/node-batch-20261017/batch.json、ledger/jp03-jp04-jp05-20261017-reviewed；证据包括*-evidence.json、formal-evidence.json、final-metadata.json和后台截图，均不入Git。远端面板/节点私有备份为/root/codex-backups/dpl/node-onboarding-20261017-jp03/04/05；面板全批旧资源基线在/root/codex-backups/dpl/node-batch-20261017；网关正式验收在/var/lib/dpl-sgw/qa-batch-20261017。面板回退按各自promotion-journal逐字段预览，不恢复整库；DNS/中转/节点服务分别专项回退。
|
||||||
|
|
||||||
|
自动audit会检查三条新端点集合、报告摘要和QA403；冻结任务适配器未自动逐一核对旧两节点结果、每个出口IP及QA拒绝文件摘要/身份/时间，须由主协调与助手独立对照完整五节点实测报告及原始文件补齐，本次按此执行。三节点账本已全部complete。管理面板浏览器已验证节点/服务器列表切换、五台在线及保存后的名称、端口、基础节点组显示，未额外提交编辑表单默认值。未进行本机/手机客户端GUI、TUN、多地区或多设备并发测试；三节点调度验证不等于十节点耗时承诺。
|
||||||
|
|
||||||
## 2026-10-16 日本节点接入
|
## 2026-10-16 日本节点接入
|
||||||
|
|
||||||
本轮选择 `node-jp02`；现场确认 IPv4 出口地区 JP、24443 空闲、无既有 DPL 服务，生产原来只有美国节点，故未重复接入。新节点运行接入及正式开放完成，管理面板浏览器验收已通过；下文首节点历史验收范围不自动扩大到本轮。
|
本轮选择 `node-jp02`；现场确认 IPv4 出口地区 JP、24443 空闲、无既有 DPL 服务，生产原来只有美国节点，故未重复接入。新节点运行接入及正式开放完成，管理面板浏览器验收已通过；下文首节点历史验收范围不自动扩大到本轮。
|
||||||
@ -23,12 +41,26 @@
|
|||||||
|
|
||||||
订阅：客户端 → c2.deeplink-subscribe.com（Cloudflare 橙云 HTTPS）→ gateway-01 宿主 Nginx → dpl 独立 SubSieve → dpl 面板。
|
订阅：客户端 → c2.deeplink-subscribe.com（Cloudflare 橙云 HTTPS）→ gateway-01 宿主 Nginx → dpl 独立 SubSieve → dpl 面板。
|
||||||
|
|
||||||
流量：客户端 → haiwi.lshan.eu.org:47101 → node-us01 64.44.115.18:24443 → 目标网络。AnyTLS 使用 us01.deeplink-subscribe.com 的有效证书，订阅要求严格校验。普通 Cloudflare 橙云只用于订阅 HTTP，不用于原生 AnyTLS 流量。
|
流量链路按部署key分列；2026-10-17正式五节点订阅逐条联网复验通过：
|
||||||
|
|
||||||
| 资源 | 本次新增 | 必须保留的原有资源 |
|
| 部署 key | VPS（不代表展示编号） | 中转入口 | 落地 | TLS 名称 | 证据 |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| us01 | node-us01 | haiwi.lshan.eu.org:47101 | 64.44.115.18:24443 | us01.deeplink-subscribe.com | 2026-10-11 首节点验收 |
|
||||||
|
| jp02 | node-jp02 | haiwi.lshan.eu.org:47202 | 193.111.31.52:24443 | jp02.deeplink-subscribe.com | 本文2026-10-16日本节点接入 |
|
||||||
|
| jp03 | node-jp03 | haiwi.lshan.eu.org:47203 | 185.200.65.70:24443 | jp03.deeplink-subscribe.com | 本文2026-10-17批量接入 |
|
||||||
|
| jp04 | node-jp04 | haiwi.lshan.eu.org:47204 | 85.113.70.174:24443 | jp04.deeplink-subscribe.com | 本文2026-10-17批量接入 |
|
||||||
|
| jp05 | node-jp05 | haiwi.lshan.eu.org:47205 | 45.153.246.171:24443 | jp05.deeplink-subscribe.com | 本文2026-10-17批量接入 |
|
||||||
|
|
||||||
|
客户端 → 对应中转入口 → 对应落地 → 目标网络。五条链路均使用严格 AnyTLS TLS；普通 Cloudflare 橙云用于订阅 HTTP，不用于原生 AnyTLS 流量。完整对象关联见[CoDocs DPL档案](/Users/hellcat/Documents/CoDocs/10-项目/dpl.md)。
|
||||||
|
|
||||||
|
| 资源 | DPL独立资源（各次接入） | 必须保留的原有资源 |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| node-us01 | dpl-node，TCP 24443，机器/节点 ID 1 | XrayR，TCP 443 |
|
| node-us01 | dpl-node，TCP 24443，机器/节点 ID 1（2026-10-11） | XrayR，TCP 443 |
|
||||||
| 中转 | dpl-us01-anytls，47101 → 24443 | us01，46101 → 443；其他规则 |
|
| node-jp02 | dpl-node，TCP 24443，机器/节点 ID 2（2026-10-16） | XrayR，TCP 443；原中转 46202 → 443 |
|
||||||
|
| node-jp03 | dpl-node，TCP 24443，机器/节点 ID 4（2026-10-17） | XrayR，TCP 443；自身panic恢复例外见本次记录 |
|
||||||
|
| node-jp04 | dpl-node，TCP 24443，机器/节点 ID 5（2026-10-17） | XrayR，TCP 443 |
|
||||||
|
| node-jp05 | dpl-node，TCP 24443，机器/节点 ID 3（2026-10-17） | XrayR，TCP 443 |
|
||||||
|
| 中转 | 五条独立AnyTLS规则，链路见上表 | 原443链路及其他规则 |
|
||||||
| gateway-01 | dpl-subscribe-gateway、dpl-subscribe-admin | subscribe-gateway、subscribe-admin |
|
| gateway-01 | dpl-subscribe-gateway、dpl-subscribe-admin | subscribe-gateway、subscribe-admin |
|
||||||
| 网关回环端口 | 18080 / 18443，管理 16444 | 8080 / 8443，管理 64444 |
|
| 网关回环端口 | 18080 / 18443，管理 16444 | 8080 / 8443，管理 64444 |
|
||||||
| Docker 存储 | dpl_sgw_config、dpl_sgw_logs | sgw_gw_config、sgw_gw_logs |
|
| Docker 存储 | dpl_sgw_config、dpl_sgw_logs | sgw_gw_config、sgw_gw_logs |
|
||||||
@ -40,7 +72,7 @@
|
|||||||
|
|
||||||
## DNS、真实 IP 和证书
|
## DNS、真实 IP 和证书
|
||||||
|
|
||||||
Namecheap 已委托至 bradley.ns.cloudflare.com、jewel.ns.cloudflare.com。导入的原有记录保留。c2 指向网关并启用代理；us01 指向节点，DNS only。两张源站证书有效期至 2027-01-09，公网订阅 HTTPS 可用。
|
2026-10-11首节点记录：Namecheap 已委托至 bradley.ns.cloudflare.com、jewel.ns.cloudflare.com。导入的原有记录保留。c2 指向网关并启用代理；us01 指向节点，DNS only。两张源站证书有效期至 2027-01-09，公网订阅 HTTPS 可用。后续日本节点独立DNS与证书见各次接入记录。
|
||||||
|
|
||||||
2026-10-11 本聊天通过正常浏览器访问直接复核：dpl 区域 SSL/TLS 为 Full (strict)。初次只读审查没有配置/源站/请求或响应标头转换/URL 改写/重定向规则，旧式 Page Rules 为零。后续按具体授权新增订阅路径 BIC 配置规则，未设置 SSL，严格模式保持。
|
2026-10-11 本聊天通过正常浏览器访问直接复核：dpl 区域 SSL/TLS 为 Full (strict)。初次只读审查没有配置/源站/请求或响应标头转换/URL 改写/重定向规则，旧式 Page Rules 为零。后续按具体授权新增订阅路径 BIC 配置规则，未设置 SSL，严格模式保持。
|
||||||
|
|
||||||
@ -90,70 +122,20 @@ VPS 代号、数据库节点/机器 ID、部署 key 与展示编号各自独立
|
|||||||
|
|
||||||
## 新节点参数单
|
## 新节点参数单
|
||||||
|
|
||||||
| 参数 | 首次实例 | 后续填写规则 |
|
参数、冲突检查与模式选择统一见[执行手册](NODE_RUNBOOK.md#统一参数单与冲突检查)，不沿用首节点示例作为后续默认值。命名仍以本文前一节为准。
|
||||||
| --- | --- | --- |
|
|
||||||
| key | us01 | 唯一、稳定的部署标识；不用于推导展示序号 |
|
|
||||||
| name | 🇺🇸 美国 US - 01 | 按[节点命名与编号](#节点命名与编号)从生产清单分配，按国家从01递增，不沿用VPS序号 |
|
|
||||||
| server | node-us01 | 从 CoDocs 项目关系及连接资料确认 |
|
|
||||||
| listen-port | 24443 | 当场核对 TCP/UDP 和防火墙，不复用过期判断 |
|
|
||||||
| relay-host / relay-port | haiwi.lshan.eu.org / 47101 | 新增规则，不改原规则 |
|
|
||||||
| tls-domain | us01.deeplink-subscribe.com | DNS only，有效证书 |
|
|
||||||
| group / plans | 首次隔离组；套餐 1、6、7、8 | 当前真实 ID 和授权集合 |
|
|
||||||
| backup | /root/codex-backups/dpl/node-onboarding-20261011-us01 | 每节点独立时间/key 目录 |
|
|
||||||
|
|
||||||
## 重复执行步骤
|
## 重复执行步骤
|
||||||
|
|
||||||
1. 读取根 AGENTS、维护入口和本文。经 CoDocs 白色机中转访问服务器；本地网页使用指定访问域名。记录原服务 PID/启动时间/端口、中转规则、Docker ID/启动时间、原订阅结果；审计资源、防火墙、所有证书钩子。受保护备份本次将修改的配置和 Git 差异，不输出私密内容。
|
按[统一接入顺序](NODE_RUNBOOK.md#统一接入顺序)执行；先区分首次 initialize 与已有组 append。promote.php必须显式mode；append只修改新节点及本次QA，默认预览、apply才写入。2026-10-17已通过隔离测试并部署独立运维快照，具体版本与现场范围见[STATUS](STATUS.md)；不能用未知版本旧脚本替代。
|
||||||
2. 非敏感脚本和模板先 Git 提交/推送，再更新服务器到明确提交，保留生产原有差异。无法直接拉取时可传输已推送提交的 Git bundle 后 fetch/fast-forward，不能用压缩包覆盖。scripts/node-onboarding 可 subtree split 为独立运维仓库；面板 .env 不送到节点。
|
|
||||||
3. 在生产应用根目录按参数单运行 stage.php，创建机器、AnyTLS 节点、隔离组和 100MB/一天的临时用户。节点虽启用显示，正式用户没有隔离组权限。相同 manifest 重跑只报告已创建；同名资源存在却缺少 manifest 时停止，不自动接管。
|
|
||||||
|
|
||||||
```sh
|
依赖检查在创建面板资源前完成；证书演练、完整订阅验收、精确回退与现有脚本限制均在执行手册维护，不在多处复制命令。面板正式开放/精确回退与多节点验证入口的测试、部署及现场范围见执行手册和STATUS。远程服务/DNS/中转仍按专项恢复。
|
||||||
php scripts/node-onboarding/stage.php \
|
|
||||||
--key=us01 --name='🇺🇸 美国 US - 01' \
|
|
||||||
--relay-host=haiwi.lshan.eu.org --relay-port=47101 \
|
|
||||||
--listen-port=24443 --tls-domain=us01.deeplink-subscribe.com \
|
|
||||||
--backup=/root/codex-backups/dpl/node-onboarding-20261011-us01
|
|
||||||
```
|
|
||||||
|
|
||||||
4. 将受保护备份的 node-config.yml、node-secret.env 经白机安装到节点 /etc/dpl-node/config.yml、secret.env，均 600；运维仓库 checkout 明确 Git 版本到 /opt/dpl-node/provision。首次安装使用下面命令。脚本检查参数、配置、端口、已有后端及 XrayR 状态，校验官方二进制并安装独立 unit，不自动启动。已有 dpl-node 时不能重跑覆盖；升级须单独复核。
|
|
||||||
|
|
||||||
```sh
|
|
||||||
bash /opt/dpl-node/provision/install-node.sh \
|
|
||||||
--listen-port=24443 --tls-domain=us01.deeplink-subscribe.com
|
|
||||||
```
|
|
||||||
|
|
||||||
5. 配置 DNS only、有效证书并审计/安装仅匹配新域名的续期钩子，再启用 dpl-node。在中转后台只新增 dpl-<key>-anytls，核对保存后的入口/目标和诊断结果。本次未启用收发 Proxy Protocol，以后启用前须同时确认入口和后端支持。
|
|
||||||
6. 后续 dpl 节点复用 c2 及当前网关，不再次更换 NS。另一个机场需要不同上游时才新增独立实例；不得改已有上游、运行原项目 setup/update/down 或复用旧卷。
|
|
||||||
7. 在网关创建本次验收目录（700），以 600 保存 stage.php 生成的 QA 私密文件和实际订阅；准备官方客户端并校验官方发布摘要。首轮已校验的 Linux 1.14.3 客户端保留于 /opt/dpl-gateway/acceptance/，下述命令可复用它。拉取临时用户完整订阅到私密文件；sing-box 使用正常版本 UA（如 sing-box/1.14.3），不要使用覆盖版本识别的裸 flag=sing-box。核验名称、入口、SNI、严格 TLS、缓存头和真实 IP；先官方客户端 check，再完整配置实际联网，核对出口、DNS、上下行计费与上报。
|
|
||||||
|
|
||||||
```sh
|
|
||||||
python3 /opt/dpl-gateway/provision/verify-node.py \
|
|
||||||
--qa /var/lib/dpl-sgw/qa/qa.json \
|
|
||||||
--binary /opt/dpl-gateway/acceptance/sing-box-1.14.3-linux-amd64/sing-box \
|
|
||||||
--subscription /var/lib/dpl-sgw/qa/subscription.json \
|
|
||||||
--expected-exit 64.44.115.18
|
|
||||||
```
|
|
||||||
|
|
||||||
helper 仅替换 TUN 等入站为临时本机 SOCKS，保留订阅 DNS、分流及出站，退出时停止客户端；失败日志留在同一私密目录。这不等于手机 GUI、系统 TUN、多地区、超额或多设备并发验收。某环境到入口不通时分段诊断并换环境核验，HTTP 200 不能替代联网。
|
|
||||||
|
|
||||||
8. 上述验收通过后，在生产根目录运行 readiness.php 和 promote.php，显式指定当前套餐集合。已有正式组时额外传 --group=<ID>。只关联所选尚无组的套餐及这些套餐现有无组用户，不改价格、周期、流量或设备限制。
|
|
||||||
|
|
||||||
```sh
|
|
||||||
php scripts/node-onboarding/readiness.php \
|
|
||||||
--backup=/root/codex-backups/dpl/node-onboarding-20261011-us01
|
|
||||||
php scripts/node-onboarding/promote.php \
|
|
||||||
--backup=/root/codex-backups/dpl/node-onboarding-20261011-us01 \
|
|
||||||
--plans=1,6,7,8
|
|
||||||
```
|
|
||||||
|
|
||||||
首次隔离组改为基础节点、订阅域名设置为 c2，以后复用正式组。脚本要求节点在线且临时用户已有流量，保存 before-promotion.json，最后禁用临时用户并写 promoted.json。清理本次客户端和私密传输副本，保留服务器备份、禁用用户与审计记录。
|
|
||||||
|
|
||||||
9. 回归原机场服务/规则/订阅，实际核对正式组权限、面板节点及订阅域名保存，确认测试用户禁用且不再下发。更新本文及 STATUS，分别记录提交、发布、真实验收及未完成项。
|
|
||||||
|
|
||||||
数据库默认 sing-box 模板已迁移新 DNS/路由格式，保留原分流和选择逻辑；现代本地 DoH 使用默认 dialer，不绕到空 direct 出站。仓库原默认资源和其他机场模板不改。apply-template.php 仅接受原默认模板，或与显式 expected-current-sha256 完全匹配的旧版本，逐次备份后只更新该模板/缓存。用户定制或未知模板须先审查，不能强行覆盖。
|
数据库默认 sing-box 模板已迁移新 DNS/路由格式，保留原分流和选择逻辑；现代本地 DoH 使用默认 dialer，不绕到空 direct 出站。仓库原默认资源和其他机场模板不改。apply-template.php 仅接受原默认模板，或与显式 expected-current-sha256 完全匹配的旧版本，逐次备份后只更新该模板/缓存。用户定制或未知模板须先审查，不能强行覆盖。
|
||||||
|
|
||||||
## 回退与排障
|
## 回退与排障
|
||||||
|
|
||||||
|
节点变更先按[精确回退入口与证据契约](NODE_RUNBOOK.md#精确回退入口与证据契约)确定本次对象及 before/after。下列专项恢复说明继续适用；历史 before-promotion 不保证包含所有 after 值，不能盲目恢复。
|
||||||
|
|
||||||
- 单节点：停用本次节点/新增中转，停止新 dpl-node；保留 XrayR、原规则和审计数据。
|
- 单节点：停用本次节点/新增中转，停止新 dpl-node；保留 XrayR、原规则和审计数据。
|
||||||
- 正式关联：依据 before-promotion.json 精确恢复本次套餐/用户组及订阅域名，先确认当前值仍是本次写入值，避免覆盖后续业务。不恢复整库。
|
- 正式关联：依据 before-promotion.json 精确恢复本次套餐/用户组及订阅域名，先确认当前值仍是本次写入值，避免覆盖后续业务。不恢复整库。
|
||||||
- 模板：从对应私密模板备份恢复该条内容并清理模板缓存，不替换整个设置表。
|
- 模板：从对应私密模板备份恢复该条内容并清理模板缓存，不替换整个设置表。
|
||||||
@ -168,6 +150,9 @@ php scripts/node-onboarding/promote.php \
|
|||||||
| --- | --- | --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- | --- | --- |
|
||||||
| dpl | 🇺🇸 美国 US - 01 / node-us01 | AnyTLS | 24443 | haiwi.lshan.eu.org:47101 | 1 | 已上线；基础节点组，正式套餐 1/6/7/8 |
|
| dpl | 🇺🇸 美国 US - 01 / node-us01 | AnyTLS | 24443 | haiwi.lshan.eu.org:47101 | 1 | 已上线；基础节点组，正式套餐 1/6/7/8 |
|
||||||
| dpl | 🇯🇵 日本 JP - 01 / node-jp02 | AnyTLS | 24443 | haiwi.lshan.eu.org:47202 | 2 | 2026-10-16 运行上线；基础节点组 1，正式套餐 1/6/7/8；浏览器验收通过 |
|
| dpl | 🇯🇵 日本 JP - 01 / node-jp02 | AnyTLS | 24443 | haiwi.lshan.eu.org:47202 | 2 | 2026-10-16 运行上线；基础节点组 1，正式套餐 1/6/7/8；浏览器验收通过 |
|
||||||
|
| dpl | 🇯🇵 日本 JP - 02 / node-jp03 | AnyTLS | 24443 | haiwi.lshan.eu.org:47203 | 4 | 2026-10-17 append上线；组1，正式五节点订阅及QA403通过 |
|
||||||
|
| dpl | 🇯🇵 日本 JP - 03 / node-jp04 | AnyTLS | 24443 | haiwi.lshan.eu.org:47204 | 5 | 2026-10-17 append上线；组1，正式五节点订阅及QA403通过 |
|
||||||
|
| dpl | 🇯🇵 日本 JP - 04 / node-jp05 | AnyTLS | 24443 | haiwi.lshan.eu.org:47205 | 3 | 2026-10-17 append上线；组1，正式五节点订阅及QA403通过 |
|
||||||
|
|
||||||
已完成：独立服务、节点 TLS、新中转、新网关、DNS/橙云、真实 IP 比对、新旧订阅 vhost 官方 IP 自动同步、两张证书续期演练。官方 sing-box 1.14.3 的完整实际订阅（仅把入站替换为临时本机 SOCKS）经中转联网，出口 64.44.115.18、US、HTTPS TLSv1.3，严格节点证书校验；完整 DNS 和远程分流规则实际启动通过，面板已记录上下行流量。
|
已完成：独立服务、节点 TLS、新中转、新网关、DNS/橙云、真实 IP 比对、新旧订阅 vhost 官方 IP 自动同步、两张证书续期演练。官方 sing-box 1.14.3 的完整实际订阅（仅把入站替换为临时本机 SOCKS）经中转联网，出口 64.44.115.18、US、HTTPS TLSv1.3，严格节点证书校验；完整 DNS 和远程分流规则实际启动通过，面板已记录上下行流量。
|
||||||
|
|
||||||
|
|||||||
258
docs/maintenance/NODE_RUNBOOK.md
Normal file
258
docs/maintenance/NODE_RUNBOOK.md
Normal file
@ -0,0 +1,258 @@
|
|||||||
|
# DPL 节点接入执行手册
|
||||||
|
|
||||||
|
入口：[节点关系与历史验收](NODE_ONBOARDING.md)。本手册定义后续授权任务的步骤。2026-10-16 已补齐本地脚本草稿；用户要求测试交给奶油，本轮未运行脚本、语法检查或功能测试，未触及线上。**新脚本必须先在隔离环境测试，再按Git流程审阅发布；本地存在不代表线上已更新。**已有节点展示命名按原入口的[节点命名与编号](NODE_ONBOARDING.md#节点命名与编号)，此处不另建规则。
|
||||||
|
|
||||||
|
## 先选择变更模式
|
||||||
|
|
||||||
|
| 模式 | 适用情形 | 允许修改的范围 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| initialize | 首次建立 DPL 网关、基础节点组及明确授权的套餐可见范围 | 本次参数单与授权清单逐项列明；订阅地址、套餐/用户关联必须分别确认 |
|
||||||
|
| append | 已有网关/正式节点组上增加节点 | 新机器、新节点、新独立中转、节点 DNS/证书和 QA 隔离资源；验收后只把新节点加入指定已有组并禁用 QA |
|
||||||
|
|
||||||
|
默认判断依据是现有资源与目标，不把“新节点”当作首次初始化。append 保持现有订阅地址、网关、套餐全部字段、正式用户分组及旧节点不变。扩展套餐覆盖另列变更任务。未知已有组/节点状态时先解决该依赖，不能通过重跑初始化猜测。
|
||||||
|
|
||||||
|
## 统一参数单与冲突检查
|
||||||
|
|
||||||
|
在本次服务器受保护任务目录保存参数，目录 700、文件 600；非秘密参数也不进公开日志。以下是待填写模板，没有可直接用于新节点的默认值。文件只使用赋值，不含执行命令；人工审阅后才加载。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
MODE='' # initialize 或 append
|
||||||
|
KEY='' # 稳定部署 key
|
||||||
|
NODE_NAME='' # 按既有命名规则确定；与 VPS 代号独立
|
||||||
|
VPS_ID='' # CoDocs 稳定 ID
|
||||||
|
NODE_ADDRESS='' # 实际落地地址，按私有资源读取
|
||||||
|
LISTEN_PORT=''
|
||||||
|
RELAY_HOST=''
|
||||||
|
RELAY_PORT=''
|
||||||
|
TLS_DOMAIN=''
|
||||||
|
FORMAL_GROUP_ID='' # append 必须填已有正式组
|
||||||
|
PLAN_IDS='' # 仅 initialize 或另有明确套餐变更授权时填
|
||||||
|
BACKUP='' # 绝对路径，每次任务独立，不复用其他节点目录
|
||||||
|
QA_DIR='' # 网关本次独立验收目录
|
||||||
|
CLIENT_BINARY='' # 已校验摘要的官方客户端绝对路径
|
||||||
|
EXPECTED_EXIT=''
|
||||||
|
EXPECTED_COUNTRY=''
|
||||||
|
LOCAL_SOCKS_PORT='' # 网关上本次隔离客户端空闲端口
|
||||||
|
APP_ROOT='' # 面板应用路径，从项目档案核实
|
||||||
|
PROVISION_ROOT='' # 面板上本次已确认脚本路径
|
||||||
|
NODE_PROVISION='' # 节点上已确认的运维 Git 路径
|
||||||
|
GIT_COMMIT='' # 已推送且经过审阅的明确提交
|
||||||
|
PANEL_URL='' # 节点回连面板的明确HTTPS地址
|
||||||
|
SUBSCRIBE_URL='' # 仅initialize明确修改的订阅地址
|
||||||
|
QA_REPORT='' # 本次隔离QA的私有实时验收报告
|
||||||
|
FORMAL_EXPECTATIONS='' # 正式多节点预期的私有JSON文件
|
||||||
|
FORMAL_REPORT='' # 正式多节点验证报告
|
||||||
|
```
|
||||||
|
|
||||||
|
填写后逐项核对：key/展示名、机器名、面板 ID、VPS、落地监听端口、中转入口的域名+端口、源站目标、SNI、正式组、DNS 现值、备份路径和 QA 端口。相同 key 有 manifest 时，只回读已有对象并确认参数完全一致；新 stage.php 重跑要求 manifest 记录的完整参数一致，并核对节点、机器和中转端点；旧 manifest 没有 parameters 时停止，不能自动采纳。相同资源无 manifest、冲突端口或未授权的覆盖均停止，不自动接管。
|
||||||
|
|
||||||
|
机密字段单独沿用 stage.php 生成的私密文件；不把机器令牌、QA UUID、订阅令牌写进上述参数单、文档或命令行。参数按角色分发，不把面板 .env 复制到节点。
|
||||||
|
|
||||||
|
可在审阅参数文件后运行以下本地字段检查；它不验证线上冲突或资源归属：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
set -eu
|
||||||
|
: "${MODE:?}" "${KEY:?}" "${NODE_NAME:?}" "${VPS_ID:?}" "${NODE_ADDRESS:?}"
|
||||||
|
: "${LISTEN_PORT:?}" "${RELAY_HOST:?}" "${RELAY_PORT:?}" "${TLS_DOMAIN:?}"
|
||||||
|
: "${BACKUP:?}" "${QA_DIR:?}" "${CLIENT_BINARY:?}" "${EXPECTED_EXIT:?}" "${EXPECTED_COUNTRY:?}"
|
||||||
|
: "${LOCAL_SOCKS_PORT:?}" "${APP_ROOT:?}" "${PROVISION_ROOT:?}" "${NODE_PROVISION:?}" "${GIT_COMMIT:?}"
|
||||||
|
: "${PANEL_URL:?}" "${QA_REPORT:?}"
|
||||||
|
case "$MODE" in
|
||||||
|
append) : "${FORMAL_GROUP_ID:?}"; test -z "$PLAN_IDS" ;;
|
||||||
|
initialize) : "${PLAN_IDS:?}" "${SUBSCRIBE_URL:?}" ;;
|
||||||
|
*) exit 1 ;;
|
||||||
|
esac
|
||||||
|
```
|
||||||
|
|
||||||
|
## 创建面板记录之前完成前置检查
|
||||||
|
|
||||||
|
只在具体接入任务授权后，通过 CoDocs 白色机跳板及登记的后台反代执行以下检查。本次文档维护没有执行这些动作。
|
||||||
|
|
||||||
|
| 位置 | 必须确认 | 未通过时 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 面板 | Git 提交、当前组和节点清单、现有订阅设置、套餐/用户分组快照、应用运行环境、脚本实际版本 | 先补证据与备份，不创建机器/节点/QA 记录 |
|
||||||
|
| 节点 | OS/架构、内存/磁盘、时间同步、systemd、Git、curl、sha256sum、ss、证书工具，以及命令实际路径 | 先明确缺失工具和安装来源；安装属于接入任务变更，文档整理不安装 |
|
||||||
|
| 软件来源 | 发行版源可下载、包版本与依赖可满足；运维仓库和官方二进制来源/摘要可验证 | 不把 apt 索引存在当可安装；不自动换不明镜像或全量升级 |
|
||||||
|
| 证书 | 已有 renewal 配置、pre/post/deploy hooks、Certbot 是 apt 还是 Snap、可执行路径、timer/cron 的真实调用参数和运行用户 | 确定新证书隔离方案及可生效调度后再签发；不能仅添加 renewal 文件就视为自动续期完成 |
|
||||||
|
| 节点隔离 | 现有 XrayR 等服务的 PID/启动时间、TCP/UDP 监听、防火墙与新端口、旧证书和目录权限 | 冲突先处理；安装脚本只检查 TCP，不能代替全项前检 |
|
||||||
|
| 中转 | 后台/登录账户按 CoDocs 读取，当前规则快照、目标落地、隧道及入口端口、Proxy Protocol 两端设置 | 缺入口先问，不改原规则；创建后回读实际值 |
|
||||||
|
| 网关 | 容器 ID/启动时间/重启计数、CPU/内存、独立 QA 路径和端口、Python3、官方客户端及其摘要 | 补齐验收依赖后再 stage，不边创建业务记录边找工具 |
|
||||||
|
|
||||||
|
历史 jp02 接入曾因 Debian11 源包 404 临时使用 Debian 官方 HTTPS 归档安装 Git；这是该次处理记录，不是以后默认改源或安装命令。先核对目标当下发行版与来源，不在本手册固化过期包 URL。
|
||||||
|
|
||||||
|
前置完成后，保存本次允许改变的对象、每个对象的 before 值、预期 after 值和精确回退依据。不要打印整库快照或秘密。
|
||||||
|
|
||||||
|
## 脚本能力与边界
|
||||||
|
|
||||||
|
后续新批次按[正式适配器](NODE_BATCH_RUNTIME.md)准备和执行：动态账号授权集合验收、缺账号时自动创建受限用户、白色机SSH复用及证据有效期续跑。旧命令适配器继续兼容。下文运维提交为上一轮三节点已验收版本，新适配器与PHP验收用户入口尚未部署，不能把旧任务验收视为新版本线上通过。
|
||||||
|
|
||||||
|
2026-10-17：下列脚本已完成 PHP 语法检查、面板隔离测试（9 tests / 65 assertions）及批量进程测试（11 tests），源码提交并推送 `6af6764`；独立运维快照 `b8d10fac963d63825c97d36d41ff7f525116bbfd` 已部署到本次面板/网关的 `/opt/dpl-batch/provision` 与三台新节点 `/opt/dpl-node/provision`。生产应用代码未随之更新。各阶段现场结果以 [STATUS](STATUS.md) 和 [接入记录](NODE_ONBOARDING.md) 为准；隔离测试中的通知替身不证明生产通知成功。
|
||||||
|
|
||||||
|
| 工具 | 新行为 | 边界 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| stage.php | 明确panel-url、完整参数/对象重跑核对、端点冲突检查；受保护manifest可恢复传输文件 | 旧manifest不自动采纳；依赖与实际资源前检仍由接入任务完成 |
|
||||||
|
| promote.php | 强制mode；默认只预览；append只写新节点group_ids及QA banned；initialize显式plans/subscribe-url/可选group-name | 真正写入需apply及15分钟内匹配该QA、端点与SNI的实时验证报告；不根据节点名推导身份 |
|
||||||
|
| readiness.php | 只统计显式plans，缺省不统计套餐；可检查给定group的冲突套餐 | 只读快照，不代替真实订阅/计费/界面验收 |
|
||||||
|
| verify-node.py | 默认离线结构核对；显式run才启动客户端/联网；可按预期逐一验证多节点、SNI、密码、出口IP及国家 | 正式多节点需要完整预期清单；只证明实际执行的探针，不代替业务/手机/TUN/全路由验收 |
|
||||||
|
| rollback.php | 默认预览；基于新promotion-journal逐字段比较after再恢复before；保持QA禁用，隔离本次节点 | 冲突整次停止，不覆盖后续变化；只覆盖面板变更，不停止远程服务或修改DNS/中转；旧before-promotion需专项人工核对 |
|
||||||
|
| operations.php | 私有文件锁、原子写日志、事务内记录写前意图、提交后完成标记；中断后按before/after识别已应用或冲突 | 文件与数据库不是同一个事务；状态未知需依据日志恢复，不盲目重新stage |
|
||||||
|
| install-node.sh / renew-node.sh | 沿用已登记的独立安装与专用续期hook | 本次未更改，不代表前置工具/安装源/实际timer已经验证 |
|
||||||
|
|
||||||
|
新脚本不修改已有节点展示名、不自动分配编号。操作日志不包含密码/UUID/机器令牌；脚本不在错误输出中打印数据库异常全文或客户端配置。上述隔离测试覆盖事务、幂等、崩溃恢复与冲突拒绝；缓存/节点通知与用户下发仍按本次现场证据验收。
|
||||||
|
|
||||||
|
## 统一接入顺序
|
||||||
|
|
||||||
|
1. 完成参数单、模式确认、前检和恢复点；非秘密运维代码先提交/推送，再更新目标至明确 Git 提交。已有线上差异保留。拉取受限时只能传输已推送提交的 Git bundle 后 fetch/fast-forward，不用归档覆盖。
|
||||||
|
2. 在确认的面板应用根目录，用下列统一参数创建隔离资源；创建后把实际 node_id、machine_id、group_id、test_user_id 与 key/VPS 对应写回本次任务 manifest，不能从服务器名推导面板 ID。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd "$APP_ROOT"
|
||||||
|
php "$PROVISION_ROOT/stage.php" \
|
||||||
|
--key="$KEY" --name="$NODE_NAME" \
|
||||||
|
--relay-host="$RELAY_HOST" --relay-port="$RELAY_PORT" \
|
||||||
|
--listen-port="$LISTEN_PORT" --tls-domain="$TLS_DOMAIN" \
|
||||||
|
--panel-url="$PANEL_URL" --backup="$BACKUP"
|
||||||
|
```
|
||||||
|
|
||||||
|
3. 按既有受保护传输方式安装 node-config.yml 和 node-secret.env 至节点的独立目录，文件 600、父目录 700；实际运维 Git 路径使用 NODE_PROVISION，保留二进制固定版本/摘要核验。节点未安装本服务时才运行：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
bash "$NODE_PROVISION/install-node.sh" \
|
||||||
|
--listen-port="$LISTEN_PORT" --tls-domain="$TLS_DOMAIN"
|
||||||
|
```
|
||||||
|
|
||||||
|
4. 配置本次 DNS only、证书和隔离续期策略，再启动新服务；中转只新增本次规则，保存后回读入口、目标及诊断结果。Proxy Protocol 默认保持本次参数单记录，不能单端打开。已有网关按 append 复用，不重建网关、NS、旧卷或订阅配置。
|
||||||
|
5. 按下节执行隔离验收；不因 service active、内核磁盘无 JSON 或 HTTP 200 单独判断成功/故障。Xboard-Node 可使用内存配置，须按实际版本、启动日志、证书和请求链路判断。
|
||||||
|
6. 验收通过后按下方“面板变更命令”预览，再对同一组参数明确apply。append只改新节点与QA；initialize才允许修改明确列出的套餐/用户分组及订阅地址。保存前后日志并回读。未通过隔离测试的新版本不能直接带到生产执行。
|
||||||
|
7. 回读正式订阅与原资源，检查 QA 禁用、订阅拒绝、下发集合无 QA，确认旧节点、旧中转、套餐/用户和订阅设置保持预期。同步 CoDocs 对象关系、节点登记及 STATUS；提交/推送/部署/现场验收分别报告。
|
||||||
|
|
||||||
|
## 证书钩子与演练
|
||||||
|
|
||||||
|
新证书只挂本项目专用 hook。历史已验证的调用方式为 `/bin/sh /opt/dpl-node/provision/renew-node.sh`；不能假定脚本有执行权限后直接执行。路径实际不同时按当前运维检出路径填写。renew-node.sh 依据 `/etc/dpl-node/certificate-domain` 和 RENEWED_LINEAGE 精确匹配，只操作 dpl-node。
|
||||||
|
|
||||||
|
目标实际 Certbot 命令路径记为 CERTBOT_BIN；先审计目录钩子、renewal 配置及 Snap/apt timer 的调用方式。经确认的演练入口：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
: "${CERTBOT_BIN:?}"
|
||||||
|
"$CERTBOT_BIN" renew --cert-name "$TLS_DOMAIN" --dry-run \
|
||||||
|
--no-random-sleep-on-renew --run-deploy-hooks --no-directory-hooks
|
||||||
|
```
|
||||||
|
|
||||||
|
`--no-directory-hooks` 不替代审计 renewal 配置内的显式钩子。必须确认该证书配置仍明确调用专用 `/bin/sh …/renew-node.sh`，并检查演练后的服务状态及旧服务 PID/启动时间。关闭随机等待只用于本次演练可控等待，不等于修改常规定时调度。BrokenPipe/超时只记未确认，回查进程和最终结果后再决定是否重试，不重复发起签发。
|
||||||
|
|
||||||
|
续期完成条件包括：证书配置正确、调度 enabled/下一次触发可见、调度参数确实覆盖新证书及专用 hook、dry-run 退出状态和旧服务保护证据齐全。证书到期时间引用证书/域名对应记录，不把一次演练日期当作长期验收。
|
||||||
|
|
||||||
|
## 统一验收入口
|
||||||
|
|
||||||
|
每次在 QA_DIR 保存 700/600 的 QA 私密文件与完整实际订阅；文件名统一为 qa.json、subscription.json、验收结果和受保护日志，参数与 stage manifest 对照，不使用首节点默认路径。本地浏览器、服务器请求均按各自授权路线执行。
|
||||||
|
|
||||||
|
在已授权的网关环境，先用本次参数里的官方客户端检查完整配置，再运行完整隔离订阅：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
"$CLIENT_BINARY" check -c "$QA_DIR/subscription.json"
|
||||||
|
python3 /opt/dpl-gateway/provision/verify-node.py \
|
||||||
|
--qa "$QA_DIR/qa.json" --subscription "$QA_DIR/subscription.json" \
|
||||||
|
--binary "$CLIENT_BINARY" --port "$LOCAL_SOCKS_PORT" \
|
||||||
|
--expected-exit "$EXPECTED_EXIT" --expected-country "$EXPECTED_COUNTRY" \
|
||||||
|
--report "$QA_REPORT" --run
|
||||||
|
```
|
||||||
|
|
||||||
|
上面脚本路径是既有网关检出约定，执行前核对实际 Git 版本与路径，若不同则使用本次确认路径，不复制临时代码替代。QA可按host/port唯一匹配本次节点，冲突时显式target-tag；正式多节点使用下方expectations。验证保留DNS/分流及全部出站，替换临时SOCKS入站，并仅为api.ipify.org与www.cloudflare.com两个诊断目的地添加指定出站规则，从而逐节点验证；不是原订阅所有路由均已验收。
|
||||||
|
|
||||||
|
| 验收层 | 必须保留的结果 |
|
||||||
|
| --- | --- |
|
||||||
|
| 参数一致 | 订阅包含本次入口/端口/SNI、严格 TLS 开启；机器/节点与 VPS 对应正确 |
|
||||||
|
| 协议/联网 | 官方客户端配置检查与完整启动结果、严格 TLS、目标 HTTPS、出口 IP 与 EXPECTED_EXIT 一致；脚本强制比对 EXPECTED_COUNTRY；仅结构核对不产生live_verified报告 |
|
||||||
|
| 订阅 | 有效用户标准/短路径按本次范围检查，JSON/gzip、private/no-store、无 HIT/Age/挑战；不要在日志保存令牌 |
|
||||||
|
| 上报 | readiness.php回读节点在线、last_check/last_push及QA上下行；按显式plans/group统计，缺省不选任何套餐 |
|
||||||
|
| 正式开放 | 正式有效用户订阅与完整expectations逐一对应，并逐节点验证；不把单节点QA结果扩大成全部可用 |
|
||||||
|
| QA 清理 | QA banned、其订阅拒绝、实际下发用户集合无 QA；客户端退出、临时本地秘密副本清理，服务器审计文件保留 |
|
||||||
|
| 回归 | 原节点/中转、套餐字段/用户组、订阅 URL、网关容器 ID/启动时间/重启计数与 before 快照对照 |
|
||||||
|
| 浏览器 | 正确反代入口，管理列表与保存结果回读；只查看后取消不算提交验收 |
|
||||||
|
|
||||||
|
面板回读命令入口如下；它只覆盖该脚本实际返回的节点/QA 字段，其他行须另留证据：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd "$APP_ROOT"
|
||||||
|
php "$PROVISION_ROOT/readiness.php" --backup="$BACKUP" --group="$FORMAL_GROUP_ID"
|
||||||
|
```
|
||||||
|
|
||||||
|
GUI/手机、TUN、多地区、并发及用户明确跳过的测试分别标为未执行或跳过。未确认结果不补成通过；统一表格不替代真实执行。
|
||||||
|
|
||||||
|
### 正式多节点验证
|
||||||
|
|
||||||
|
在私有文件 FORMAL_EXPECTATIONS 保存每个AnyTLS出站的明确预期，完整覆盖订阅中的全部AnyTLS标签。每条结构：`tag`、`host`、整数`port`、`sni`、`uuid`、`expected_exit`、`expected_country`；uuid是该正式用户的实际密码，不能复制进普通参数单或文档。标签必须与订阅精确一致；不重命名节点。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 /opt/dpl-gateway/provision/verify-node.py \
|
||||||
|
--subscription "$QA_DIR/formal-subscription.json" \
|
||||||
|
--expectations "$FORMAL_EXPECTATIONS" --binary "$CLIENT_BINARY" \
|
||||||
|
--port "$LOCAL_SOCKS_PORT" --report "$FORMAL_REPORT" --run
|
||||||
|
```
|
||||||
|
|
||||||
|
不加run时只读取私有JSON并核对结构，不启动客户端、不发网络请求；run每次会先将旧报告失效，避免失败后误用旧成功结果。实时报告记录具体结果与时间，面板正式开放仅接受匹配manifest QA及端点、15分钟内的实时QA报告，正式多节点报告不能代替QA报告。
|
||||||
|
|
||||||
|
### 面板变更命令
|
||||||
|
|
||||||
|
以下命令默认只预览数据库字段变化；确认输出范围后，对同一命令追加 `--verification="$QA_REPORT" --apply` 才写入。append禁止传plans、subscribe-url和group-name；它不会修改现有套餐/正式用户/订阅地址。initialize对应已有套餐中的无组用户关联也属于实际写入范围，必须先看预览。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd "$APP_ROOT"
|
||||||
|
php "$PROVISION_ROOT/promote.php" --backup="$BACKUP" --mode=append \
|
||||||
|
--group="$FORMAL_GROUP_ID"
|
||||||
|
```
|
||||||
|
|
||||||
|
首次初始化另用以下命令，按任务需要显式传group/group-name；没有这些参数时不自动改组名：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd "$APP_ROOT"
|
||||||
|
php "$PROVISION_ROOT/promote.php" --backup="$BACKUP" --mode=initialize \
|
||||||
|
--plans="$PLAN_IDS" --subscribe-url="$SUBSCRIBE_URL"
|
||||||
|
```
|
||||||
|
|
||||||
|
promotion-journal.json记录允许写入字段、before/after与manifest。相同日志/参数重跑会比较实际字段，已应用则只确认；参数变化、已启动回退、旧promoted.json无新日志或对象冲突时停止。prepared可能意味着数据库未提交、已提交但后续通知/缓存/落盘失败；只有完整before或完整after匹配才恢复，中间/未知状态先人工处理。不要删除日志重新执行。
|
||||||
|
|
||||||
|
## 精确回退入口与证据契约
|
||||||
|
|
||||||
|
触发失败先停止正式开放，隔离新节点。回退按本次 manifest 指定对象执行；不恢复整库、不删除原卷、不停止旧节点。执行前取回当前值，只有当前值仍等于本次 after 才恢复 before；存在后续变化时停止本次自动回退，不写任何字段；专项人工恢复另行核对。
|
||||||
|
|
||||||
|
| 对象 | 必需恢复证据 | 动作及回读 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| append 新节点正式组 | 该 node_id 的 before/after group_ids、enabled/show 及实际变动字段 | 新rollback.php按日志恢复本次字段并禁用/隐藏节点；不动已有套餐/用户/订阅地址 |
|
||||||
|
| initialize 正式关联 | 新promotion-journal.json、manifest.json；旧before-promotion/promoted仅用于历史专项回退 | 按 ID 逐条恢复本次变动的套餐/用户组、节点组、组名、订阅 URL；缺 after 不盲目回填，以实际变更证据核对 |
|
||||||
|
| QA | test_user_id 与实际禁用状态 | 始终保持本次测试账号禁用、订阅拒绝、不下发；不因回退恢复 QA 活跃 |
|
||||||
|
| 中转 | 新增规则的真实 ID、入口和目标 before/after | 只停用本次新增规则，再回读；原规则保持 |
|
||||||
|
| 节点服务 | 新服务/unit/配置及安装前状态、原服务 PID/启动时间 | 在已授权节点上执行下方停止入口，只停止新 dpl-node，回查旧服务；不删证书/审计目录 |
|
||||||
|
| DNS/证书续期 | 本次记录 ID/旧值/新值、renewal/hook/timer 原文件及权限 | 仅撤回本次增改；已共用证书/调度不能直接删除，恢复后核对下一次调度及原服务 |
|
||||||
|
| 网关/模板/CF | 对应专项恢复点 | append 不应修改；initialize 或专项变更按原[恢复专题](NODE_ONBOARDING.md#回退与排障)逐项执行 |
|
||||||
|
|
||||||
|
节点上的停止入口（确认本服务确为本次新增后）：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
systemctl stop dpl-node
|
||||||
|
systemctl is-active dpl-node || test "$?" -eq 3
|
||||||
|
```
|
||||||
|
|
||||||
|
新日志的面板回退统一入口（默认预览，加apply才执行）：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd "$APP_ROOT"
|
||||||
|
php "$PROVISION_ROOT/rollback.php" --backup="$BACKUP"
|
||||||
|
```
|
||||||
|
|
||||||
|
回退逐字段核对当前值仍等于promotion记录的after，任何冲突整体停止；写回本次before、保持QA禁用，并禁用/隐藏本次节点。rollback-journal.json保存回退本身的意图与完成状态，重复执行也需匹配完整结果。旧日志不能自动补造after或升级成新日志。DNS、中转、远程服务、证书、网关/CF属于其他对象，仍按对应专项步骤和恢复证据处理；该面板脚本不包含跨服务器/服务商变更。
|
||||||
|
|
||||||
|
## 原交接测试清单与当前证据
|
||||||
|
|
||||||
|
2026-10-16编写时尚未测试或发布；2026-10-17已执行本节上方所列隔离测试、语法检查和独立运维快照部署，现场任务仍以STATUS区分完成范围。下列保留为完整验证清单，不表示每一项都已有独立用例或实际故障演练；未执行的项不能因部署而记为通过。
|
||||||
|
|
||||||
|
离线测试应在全新SQLite内存库/隔离应用容器中进行，禁止加载项目本地真实.env、默认数据库/Redis/队列或使用正式凭据：
|
||||||
|
|
||||||
|
1. append仅改变本次node.group_ids、QA.banned，现有订阅设置/全部套餐/正式用户/旧节点逐字段不变；拒绝append夹带plans/url/group-name。
|
||||||
|
2. initialize只修改明确套餐及其无组用户、指定订阅地址/可选组名；拒绝不存在ID、其他组冲突，验证原null订阅设置的创建及回退。
|
||||||
|
3. 默认预览无数据库写入；apply要求近期真实QA报告且QA身份、端点/SNI正确；结构报告、过期/错误QA报告不能通过。
|
||||||
|
4. 回读仅使用本次显式plans；无plans时不查询/统计隐含套餐集合。
|
||||||
|
5. 应用/回退幂等；数据库提交前失败、提交后通知/缓存/日志失败、prepared+完整before/after与混合冲突，不能重复创建或覆盖后续变化；QA始终不复活。
|
||||||
|
6. 多节点JSON用虚构身份；缺失/重复标签、端点/SNI/密码/严格TLS/国家/IP错误均失败。模拟子进程和传输，不发公网请求；默认无run绝不启动进程/开socket，失败后不能留下旧成功报告。
|
||||||
|
7. stage相同完整参数恢复缺失传输文件；参数变化、旧manifest、现有同名/同中转端点资源或已promote时拒绝接管；检查所有秘密文件权限。
|
||||||
|
8. 通过隔离测试之后，线上发布/验收仍需具体任务授权；测试本身不授权生产变更。节点命名按NODE_ONBOARDING执行，不因测试改名或分配编号。
|
||||||
@ -4,7 +4,7 @@
|
|||||||
|
|
||||||
## 接手顺序
|
## 接手顺序
|
||||||
|
|
||||||
1. 阅读全局及项目根目录 AGENTS.md、当前用户要求和本目录说明。用户最新明确边界优先，不扩大本次授权。
|
1. 按全局规则先读 CoDocs 项目总览、首次会话的持续维护协议与 dpl 档案，再读项目根目录 AGENTS.md、当前用户要求和本目录说明。用户最新明确边界优先，不扩大本次授权。
|
||||||
2. 确认所在仓库、分支、工作区差异与远端；分别识别用户既有修改、本次任务与其他聊天修改。不要覆盖、清理或混入其他任务。
|
2. 确认所在仓库、分支、工作区差异与远端；分别识别用户既有修改、本次任务与其他聊天修改。不要覆盖、清理或混入其他任务。
|
||||||
3. 按 [架构与配置](ARCHITECTURE.md) 沿实际调用链检查问题；按 [业务约束](BUSINESS_RULES.md) 判断影响。
|
3. 按 [架构与配置](ARCHITECTURE.md) 沿实际调用链检查问题；按 [业务约束](BUSINESS_RULES.md) 判断影响。
|
||||||
4. 修改后运行与影响范围相关的检查；后台界面必须浏览器验收，包括页面显示、保存、切换、权限。HTTP 200 不代表前端可用。
|
4. 修改后运行与影响范围相关的检查；后台界面必须浏览器验收，包括页面显示、保存、切换、权限。HTTP 200 不代表前端可用。
|
||||||
@ -21,9 +21,14 @@
|
|||||||
| TELEGRAM_TICKETS.md | Telegram 工单接入、绑定、权限及故障处理 |
|
| TELEGRAM_TICKETS.md | Telegram 工单接入、绑定、权限及故障处理 |
|
||||||
| STATUS.md | 最近验证基线、已知限制与待验收项目 |
|
| STATUS.md | 最近验证基线、已知限制与待验收项目 |
|
||||||
| CLOUD_FLARE_AUDIT.md | 两机场 Cloudflare 审查、已部署调优、域名轮换与回退及可选未实施项 |
|
| CLOUD_FLARE_AUDIT.md | 两机场 Cloudflare 审查、已部署调优、域名轮换与回退及可选未实施项 |
|
||||||
| NODE_ONBOARDING.md | 共享资源隔离、订阅网关与新增节点流程及待验收状态 |
|
| NODE_ONBOARDING.md | 五节点链路、命名规则、共享资源与验收 |
|
||||||
|
| NODE_RUNBOOK.md | initialize/append参数、脚本入口、前检/证书/验收与面板回退；隔离测试/运维版本与现场边界，含验证清单 |
|
||||||
|
| NODE_BATCH.md | 批量接入默认一个子助手的分工与关卡；append并发锁/查重/断点恢复、命令协议与测试边界 |
|
||||||
|
| NODE_BATCH_RUNTIME.md | 新批次默认正式适配器、动态集合验收、自动受限测试用户、白色机SSH复用、证据续跑及部署准备 |
|
||||||
| GATEWAY_ADMIN_PROXY.md | 两网关管理域名、白色机新旧区域反代、独立证书续期、恢复和验收边界 |
|
| GATEWAY_ADMIN_PROXY.md | 两网关管理域名、白色机新旧区域反代、独立证书续期、恢复和验收边界 |
|
||||||
|
|
||||||
这些说明用于辅助 AI 接手，不代表 AI 会在聊天结束后自动持续运行。应用定时任务在服务器执行，AI 维护任务需要会话或明确配置的自动化来启动。
|
这些说明用于辅助 AI 接手，不代表 AI 会在聊天结束后自动持续运行。应用定时任务在服务器执行，AI 维护任务需要会话或明确配置的自动化来启动。
|
||||||
|
|
||||||
正式维护文档放在 docs/maintenance/，纳入 Git 并随相关代码一起更新。项目根目录 AGENTS.md 是 AI 自动接手入口；运行日志、截图、临时脚本与任务产物放在 codex/output/，继续默认忽略。受保护备份和私密配置按运行环境保存，不提交 Git。
|
正式维护文档放在 docs/maintenance/，纳入 Git 并随相关代码一起更新。项目根目录 AGENTS.md 是 AI 自动接手入口；运行日志、截图、临时脚本与任务产物放在 codex/output/，继续默认忽略。受保护备份和私密配置按运行环境保存，不提交 Git。
|
||||||
|
|
||||||
|
本机共享 wwwroot 权限已于 2026-10-13 收紧；策略、服务身份、限制和恢复入口见 [CoDocs 本地目录权限](/Users/hellcat/Documents/CoDocs/20-基础设施/本地目录权限.md)。仅本机适用，未变更本项目业务或生产权限。
|
||||||
|
|||||||
@ -2,6 +2,46 @@
|
|||||||
|
|
||||||
文档核对日期：2026-10-11。以下区分本会话检查结果与用户验收确认，不承诺服务器一直保持此状态。
|
文档核对日期：2026-10-11。以下区分本会话检查结果与用户验收确认，不承诺服务器一直保持此状态。
|
||||||
|
|
||||||
|
## 2026-10-17 后续五节点接入（进行中）
|
||||||
|
|
||||||
|
本批授权新增5个节点，复用一个核查验收助手。准备阶段独立审查发现并修复正式适配器的两处调用链阻断：QA订阅远端Python脚本缩进错误、标准化快照未保留协议类型导致最终audit无法重复校验。Python全套34项通过，新增覆盖实际发送的QA脚本编译，以及正式source→snapshot→audit→缓存audit链路；这些是离线回归，不表示本批线上接入或验收完成。
|
||||||
|
|
||||||
|
候选jp06/07/08、us03现场只读核查通过；us02因SSH主机密钥与既有信任不符停止，用户批准备用us04精确首次指纹后核查通过。5台原XrayR均在运行，新接入端口24443空闲；Debian11/10的Git依赖按各自发行版准备，不沿用硬编码bullseye源。本批账本与现场证据在忽略目录`codex/output/node-batch-five-20261017/`。正式开放、动态全量订阅、QA清理、旧资源回归及资源文档同步尚待完成。
|
||||||
|
|
||||||
|
现场进展：面板、网关和5台候选的运维目录已Git部署至2547ea25；修复后的本地适配器源码ac9f249已推送。日本3台QA及独立审查通过，尚未开放；美国两台节点程序已安装，但既有Snap Certbot因AppArmor profile未加载而不能签证书。已准备限定Certbot/core profiles的安全加载脚本与启动/续期unit，尚待提交推送、独立运维部署及实际验证；不将准备状态写成修复完成。
|
||||||
|
|
||||||
|
## 2026-10-17 批量接入流程优化（本地验证）
|
||||||
|
|
||||||
|
已实现正式`ssh-site-v1`适配器及`configure-site.py`入口，去除临时handler中的固定日期、日本、组1和节点数量。正式验收从真实账号授权集合生成快照，完整校验新旧节点、预期出口、严格TLS、原始证据摘要、QA身份/403/禁用/下发排除及时效；集合变化停止。没有有效正式组账号时自动创建100MiB/1小时/1设备非管理员测试用户，journal恢复同一身份，正常成功/失败/等待均清理本任务用户，关闭后不复活；异常中断依靠日志恢复和时限兜底。已批准指纹匹配后才读取凭据，SSH始终经白色机，按线程复用会话、按阶段延迟连接，不自动重放断线命令。
|
||||||
|
|
||||||
|
续跑保留全批前检及publish/audit现场关口，以请求/代码/当前依赖摘要和有效期复用已完成步骤，前驱变化向下游失效；事件输出替代重复状态轮询。原服务以unit/程序/配置/监听和进程信息区分实际冲突与待调查重启，原基线不覆盖。独立运维打包工具从已提交源码生成可追踪快照，保留工作区/真实索引及旧运维基础树；旧命令适配器及旧冻结任务文件不修改。
|
||||||
|
|
||||||
|
本轮Python 32项测试通过；PHP新增10项/38断言及既有9项/65断言隔离测试通过，均未加载真实.env或联网。CoDocs全库离线检查通过，前次.obsidian权限问题已由用户处理，本轮未读写其配置正文。新版本尚未部署至远端或进行真实SSH/客户端性能演练，不能把上一轮三节点验收复用为新实现线上通过，也不承诺十节点耗时。准备/部署/执行与边界见[正式适配器专题](NODE_BATCH_RUNTIME.md)；后续新批次默认按该入口执行。实现源码已提交推送，独立运维分支`codex/node-runtime-20261017`已准备并经Git bundle校验；精确版本按该分支Git记录读取，仍未切换远端旧任务的运行目录。
|
||||||
|
|
||||||
|
## 2026-10-17 三节点批量接入完成
|
||||||
|
|
||||||
|
新增日本JP - 02/03/04（node-jp03/04/05），均已append正式组1、QA禁用。五节点正式完整订阅逐条严格TLS联网、预期IPv4出口/国家及三个QA订阅403通过；旧节点、套餐/用户关联、订阅设置与四个网关容器基线保持。续期演练逐台通过。源码6af6764、独立运维b8d10fac已提交推送并部署运维目录，生产应用代码未随之更新；9项/65断言面板隔离测试及11项批量进程测试通过。
|
||||||
|
|
||||||
|
jp03旧XrayR接入前自身QUIC sniffer panic自动恢复的基线例外、原证据与恢复边界，以及完整参数/验收/备份/自动audit不足的人工补验，见[本次接入记录](NODE_ONBOARDING.md#2026-10-17-三个日本节点批量接入)。GUI/TUN/多地区/多设备并发与生产回退演练未执行。下方2026-10-16未测试/未发布为当时历史，当前以本节为准。CoDocs新增三实例/三域名、服务器核实字段及关联已同步；resources.py check通过（166对象/299关系），check_docs.py全库检查发现6项.obsidian目录/文件权限不合要求；本任务未修改该目录，不能把全库检查写成通过。
|
||||||
|
|
||||||
|
## 2026-10-16 批量接入默认单子助手规则
|
||||||
|
|
||||||
|
按用户明确要求，在项目 AGENTS.md 登记批量接入触发规则，在 [批量专题](NODE_BATCH.md#多智能体协作流程) 维护唯一完整流程：主协调者统一参数/编号/端口/账本与写入；按用户为节省 token 的最新要求，只启动一个核查验收助手，分阶段核查候选及冲突、审查脚本、测试恢复并独立验收。规定复用、失败/暂停处理、写入及正式开放关卡、最小上下文与避免重复检查。此次仅更新文档，没有启动新接入任务或工作助手，没有生产变更；先前单个显示测试助手不算本批分工验收。规则本地落盘不代表自动执行或生产验证；尚未提交/推送。
|
||||||
|
|
||||||
|
## 2026-10-16 批量调度本地验证
|
||||||
|
|
||||||
|
新增 `batch.py`、`command-adapter.py` 与[批量专题](NODE_BATCH.md)：append 模式，全批前检屏障，独立 VPS 并发，共享面板/DNS/中转锁，跨批资源预留，固定计划与逐阶段现场回读恢复。11 项独立进程离线测试通过，覆盖并发/串行边界、双进程去重、回包丢失与 SIGTERM 中断恢复、参数冲突、错误回执和命令摘要变更；没有加载 Laravel/.env 或访问网络。CoDocs 离线结构检查通过。该测试结果只覆盖新调度器与命令适配器，不覆盖下节其他聊天新写的 PHP 面板脚本及真实订阅验证器。
|
||||||
|
|
||||||
|
当前未提交/推送/部署，未新增线上节点。生产操作通过经过审阅的任务命令 profile 接入；没有内置 DNS/中转后台自动写入或 SSH 登录工具，缺少 handler 的阶段返回等待。真实生产命令、十节点耗时、远端现场及业务验收尚未验证，不能宣称已经支持现网一键接入。
|
||||||
|
|
||||||
|
## 2026-10-16 节点脚本本地草稿（未测试/未发布）
|
||||||
|
|
||||||
|
已在本地补promote.php的append/initialize模式隔离和默认预览，新增operations.php字段日志及rollback.php面板精确回退；readiness.php按显式套餐回读，verify-node.py支持完整多节点预期及IP/国家断言、默认不联网，stage.php移除固定面板地址并核对完整重跑参数。使用与限制见[NODE_RUNBOOK](NODE_RUNBOOK.md)。用户要求测试交给奶油，本次未运行测试/语法检查或应用命令，未改线上，未提交/推送/部署，不能视为已验证可用版本。节点命名未改；本次代码不更改展示名或分配编号。
|
||||||
|
|
||||||
|
## 2026-10-16 节点文档核对（仅离线）
|
||||||
|
|
||||||
|
核对两条既有节点链路并补 CoDocs 美国节点关联；新[执行手册](NODE_RUNBOOK.md)统一参数/前检/证书钩子与演练/验收及精确回退，首次初始化与追加模式分开。现有 promote.php 会写固定订阅地址并关联无组套餐/用户，不适合作为追加默认入口；readiness.php 套餐统计硬编码，verify-node.py 只接受单个 AnyTLS 出站，自动追加与统一回退工具仍缺失。文档已明确限制与现有管理界面操作入口，本次未改脚本或线上。以下运行结果均引用原任务证据，不是本轮复验；节点命名继续由原维护任务处理。
|
||||||
|
|
||||||
## 2026-10-16 日本节点
|
## 2026-10-16 日本节点
|
||||||
|
|
||||||
`node-jp02` 已按既有 Git 运维版本独立部署并加入基础节点组 1，节点/机器 ID 2、AnyTLS、名称 `🇯🇵 日本 JP - 01`。完整隔离订阅经中转实际联网、严格 TLS、日本出口、DNS/路由、上下行上报均通过；证书续期演练及专用 deploy hook 通过。正式有效订阅同时包含美国、日本节点，测试账号禁用且订阅 403。未改价格/套餐限制；原 XrayR 与原中转保留。管理面板实际登录、节点/服务器列表切换、在线状态及已保存参数回读通过；未额外提交编辑表单默认值。参数、Git 版本、备份和边界见 [节点接入记录](NODE_ONBOARDING.md#2026-10-16-日本节点接入)。
|
`node-jp02` 已按既有 Git 运维版本独立部署并加入基础节点组 1，节点/机器 ID 2、AnyTLS、名称 `🇯🇵 日本 JP - 01`。完整隔离订阅经中转实际联网、严格 TLS、日本出口、DNS/路由、上下行上报均通过；证书续期演练及专用 deploy hook 通过。正式有效订阅同时包含美国、日本节点，测试账号禁用且订阅 403。未改价格/套餐限制；原 XrayR 与原中转保留。管理面板实际登录、节点/服务器列表切换、在线状态及已保存参数回读通过；未额外提交编辑表单默认值。参数、Git 版本、备份和边界见 [节点接入记录](NODE_ONBOARDING.md#2026-10-16-日本节点接入)。
|
||||||
|
|||||||
93
scripts/node-onboarding/acceptance.py
Normal file
93
scripts/node-onboarding/acceptance.py
Normal file
@ -0,0 +1,93 @@
|
|||||||
|
"""Token-free dynamic evidence validation shared by the reusable site adapter."""
|
||||||
|
import hashlib
|
||||||
|
import ipaddress
|
||||||
|
import re
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
|
||||||
|
|
||||||
|
def digest(value):
|
||||||
|
return hashlib.sha256(json.dumps(value, sort_keys=True, ensure_ascii=False, separators=(',', ':')).encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def snapshot(nodes):
|
||||||
|
if not isinstance(nodes, list) or not nodes:
|
||||||
|
raise ValueError('Empty authorized node set')
|
||||||
|
fields = ('id', 'type', 'tag', 'relay', 'relay_port', 'sni', 'expected_exit', 'country')
|
||||||
|
result = []
|
||||||
|
for node in nodes:
|
||||||
|
if node.get('type') != 'anytls' or any(node.get(k) in (None, '') for k in fields):
|
||||||
|
raise ValueError('Unsupported protocol or incomplete independently registered expectations')
|
||||||
|
if type(node['id']) is not int or node['id']<1 or type(node['relay_port']) is not int or not 1<=node['relay_port']<=65535 or not isinstance(node['country'],str) or not re.fullmatch('[A-Z]{2}',node['country']):
|
||||||
|
raise ValueError('Invalid authorized node identity or expectation')
|
||||||
|
address=ipaddress.ip_address(node['expected_exit'])
|
||||||
|
if getattr(address,'scope_id',None):raise ValueError('Scoped exit expectation unsupported')
|
||||||
|
result.append({k: node[k] for k in fields})
|
||||||
|
for key in ('id', 'tag'):
|
||||||
|
if len({n[key] for n in result}) != len(result):
|
||||||
|
raise ValueError('Duplicate authorized node identity')
|
||||||
|
if len({(n['relay'], n['relay_port'], n['sni']) for n in result}) != len(result):
|
||||||
|
raise ValueError('Ambiguous authorized endpoint')
|
||||||
|
return sorted(result, key=lambda n: n['id'])
|
||||||
|
|
||||||
|
|
||||||
|
def audit(before, after, report_bytes, qa_bytes, binding, plan_sha, expected_qas, now=None, ttl=900):
|
||||||
|
now = time.time() if now is None else now
|
||||||
|
before, after = snapshot(before), snapshot(after)
|
||||||
|
if before != after or binding.get('snapshot_sha256') != digest(before):
|
||||||
|
raise ValueError('Authorized node set changed; regenerate and verify the complete set')
|
||||||
|
if binding.get('plan_sha256') != plan_sha:
|
||||||
|
raise ValueError('Wrong plan binding')
|
||||||
|
for label, raw in [('report', report_bytes), ('qa_rejections', qa_bytes)]:
|
||||||
|
if binding.get(label + '_sha256') != hashlib.sha256(raw).hexdigest():
|
||||||
|
raise ValueError('Evidence bytes changed')
|
||||||
|
report, qas = json.loads(report_bytes), json.loads(qa_bytes)
|
||||||
|
for stamp in (binding.get('verified_at'), report.get('verified_at'), qas.get('captured_at')):
|
||||||
|
if type(stamp) not in (int, float) or not 0 <= now-stamp <= ttl:
|
||||||
|
raise ValueError('Evidence expired or future dated')
|
||||||
|
if report.get('state') != 'live_verified' or len(report.get('results', [])) != len(before):
|
||||||
|
raise ValueError('Incomplete live verification')
|
||||||
|
expected = {(n['tag'], n['relay'], n['relay_port'], n['sni'], n['expected_exit'], n['country']) for n in before}
|
||||||
|
actual = set()
|
||||||
|
for n in report['results']:
|
||||||
|
if n.get('live') is not True or n.get('strict_tls') is not True:
|
||||||
|
raise ValueError('Live strict TLS required for every node')
|
||||||
|
actual.add(tuple(n.get(k) for k in ('tag', 'relay', 'relay_port', 'sni', 'exit_ip', 'country')))
|
||||||
|
if actual != expected:
|
||||||
|
raise ValueError('Node/endpoint/exit/country mismatch')
|
||||||
|
if qas.get('plan_sha256') != plan_sha or set(qas.get('users', {})) != set(expected_qas):
|
||||||
|
raise ValueError('QA evidence identity mismatch')
|
||||||
|
for key, identity in expected_qas.items():
|
||||||
|
q = qas['users'][key]
|
||||||
|
if q.get('id') != identity or q.get('http_status') != 403 or q.get('banned') is not True or q.get('excluded') is not True:
|
||||||
|
raise ValueError('QA account not fully revoked')
|
||||||
|
return {'nodes': len(before), 'snapshot_sha256': digest(before), 'verified_at': now}
|
||||||
|
|
||||||
|
|
||||||
|
def service_change(before, current):
|
||||||
|
"""Process movement is investigatory; never silently accept/rebase it."""
|
||||||
|
required = ('unit_sha256', 'binary_sha256', 'config_sha256', 'listeners')
|
||||||
|
if any(not before.get(k) or not current.get(k) for k in required):
|
||||||
|
return 'unknown'
|
||||||
|
if any(before[k] != current[k] for k in required) or current.get('active') is not True:
|
||||||
|
return 'conflict'
|
||||||
|
if any(before.get(k) != current.get(k) for k in ('pid', 'started', 'restarts')):
|
||||||
|
return 'investigate_restart'
|
||||||
|
return 'ready'
|
||||||
|
|
||||||
|
|
||||||
|
def provider_state(observed, node, now=None):
|
||||||
|
"""Canonical observations from the registered provider UI/API, never HTML substrings."""
|
||||||
|
now=time.time() if now is None else now
|
||||||
|
stamp=observed.get('captured_at')
|
||||||
|
if type(stamp) not in (int,float) or not 0<=now-stamp<=900:
|
||||||
|
raise ValueError('Provider observation expired')
|
||||||
|
if not isinstance(observed.get('dns'),list) or not isinstance(observed.get('relays'),list):
|
||||||
|
raise ValueError('Canonical DNS and relay observations required')
|
||||||
|
dns=[d for d in observed['dns'] if d.get('name')==node['tls_domain']]
|
||||||
|
relays=[r for r in observed['relays'] if r.get('host')==node['relay_host'] and r.get('port')==node['relay_port']]
|
||||||
|
if dns and (len(dns)!=1 or dns[0].get('type')!='A' or dns[0].get('address')!=node['node_address'] or dns[0].get('proxy') is not False):
|
||||||
|
raise ValueError('DNS provider resource conflict')
|
||||||
|
if relays and (len(relays)!=1 or relays[0].get('target')!=node['node_address'] or relays[0].get('listen_port')!=node['listen_port'] or relays[0].get('name')!='dpl-'+node['key']+'-anytls' or relays[0].get('proxy_protocol_in') is not False or relays[0].get('proxy_protocol_out') is not False):
|
||||||
|
raise ValueError('Relay provider resource conflict')
|
||||||
|
return {'dns':dns,'relays':relays}
|
||||||
461
scripts/node-onboarding/batch.py
Normal file
461
scripts/node-onboarding/batch.py
Normal file
@ -0,0 +1,461 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""DPL append-only batch coordinator. Adapters must probe before and after writes."""
|
||||||
|
import argparse
|
||||||
|
import concurrent.futures
|
||||||
|
import contextlib
|
||||||
|
import fcntl
|
||||||
|
import hashlib
|
||||||
|
import ipaddress
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import signal
|
||||||
|
import sqlite3
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
|
||||||
|
PHASES = ('stage', 'dns', 'provision', 'relay', 'verify', 'publish', 'audit')
|
||||||
|
SHARED = {'stage': 'panel', 'dns': 'dns', 'relay': 'relay', 'publish': 'panel'}
|
||||||
|
STOP = threading.Event()
|
||||||
|
EVENTS = False
|
||||||
|
|
||||||
|
|
||||||
|
def digest(value):
|
||||||
|
return hashlib.sha256(json.dumps(value, sort_keys=True, ensure_ascii=False,
|
||||||
|
separators=(',', ':')).encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def read_json(path):
|
||||||
|
return json.loads(Path(path).read_text())
|
||||||
|
|
||||||
|
|
||||||
|
def atomic_json(path, value):
|
||||||
|
path = Path(path)
|
||||||
|
tmp = path.with_name(path.name + '.tmp-' + str(os.getpid()) + '-' + str(threading.get_ident()))
|
||||||
|
with tmp.open('x') as f:
|
||||||
|
os.chmod(tmp, 0o600)
|
||||||
|
json.dump(value, f, ensure_ascii=False, sort_keys=True)
|
||||||
|
f.flush()
|
||||||
|
os.fsync(f.fileno())
|
||||||
|
os.replace(tmp, path)
|
||||||
|
|
||||||
|
|
||||||
|
def validate(plan):
|
||||||
|
if plan.get('version') != 1 or plan.get('mode') != 'append':
|
||||||
|
raise ValueError('Only version=1, mode=append is supported')
|
||||||
|
if not re.fullmatch(r'[a-z0-9-]{1,64}', plan.get('batch_id', '')):
|
||||||
|
raise ValueError('Invalid batch_id')
|
||||||
|
context = plan.get('context', {})
|
||||||
|
required = ('panel_vps', 'gateway_vps', 'jump_vps', 'app_root', 'panel_url',
|
||||||
|
'subscription_url', 'provision_commit', 'bundle', 'client_binary')
|
||||||
|
if any(not context.get(k) for k in required):
|
||||||
|
raise ValueError('Missing explicit context fields')
|
||||||
|
if not re.fullmatch(r'[0-9a-f]{40}', context['provision_commit']):
|
||||||
|
raise ValueError('A full reviewed, pushed Git commit is required')
|
||||||
|
for k in ('panel_url', 'subscription_url'):
|
||||||
|
if not re.fullmatch(r'https://[a-z0-9.-]+', context[k]):
|
||||||
|
raise ValueError('HTTPS origin without token/path required: ' + k)
|
||||||
|
adapter = plan.get('adapter')
|
||||||
|
if not isinstance(adapter, list) or not adapter or not all(isinstance(x, str) for x in adapter):
|
||||||
|
raise ValueError('adapter must be an argv array, never a shell string')
|
||||||
|
if not plan.get('nodes') or not isinstance(plan['nodes'], list):
|
||||||
|
raise ValueError('A nonempty node list is required')
|
||||||
|
seen = {k: set() for k in ('key', 'name', 'vps_id', 'node_address', 'tls_domain', 'relay', 'socks_port', 'number')}
|
||||||
|
for n in plan['nodes']:
|
||||||
|
if any(k not in n for k in ('key', 'country', 'sequence', 'flag', 'country_name', 'name',
|
||||||
|
'vps_id', 'node_address', 'listen_port', 'relay_host',
|
||||||
|
'relay_port', 'tls_domain', 'formal_group_id',
|
||||||
|
'expected_exit', 'socks_port')):
|
||||||
|
raise ValueError('Incomplete node specification')
|
||||||
|
if not re.fullmatch(r'[a-z0-9-]{1,48}', n['key']):
|
||||||
|
raise ValueError('Invalid deployment key')
|
||||||
|
if not re.fullmatch(r'[A-Z]{2}', n['country']):
|
||||||
|
raise ValueError('Country must be a two-letter code')
|
||||||
|
if type(n['sequence']) is not int or not 1 <= n['sequence'] <= 99:
|
||||||
|
raise ValueError('Display sequence must be 01..99')
|
||||||
|
expected = f"{n['flag']} {n['country_name']} {n['country']} - {n['sequence']:02d}"
|
||||||
|
flag = ''.join(chr(0x1F1E6 + ord(x) - ord('A')) for x in n['country'])
|
||||||
|
if n['name'] != expected or n['flag'] != flag:
|
||||||
|
raise ValueError('Display name/flag does not match country and sequence')
|
||||||
|
for k in ('listen_port', 'relay_port', 'socks_port'):
|
||||||
|
if type(n[k]) is not int or not 1024 <= n[k] <= 65535:
|
||||||
|
raise ValueError('Invalid port: ' + k)
|
||||||
|
if type(n['formal_group_id']) is not int or n['formal_group_id'] < 1:
|
||||||
|
raise ValueError('An existing formal group is required')
|
||||||
|
ipaddress.ip_address(n['node_address'])
|
||||||
|
exit_address=ipaddress.ip_address(n['expected_exit'])
|
||||||
|
if getattr(exit_address,'scope_id',None):raise ValueError('Scoped/link-local exit addresses are not supported')
|
||||||
|
for k in ('relay_host', 'tls_domain'):
|
||||||
|
if not re.fullmatch(r'[a-z0-9](?:[a-z0-9.-]*[a-z0-9])?\.[a-z]{2,}', n[k]):
|
||||||
|
raise ValueError('Invalid DNS name: ' + k)
|
||||||
|
for k in ('vps_id',):
|
||||||
|
if not re.fullmatch(r'VPS-[0-9]{3}', n[k]):
|
||||||
|
raise ValueError('Use the exact CoDocs VPS ID')
|
||||||
|
identities = {k: n[k] for k in ('key', 'name', 'vps_id', 'node_address', 'tls_domain', 'socks_port')}
|
||||||
|
identities.update(relay=(n['relay_host'], n['relay_port']), number=(n['country'], n['sequence']))
|
||||||
|
for k, value in identities.items():
|
||||||
|
if value in seen[k]:
|
||||||
|
raise ValueError('Duplicate batch identity: ' + k)
|
||||||
|
seen[k].add(value)
|
||||||
|
# The plan carries references, never secret values. Adapters load credentials themselves.
|
||||||
|
def check(value):
|
||||||
|
if isinstance(value, dict):
|
||||||
|
for k, v in value.items():
|
||||||
|
if re.search(r'password|secret|token|private_key', k, re.I):
|
||||||
|
raise ValueError('Secret fields are not allowed in a batch plan')
|
||||||
|
check(v)
|
||||||
|
elif isinstance(value, list):
|
||||||
|
for v in value:
|
||||||
|
check(v)
|
||||||
|
check(plan)
|
||||||
|
return plan
|
||||||
|
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def lock(root, name):
|
||||||
|
path = root / ('lock-' + hashlib.sha256(name.encode()).hexdigest())
|
||||||
|
with path.open('a+') as f:
|
||||||
|
os.chmod(path, 0o600)
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||||
|
break
|
||||||
|
except BlockingIOError:
|
||||||
|
if STOP.wait(.1):
|
||||||
|
raise RuntimeError('Stopped while waiting for resource lock')
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
fcntl.flock(f, fcntl.LOCK_UN)
|
||||||
|
|
||||||
|
|
||||||
|
class Ledger:
|
||||||
|
def __init__(self, root):
|
||||||
|
self.root = Path(root).resolve()
|
||||||
|
if self.root == Path('/'):
|
||||||
|
raise ValueError('Invalid state root')
|
||||||
|
self.root.mkdir(parents=True, mode=0o700, exist_ok=True)
|
||||||
|
os.chmod(self.root, 0o700)
|
||||||
|
self.path = self.root / 'registry.sqlite3'
|
||||||
|
with self.db() as c:
|
||||||
|
c.executescript('''
|
||||||
|
CREATE TABLE IF NOT EXISTS batches(id TEXT PRIMARY KEY, digest TEXT NOT NULL);
|
||||||
|
CREATE TABLE IF NOT EXISTS claims(namespace TEXT, kind TEXT, value TEXT,
|
||||||
|
owner TEXT NOT NULL, PRIMARY KEY(namespace,kind,value));
|
||||||
|
CREATE TABLE IF NOT EXISTS evidence(batch TEXT,node TEXT,phase TEXT,
|
||||||
|
dependency TEXT,expires REAL,request TEXT,PRIMARY KEY(batch,node,phase));
|
||||||
|
CREATE TABLE IF NOT EXISTS phases(batch TEXT, node TEXT, phase TEXT,
|
||||||
|
status TEXT NOT NULL, attempts INTEGER NOT NULL DEFAULT 0,
|
||||||
|
updated REAL NOT NULL, PRIMARY KEY(batch,node,phase));
|
||||||
|
''')
|
||||||
|
os.chmod(self.path, 0o600)
|
||||||
|
|
||||||
|
@contextlib.contextmanager
|
||||||
|
def db(self):
|
||||||
|
c = sqlite3.connect(self.path, timeout=30)
|
||||||
|
try:
|
||||||
|
c.execute('PRAGMA synchronous=FULL')
|
||||||
|
with c: yield c
|
||||||
|
finally: c.close()
|
||||||
|
|
||||||
|
def reserve(self, plan):
|
||||||
|
namespace = plan['context']['panel_vps'] + ':' + plan['context']['app_root']
|
||||||
|
with self.db() as c:
|
||||||
|
c.execute('BEGIN IMMEDIATE')
|
||||||
|
prior = c.execute('SELECT digest FROM batches WHERE id=?', (plan['batch_id'],)).fetchone()
|
||||||
|
if prior and prior[0] != digest(plan):
|
||||||
|
raise ValueError('Batch plan changed; resume requires the original exact plan')
|
||||||
|
for n in plan['nodes']:
|
||||||
|
owner = digest({'context': plan['context'], 'node': n})
|
||||||
|
claims = {k: n[k] for k in ('key', 'name', 'vps_id', 'node_address', 'tls_domain')}
|
||||||
|
claims['relay'] = [n['relay_host'], n['relay_port']]
|
||||||
|
claims['number'] = [n['country'], n['sequence']]
|
||||||
|
for kind, value in claims.items():
|
||||||
|
scope = namespace
|
||||||
|
if kind == 'relay':
|
||||||
|
scope = 'relay:' + n['relay_host']
|
||||||
|
elif kind in ('vps_id', 'node_address', 'tls_domain'):
|
||||||
|
scope = 'infrastructure'
|
||||||
|
encoded = json.dumps(value, ensure_ascii=False, sort_keys=True)
|
||||||
|
existing = c.execute('SELECT owner FROM claims WHERE namespace=? AND kind=? AND value=?',
|
||||||
|
(scope, kind, encoded)).fetchone()
|
||||||
|
if existing and existing[0] != owner:
|
||||||
|
raise ValueError('Another batch reserved identity: ' + kind)
|
||||||
|
c.execute('INSERT OR IGNORE INTO claims VALUES(?,?,?,?)', (scope, kind, encoded, owner))
|
||||||
|
# SOCKS listeners are shared by all batches using this gateway.
|
||||||
|
scope, value = 'gateway:' + plan['context']['gateway_vps'], str(n['socks_port'])
|
||||||
|
old = c.execute('SELECT owner FROM claims WHERE namespace=? AND kind=? AND value=?',
|
||||||
|
(scope, 'socks_port', value)).fetchone()
|
||||||
|
if old and old[0] != owner:
|
||||||
|
raise ValueError('Another batch reserved the gateway SOCKS port')
|
||||||
|
c.execute('INSERT OR IGNORE INTO claims VALUES(?,?,?,?)', (scope, 'socks_port', value, owner))
|
||||||
|
formal_port=plan['context'].get('formal_socks_port')
|
||||||
|
if formal_port is not None:
|
||||||
|
if type(formal_port) is not int or not 1024<=formal_port<=65535 or any(n['socks_port']==formal_port for n in plan['nodes']):
|
||||||
|
raise ValueError('A distinct formal SOCKS port is required')
|
||||||
|
scope,value='gateway:'+plan['context']['gateway_vps'],str(formal_port)
|
||||||
|
owner=digest({'context':plan['context'],'formal_port':formal_port})
|
||||||
|
old=c.execute('SELECT owner FROM claims WHERE namespace=? AND kind=? AND value=?',(scope,'socks_port',value)).fetchone()
|
||||||
|
if old and old[0]!=owner:raise ValueError('Formal SOCKS port already reserved')
|
||||||
|
c.execute('INSERT OR IGNORE INTO claims VALUES(?,?,?,?)',(scope,'socks_port',value,owner))
|
||||||
|
c.execute('INSERT OR IGNORE INTO batches VALUES(?,?)', (plan['batch_id'], digest(plan)))
|
||||||
|
|
||||||
|
def state(self, batch, key, phase):
|
||||||
|
with self.db() as c:
|
||||||
|
row = c.execute('SELECT status FROM phases WHERE batch=? AND node=? AND phase=?',
|
||||||
|
(batch, key, phase)).fetchone()
|
||||||
|
return row[0] if row else 'pending'
|
||||||
|
|
||||||
|
def reusable(self, batch, key, phase, dependency, request):
|
||||||
|
if not dependency or self.state(batch, key, phase) != 'done':
|
||||||
|
return False
|
||||||
|
with self.db() as c:
|
||||||
|
row = c.execute('SELECT dependency,expires,request FROM evidence WHERE batch=? AND node=? AND phase=?', (batch,key,phase)).fetchone()
|
||||||
|
return bool(row and row[0] == dependency and time.time() <= row[1] <= time.time()+900 and row[2] == request)
|
||||||
|
|
||||||
|
def proof(self, batch, key, phase, dependency, expires, request):
|
||||||
|
with self.db() as c:
|
||||||
|
c.execute('INSERT OR REPLACE INTO evidence VALUES(?,?,?,?,?,?)',(batch,key,phase,dependency,expires,request))
|
||||||
|
|
||||||
|
def mark(self, batch, key, phase, status, attempt=False):
|
||||||
|
previous=self.state(batch,key,phase)
|
||||||
|
with self.db() as c:
|
||||||
|
c.execute('''INSERT INTO phases VALUES(?,?,?,?,?,?) ON CONFLICT(batch,node,phase)
|
||||||
|
DO UPDATE SET status=excluded.status,attempts=phases.attempts+?,updated=excluded.updated''',
|
||||||
|
(batch, key, phase, status, int(attempt), time.time(), int(attempt)))
|
||||||
|
if previous!=status or attempt:
|
||||||
|
fd=os.open(self.root/'events.jsonl',os.O_CREAT|os.O_WRONLY|os.O_APPEND,0o600)
|
||||||
|
try:
|
||||||
|
os.fchmod(fd,0o600)
|
||||||
|
event=json.dumps({'batch':batch,'node':key,'phase':phase,'status':status,'time':time.time()})+'\n'
|
||||||
|
os.write(fd,event.encode())
|
||||||
|
if EVENTS:sys.stdout.write(event);sys.stdout.flush()
|
||||||
|
finally:os.close(fd)
|
||||||
|
|
||||||
|
|
||||||
|
class Coordinator:
|
||||||
|
def __init__(self, plan, root, jobs=4, timeout=900):
|
||||||
|
self.plan = validate(plan)
|
||||||
|
self.ledger = Ledger(root)
|
||||||
|
self.jobs = jobs
|
||||||
|
self.timeout = timeout
|
||||||
|
self.dependencies = {}
|
||||||
|
self.runtime = None
|
||||||
|
self.ledger.reserve(plan)
|
||||||
|
self.dir = self.ledger.root / plan['batch_id']
|
||||||
|
self.dir.mkdir(mode=0o700, exist_ok=True)
|
||||||
|
self.plan_file = self.dir / 'plan.json'
|
||||||
|
atomic_json(self.plan_file, plan)
|
||||||
|
if plan.get('runtime'):
|
||||||
|
self.check_runtime()
|
||||||
|
directory = str(Path(__file__).resolve().parent)
|
||||||
|
if directory not in sys.path: sys.path.insert(0, directory)
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
self.runtime = SiteAdapter(plan, self.dir, atomic_writer=atomic_json, resource_lock=lock)
|
||||||
|
|
||||||
|
def check_runtime(self):
|
||||||
|
runtime = self.plan['runtime']
|
||||||
|
if runtime.get('kind') != 'ssh-site-v1': raise ValueError('Unknown runtime')
|
||||||
|
if runtime.get('python') != str(Path(sys.executable).resolve()):raise ValueError('Use the frozen runtime interpreter')
|
||||||
|
import paramiko
|
||||||
|
if paramiko.__version__ != runtime.get('paramiko_version'):raise ValueError('Runtime dependency version changed')
|
||||||
|
base = Path(__file__).resolve().parent
|
||||||
|
required = [base/n for n in ('batch.py','site_adapter.py','ssh_pool.py','acceptance.py')]
|
||||||
|
required += [Path(self.plan['context'][k]) for k in ('ssh_config','exit_registry')]
|
||||||
|
files = runtime.get('files', {})
|
||||||
|
if any(str(f) not in files for f in required): raise ValueError('Runtime source pin missing')
|
||||||
|
for f, sha in files.items():
|
||||||
|
if not Path(f).is_absolute() or hashlib.sha256(Path(f).read_bytes()).hexdigest() != sha:
|
||||||
|
raise ValueError('Runtime code/config changed')
|
||||||
|
|
||||||
|
|
||||||
|
def call(self, n, phase, action):
|
||||||
|
fingerprint = digest({'context': self.plan['context'], 'node': n})
|
||||||
|
path = self.dir / (n['key'] + '.json')
|
||||||
|
atomic_json(path, dict(n, request_sha256=fingerprint))
|
||||||
|
argv = self.plan['adapter'] + ['--plan', str(self.plan_file), '--node', str(path),
|
||||||
|
'--phase', phase, '--action', action, '--state-dir', str(self.dir)]
|
||||||
|
if self.runtime:
|
||||||
|
self.check_runtime()
|
||||||
|
r = self.runtime.call(dict(n, request_sha256=fingerprint), phase, action)
|
||||||
|
if (r.get('key'),r.get('phase'),r.get('request_sha256')) != (n['key'],phase,fingerprint):
|
||||||
|
raise RuntimeError('Runtime identity mismatch')
|
||||||
|
if r.get('status') not in ('ready','absent','waiting','unknown','conflict'):
|
||||||
|
raise RuntimeError('Runtime status invalid')
|
||||||
|
return r
|
||||||
|
with subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
|
||||||
|
start_new_session=True) as p:
|
||||||
|
deadline = time.monotonic() + self.timeout
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
output, _ = p.communicate(timeout=.2)
|
||||||
|
break
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
if STOP.is_set() or time.monotonic() >= deadline:
|
||||||
|
os.killpg(p.pid, signal.SIGTERM)
|
||||||
|
try:
|
||||||
|
p.communicate(timeout=5)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
os.killpg(p.pid, signal.SIGKILL)
|
||||||
|
p.communicate()
|
||||||
|
raise RuntimeError('Adapter interrupted; remote outcome must be probed')
|
||||||
|
if p.returncode or len(output) > 65536:
|
||||||
|
raise RuntimeError('Adapter failed; no stdout/stderr persisted')
|
||||||
|
r = json.loads(output)
|
||||||
|
if (r.get('key'), r.get('phase'), r.get('request_sha256')) != (n['key'], phase, fingerprint):
|
||||||
|
raise RuntimeError('Adapter receipt identity mismatch')
|
||||||
|
if r.get('status') not in ('ready', 'absent', 'waiting', 'unknown', 'conflict'):
|
||||||
|
raise RuntimeError('Invalid adapter status')
|
||||||
|
return r
|
||||||
|
|
||||||
|
def phase(self, n, phase):
|
||||||
|
try:
|
||||||
|
return self._phase(n, phase)
|
||||||
|
except Exception:
|
||||||
|
self.ledger.mark(self.plan['batch_id'], n['key'], phase, 'unknown')
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _phase(self, n, phase):
|
||||||
|
batch, key = self.plan['batch_id'], n['key']
|
||||||
|
resource = SHARED.get(phase, n['vps_id'])
|
||||||
|
# Common panel/dns/relay lock names across batches and processes.
|
||||||
|
namespace = self.plan['context']['panel_vps']
|
||||||
|
if resource in ('dns', 'relay'):
|
||||||
|
namespace = 'dpl-external'
|
||||||
|
with lock(self.ledger.root, namespace + ':' + resource):
|
||||||
|
dependency = self.dependencies.get(key, {}).get(phase)
|
||||||
|
request = digest({'plan': self.plan, 'node': n})
|
||||||
|
# Publish/audit/preflight are always live; unknown or running phases cannot use a cache.
|
||||||
|
if phase in ('stage','dns','provision','relay','verify') and self.ledger.reusable(batch,key,phase,dependency,request):
|
||||||
|
if self.runtime: self.check_runtime()
|
||||||
|
return True
|
||||||
|
r = self.call(n, phase, 'probe')
|
||||||
|
if phase == 'preflight' and r['status'] == 'ready':
|
||||||
|
deps = r.get('resume_dependencies', {})
|
||||||
|
clean = {k:v for k,v in deps.items() if k in PHASES and isinstance(v,str) and re.fullmatch('[0-9a-f]{64}',v)}
|
||||||
|
self.dependencies[key] = {k:digest({p:clean.get(p) for p in PHASES[:PHASES.index(k)+1]}) for k in clean}
|
||||||
|
if r['status'] == 'ready':
|
||||||
|
if dependency and type(r.get('valid_until')) in (int,float) and time.time() < r['valid_until'] <= time.time()+900:
|
||||||
|
self.ledger.proof(batch,key,phase,dependency,r['valid_until'],request)
|
||||||
|
self.ledger.mark(batch, key, phase, 'done')
|
||||||
|
return True
|
||||||
|
if r['status'] == 'conflict':
|
||||||
|
STOP.set()
|
||||||
|
if phase in ('preflight', 'verify', 'audit') or r['status'] != 'absent' or r.get('retry_safe') is not True:
|
||||||
|
self.ledger.mark(batch, key, phase, r['status'])
|
||||||
|
return False
|
||||||
|
if STOP.is_set():
|
||||||
|
return False
|
||||||
|
# Persist intent before a write; never infer success from the command exit code.
|
||||||
|
self.ledger.mark(batch, key, phase, 'running', attempt=True)
|
||||||
|
applied = self.call(n, phase, 'apply')
|
||||||
|
if applied['status'] == 'conflict':
|
||||||
|
self.ledger.mark(batch, key, phase, 'conflict')
|
||||||
|
STOP.set()
|
||||||
|
return False
|
||||||
|
r = self.call(n, phase, 'probe')
|
||||||
|
self.ledger.mark(batch, key, phase, 'done' if r['status'] == 'ready' else r['status'])
|
||||||
|
if r['status'] == 'ready' and dependency and type(r.get('valid_until')) in (int,float) and time.time()<r['valid_until']<=time.time()+900:
|
||||||
|
self.ledger.proof(batch,key,phase,dependency,r['valid_until'],request)
|
||||||
|
if r['status'] == 'conflict':
|
||||||
|
STOP.set()
|
||||||
|
return r['status'] == 'ready'
|
||||||
|
|
||||||
|
def worker(self, n):
|
||||||
|
try:
|
||||||
|
with lock(self.ledger.root, 'node:' + self.plan['context']['panel_vps'] + ':' + n['key']):
|
||||||
|
self.ledger.mark(self.plan['batch_id'], n['key'], 'coordinator', 'running')
|
||||||
|
# Already finished batches are re-audited; disabled QA need not be re-enabled.
|
||||||
|
if self.ledger.state(self.plan['batch_id'], n['key'], 'audit') == 'done':
|
||||||
|
success = self.phase(n, 'audit')
|
||||||
|
else:
|
||||||
|
success = True
|
||||||
|
for phase in PHASES:
|
||||||
|
if STOP.is_set() or not self.phase(n, phase):
|
||||||
|
success = False
|
||||||
|
break
|
||||||
|
self.ledger.mark(self.plan['batch_id'], n['key'], 'coordinator', 'done' if success else 'waiting')
|
||||||
|
return success
|
||||||
|
except Exception:
|
||||||
|
# Never print adapter payloads, commands, credentials or exception contents.
|
||||||
|
self.ledger.mark(self.plan['batch_id'], n['key'], 'coordinator', 'unknown')
|
||||||
|
return False
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
# Whole-batch barrier: no staging until every candidate passed current preflight.
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=self.jobs) as pool:
|
||||||
|
def check(n):
|
||||||
|
try:
|
||||||
|
return self.phase(n, 'preflight')
|
||||||
|
except Exception:
|
||||||
|
self.ledger.mark(self.plan['batch_id'], n['key'], 'preflight', 'unknown')
|
||||||
|
return False
|
||||||
|
checks = list(pool.map(check, self.plan['nodes']))
|
||||||
|
if not all(checks):
|
||||||
|
return False
|
||||||
|
return all(list(pool.map(self.worker, self.plan['nodes'])))
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
p.add_argument('command', choices=('plan', 'run', 'status'))
|
||||||
|
p.add_argument('--plan', required=True)
|
||||||
|
p.add_argument('--state-root', required=True)
|
||||||
|
p.add_argument('--jobs', type=int, default=4)
|
||||||
|
p.add_argument('--timeout', type=int, default=900)
|
||||||
|
p.add_argument('--events', action='store_true', help='Emit token-free phase transition JSON lines; final result is last line')
|
||||||
|
p.add_argument('--execute', action='store_true', help='Required for a run that may write remotely')
|
||||||
|
a = p.parse_args()
|
||||||
|
global EVENTS
|
||||||
|
EVENTS = a.events
|
||||||
|
os.umask(0o077)
|
||||||
|
if not 1 <= a.jobs <= 16 or not 1 <= a.timeout <= 3600:
|
||||||
|
p.error('jobs must be 1..16; timeout 1..3600')
|
||||||
|
try:
|
||||||
|
plan = validate(read_json(a.plan))
|
||||||
|
if a.command == 'plan':
|
||||||
|
print(json.dumps({'batch_id': plan['batch_id'], 'nodes': len(plan['nodes']),
|
||||||
|
'plan_sha256': digest(plan), 'writes': False}))
|
||||||
|
return 0
|
||||||
|
if a.command == 'run' and not a.execute:
|
||||||
|
p.error('run requires --execute; plan does not perform remote requests')
|
||||||
|
if a.command == 'status':
|
||||||
|
path = Path(a.state_root).resolve() / 'registry.sqlite3'
|
||||||
|
rows = {}
|
||||||
|
if path.exists():
|
||||||
|
with sqlite3.connect(path.as_uri() + '?mode=ro', uri=True) as c:
|
||||||
|
prior = c.execute('SELECT digest FROM batches WHERE id=?', (plan['batch_id'],)).fetchone()
|
||||||
|
if prior and prior[0] != digest(plan):
|
||||||
|
raise ValueError('Batch plan differs from its reserved plan')
|
||||||
|
rows = {(key, phase): status for key, phase, status in c.execute(
|
||||||
|
'SELECT node,phase,status FROM phases WHERE batch=?', (plan['batch_id'],))}
|
||||||
|
print(json.dumps({n['key']: {phase: rows.get((n['key'], phase), 'pending')
|
||||||
|
for phase in ('preflight',) + PHASES + ('coordinator',)}
|
||||||
|
for n in plan['nodes']}))
|
||||||
|
return 0
|
||||||
|
runner = Coordinator(plan, a.state_root, a.jobs, a.timeout)
|
||||||
|
for sig in (signal.SIGINT, signal.SIGTERM):
|
||||||
|
signal.signal(sig, lambda *_: STOP.set())
|
||||||
|
try:
|
||||||
|
success = runner.run()
|
||||||
|
finally:
|
||||||
|
if runner.runtime:
|
||||||
|
try:runner.runtime.close()
|
||||||
|
except Exception:
|
||||||
|
for node in plan['nodes']:runner.ledger.mark(plan['batch_id'],node['key'],'coordinator','unknown')
|
||||||
|
raise
|
||||||
|
print(json.dumps({'batch_id': plan['batch_id'], 'complete': success,
|
||||||
|
'next': None if success else 'Inspect status, resolve waiting/conflict, rerun the exact plan'}))
|
||||||
|
return 0 if success else 2
|
||||||
|
except Exception as exc:
|
||||||
|
# Validation errors have no adapter/credential material.
|
||||||
|
print(json.dumps({'error': str(exc) if isinstance(exc, ValueError) else 'Coordinator failed; inspect protected state'}))
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
60
scripts/node-onboarding/bundle-ops.py
Normal file
60
scripts/node-onboarding/bundle-ops.py
Normal file
@ -0,0 +1,60 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Build a versioned flat operations bundle from committed files, without touching the worktree/index."""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
FILES=('stage.php','promote.php','operations.php','rollback.php','readiness.php',
|
||||||
|
'verify-node.py','panel-acceptance.php','install-node.sh','renew-node.sh','dpl-node.service')
|
||||||
|
|
||||||
|
|
||||||
|
def build(repo,source,base,branch,bundle,advance=False):
|
||||||
|
if not all(re.fullmatch('[0-9a-f]{40}',r) for r in (source,base)) or not re.fullmatch('codex/[A-Za-z0-9_/-]+',branch) or '..' in branch:
|
||||||
|
raise ValueError('Explicit full commits and a new codex branch required')
|
||||||
|
destination=Path(bundle)
|
||||||
|
if not destination.is_absolute() or destination.exists() or destination.is_symlink():raise ValueError('New absolute bundle path required')
|
||||||
|
def git(*args,input=None,env=None):
|
||||||
|
r=subprocess.run(['git','-C',str(repo),*args],input=input,stdout=subprocess.PIPE,stderr=subprocess.DEVNULL,env=env)
|
||||||
|
if r.returncode:raise ValueError('Git source/base/ref check failed')
|
||||||
|
return r.stdout
|
||||||
|
git('cat-file','-e',source+'^{commit}');git('cat-file','-e',base+'^{commit}');git('check-ref-format','refs/heads/'+branch)
|
||||||
|
exists=subprocess.run(['git','-C',str(repo),'show-ref','--verify','--quiet','refs/heads/'+branch]).returncode
|
||||||
|
expected_old='0'*40
|
||||||
|
if exists==0 and advance:
|
||||||
|
expected_old=git('rev-parse','refs/heads/'+branch).decode().strip()
|
||||||
|
if expected_old!=base:raise ValueError('Advance requires the exact current branch head as reviewed base')
|
||||||
|
elif exists!=1:raise ValueError('Branch exists or outcome unknown; inspect it, do not overwrite')
|
||||||
|
with tempfile.TemporaryDirectory(prefix='dpl-ops-index-',dir=destination.parent) as temp:
|
||||||
|
env=dict(os.environ,GIT_INDEX_FILE=str(Path(temp)/'index'))
|
||||||
|
git('read-tree',base,env=env)
|
||||||
|
for name in FILES:
|
||||||
|
entry=git('ls-tree',source,'--','scripts/node-onboarding/'+name).decode().strip()
|
||||||
|
if not entry:raise ValueError('Required committed operations file missing')
|
||||||
|
mode,kind,rest=entry.split(' ',2);sha,_=rest.split('\t',1)
|
||||||
|
if kind!='blob' or mode not in ('100644','100755'):raise ValueError('Regular committed operations file required')
|
||||||
|
git('update-index','--add','--cacheinfo',mode+','+sha+','+name,env=env)
|
||||||
|
tree=git('write-tree',env=env).decode().strip()
|
||||||
|
commit=git('commit-tree',tree,'-p',base,input=('DPL operations snapshot\n\nSource commit: '+source+'\n').encode()).decode().strip()
|
||||||
|
git('update-ref','refs/heads/'+branch,commit,expected_old)
|
||||||
|
git('bundle','create',str(destination),'refs/heads/'+branch)
|
||||||
|
destination.chmod(0o600)
|
||||||
|
return {'commit':commit,'source_commit':source,'branch':branch,'bundle_sha256':hashlib.sha256(destination.read_bytes()).hexdigest(),'files':len(FILES),'network':False}
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p=argparse.ArgumentParser(description=__doc__)
|
||||||
|
p.add_argument('--repo',default=str(Path(__file__).resolve().parents[2]))
|
||||||
|
for k in ('source','base','branch','bundle'):p.add_argument('--'+k,required=True)
|
||||||
|
p.add_argument('--advance',action='store_true',help='Explicit fast-forward only from the exact --base current branch head')
|
||||||
|
a=p.parse_args()
|
||||||
|
try:
|
||||||
|
print(json.dumps(build(a.repo,a.source,a.base,a.branch,a.bundle,a.advance)));return 0
|
||||||
|
except Exception:
|
||||||
|
print(json.dumps({'state':'blocked','reason':'Inspect source/base, branch and bundle outcome before retry; no network performed'}));return 2
|
||||||
|
|
||||||
|
if __name__=='__main__':raise SystemExit(main())
|
||||||
66
scripts/node-onboarding/command-adapter.py
Normal file
66
scripts/node-onboarding/command-adapter.py
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run reviewed, pinned site-specific commands for batch.py without a shell.
|
||||||
|
|
||||||
|
Missing integration is a waiting gate, never an assumed successful operation.
|
||||||
|
Handlers receive paths to the frozen request, not passwords on the command line.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p = argparse.ArgumentParser(description=__doc__)
|
||||||
|
for field in ('plan', 'node', 'phase', 'action', 'state-dir'):
|
||||||
|
p.add_argument('--' + field, required=True)
|
||||||
|
a = p.parse_args()
|
||||||
|
node = json.loads(Path(a.node).read_text())
|
||||||
|
plan = json.loads(Path(a.plan).read_text())
|
||||||
|
identity = {'key': node['key'], 'phase': a.phase, 'request_sha256': node['request_sha256']}
|
||||||
|
try:
|
||||||
|
profile = plan['command_profile']
|
||||||
|
# Every runnable local script/config must be pinned; pinning the interpreter
|
||||||
|
# alone does not pin scripts, remote code or the commands' dependencies.
|
||||||
|
files = profile['files']
|
||||||
|
if not files:
|
||||||
|
raise ValueError('No pinned handler files')
|
||||||
|
for filename, expected in files.items():
|
||||||
|
if not Path(filename).is_absolute():
|
||||||
|
raise ValueError('Absolute handler paths required')
|
||||||
|
if hashlib.sha256(Path(filename).read_bytes()).hexdigest() != expected:
|
||||||
|
raise ValueError('Handler changed')
|
||||||
|
command = profile['phases'].get(a.phase, {}).get(a.action)
|
||||||
|
if command is None:
|
||||||
|
result = dict(identity, status='waiting', reason='handler_not_configured')
|
||||||
|
else:
|
||||||
|
if not isinstance(command, list) or not command or not all(isinstance(x, str) for x in command):
|
||||||
|
raise ValueError('Command must be argv')
|
||||||
|
values = dict(plan=a.plan, node=a.node, state_dir=a.state_dir,
|
||||||
|
key=node['key'], phase=a.phase, action=a.action)
|
||||||
|
argv = [x.format_map(values) for x in command]
|
||||||
|
if not Path(argv[0]).is_absolute():
|
||||||
|
raise ValueError('Absolute executable required')
|
||||||
|
completed = subprocess.run(argv, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||||
|
if completed.returncode or len(completed.stdout) > 65536:
|
||||||
|
raise ValueError('Handler failed')
|
||||||
|
result = json.loads(completed.stdout)
|
||||||
|
if any(result.get(k) != v for k, v in identity.items()):
|
||||||
|
raise ValueError('Wrong handler receipt')
|
||||||
|
if result.get('status') not in ('ready', 'absent', 'waiting', 'unknown', 'conflict'):
|
||||||
|
raise ValueError('Wrong handler status')
|
||||||
|
# Only structured nonsecret receipt fields are allowed through to the runner.
|
||||||
|
safe = {**identity, 'status': result['status']}
|
||||||
|
if result.get('retry_safe') is True:
|
||||||
|
safe['retry_safe'] = True
|
||||||
|
print(json.dumps(safe))
|
||||||
|
return 0
|
||||||
|
except Exception:
|
||||||
|
print(json.dumps(dict(identity, status='unknown')))
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
44
scripts/node-onboarding/configure-site.py
Normal file
44
scripts/node-onboarding/configure-site.py
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Freeze a reusable SSH-site plan locally. No network, credentials or ledger writes."""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import importlib
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from batch import validate,atomic_json
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
|
||||||
|
|
||||||
|
def configure(plan,site):
|
||||||
|
allowed={'ops_root','node_root','batch_backup_root','node_backup_root','qa_root','provider_snapshot','exit_registry','ssh_config','preserved_service','formal_socks_port','gateway_containers','client_sha256'}
|
||||||
|
if set(site)-allowed:raise ValueError('Unexpected site configuration fields')
|
||||||
|
library=importlib.import_module('paramiko')
|
||||||
|
if library.__version__!='5.0.0':raise ValueError('Use the reviewed Paramiko 5.0.0 environment')
|
||||||
|
plan=dict(plan,context={**plan['context'],**site})
|
||||||
|
validate(plan)
|
||||||
|
adapter=SiteAdapter(plan,Path('/private/tmp/dpl-validation-only')) # Creates no directory or connections.
|
||||||
|
adapter.pool.close()
|
||||||
|
base=Path(__file__).resolve().parent
|
||||||
|
plan.pop('command_profile',None)
|
||||||
|
plan['adapter']=[str(Path(sys.executable).resolve()),str(base/'command-adapter.py')]
|
||||||
|
paths=[base/x for x in ('batch.py','site_adapter.py','ssh_pool.py','acceptance.py')]+[Path(site[k]) for k in ('ssh_config','exit_registry')]
|
||||||
|
plan['runtime']={'kind':'ssh-site-v1','python':str(Path(sys.executable).resolve()),'paramiko_version':library.__version__,'files':{str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in paths}}
|
||||||
|
return plan
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p=argparse.ArgumentParser(description=__doc__)
|
||||||
|
for k in ('plan','site','output'):p.add_argument('--'+k,required=True)
|
||||||
|
a=p.parse_args()
|
||||||
|
try:
|
||||||
|
output=Path(a.output)
|
||||||
|
if not output.is_absolute() or output.exists() or output.is_symlink():raise ValueError('New absolute output file required; never mutate a frozen plan')
|
||||||
|
plan=configure(json.loads(Path(a.plan).read_text()),json.loads(Path(a.site).read_text()))
|
||||||
|
atomic_json(output,plan)
|
||||||
|
print(json.dumps({'state':'configured','nodes':len(plan['nodes']),'network':False,'ledger_writes':False}))
|
||||||
|
return 0
|
||||||
|
except Exception:
|
||||||
|
print(json.dumps({'state':'blocked','reason':'Inspect explicit site paths, registry and plan; no network performed'}));return 2
|
||||||
|
|
||||||
|
if __name__=='__main__':raise SystemExit(main())
|
||||||
7
scripts/node-onboarding/dpl-certbot-apparmor-renew.conf
Normal file
7
scripts/node-onboarding/dpl-certbot-apparmor-renew.conf
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
[Unit]
|
||||||
|
Requires=dpl-certbot-apparmor.service
|
||||||
|
After=dpl-certbot-apparmor.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
# Recheck generated profiles after a snap refresh, before the existing renewal command.
|
||||||
|
ExecStartPre=/bin/bash /opt/dpl-certbot-security/provision/load-certbot-apparmor.sh --load
|
||||||
14
scripts/node-onboarding/dpl-certbot-apparmor.service
Normal file
14
scripts/node-onboarding/dpl-certbot-apparmor.service
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Load existing Certbot AppArmor profiles for DPL certificate operations
|
||||||
|
After=local-fs.target
|
||||||
|
RequiresMountsFor=/snap/core/current /snap/certbot/current /var/lib/snapd/apparmor/profiles
|
||||||
|
Before=snap.certbot.renew.service
|
||||||
|
ConditionPathExists=/var/lib/snapd/apparmor/profiles/snap.certbot.certbot
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/bin/bash /opt/dpl-certbot-security/provision/load-certbot-apparmor.sh --load
|
||||||
|
RemainAfterExit=yes
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
30
scripts/node-onboarding/load-certbot-apparmor.sh
Normal file
30
scripts/node-onboarding/load-certbot-apparmor.sh
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -euo pipefail
|
||||||
|
# Restore only profiles already generated by the installed Certbot/core snaps.
|
||||||
|
# This does not enable the global AppArmor service or change profile modes.
|
||||||
|
[[ $(id -u) == 0 ]] || exit 1
|
||||||
|
mode=${1:---load}
|
||||||
|
[[ $# -le 1 && ( $mode == --load || $mode == --check ) ]] || exit 2
|
||||||
|
[[ $(cat /sys/module/apparmor/parameters/enabled) == Y ]] || exit 3
|
||||||
|
directory=/var/lib/snapd/apparmor/profiles
|
||||||
|
[[ -d $directory && ! -L $directory && $(stat -c %u "$directory") == 0 ]] || exit 4
|
||||||
|
[[ -z $(find "$directory" -maxdepth 0 -perm /022 -print) ]] || exit 4
|
||||||
|
[[ -f $directory/snap.certbot.certbot && -f $directory/snap.certbot.renew ]] || exit 5
|
||||||
|
shopt -s nullglob
|
||||||
|
core=$(readlink -f /snap/core/current)
|
||||||
|
[[ $core =~ ^/(var/lib/snapd/)?snap/core/([0-9]+)$ ]] || exit 5
|
||||||
|
revision=${BASH_REMATCH[2]}
|
||||||
|
profiles=("$directory"/snap.certbot.* "$directory"/snap-update-ns.certbot "$directory/snap-confine.core.$revision")
|
||||||
|
(( ${#profiles[@]} >= 4 )) || exit 5
|
||||||
|
# Validate every input before loading any profile. Never use complain/disable flags.
|
||||||
|
for profile in "${profiles[@]}"; do
|
||||||
|
[[ -f $profile && ! -L $profile && $(stat -c %u "$profile") == 0 ]] || exit 6
|
||||||
|
[[ -z $(find "$profile" -maxdepth 0 -perm /022 -print) ]] || exit 6
|
||||||
|
/usr/sbin/apparmor_parser --skip-kernel-load --skip-cache "$profile"
|
||||||
|
done
|
||||||
|
if [[ $mode == --load ]]; then
|
||||||
|
for profile in "${profiles[@]}"; do
|
||||||
|
/usr/sbin/apparmor_parser --replace --skip-cache "$profile"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
printf 'Certbot/core profiles checked: %s; mode: %s\n' "${#profiles[@]}" "$mode"
|
||||||
332
scripts/node-onboarding/operations.php
Normal file
332
scripts/node-onboarding/operations.php
Normal file
@ -0,0 +1,332 @@
|
|||||||
|
<?php
|
||||||
|
/** Shared CLI helpers; loading this file does not bootstrap Laravel or open a connection. */
|
||||||
|
namespace DplOnboarding;
|
||||||
|
|
||||||
|
use App\Models\{Server, ServerGroup, User, Plan, Setting};
|
||||||
|
use Illuminate\Support\Facades\{DB, Cache};
|
||||||
|
use RuntimeException;
|
||||||
|
|
||||||
|
function ids(string $value): array
|
||||||
|
{
|
||||||
|
if (!preg_match('/^[1-9][0-9]*(,[1-9][0-9]*)*$/D', $value)) {
|
||||||
|
throw new RuntimeException('Expected explicit comma-separated positive IDs');
|
||||||
|
}
|
||||||
|
$ids = array_map('intval', explode(',', $value));
|
||||||
|
if (count($ids) !== count(array_unique($ids))) throw new RuntimeException('Duplicate IDs');
|
||||||
|
return $ids;
|
||||||
|
}
|
||||||
|
|
||||||
|
function readJson(string $path): array
|
||||||
|
{
|
||||||
|
if (!is_file($path) || is_link($path)) throw new RuntimeException('Required regular journal/manifest missing');
|
||||||
|
$data = json_decode(file_get_contents($path), true, flags: JSON_THROW_ON_ERROR);
|
||||||
|
if (!is_array($data)) throw new RuntimeException('Invalid JSON document');
|
||||||
|
return $data;
|
||||||
|
}
|
||||||
|
|
||||||
|
function writeJson(string $path, array $data): void
|
||||||
|
{
|
||||||
|
if (is_link($path)) throw new RuntimeException('Refusing journal symlink');
|
||||||
|
$tmp = tempnam(dirname($path), '.onboarding-');
|
||||||
|
if ($tmp === false) throw new RuntimeException('Cannot create journal');
|
||||||
|
try {
|
||||||
|
chmod($tmp, 0600);
|
||||||
|
$body = json_encode($data, JSON_THROW_ON_ERROR | JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE) . "\n";
|
||||||
|
if (file_put_contents($tmp, $body) !== strlen($body) || !rename($tmp, $path)) {
|
||||||
|
throw new RuntimeException('Cannot persist journal');
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
if (is_file($tmp)) unlink($tmp);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function manifest(string $dir): array
|
||||||
|
{
|
||||||
|
$m = readJson($dir . '/manifest.json');
|
||||||
|
foreach (['node_id','machine_id','group_id','test_user_id'] as $field) {
|
||||||
|
if (!isset($m[$field]) || !preg_match('/^[1-9][0-9]*$/D', (string)$m[$field])) {
|
||||||
|
throw new RuntimeException('Manifest missing positive object ID');
|
||||||
|
}
|
||||||
|
$m[$field] = (int)$m[$field];
|
||||||
|
}
|
||||||
|
if (!isset($m['key']) || !preg_match('/^[a-z0-9-]+$/D', $m['key'])) throw new RuntimeException('Invalid manifest key');
|
||||||
|
return $m;
|
||||||
|
}
|
||||||
|
|
||||||
|
function locked(string $dir, callable $fn): mixed
|
||||||
|
{
|
||||||
|
if (!is_dir($dir) || is_link($dir) || (fileperms($dir)&0077)!==0) throw new RuntimeException('Existing private backup directory required');
|
||||||
|
$path = $dir . '/operation.lock';
|
||||||
|
if (is_link($path)) throw new RuntimeException('Refusing lock symlink');
|
||||||
|
$h = fopen($path, 'c');
|
||||||
|
if ($h === false) throw new RuntimeException('Cannot open operation lock');
|
||||||
|
chmod($path, 0600);
|
||||||
|
try {
|
||||||
|
if (!flock($h, LOCK_EX | LOCK_NB)) throw new RuntimeException('Another operation is active');
|
||||||
|
return $fn();
|
||||||
|
} finally { flock($h, LOCK_UN); fclose($h); }
|
||||||
|
}
|
||||||
|
|
||||||
|
function boot(): void
|
||||||
|
{
|
||||||
|
require getcwd() . '/vendor/autoload.php';
|
||||||
|
$app = require getcwd() . '/bootstrap/app.php';
|
||||||
|
$app->make(\Illuminate\Contracts\Console\Kernel::class)->bootstrap();
|
||||||
|
}
|
||||||
|
|
||||||
|
final class PanelChange
|
||||||
|
{
|
||||||
|
private const MODELS = ['node'=>Server::class,'qa'=>User::class,'user'=>User::class,
|
||||||
|
'plan'=>Plan::class,'group'=>ServerGroup::class,'setting'=>Setting::class];
|
||||||
|
private const FIELDS = ['node'=>['group_ids'],'qa'=>['banned'],'user'=>['group_id'],
|
||||||
|
'plan'=>['group_id'],'group'=>['name'],'setting'=>['value']];
|
||||||
|
|
||||||
|
private static function normalize(string $field, mixed $value): mixed
|
||||||
|
{
|
||||||
|
return match ($field) {
|
||||||
|
'group_ids' => array_map('strval', $value ?? []),
|
||||||
|
'group_id' => $value === null ? null : (int)$value,
|
||||||
|
'banned','enabled','show' => (bool)$value,
|
||||||
|
default => $value,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function object(string $kind, int|string $id): mixed
|
||||||
|
{
|
||||||
|
$class = self::MODELS[$kind] ?? throw new RuntimeException('Unsupported journal object');
|
||||||
|
return $class::where($kind === 'setting' ? 'name' : 'id', $id)->lockForUpdate()->first();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function value(mixed $model, string $field): mixed
|
||||||
|
{
|
||||||
|
return $model === null ? null : self::normalize($field, $model->getAttribute($field));
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function operation(string $kind, mixed $model, string $field, mixed $after, int|string|null $id = null): ?array
|
||||||
|
{
|
||||||
|
if (!in_array($field, self::FIELDS[$kind] ?? [], true)) throw new RuntimeException('Unsupported journal field');
|
||||||
|
$before = self::value($model, $field);
|
||||||
|
$after = self::normalize($field, $after);
|
||||||
|
if ($model !== null && $before === $after) return null;
|
||||||
|
return ['kind'=>$kind,'id'=>$id ?? $model->getKey(),'field'=>$field,
|
||||||
|
'before_exists'=>$model !== null,'before'=>$before,'after'=>$after];
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function context(array $m): array
|
||||||
|
{
|
||||||
|
$node = self::object('node', $m['node_id']) ?? throw new RuntimeException('Manifest node missing');
|
||||||
|
$qa = self::object('qa', $m['test_user_id']) ?? throw new RuntimeException('Manifest QA missing');
|
||||||
|
if ((int)$node->machine_id !== $m['machine_id'] || $node->type !== 'anytls') {
|
||||||
|
throw new RuntimeException('Manifest node/machine mismatch');
|
||||||
|
}
|
||||||
|
if ($qa->is_admin || (int)$qa->group_id !== $m['group_id'] ||
|
||||||
|
!str_starts_with($qa->email, 'node-qa-' . $m['key'] . '-') ||
|
||||||
|
!str_ends_with($qa->email, '@example.invalid')) {
|
||||||
|
throw new RuntimeException('Manifest does not identify an isolated QA user');
|
||||||
|
}
|
||||||
|
return [$node, $qa];
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function checkJournal(array $j, array $m): void
|
||||||
|
{
|
||||||
|
if (($j['version'] ?? null) !== 1 || ($j['manifest'] ?? null) !== $m || !is_array($j['operations'] ?? null)) {
|
||||||
|
throw new RuntimeException('Journal/manifest mismatch or unsupported version');
|
||||||
|
}
|
||||||
|
foreach ($j['operations'] as $op) {
|
||||||
|
if (!in_array($op['field'], self::FIELDS[$op['kind']] ?? [], true)) throw new RuntimeException('Invalid journal operation');
|
||||||
|
if ($op['kind']==='node' && $op['id']!==$m['node_id']) throw new RuntimeException('Wrong node in journal');
|
||||||
|
if ($op['kind']==='qa' && $op['id']!==$m['test_user_id']) throw new RuntimeException('Wrong QA in journal');
|
||||||
|
if ($op['kind']==='setting' && $op['id']!=='subscribe_url') throw new RuntimeException('Wrong setting in journal');
|
||||||
|
if ($j['request']['mode']==='append' && !in_array($op['kind'], ['node','qa'], true)) {
|
||||||
|
throw new RuntimeException('Append journal contains shared-resource writes');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function current(array $op): array
|
||||||
|
{
|
||||||
|
$model = self::object($op['kind'], $op['id']);
|
||||||
|
return [$model, $model !== null, self::value($model, $op['field'])];
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function applyOperation(array $op): void
|
||||||
|
{
|
||||||
|
[$model, $exists, $value] = self::current($op);
|
||||||
|
if ($exists !== $op['before_exists'] || $value !== $op['before']) throw new RuntimeException('Object changed before application');
|
||||||
|
if (!$exists) {
|
||||||
|
if ($op['kind'] !== 'setting') throw new RuntimeException('Missing target');
|
||||||
|
$model = new Setting(['name'=>$op['id']]);
|
||||||
|
}
|
||||||
|
$model->setAttribute($op['field'], $op['after']);
|
||||||
|
// Keep model observers for node full-sync notifications; QA does not send notifications.
|
||||||
|
$op['kind'] === 'qa' ? $model->saveQuietly() : $model->save();
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function flushSettings(array $ops): void
|
||||||
|
{
|
||||||
|
if (array_filter($ops, fn($op) => $op['kind']==='setting')) {
|
||||||
|
// Called after commit; clearing request-local state also makes retry recovery safe.
|
||||||
|
Cache::store('redis')->forget(\App\Support\Setting::CACHE_KEY);
|
||||||
|
app()->forgetInstance(\App\Support\Setting::class);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function notifyCommitted(array $m): void
|
||||||
|
{
|
||||||
|
// Explicitly repeat idempotent invalidations after commit. If an observer's
|
||||||
|
// after-commit callback failed, prepared+after recovery must still notify.
|
||||||
|
\App\Services\NodeSyncService::notifyFullSync($m['node_id']);
|
||||||
|
\App\Services\NodeSyncService::notifyMachineNodesChanged($m['machine_id']);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static function verifyEvidence(array $request, array $m, mixed $node): void
|
||||||
|
{
|
||||||
|
$report = readJson($request['verification'] ?? throw new RuntimeException('Missing --verification live QA report'));
|
||||||
|
$at = $report['verified_at'] ?? 0;
|
||||||
|
if (($report['state'] ?? '') !== 'live_verified' || ($report['qa_user_id'] ?? null) !== $m['test_user_id'] ||
|
||||||
|
!is_int($at) || $at > time() || time()-$at > 900) {
|
||||||
|
throw new RuntimeException('Recent live verification for this manifest QA required');
|
||||||
|
}
|
||||||
|
$sni = $node->protocol_settings['tls']['server_name'] ?? null;
|
||||||
|
$targets = array_filter($report['results'] ?? [], fn($row) =>
|
||||||
|
($row['live'] ?? false) === true && ($row['strict_tls'] ?? false) === true &&
|
||||||
|
($row['relay'] ?? null) === $node->host && (int)($row['relay_port'] ?? 0) === (int)$node->port &&
|
||||||
|
($row['sni'] ?? null) === $sni);
|
||||||
|
if (count($targets)!==1) throw new RuntimeException('Live evidence endpoint/SNI mismatch');
|
||||||
|
}
|
||||||
|
|
||||||
|
public static function promote(string $dir, array $request, bool $apply): array
|
||||||
|
{
|
||||||
|
$mode = $request['mode'] ?? '';
|
||||||
|
if (!in_array($mode, ['append','initialize'], true)) throw new RuntimeException('Explicit append/initialize mode required');
|
||||||
|
$request['group'] = isset($request['group']) ? (int)$request['group'] : null;
|
||||||
|
if ($mode==='append' && (!$request['group'] || isset($request['plans']) || isset($request['subscribe-url']) || isset($request['group-name']))) {
|
||||||
|
throw new RuntimeException('Append requires group and forbids plans, subscribe-url and group-name');
|
||||||
|
}
|
||||||
|
if ($mode==='initialize') {
|
||||||
|
$request['plan_ids'] = ids($request['plans'] ?? '');
|
||||||
|
$url = $request['subscribe-url'] ?? '';
|
||||||
|
$parts = parse_url($url);
|
||||||
|
if (!is_array($parts) || ($parts['scheme'] ?? '')!=='https' || empty($parts['host']) || isset($parts['user']) || isset($parts['pass']) || isset($parts['query']) || isset($parts['fragment'])) {
|
||||||
|
throw new RuntimeException('Initialize requires an explicit HTTPS subscribe-url without credentials/query');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ksort($request);
|
||||||
|
$m = manifest($dir); $path = $dir . '/promotion-journal.json';
|
||||||
|
$result = DB::transaction(function () use ($dir,$request,$mode,$m,$path,$apply) {
|
||||||
|
[$node,$qa] = self::context($m);
|
||||||
|
if (is_file($path)) {
|
||||||
|
$j = readJson($path); self::checkJournal($j,$m);
|
||||||
|
if ($j['request'] !== $request) throw new RuntimeException('Existing operation has different parameters');
|
||||||
|
if (is_file($dir.'/rollback-journal.json')) throw new RuntimeException('Rollback started; use a new reviewed task');
|
||||||
|
$before = $after = true;
|
||||||
|
foreach ($j['operations'] as $op) {
|
||||||
|
[, $exists,$value] = self::current($op);
|
||||||
|
$before = $before && $exists===$op['before_exists'] && $value===$op['before'];
|
||||||
|
$after = $after && $exists && $value===$op['after'];
|
||||||
|
}
|
||||||
|
if (!$after && (!$before || $j['state']!=='prepared')) throw new RuntimeException('Journal conflict; inspect before retrying');
|
||||||
|
if (!$apply) return ['state'=>$after?'already_applied':'preview','journal'=>$j];
|
||||||
|
if (!$after) {
|
||||||
|
self::verifyEvidence($request,$m,$node);
|
||||||
|
if (!$node->is_online || (int)$qa->u+(int)$qa->d<=0 || $qa->banned) throw new RuntimeException('QA evidence no longer ready');
|
||||||
|
foreach ($j['operations'] as $op) self::applyOperation($op);
|
||||||
|
}
|
||||||
|
return ['state'=>'applied','journal'=>$j];
|
||||||
|
}
|
||||||
|
if (is_file($dir.'/promoted.json')) throw new RuntimeException('Legacy promotion exists without new journal; do not adopt automatically');
|
||||||
|
if (!$node->enabled || !$node->show || !$node->is_online || (int)$qa->u + (int)$qa->d <= 0 || $qa->banned) throw new RuntimeException('Fresh online/active QA traffic evidence missing');
|
||||||
|
if (self::value($node,'group_ids') !== [(string)$m['group_id']]) throw new RuntimeException('Node no longer in its isolated stage group');
|
||||||
|
$groupId = $request['group'] ?? $m['group_id'];
|
||||||
|
$group = self::object('group',$groupId) ?? throw new RuntimeException('Target group missing');
|
||||||
|
if ($mode==='append' && $groupId===$m['group_id']) throw new RuntimeException('Append requires an existing formal group');
|
||||||
|
$ops = [self::operation('node',$node,'group_ids',[(string)$groupId]), self::operation('qa',$qa,'banned',true)];
|
||||||
|
if ($mode==='initialize') {
|
||||||
|
$plans = Plan::whereIn('id',$request['plan_ids'])->orderBy('id')->lockForUpdate()->get();
|
||||||
|
if ($plans->count()!==count($request['plan_ids'])) throw new RuntimeException('Plan IDs do not all exist');
|
||||||
|
foreach ($plans as $plan) {
|
||||||
|
if ($plan->group_id !== null && (int)$plan->group_id!==$groupId) throw new RuntimeException('Plan belongs to another group');
|
||||||
|
$ops[] = self::operation('plan',$plan,'group_id',$groupId);
|
||||||
|
}
|
||||||
|
// Only the explicitly selected plans' unassigned users can change in initialize mode.
|
||||||
|
foreach (User::whereIn('plan_id',$request['plan_ids'])->whereNull('group_id')->orderBy('id')->lockForUpdate()->get() as $user) {
|
||||||
|
$ops[] = self::operation('user',$user,'group_id',$groupId);
|
||||||
|
}
|
||||||
|
if (isset($request['group-name'])) $ops[] = self::operation('group',$group,'name',$request['group-name']);
|
||||||
|
$setting = self::object('setting','subscribe_url');
|
||||||
|
$ops[] = self::operation('setting',$setting,'value',$request['subscribe-url'],'subscribe_url');
|
||||||
|
}
|
||||||
|
if ($apply) self::verifyEvidence($request,$m,$node);
|
||||||
|
$j = ['version'=>1,'state'=>'prepared','manifest'=>$m,'request'=>$request,
|
||||||
|
'operations'=>array_values(array_filter($ops)),
|
||||||
|
'node_guard'=>['enabled'=>(bool)$node->enabled,'show'=>(bool)$node->show]];
|
||||||
|
if (!$apply) return ['state'=>'preview','journal'=>$j];
|
||||||
|
writeJson($path,$j); // Durable intent before any DB write; recover prepared+after on retry.
|
||||||
|
foreach ($j['operations'] as $op) self::applyOperation($op);
|
||||||
|
return ['state'=>'applied','journal'=>$j];
|
||||||
|
},1);
|
||||||
|
if ($apply) {
|
||||||
|
self::flushSettings($result['journal']['operations']);
|
||||||
|
self::notifyCommitted($m);
|
||||||
|
$result['journal']['state']='applied'; writeJson($path,$result['journal']);
|
||||||
|
}
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static function rollback(string $dir, bool $apply): array
|
||||||
|
{
|
||||||
|
$m = manifest($dir); $j = readJson($dir.'/promotion-journal.json'); self::checkJournal($j,$m);
|
||||||
|
$rollbackPath=$dir.'/rollback-journal.json';
|
||||||
|
$result=DB::transaction(function () use ($m,$j,$apply,$rollbackPath) {
|
||||||
|
[$node,$qa]=self::context($m);
|
||||||
|
if (!$qa->banned) throw new RuntimeException('QA must remain disabled during rollback');
|
||||||
|
if (is_file($rollbackPath)) {
|
||||||
|
$r=readJson($rollbackPath);
|
||||||
|
if (($r['promotion']??null)!==$j) throw new RuntimeException('Rollback journal mismatch');
|
||||||
|
} else {
|
||||||
|
// Rollback itself has a durable before/after journal, including node containment.
|
||||||
|
$undo=[];
|
||||||
|
foreach (array_reverse($j['operations']) as $op) {
|
||||||
|
if ($op['kind']==='qa') continue; // Never reactivate test users.
|
||||||
|
[$model,$exists,$value]=self::current($op);
|
||||||
|
if (!$exists || $value!==$op['after']) throw new RuntimeException('Rollback conflict: current value differs from recorded after');
|
||||||
|
$undo[]=['kind'=>$op['kind'],'id'=>$op['id'],'field'=>$op['field'],
|
||||||
|
'before_exists'=>true,'before'=>$op['after'],'after'=>$op['before'],
|
||||||
|
'delete'=>!$op['before_exists']];
|
||||||
|
}
|
||||||
|
foreach (['enabled','show'] as $field) {
|
||||||
|
if ((bool)$node->$field!==$j['node_guard'][$field]) throw new RuntimeException('Node containment field changed after promotion');
|
||||||
|
if ($node->$field) $undo[]=['kind'=>'node','id'=>$m['node_id'],'field'=>$field,
|
||||||
|
'before_exists'=>true,'before'=>true,'after'=>false,'delete'=>false];
|
||||||
|
}
|
||||||
|
if (!$qa->banned) throw new RuntimeException('QA reactivated after promotion; resolve conflict before rollback');
|
||||||
|
$r=['version'=>1,'state'=>'prepared','promotion'=>$j,'operations'=>$undo];
|
||||||
|
}
|
||||||
|
$before=$after=true;
|
||||||
|
foreach ($r['operations'] as $op) {
|
||||||
|
// Only rollback containment adds these two node fields.
|
||||||
|
if (!in_array($op['field'],self::FIELDS[$op['kind']]??[],true) && !($op['kind']==='node' && $op['id']===$m['node_id'] && in_array($op['field'],['enabled','show'],true))) {
|
||||||
|
throw new RuntimeException('Invalid rollback field');
|
||||||
|
}
|
||||||
|
[, $exists,$value]=self::current($op);
|
||||||
|
$before=$before && $exists && $value===$op['before'];
|
||||||
|
$after=$after && ($op['delete'] ? !$exists : ($exists && $value===$op['after']));
|
||||||
|
}
|
||||||
|
if (!$after && (!$before || $r['state']!=='prepared')) throw new RuntimeException('Rollback conflicts; no objects restored');
|
||||||
|
if (!$apply) return ['state'=>$after?'already_rolled_back':'preview','journal'=>$r];
|
||||||
|
if (!is_file($rollbackPath)) writeJson($rollbackPath,$r);
|
||||||
|
if (!$after) foreach ($r['operations'] as $op) {
|
||||||
|
[$model]=self::current($op);
|
||||||
|
if ($op['delete']) $model->delete();
|
||||||
|
else { $model->setAttribute($op['field'],$op['after']); $model->save(); }
|
||||||
|
}
|
||||||
|
return ['state'=>'rolled_back','journal'=>$r];
|
||||||
|
},1);
|
||||||
|
if ($apply) {
|
||||||
|
self::flushSettings($j['operations']);
|
||||||
|
self::notifyCommitted($m);
|
||||||
|
$result['journal']['state']='rolled_back';writeJson($rollbackPath,$result['journal']);
|
||||||
|
}
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
}
|
||||||
100
scripts/node-onboarding/panel-acceptance.php
Normal file
100
scripts/node-onboarding/panel-acceptance.php
Normal file
@ -0,0 +1,100 @@
|
|||||||
|
<?php
|
||||||
|
/** On-demand batch acceptance lease. No bootstrap occurs when included by tests. */
|
||||||
|
namespace DplOnboarding;
|
||||||
|
require_once __DIR__.'/operations.php';
|
||||||
|
use App\Models\{User,Server,ServerGroup};
|
||||||
|
use App\Services\ServerService;
|
||||||
|
use Illuminate\Support\Facades\DB;
|
||||||
|
use Illuminate\Support\Str;
|
||||||
|
use RuntimeException;
|
||||||
|
|
||||||
|
final class Acceptance {
|
||||||
|
public static function eligible(User $u,int $group): bool {
|
||||||
|
return !$u->banned && (int)$u->group_id===$group && (int)$u->u+(int)$u->d<(int)$u->transfer_enable && ($u->expired_at===null || (int)$u->expired_at>=time());
|
||||||
|
}
|
||||||
|
public static function prepare(string $dir,int $group,bool $apply): array {
|
||||||
|
return locked($dir,function()use($dir,$group,$apply){
|
||||||
|
if(!ServerGroup::find($group))throw new RuntimeException('Formal group missing');
|
||||||
|
$path=$dir.'/acceptance-user.json';
|
||||||
|
if(is_file($path) && readJson($path)['state']==='closed' && $apply){
|
||||||
|
$old=readJson($path);$prior=isset($old['id'])?User::find($old['id']):null;
|
||||||
|
if($old['owned'] && $prior && (!$prior->banned || $prior->email!==$old['email'] || $prior->uuid!==$old['uuid'] || $prior->token!==$old['token'] || $prior->is_admin || (int)$prior->group_id!==$old['group']))throw new RuntimeException('Prior lease not revoked');
|
||||||
|
writeJson($dir.'/acceptance-closed-'.($old['id']??'absent').'.json',$old);unlink($path);
|
||||||
|
}
|
||||||
|
if(is_file($path)){
|
||||||
|
$j=readJson($path);
|
||||||
|
if($j['group']!==$group || $j['state']==='closed')throw new RuntimeException('Lease closed or parameters changed; never reactivate');
|
||||||
|
$u=User::where('email',$j['email'])->first();
|
||||||
|
if(!$u && $j['owned'] && $j['state']==='prepared' && $apply){
|
||||||
|
$u=DB::transaction(fn()=>self::create($j),1);
|
||||||
|
}
|
||||||
|
if(!$u || $u->uuid!==$j['uuid'] || $u->token!==$j['token'] || !self::eligible($u,$group) || ($j['owned'] && $u->is_admin))throw new RuntimeException('Acceptance lease drifted');
|
||||||
|
if($j['owned'])self::limits($u,$j);
|
||||||
|
$j['id']=$u->id;$j['state']='active';if($apply)writeJson($path,$j);
|
||||||
|
return ['state'=>'ready','id'=>$u->id,'owned'=>$j['owned']];
|
||||||
|
}
|
||||||
|
$u=null;
|
||||||
|
$candidates=User::where('group_id',$group)->where('banned',0)->whereRaw('u+d<transfer_enable')->where(fn($q)=>$q->whereNull('expired_at')->orWhere('expired_at','>=',time()))->where('email','not like','node-qa-%')->where('email','not like','batch-acceptance-%')->orderByDesc('is_admin')->orderBy('id')->limit(20)->get();
|
||||||
|
foreach($candidates as $candidate){
|
||||||
|
$effective=true;
|
||||||
|
foreach(ServerService::getAvailableServers($candidate) as $s)if(!ServerService::getAvailableUsers(Server::findOrFail($s['id']))->contains('id',$candidate->id)){$effective=false;break;}
|
||||||
|
if($effective){$u=$candidate;break;}
|
||||||
|
}
|
||||||
|
if(!$apply)return ['state'=>$u?'existing':'would_create'];
|
||||||
|
$j=['group'=>$group,'state'=>'prepared','owned'=>!$u,'email'=>$u?->email??'batch-acceptance-'.substr(hash('sha256',$dir),0,12).'-'.Str::lower(Str::random(8)).'@example.invalid','uuid'=>$u?->uuid??(string)Str::uuid(),'token'=>$u?->token??Str::random(32),'quota'=>104857600,'expires_at'=>time()+3600];
|
||||||
|
// Persist identity before commit. A lost response resumes the same row, never another user.
|
||||||
|
if(!$u && User::where('email',$j['email'])->exists())throw new RuntimeException('Unowned acceptance email conflict');
|
||||||
|
writeJson($path,$j);
|
||||||
|
if(!$u)$u=DB::transaction(fn()=>self::create($j),1);
|
||||||
|
$j['id']=$u->id;$j['state']='active';writeJson($path,$j);
|
||||||
|
return ['state'=>'ready','id'=>$u->id,'owned'=>$j['owned']];
|
||||||
|
});
|
||||||
|
}
|
||||||
|
private static function limits(User $u,array $j):void {
|
||||||
|
if((int)$u->transfer_enable!==$j['quota'] || (int)$u->expired_at!==$j['expires_at'] || (int)$u->device_limit!==1)throw new RuntimeException('Owned lease limits changed; inspect before mutation');
|
||||||
|
}
|
||||||
|
private static function create(array $j): User {
|
||||||
|
$u=new User(['email'=>$j['email'],'uuid'=>$j['uuid'],'token'=>$j['token'],'password'=>password_hash(Str::random(40),PASSWORD_BCRYPT),'group_id'=>$j['group'],'transfer_enable'=>$j['quota'],'expired_at'=>$j['expires_at'],'banned'=>0,'is_admin'=>0,'remind_expire'=>0,'remind_traffic'=>0,'device_limit'=>1]);$u->saveQuietly();return $u;
|
||||||
|
}
|
||||||
|
public static function source(string $dir,int $group,bool $inspect=false): array {
|
||||||
|
$j=readJson($dir.'/acceptance-user.json');$u=User::findOrFail($j['id']);
|
||||||
|
if($j['group']!==$group || (!$inspect && $j['state']!=='active') || $u->uuid!==$j['uuid'] || $u->token!==$j['token'] || (!$inspect && !self::eligible($u,$group)) || ($inspect && !$j['owned'] && !self::eligible($u,$group)) || (int)$u->group_id!==$group)throw new RuntimeException('Acceptance user unavailable');
|
||||||
|
if($j['owned'])self::limits($u,$j);
|
||||||
|
$nodes=[];
|
||||||
|
foreach(ServerService::getAvailableServers($u) as $s){
|
||||||
|
if(isset($s['ports']))throw new RuntimeException('Port-range nodes require an explicit verifier');
|
||||||
|
$model=Server::findOrFail($s['id']);
|
||||||
|
if(!$inspect && !ServerService::getAvailableUsers($model)->contains('id',$u->id))throw new RuntimeException('User excluded by effective node filter');
|
||||||
|
$nodes[]=['id'=>$s['id'],'type'=>$s['type'],'tag'=>$s['name'],'relay'=>$s['host'],'relay_port'=>(int)$s['port'],'sni'=>$s['protocol_settings']['tls']['server_name']??null,'uuid'=>$s['password']];
|
||||||
|
}
|
||||||
|
if($inspect){foreach($nodes as &$n)unset($n['uuid']);unset($n);return ['nodes'=>$nodes];}
|
||||||
|
return ['user_id'=>$u->id,'owned'=>$j['owned'],'token'=>$u->token,'nodes'=>$nodes];
|
||||||
|
}
|
||||||
|
public static function cleanup(string $dir,bool $apply): array {
|
||||||
|
return locked($dir,function()use($dir,$apply){
|
||||||
|
$p=$dir.'/acceptance-user.json';if(!is_file($p))return ['state'=>'absent'];$j=readJson($p);
|
||||||
|
$u=isset($j['id'])?User::find($j['id']):User::where('email',$j['email'])->first();
|
||||||
|
if($j['owned'] && $u){
|
||||||
|
if($u->email!==$j['email'] || $u->uuid!==$j['uuid'] || $u->token!==$j['token'] || $u->is_admin || (int)$u->group_id!==$j['group'])throw new RuntimeException('Cleanup ownership mismatch');
|
||||||
|
self::limits($u,$j);
|
||||||
|
if($apply){$u->banned=true;$u->saveQuietly();}
|
||||||
|
if($apply && !$u->fresh()->banned)throw new RuntimeException('Cleanup failed');
|
||||||
|
}
|
||||||
|
if($apply){$j['state']='closed';$j['id']=$u?->id;writeJson($p,$j);}
|
||||||
|
return ['state'=>$apply?'closed':'preview','owned'=>$j['owned'],'id'=>$u?->id];
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if(realpath($_SERVER['SCRIPT_FILENAME']??'')===__FILE__){
|
||||||
|
$o=getopt('',['action:','dir:','group:','apply','help']);
|
||||||
|
if(isset($o['help'])){echo "--action=prepare|source|snapshot|cleanup --dir=PRIVATE_DIR --group=EXISTING_ID [--apply]\n";exit;}
|
||||||
|
try{
|
||||||
|
$dir=$o['dir']??'';$group=(int)($o['group']??0);
|
||||||
|
if(!preg_match('~^/[A-Za-z0-9_./-]+$~D',$dir)||str_contains($dir,'..')||$group<1)throw new RuntimeException('Explicit private directory and group required');
|
||||||
|
umask(0077);if(!is_dir($dir))mkdir($dir,0700,true);
|
||||||
|
if(is_link($dir)||(fileperms($dir)&0077))throw new RuntimeException('Private directory required');
|
||||||
|
boot();$apply=isset($o['apply']);
|
||||||
|
$r=match($o['action']??''){'prepare'=>Acceptance::prepare($dir,$group,$apply),'source'=>Acceptance::source($dir,$group),'snapshot'=>Acceptance::source($dir,$group,true),'cleanup'=>Acceptance::cleanup($dir,$apply),default=>throw new RuntimeException('Unknown action')};
|
||||||
|
echo json_encode($r,JSON_THROW_ON_ERROR).PHP_EOL;
|
||||||
|
}catch(\Throwable $e){fwrite(STDERR,json_encode(['state'=>'blocked','reason'=>'Inspect protected lease and effective authorization; no automatic overwrite']).PHP_EOL);exit(2);}
|
||||||
|
}
|
||||||
@ -1,43 +1,25 @@
|
|||||||
<?php
|
<?php
|
||||||
// Run only after real relay, subscription, certificate and traffic checks pass.
|
// Default is preview. Apply only in a separately authorized target environment.
|
||||||
require getcwd() . '/vendor/autoload.php';
|
require __DIR__ . '/operations.php';
|
||||||
$app = require getcwd() . '/bootstrap/app.php';
|
use DplOnboarding\PanelChange;
|
||||||
$app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();
|
$opts = getopt('', ['backup:', 'mode:', 'plans:', 'group:', 'subscribe-url:', 'group-name:', 'verification:', 'apply', 'help']);
|
||||||
use App\Models\Plan;
|
if (isset($opts['help'])) {
|
||||||
use App\Models\Server;
|
echo "--backup=DIR --mode=append --group=ID [--verification=QA_REPORT --apply]\n";
|
||||||
use App\Models\ServerGroup;
|
echo "--backup=DIR --mode=initialize --plans=IDS --subscribe-url=HTTPS_URL [--group=ID] [--group-name=NAME] [--verification=QA_REPORT --apply]\n";
|
||||||
use App\Models\User;
|
exit;
|
||||||
use Illuminate\Support\Facades\DB;
|
}
|
||||||
$opts = getopt('', ['backup:', 'plans:', 'group:']);
|
try {
|
||||||
$dir = $opts['backup'] ?? throw new RuntimeException('Missing backup');
|
$dir = $opts['backup'] ?? throw new RuntimeException('Missing --backup');
|
||||||
$ids = array_map('intval', explode(',', $opts['plans'] ?? ''));
|
$apply = array_key_exists('apply', $opts);
|
||||||
if (!$ids || in_array(0, $ids, true)) throw new RuntimeException('Explicit plan IDs required');
|
unset($opts['backup'], $opts['apply']);
|
||||||
$m = json_decode(file_get_contents("$dir/manifest.json"), true, flags: JSON_THROW_ON_ERROR);
|
if (!isset($opts['mode']) || !in_array($opts['mode'], ['append','initialize'], true)) throw new RuntimeException('Explicit --mode required');
|
||||||
$groupId = (int)($opts['group'] ?? $m['group_id']);
|
if (isset($opts['group'])) { DplOnboarding\ids($opts['group']); if (str_contains($opts['group'],',')) throw new RuntimeException('One group required'); }
|
||||||
$node = Server::findOrFail($m['node_id']);
|
DplOnboarding\boot();
|
||||||
$qa = User::findOrFail($m['test_user_id']);
|
$result = DplOnboarding\locked($dir, fn() => PanelChange::promote($dir, $opts, $apply));
|
||||||
if (!$node->is_online || $qa->u + $qa->d <= 0) throw new RuntimeException('Online and traffic evidence missing');
|
// Journal is field-scoped and contains no passwords/UUIDs/tokens.
|
||||||
if (is_file("$dir/promoted.json")) { echo "Already promoted; verify current state separately\n"; exit; }
|
echo json_encode($result, JSON_THROW_ON_ERROR | JSON_UNESCAPED_UNICODE) . PHP_EOL;
|
||||||
$plans = Plan::whereIn('id', $ids)->get();
|
} catch (Throwable $e) {
|
||||||
if ($plans->count() !== count($ids)) throw new RuntimeException('Plan list mismatch');
|
$reason=get_class($e)===RuntimeException::class ? $e->getMessage() : "Inspect protected journal/current state";
|
||||||
foreach ($plans as $p) if ($p->group_id !== null && $p->group_id !== $groupId) throw new RuntimeException('Plan already assigned elsewhere');
|
fwrite(STDERR, "Promotion stopped: " . $reason . "\n");
|
||||||
$users = User::whereIn('plan_id', $ids)->whereNull('group_id')->get(['id','group_id','updated_at']);
|
exit(1);
|
||||||
umask(0077);
|
}
|
||||||
file_put_contents("$dir/before-promotion.json", json_encode([
|
|
||||||
'plans' => $plans->map->getAttributes(), 'user_groups' => $users,
|
|
||||||
'node_groups' => $node->group_ids, 'group' => ServerGroup::findOrFail($groupId)->getAttributes(),
|
|
||||||
'subscribe_url' => admin_setting('subscribe_url'), 'qa_banned' => $qa->banned,
|
|
||||||
], JSON_THROW_ON_ERROR));
|
|
||||||
DB::transaction(function () use ($ids, $users, $node, $qa, $groupId) {
|
|
||||||
$group = ServerGroup::findOrFail($groupId);
|
|
||||||
if (str_starts_with($group->name, 'dpl 接入验收 ')) { $group->name = '基础节点'; $group->save(); }
|
|
||||||
Plan::whereIn('id', $ids)->whereNull('group_id')->update(['group_id' => $groupId]);
|
|
||||||
User::whereIn('id', $users->pluck('id'))->whereNull('group_id')->update(['group_id' => $groupId]);
|
|
||||||
$node->group_ids = [(string)$groupId]; $node->save();
|
|
||||||
$qa->banned = 1; $qa->saveQuietly();
|
|
||||||
admin_setting(['subscribe_url' => 'https://c2.deeplink-subscribe.com']);
|
|
||||||
});
|
|
||||||
$result = ['node_id' => $node->id, 'group_id' => $groupId, 'plan_ids' => $ids,
|
|
||||||
'existing_users_assigned' => $users->count(), 'qa_disabled' => true];
|
|
||||||
file_put_contents("$dir/promoted.json", json_encode($result, JSON_THROW_ON_ERROR));
|
|
||||||
echo json_encode($result) . PHP_EOL;
|
|
||||||
|
|||||||
@ -1,16 +1,32 @@
|
|||||||
<?php
|
<?php
|
||||||
require getcwd() . '/vendor/autoload.php';
|
require __DIR__ . '/operations.php';
|
||||||
$app = require getcwd() . '/bootstrap/app.php';
|
$opts=getopt('', ['backup:', 'plans:', 'group:', 'help']);
|
||||||
$app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();
|
if (isset($opts['help'])) { echo "--backup=DIR [--plans=ID,ID] [--group=ID]; no implicit plan IDs\n"; exit; }
|
||||||
$opts = getopt('', ['backup:']);
|
try {
|
||||||
$dir = $opts['backup'] ?? throw new RuntimeException('Missing backup');
|
$dir=$opts['backup'] ?? throw new RuntimeException('Missing --backup');
|
||||||
$m = json_decode(file_get_contents("$dir/manifest.json"), true, flags: JSON_THROW_ON_ERROR);
|
$ids=isset($opts['plans']) ? DplOnboarding\ids($opts['plans']) : [];
|
||||||
$node = App\Models\Server::findOrFail($m['node_id']);
|
$group=null;
|
||||||
$qa = App\Models\User::findOrFail($m['test_user_id']);
|
if (isset($opts['group'])) {
|
||||||
echo json_encode([
|
$groups=DplOnboarding\ids($opts['group']);
|
||||||
'node_id' => $node->id, 'name' => $node->name, 'is_online' => $node->is_online,
|
if (count($groups)!==1) throw new RuntimeException('One group required');
|
||||||
'last_check_at' => $node->last_check_at, 'last_push_at' => $node->last_push_at,
|
$group=$groups[0];
|
||||||
'qa_upload' => $qa->u, 'qa_download' => $qa->d, 'qa_banned' => $qa->banned,
|
}
|
||||||
'plans' => App\Models\Plan::orderBy('id')->get(['id', 'group_id']),
|
$m=DplOnboarding\manifest($dir);
|
||||||
'existing_users_with_selected_plan_and_null_group' => App\Models\User::whereIn('plan_id', [1,6,7,8])->whereNull('group_id')->count(),
|
DplOnboarding\boot();
|
||||||
], JSON_UNESCAPED_UNICODE) . PHP_EOL;
|
$node=App\Models\Server::findOrFail($m['node_id']);
|
||||||
|
$qa=App\Models\User::findOrFail($m['test_user_id']);
|
||||||
|
if ((int)$node->machine_id!==$m['machine_id']) throw new RuntimeException('Manifest machine mismatch');
|
||||||
|
$plans=App\Models\Plan::whereIn('id',$ids)->orderBy('id')->get(['id','group_id']);
|
||||||
|
if ($plans->count()!==count($ids)) throw new RuntimeException('Some selected plans missing');
|
||||||
|
if ($group!==null) App\Models\ServerGroup::findOrFail($group);
|
||||||
|
$result=[
|
||||||
|
'node_id'=>$node->id, 'name'=>$node->name, 'is_online'=>(bool)$node->is_online,
|
||||||
|
'last_check_at'=>$node->last_check_at, 'last_push_at'=>$node->last_push_at,
|
||||||
|
'group_ids'=>$node->group_ids, 'qa_upload'=>(int)$qa->u, 'qa_download'=>(int)$qa->d,
|
||||||
|
'qa_banned'=>(bool)$qa->banned, 'selected_plan_ids'=>$ids, 'plans'=>$plans->toArray(),
|
||||||
|
'existing_users_with_selected_plan_and_null_group'=>$ids ? App\Models\User::whereIn('plan_id',$ids)->whereNull('group_id')->count() : null,
|
||||||
|
'selected_plans_in_other_group'=>$group===null ? null : $plans->filter(fn($p) => $p->group_id!==null && (int)$p->group_id!==$group)->pluck('id')->all(),
|
||||||
|
'scope'=>'Read-only snapshot; not subscription/traffic/GUI acceptance',
|
||||||
|
];
|
||||||
|
echo json_encode($result,JSON_THROW_ON_ERROR|JSON_UNESCAPED_UNICODE).PHP_EOL;
|
||||||
|
} catch (Throwable $e) { fwrite(STDERR,"Readiness failed; verify parameters and manifest.\n"); exit(1); }
|
||||||
|
|||||||
1
scripts/node-onboarding/requirements.txt
Normal file
1
scripts/node-onboarding/requirements.txt
Normal file
@ -0,0 +1 @@
|
|||||||
|
paramiko==5.0.0
|
||||||
16
scripts/node-onboarding/rollback.php
Normal file
16
scripts/node-onboarding/rollback.php
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
<?php
|
||||||
|
// Field-scoped compare-before-restore. Preview unless explicitly --apply.
|
||||||
|
require __DIR__ . '/operations.php';
|
||||||
|
use DplOnboarding\PanelChange;
|
||||||
|
$opts=getopt('', ['backup:', 'apply', 'help']);
|
||||||
|
if (isset($opts['help'])) { echo "--backup=DIR [--apply]; restores new-format promotion journal only\n"; exit; }
|
||||||
|
try {
|
||||||
|
$dir=$opts['backup'] ?? throw new RuntimeException('Missing --backup');
|
||||||
|
DplOnboarding\boot();
|
||||||
|
$result=DplOnboarding\locked($dir, fn() => PanelChange::rollback($dir, array_key_exists('apply',$opts)));
|
||||||
|
echo json_encode($result, JSON_THROW_ON_ERROR | JSON_UNESCAPED_UNICODE) . PHP_EOL;
|
||||||
|
} catch (Throwable $e) {
|
||||||
|
$reason=get_class($e)===RuntimeException::class ? $e->getMessage() : "Inspect journal/conflicts";
|
||||||
|
fwrite(STDERR, "Rollback stopped: " . $reason . "\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
440
scripts/node-onboarding/site_adapter.py
Normal file
440
scripts/node-onboarding/site_adapter.py
Normal file
@ -0,0 +1,440 @@
|
|||||||
|
"""Reusable AnyTLS site adapter; invoked in-process by batch.py, never a daemon."""
|
||||||
|
import base64,hashlib,json,os,shlex,threading,time
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
from ssh_pool import Pool,private_json
|
||||||
|
from acceptance import audit,digest,snapshot,service_change,provider_state
|
||||||
|
|
||||||
|
class SiteAdapter:
|
||||||
|
def __init__(self,plan,directory,pool=None,atomic_writer=None,resource_lock=None):
|
||||||
|
if atomic_writer is None or resource_lock is None:
|
||||||
|
from batch import atomic_json,lock
|
||||||
|
atomic_writer,resource_lock=atomic_writer or atomic_json,resource_lock or lock
|
||||||
|
self.atomic_json,self.resource_lock=atomic_writer,resource_lock
|
||||||
|
self.plan,self.directory=plan,Path(directory)
|
||||||
|
self.ctx=plan['context'];self.lock=threading.RLock();self.leases={};self.formal_done={}
|
||||||
|
for field in ('ops_root','node_root','batch_backup_root','node_backup_root','qa_root','provider_snapshot','exit_registry','ssh_config','client_binary'):
|
||||||
|
value=self.ctx.get(field,'')
|
||||||
|
if not isinstance(value,str) or not value.startswith('/') or '..' in value.split('/') or any(c not in 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_./-' for c in value):
|
||||||
|
raise ValueError('Explicit safe absolute site path required: '+field)
|
||||||
|
config=private_json(self.ctx['ssh_config'])
|
||||||
|
for candidate in plan['nodes']:
|
||||||
|
if config['hosts'].get(candidate['vps_id'],{}).get('host')!=candidate['node_address']:raise ValueError('SSH target differs from frozen candidate address')
|
||||||
|
for role in ('jump_vps','panel_vps','gateway_vps'):
|
||||||
|
if self.ctx[role] not in config['hosts']:raise ValueError('Missing registered SSH role')
|
||||||
|
self.pool=pool or Pool(config,self.ctx['jump_vps'])
|
||||||
|
if not __import__('re').fullmatch('[0-9a-f]{64}',self.ctx.get('client_sha256','')):raise ValueError('Reviewed official client SHA256 required')
|
||||||
|
self.exit_registry=private_json(self.ctx['exit_registry'])
|
||||||
|
for candidate in plan['nodes']:
|
||||||
|
if self.exit_registry.get(candidate['tls_domain'])!={'expected_exit':candidate['expected_exit'],'country':candidate['country']}:raise ValueError('Candidate exit registry missing or changed')
|
||||||
|
if any(n['vps_id'] in [self.ctx['jump_vps'],self.ctx['panel_vps'],self.ctx['gateway_vps']] for n in plan['nodes']):raise ValueError('Dedicated node host required')
|
||||||
|
for key in ('app_root',):
|
||||||
|
if not __import__('re').fullmatch('/[A-Za-z0-9_./-]+',self.ctx[key]) or '..' in self.ctx[key].split('/'):raise ValueError('Unsafe application root')
|
||||||
|
if type(self.ctx.get('formal_socks_port')) is not int or not 1024<=self.ctx['formal_socks_port']<=65535 or self.ctx['formal_socks_port'] in [n['socks_port'] for n in plan['nodes']]:raise ValueError('Reserve a distinct formal SOCKS port')
|
||||||
|
if not self.ctx.get('gateway_containers') or any(not __import__('re').fullmatch('[A-Za-z0-9_.-]+',v) for v in self.ctx['gateway_containers']):raise ValueError('Explicit gateway container inventory required')
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
# On waiting, failure and success, close owned formal leases before closing SSH.
|
||||||
|
failed=False
|
||||||
|
for group in list(self.leases):
|
||||||
|
try:self.account_command(group,'cleanup',True)
|
||||||
|
except Exception:failed=True
|
||||||
|
self.pool.close()
|
||||||
|
if failed:
|
||||||
|
self.atomic_json(self.directory/'cleanup-error.json',{'groups':list(self.leases),'reason':'cleanup_outcome_unknown','needs_probe':True})
|
||||||
|
raise RuntimeError('Acceptance cleanup incomplete; inspect protected leases before retry')
|
||||||
|
|
||||||
|
def account_command(self,group,action,apply=False,client=None):
|
||||||
|
panel=client or self.pool.get(self.ctx['panel_vps'])
|
||||||
|
directory=self.ctx['batch_backup_root']+'/acceptance-group-'+str(group)
|
||||||
|
cmd='cd '+self.ctx['app_root']+' && php '+self.ctx['ops_root']+'/panel-acceptance.php --action='+action+' --dir='+directory+' --group='+str(group)+(' --apply' if apply else '')
|
||||||
|
_,out,err=panel.exec_command(cmd,timeout=90);data=out.read();err.read()
|
||||||
|
if out.channel.recv_exit_status():raise RuntimeError('acceptance_account_blocked')
|
||||||
|
try:return json.loads(data)
|
||||||
|
except ValueError:raise RuntimeError('acceptance_account_response_not_json') from None
|
||||||
|
|
||||||
|
def prepare_account(self,n,panel,execute):
|
||||||
|
with self.lock:
|
||||||
|
group=n['formal_group_id']
|
||||||
|
for spec in self.plan['nodes']:
|
||||||
|
registered=self.exit_registry.get(spec['tls_domain'],{})
|
||||||
|
if registered != {'expected_exit':spec['expected_exit'],'country':spec['country']}:
|
||||||
|
raise ValueError('Exit registry must independently cover the candidate plan')
|
||||||
|
if group not in self.leases:
|
||||||
|
# Register cleanup obligation before any possibly successful remote create.
|
||||||
|
self.leases[group]=True
|
||||||
|
self.account_command(group,'prepare',True,panel)
|
||||||
|
source=self.account_command(group,'source',client=panel)
|
||||||
|
for spec in source['nodes']:
|
||||||
|
if spec['sni'] not in self.exit_registry:raise ValueError('Missing old-node exit expectation')
|
||||||
|
|
||||||
|
def certificate_gate(self,n,node,read,execute):
|
||||||
|
owner=read(node,'/etc/dpl-node/batch-owner.json')
|
||||||
|
owned=bool(owner and json.loads(owner).get('request_sha256')==n['request_sha256'])
|
||||||
|
script=r"""from pathlib import Path
|
||||||
|
import re
|
||||||
|
for name in ['/etc/letsencrypt/cli.ini','/root/.config/letsencrypt/cli.ini']:
|
||||||
|
p=Path(name)
|
||||||
|
if p.exists():
|
||||||
|
for line in p.read_text().splitlines():
|
||||||
|
if re.match(r'^\s*(pre[-_]hook|post[-_]hook|deploy[-_]hook)\s*=\s*\S',line):raise SystemExit(2)
|
||||||
|
domain=DOMAIN
|
||||||
|
paths=[Path('/etc/letsencrypt')/kind/domain for kind in ['live','archive']]+[Path('/etc/letsencrypt/renewal')/(domain+'.conf')]
|
||||||
|
if not OWNED and any(p.exists() or p.is_symlink() for p in paths):raise SystemExit(3)
|
||||||
|
p=paths[-1]
|
||||||
|
if p.exists():
|
||||||
|
for line in p.read_text().splitlines():
|
||||||
|
if re.match(r'^\s*(pre_hook|post_hook)\s*=\s*\S',line):raise SystemExit(4)
|
||||||
|
if re.match(r'^\s*deploy_hook\s*=',line) and line.split('=',1)[1].strip()!=HOOK:raise SystemExit(5)
|
||||||
|
"""
|
||||||
|
prefix='DOMAIN='+repr(n['tls_domain'])+';OWNED='+repr(owned)+';HOOK='+repr('/bin/sh '+self.ctx['node_root']+'/renew-node.sh')+'\n'
|
||||||
|
execute(node,"python3 - <<'DPLCERTGATE'\n"+prefix+script+"\nDPLCERTGATE")
|
||||||
|
execute(node,'command -v git >/dev/null && command -v curl >/dev/null && command -v certbot >/dev/null && command -v ss >/dev/null && command -v sha256sum >/dev/null && command -v openssl >/dev/null')
|
||||||
|
|
||||||
|
def guard_service(self,node,backup,read,write,execute):
|
||||||
|
spec=self.ctx['preserved_service']
|
||||||
|
for field in ('unit','binary','config','port'):
|
||||||
|
if field not in spec:raise ValueError('Explicit preserved service specification required')
|
||||||
|
if not all(isinstance(spec[k],str) and __import__('re').fullmatch('[A-Za-z0-9_./@-]+',spec[k]) for k in ('unit','binary','config')) or type(spec['port']) is not int:
|
||||||
|
raise ValueError('Unsafe preserved service specification')
|
||||||
|
script="""import hashlib,json,pathlib,subprocess
|
||||||
|
s=SPEC
|
||||||
|
v=dict(x.split('=',1) for x in subprocess.check_output(['systemctl','show',s['unit'],'-p','MainPID','-p','ActiveEnterTimestamp','-p','NRestarts','-p','ActiveState','-p','FragmentPath'],text=True).splitlines())
|
||||||
|
def h(p):return hashlib.sha256(pathlib.Path(p).read_bytes()).hexdigest()
|
||||||
|
listeners=subprocess.check_output(['ss','-lntH','( sport = :'+str(s['port'])+' )'],text=True)
|
||||||
|
print(json.dumps({'unit_sha256':h(v['FragmentPath']),'binary_sha256':h(s['binary']),'config_sha256':h(s['config']),'listeners':sorted(x.split()[3] for x in listeners.splitlines()),'pid':v['MainPID'],'started':v['ActiveEnterTimestamp'],'restarts':v['NRestarts'],'active':v['ActiveState']=='active'}))
|
||||||
|
""".replace('SPEC',repr(spec))
|
||||||
|
current=json.loads(execute(node,"python3 - <<'DPLSERVICE'\n"+script+"\nDPLSERVICE"))
|
||||||
|
if not current['active'] or not current['listeners']:raise RuntimeError('preserved_service_unavailable')
|
||||||
|
execute(node,'install -d -m 700 '+backup)
|
||||||
|
p=backup+'/preserved-service-original.json';raw=read(node,p)
|
||||||
|
if raw is None:write(node,p,json.dumps(current));return current
|
||||||
|
before=json.loads(raw);status=service_change(before,current)
|
||||||
|
if status=='investigate_restart':
|
||||||
|
write(node,backup+'/preserved-service-observed.json',json.dumps(current))
|
||||||
|
review=read(node,backup+'/preserved-service-review.json')
|
||||||
|
review=json.loads(review) if review else {}
|
||||||
|
if review.get('before_sha256')!=digest(before) or review.get('after_sha256')!=digest(current) or not review.get('reason') or not review.get('evidence_sha256'):
|
||||||
|
raise RuntimeError('investigate_preserved_service_restart')
|
||||||
|
elif status!='ready':raise RuntimeError('preserved_service_'+status)
|
||||||
|
return current
|
||||||
|
|
||||||
|
def dependencies(self,n,panel,gateway,node,stage_probe,read,execute,gateway_py):
|
||||||
|
"""Current dependency reads; a missing observation cannot authorize a skip."""
|
||||||
|
state=stage_probe(panel);deps={'stage':digest(state)}
|
||||||
|
ui=provider_state(private_json(self.ctx['provider_snapshot']),n)
|
||||||
|
try:
|
||||||
|
dns=json.loads(gateway_py(gateway,'import socket,json\nprint(json.dumps(sorted(set(x[4][0] for x in socket.getaddrinfo('+repr(n['tls_domain'])+',443,socket.AF_INET)))))'))
|
||||||
|
if dns==[n['node_address']]:deps['dns']=digest(dns)
|
||||||
|
except Exception:pass
|
||||||
|
try:
|
||||||
|
live=execute(node,'systemctl is-active dpl-node; git -C '+self.ctx['node_root']+' rev-parse HEAD; sha256sum /etc/dpl-node/config.yml /etc/dpl-node/secret.env /etc/systemd/system/dpl-node.service /etc/letsencrypt/live/'+n['tls_domain']+'/fullchain.pem')
|
||||||
|
deps['provision']=digest(live)
|
||||||
|
# End-to-end strict TLS at preflight is necessary to reuse a relay observation.
|
||||||
|
tls=gateway_py(gateway,'import ssl,socket\nc=ssl.create_default_context()\nwith socket.create_connection(('+repr(n['relay_host'])+','+str(n['relay_port'])+'),10) as s:\n with c.wrap_socket(s,server_hostname='+repr(n['tls_domain'])+') as t:print(t.version())')
|
||||||
|
rows=ui['relays']
|
||||||
|
deps['relay']=digest([rows,tls])
|
||||||
|
report=read(panel,self.ctx['node_backup_root']+'/'+n['key']+'/qa-report.json')
|
||||||
|
if report:deps['verify']=digest([state,live,hashlib.sha256(report).hexdigest()])
|
||||||
|
except Exception:pass
|
||||||
|
return deps
|
||||||
|
|
||||||
|
def formal(self,n,panel,gateway,php,execute,read,write):
|
||||||
|
group=n['formal_group_id'];ctx=self.ctx
|
||||||
|
with self.lock, self.resource_lock(self.directory.parent, ctx['gateway_vps']+':formal:'+str(ctx['formal_socks_port'])):
|
||||||
|
source=self.account_command(group,'snapshot' if group in self.formal_done else 'source',client=panel)
|
||||||
|
def public(source):
|
||||||
|
values=[]
|
||||||
|
for item in source['nodes']:
|
||||||
|
if item['sni'] not in self.exit_registry:raise ValueError('Unregistered current node exit')
|
||||||
|
values.append(dict(item,**self.exit_registry[item['sni']]))
|
||||||
|
return snapshot(values)
|
||||||
|
before=public(source)
|
||||||
|
# Do not run final verification while any candidate in this group is still isolated.
|
||||||
|
cohort=[x for x in self.plan['nodes'] if x['formal_group_id']==group]
|
||||||
|
wanted={(x['relay_host'],x['relay_port'],x['tls_domain']) for x in cohort}
|
||||||
|
if not wanted.issubset({(x['relay'],x['relay_port'],x['sni']) for x in before}):raise RuntimeError('formal_group_not_ready')
|
||||||
|
if group in self.formal_done:
|
||||||
|
prior=self.formal_done[group]
|
||||||
|
return audit(prior['before'],before,prior['report'],prior['qas'],prior['binding'],digest(self.plan),prior['expected_qas'])
|
||||||
|
root=ctx['qa_root']+'/formal-group-'+str(group)
|
||||||
|
execute(gateway,'install -d -m 700 '+root)
|
||||||
|
expectations=[]
|
||||||
|
for node in before:
|
||||||
|
original=next(x for x in source['nodes'] if x['id']==node['id'])
|
||||||
|
expectations.append({'tag':node['tag'],'host':node['relay'],'port':node['relay_port'],'sni':node['sni'],'uuid':original['uuid'],'expected_exit':node['expected_exit'],'expected_country':node['country']})
|
||||||
|
# Tokens/UUIDs never leave memory except the gateway's private task files.
|
||||||
|
write(gateway,root+'/source-private.json',json.dumps({'token':source['token'],'expectations':expectations,'owned':source['owned']}))
|
||||||
|
script="""import gzip,json,urllib.request,subprocess,os,time
|
||||||
|
from pathlib import Path
|
||||||
|
os.umask(0o077);root=Path(ROOT);source=json.loads((root/'source-private.json').read_text())
|
||||||
|
r=urllib.request.Request(SUB+'/api/v1/client/subscribe?token='+source['token'],headers={'User-Agent':'sing-box/1.14.3'})
|
||||||
|
with urllib.request.urlopen(r,timeout=30) as response:
|
||||||
|
data=response.read();headers={k.lower():v for k,v in response.headers.items()}
|
||||||
|
if headers.get('content-encoding','').lower()=='gzip':data=gzip.decompress(data)
|
||||||
|
assert headers.get('cf-cache-status','').upper() not in ['HIT','STALE','UPDATING'] and not headers.get('age') and not headers.get('cf-mitigated') and 'no-store' in headers.get('cache-control','').lower()
|
||||||
|
(root/'subscription.json').write_bytes(data);(root/'expectations.json').write_text(json.dumps(source['expectations']))
|
||||||
|
deadline=time.monotonic()+900
|
||||||
|
for attempt in range(3 if source['owned'] else 1):
|
||||||
|
with (root/'verify.stdout').open('wb') as out,(root/'verify.stderr').open('wb') as err:
|
||||||
|
result=subprocess.run(['python3',OPS+'/verify-node.py','--subscription',str(root/'subscription.json'),'--expectations',str(root/'expectations.json'),'--binary',BINARY,'--port',str(PORT),'--report',str(root/'report.json'),'--run'],timeout=max(1,deadline-time.monotonic()),stdout=out,stderr=err)
|
||||||
|
if result.returncode==0:break
|
||||||
|
try:reason=json.loads((root/'verify.stderr').read_text()).get('reason')
|
||||||
|
except Exception:reason=None
|
||||||
|
if not source['owned'] or attempt==2 or reason!='Verification failed; inspect protected diagnostics' or deadline-time.monotonic()<15:raise RuntimeError('Formal verification failed')
|
||||||
|
# A just-created user's UUID may need one node pull interval to propagate.
|
||||||
|
# Retry only diagnostic reads, keep the same lease; never replay production writes.
|
||||||
|
time.sleep(15)
|
||||||
|
"""
|
||||||
|
prefix='ROOT='+repr(root)+';SUB='+repr(ctx['subscription_url'])+';OPS='+repr(ctx['ops_root'])+';BINARY='+repr(ctx['client_binary'])+';PORT='+repr(ctx['formal_socks_port'])+'\n'
|
||||||
|
try:
|
||||||
|
execute(gateway,"python3 - <<'DPLFORMAL'\n"+prefix+script+"\nDPLFORMAL",1000)
|
||||||
|
after=public(self.account_command(group,'source',client=panel))
|
||||||
|
report=read(gateway,root+'/report.json')
|
||||||
|
finally:
|
||||||
|
# Never leave an owned formal test user active, including failed client runs.
|
||||||
|
self.account_command(group,'cleanup',True,panel)
|
||||||
|
identities={};tokens={}
|
||||||
|
for candidate in cohort:
|
||||||
|
b=ctx['node_backup_root']+'/'+candidate['key']
|
||||||
|
result=php(panel,'$m=json_decode(file_get_contents('+self.php_value(b+'/manifest.json')+'),true);$s=App\\Models\\Server::findOrFail($m["node_id"]);$u=App\\Models\\User::findOrFail($m["test_user_id"]);echo json_encode(["id"=>$u->id,"token"=>$u->token,"banned"=>(bool)$u->banned,"excluded"=>!App\\Services\\ServerService::getAvailableUsers($s)->contains("id",$u->id)]);')
|
||||||
|
identities[candidate['key']]=result['id'];tokens[candidate['key']]=result
|
||||||
|
# Include every owned formal lease from earlier resumptions, not just the current user.
|
||||||
|
lease_dir=ctx['batch_backup_root']+'/acceptance-group-'+str(group)
|
||||||
|
leases=php(panel,'$out=[];foreach(glob('+self.php_value(lease_dir+'/acceptance-*.json')+') as $f){$j=json_decode(file_get_contents($f),true);if(!($j["owned"]??false)||empty($j["id"]))continue;$u=App\\Models\\User::findOrFail($j["id"]);$excluded=true;foreach(App\\Models\\Server::all() as $s)if(App\\Services\\ServerService::getAvailableUsers($s)->contains("id",$u->id))$excluded=false;$out[(string)$u->id]=["id"=>$u->id,"token"=>$u->token,"banned"=>(bool)$u->banned,"excluded"=>$excluded];}echo json_encode($out);')
|
||||||
|
for ident,value in (leases.items() if isinstance(leases,dict) else []):
|
||||||
|
key='formal-'+str(ident);identities[key]=value['id'];tokens[key]=value
|
||||||
|
users={}
|
||||||
|
for key,value in tokens.items():
|
||||||
|
# Only the gateway accesses the subscription source; local HTTP never bypasses its registered route.
|
||||||
|
script='import urllib.request,urllib.error\nr=urllib.request.Request('+repr(ctx['subscription_url']+'/api/v1/client/subscribe?token='+value['token'])+',headers={"User-Agent":"sing-box/1.14.3"})\ntry:\n response=urllib.request.urlopen(r,timeout=30);print(response.status);response.close()\nexcept urllib.error.HTTPError as e:print(e.code)'
|
||||||
|
# Keep tokens out of argv: exec_command carries a private stdin script, not an OS argv.
|
||||||
|
si,o,e=gateway.exec_command('python3 -',timeout=60);si.write(script);si.channel.shutdown_write();out=o.read();e.read()
|
||||||
|
if o.channel.recv_exit_status():raise RuntimeError('qa_rejection_probe_failed')
|
||||||
|
users[key]={'id':value['id'],'banned':value['banned'],'excluded':value['excluded'],'http_status':int(out.strip())}
|
||||||
|
qas=json.dumps({'users':users,'captured_at':int(time.time()),'plan_sha256':digest(self.plan)},sort_keys=True).encode()
|
||||||
|
binding={'plan_sha256':digest(self.plan),'snapshot_sha256':digest(before),'report_sha256':hashlib.sha256(report).hexdigest(),'qa_rejections_sha256':hashlib.sha256(qas).hexdigest(),'verified_at':int(time.time())}
|
||||||
|
audit(before,after,report,qas,binding,digest(self.plan),identities)
|
||||||
|
write(gateway,root+'/qa-rejections.json',qas);write(gateway,root+'/binding.json',json.dumps(binding));write(gateway,root+'/snapshot.json',json.dumps(before))
|
||||||
|
self.formal_done[group]={'before':before,'report':report,'qas':qas,'binding':binding,'expected_qas':identities}
|
||||||
|
# Persist only nonsecret, byte-bound results for coordinator/operator review.
|
||||||
|
self.atomic_json(self.directory/('formal-group-'+str(group)+'.json'),{'snapshot':before,'report':json.loads(report),'qas':json.loads(qas),'binding':binding})
|
||||||
|
|
||||||
|
def qa_evidence(self,n,raw,state):
|
||||||
|
# Bind the exact remote bytes, not a re-serialized nested report. Persist the latest
|
||||||
|
# online/traffic state so independent review never needs a separate task collector.
|
||||||
|
report=json.loads(raw)
|
||||||
|
self.atomic_json(self.directory/(n['key']+'-evidence.json'),{
|
||||||
|
'key':n['key'],'request_sha256':n['request_sha256'],
|
||||||
|
'report_sha256':hashlib.sha256(raw).hexdigest(),'report':report,
|
||||||
|
'panel':state,'collected_at':int(time.time())})
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def php_value(value):
|
||||||
|
return "json_decode(base64_decode('"+base64.b64encode(json.dumps(value).encode()).decode()+"'),true)"
|
||||||
|
|
||||||
|
def call(self,n,phase,action):
|
||||||
|
plan=self.plan;ctx=self.ctx;key=n['key'];sha=n['request_sha256']
|
||||||
|
a=SimpleNamespace(phase=phase,action=action,state_dir=str(self.directory))
|
||||||
|
B=ctx['node_backup_root']+'/'+key;Q=ctx['qa_root']+'/'+key
|
||||||
|
OPS=ctx['ops_root'];NP=ctx['node_root'];APP=ctx['app_root']
|
||||||
|
def execute(c,cmd,timeout=120):
|
||||||
|
si,o,e=c.exec_command(cmd,timeout=timeout);out=o.read();err=e.read();code=o.channel.recv_exit_status()
|
||||||
|
if code: raise RuntimeError('remote_exit_'+str(code))
|
||||||
|
return out.decode()
|
||||||
|
def php(c,body):
|
||||||
|
si,o,e=c.exec_command('cd '+shlex.quote(APP)+' && php',timeout=90)
|
||||||
|
si.write("<?php require 'vendor/autoload.php';$app=require 'bootstrap/app.php';$app->make(Illuminate\\Contracts\\Console\\Kernel::class)->bootstrap();"+body)
|
||||||
|
si.channel.shutdown_write();out=o.read();err=e.read()
|
||||||
|
if o.channel.recv_exit_status():raise RuntimeError('panel_php_failed')
|
||||||
|
try:return json.loads(out)
|
||||||
|
except ValueError:raise RuntimeError('panel_response_not_json') from None
|
||||||
|
def payload(value):return "json_decode(base64_decode('"+base64.b64encode(json.dumps(value,ensure_ascii=False).encode()).decode()+"'),true)"
|
||||||
|
def read(c,path):
|
||||||
|
with c.open_sftp() as s:
|
||||||
|
try:
|
||||||
|
with s.open(path,'rb') as f:return f.read()
|
||||||
|
except FileNotFoundError:return None
|
||||||
|
except OSError as e:
|
||||||
|
if e.errno==2:return None
|
||||||
|
raise
|
||||||
|
def write(c,path,data):
|
||||||
|
if isinstance(data,str):data=data.encode()
|
||||||
|
with c.open_sftp() as s:
|
||||||
|
tmp=path+'.tmp-'+str(os.getpid())+'-'+str(threading.get_ident())
|
||||||
|
with s.open(tmp,'w') as f:s.chmod(tmp,0o600);f.write(data)
|
||||||
|
s.posix_rename(tmp,path)
|
||||||
|
def parameters():return dict(sorted({'key':key,'name':n['name'],'relay-host':n['relay_host'],'relay-port':n['relay_port'],'listen-port':n['listen_port'],'tls-domain':n['tls_domain'],'panel-url':ctx['panel_url']}.items()))
|
||||||
|
def canonical(value):return hashlib.sha256(json.dumps(value,sort_keys=True,ensure_ascii=False,separators=(',',':')).encode()).hexdigest()
|
||||||
|
def containers(gateway):
|
||||||
|
data=json.loads(execute(gateway,'docker inspect '+ ' '.join(ctx['gateway_containers'])))
|
||||||
|
return {v['Name']:{'id':v['Id'],'started':v['State']['StartedAt'],'restarts':v['RestartCount']} for v in data}
|
||||||
|
def receipt(status,retry=False):return {'key':key,'phase':a.phase,'request_sha256':sha,'status':status,'valid_until':time.time()+300,**({'retry_safe':True} if retry else {})}
|
||||||
|
def stage_probe(panel):
|
||||||
|
p=parameters()
|
||||||
|
code='$b='+payload(B)+';$p='+payload(p)+r''';$f=$b.'/manifest.json';if(!is_file($f)){$conf=App\Models\ServerMachine::where('name','dpl-'.$p['key'])->exists()||App\Models\Server::where('name',$p['name'])->exists()||App\Models\Server::where('host',$p['relay-host'])->where('port',(string)$p['relay-port'])->exists();echo json_encode(['status'=>$conf?'conflict':'absent','retry_safe'=>!$conf]);return;}$m=json_decode(file_get_contents($f),true);$s=App\Models\Server::find($m['node_id']);$mch=App\Models\ServerMachine::find($m['machine_id']);$ok=($m['parameters']??null)===$p && $s && $mch && $s->machine_id===$mch->id && $mch->name==='dpl-'.$p['key'] && $s->type==='anytls' && ($s->protocol_settings['tls']['allow_insecure']??null)===false && $s->name===$p['name'] && $s->host===$p['relay-host'] && (int)$s->port===$p['relay-port'] && (int)$s->server_port===$p['listen-port'] && ($s->protocol_settings['tls']['server_name']??null)===$p['tls-domain'];$u=App\Models\User::find($m['test_user_id']);$g=App\Models\ServerGroup::find($m['group_id']);$ok=$ok && $u && $g && !$u->is_admin && (int)$u->group_id===$m['group_id'] && str_starts_with($u->email,'node-qa-'.$p['key'].'-') && $g->name==='dpl 接入验收 '.$p['key'];if(!is_file($b.'/promotion-journal.json'))$ok=$ok && $s->group_ids===[(string)$m['group_id']] && !$u->banned;echo json_encode(['status'=>$ok?'ready':'conflict','manifest'=>$m]);'''
|
||||||
|
return php(panel,code)
|
||||||
|
def state(panel):
|
||||||
|
return php(panel,'$b='+payload(B)+r''';$m=json_decode(file_get_contents($b.'/manifest.json'),true);$s=App\Models\Server::findOrFail($m['node_id']);$u=App\Models\User::findOrFail($m['test_user_id']);echo json_encode(['manifest'=>$m,'node_online'=>(bool)$s->is_online,'groups'=>$s->group_ids,'qa_banned'=>(bool)$u->banned,'u'=>(int)$u->u,'d'=>(int)$u->d]);''')
|
||||||
|
def baseline(panel,gateway):
|
||||||
|
directory=ctx['batch_backup_root'];f=directory+'/baseline.json'
|
||||||
|
version=json.loads(execute(panel,"python3 - <<'DPLVERSION'\nimport subprocess,hashlib,json\np="+repr(APP)+"\nh=subprocess.check_output(['git','-C',p,'rev-parse','HEAD']).decode().strip()\nd=subprocess.check_output(['git','-C',p,'diff','HEAD'])+subprocess.check_output(['git','-C',p,'status','--porcelain'])\nprint(json.dumps({'head':h,'worktree_sha256':hashlib.sha256(d).hexdigest()}))\nDPLVERSION"))
|
||||||
|
previous=read(panel,directory+'/application-version.json')
|
||||||
|
if previous is None:write(panel,directory+'/application-version.json',json.dumps(version))
|
||||||
|
elif json.loads(previous)!=version:raise RuntimeError('application_version_changed')
|
||||||
|
# Read/write a protected snapshot only once, before any new stage.
|
||||||
|
return php(panel,'$p='+payload(f)+r''';if(is_file($p)){echo json_encode(['exists'=>true]);return;} $v=['nodes'=>App\Models\Server::orderBy('id')->get()->toArray(),'plans'=>App\Models\Plan::orderBy('id')->get()->toArray(),'users'=>App\Models\User::orderBy('id')->get(['id','group_id','plan_id'])->toArray(),'subscribe_url'=>admin_setting('subscribe_url')];file_put_contents($p,json_encode($v,JSON_THROW_ON_ERROR));chmod($p,0600);echo json_encode(['exists'=>true]);''')
|
||||||
|
def gateway_py(gateway,script,timeout=120):
|
||||||
|
return execute(gateway,"python3 - <<'DPLPY'\n"+script+"\nDPLPY",timeout)
|
||||||
|
def assert_old(node):
|
||||||
|
self.guard_service(node,B,read,write,execute)
|
||||||
|
def transfer(panel,gateway,node):
|
||||||
|
execute(node,'install -d -m 700 /etc/dpl-node')
|
||||||
|
execute(gateway,'install -d -m 700 '+Q)
|
||||||
|
for source,dest,target in [('node-config.yml',node,'/etc/dpl-node/config.yml'),('node-secret.env',node,'/etc/dpl-node/secret.env'),('qa-private.json',gateway,Q+'/qa.json')]:
|
||||||
|
data=read(panel,B+'/'+source);prior=read(dest,target)
|
||||||
|
if prior is not None and prior!=data:raise RuntimeError('private_config_conflict')
|
||||||
|
if prior is None:write(dest,target,data)
|
||||||
|
def fetch_qa(gateway):
|
||||||
|
script='''import json,urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
q=json.loads(Path(Q+'/qa.json').read_text())
|
||||||
|
req=urllib.request.Request(SUB+'/api/v1/client/subscribe?token='+q['subscribe_token'],headers={'User-Agent':'sing-box/1.14.3'})
|
||||||
|
with urllib.request.urlopen(req,timeout=30) as r:
|
||||||
|
data=r.read(); code=r.status; headers={k.lower():v for k,v in r.headers.items()}
|
||||||
|
import gzip
|
||||||
|
if headers.get('content-encoding','').lower()=='gzip':data=gzip.decompress(data)
|
||||||
|
v=json.loads(data);Path(Q+'/subscription.json').write_bytes(data);Path(Q+'/subscription.json').chmod(0o600)
|
||||||
|
print(json.dumps({'status':code,'cache':headers.get('cf-cache-status'),'control':headers.get('cache-control'),'age':headers.get('age')}))
|
||||||
|
'''
|
||||||
|
return gateway_py(gateway,'Q='+repr(Q)+';SUB='+repr(ctx['subscription_url'])+'\n'+script)
|
||||||
|
def phase_impl(panel,gateway,node):
|
||||||
|
if a.phase=='preflight':
|
||||||
|
with self.lock, self.resource_lock(self.directory.parent,ctx['panel_vps']+':panel'):
|
||||||
|
execute(panel,'install -d -m 700 '+ctx['batch_backup_root'])
|
||||||
|
execute(panel,'test "$(git -C '+OPS+' rev-parse HEAD)" = '+ctx['provision_commit']+' && test -z "$(git -C '+OPS+' status --porcelain)"')
|
||||||
|
baseline(panel,gateway)
|
||||||
|
base_path=ctx['batch_backup_root']+'/gateway-baseline.json'
|
||||||
|
current_containers=containers(gateway)
|
||||||
|
prior_gateway=read(panel,base_path)
|
||||||
|
if prior_gateway is None:write(panel,base_path,json.dumps(current_containers,sort_keys=True))
|
||||||
|
elif json.loads(prior_gateway)!=current_containers:return receipt('conflict')
|
||||||
|
ui=Path(ctx['provider_snapshot'])
|
||||||
|
if not ui.exists():return receipt('waiting')
|
||||||
|
snap=private_json(ui)
|
||||||
|
if type(snap.get('captured_at')) not in (int,float) or not 0<=time.time()-snap['captured_at']<=900:return receipt('waiting')
|
||||||
|
try:provider_state(snap,n)
|
||||||
|
except ValueError:return receipt('conflict')
|
||||||
|
gateway_py(gateway,'import socket\ns=socket.socket();s.bind(("127.0.0.1",'+str(n['socks_port'])+'));s.close()')
|
||||||
|
s=stage_probe(panel)
|
||||||
|
if s['status']=='conflict':return receipt('conflict')
|
||||||
|
assert_old(node)
|
||||||
|
self.certificate_gate(n,node,read,execute)
|
||||||
|
execute(node,'test "$(git -C '+NP+' rev-parse HEAD)" = '+ctx['provision_commit']+' && test -z "$(git -C '+NP+' status --porcelain)" && command -v git >/dev/null')
|
||||||
|
execute(gateway,"echo '"+ctx['client_sha256']+" "+ctx['client_binary']+"' | sha256sum -c --status && test -x "+shlex.quote(ctx['client_binary'])+' && test "$(git -C '+OPS+' rev-parse HEAD)" = '+ctx['provision_commit']+' && test -z "$(git -C '+OPS+' status --porcelain)"')
|
||||||
|
if s['status']=='absent':execute(node,"test ! -e /etc/dpl-node/config.yml && test ! -e /etc/systemd/system/dpl-node.service && test ! -e /etc/dpl-node/batch-owner.json && ! ss -lntH '( sport = :"+str(n['listen_port'])+" )' | grep -q .")
|
||||||
|
if not php(panel,'echo json_encode(App\\Models\\ServerGroup::where("id",'+str(n['formal_group_id'])+')->exists());'):return receipt('conflict')
|
||||||
|
self.prepare_account(n,panel,execute)
|
||||||
|
r=receipt('ready');r['resume_dependencies']=self.dependencies(n,panel,gateway,node,stage_probe,read,execute,gateway_py);return r
|
||||||
|
if a.phase=='stage':
|
||||||
|
if a.action=='probe':
|
||||||
|
r=stage_probe(panel);return receipt(r['status'],r.get('retry_safe',False))
|
||||||
|
command='cd '+APP+' && php '+OPS+'/stage.php '+ ' '.join('--'+k+'='+shlex.quote(str(v)) for k,v in {**parameters(),'backup':B}.items())
|
||||||
|
execute(panel,command);return receipt('ready')
|
||||||
|
if a.phase=='dns':
|
||||||
|
try:
|
||||||
|
result=gateway_py(gateway,'import socket,json\nprint(json.dumps(sorted(set(x[4][0] for x in socket.getaddrinfo('+repr(n['tls_domain'])+',443,socket.AF_INET)))))')
|
||||||
|
return receipt('ready' if json.loads(result)==[n['node_address']] else 'conflict')
|
||||||
|
except Exception:return receipt('waiting')
|
||||||
|
if a.phase=='provision':
|
||||||
|
owner='/etc/dpl-node/batch-owner.json';prior=read(node,owner)
|
||||||
|
if prior is not None and json.loads(prior).get('request_sha256')!=sha:return receipt('conflict')
|
||||||
|
if prior is None and read(node,'/etc/systemd/system/dpl-node.service') is not None:return receipt('conflict')
|
||||||
|
done=read(node,B+'/provision-ok.json')
|
||||||
|
if a.action=='probe':
|
||||||
|
if done and json.loads(done).get('request_sha256')==sha:
|
||||||
|
for file,target in [('node-config.yml','/etc/dpl-node/config.yml'),('node-secret.env','/etc/dpl-node/secret.env')]:
|
||||||
|
if read(panel,B+'/'+file)!=read(node,target):return receipt('conflict')
|
||||||
|
execute(node,"echo '55bf71fa9d9f2048d3255ae7c0af929a41897ca7743f6ead34132a6ca4c79043 /opt/dpl-node/xboard-node' | sha256sum -c >/dev/null")
|
||||||
|
assert_old(node);execute(node,'systemctl is-active --quiet dpl-node && openssl x509 -checkend 86400 -noout -in /etc/letsencrypt/live/'+n['tls_domain']+'/fullchain.pem')
|
||||||
|
return receipt('ready')
|
||||||
|
return receipt('absent',True)
|
||||||
|
if prior is None:
|
||||||
|
execute(node,'install -d -m 700 /etc/dpl-node');write(node,owner,json.dumps({'request_sha256':sha,'key':key}))
|
||||||
|
transfer(panel,gateway,node)
|
||||||
|
if read(node,'/etc/systemd/system/dpl-node.service') is None:
|
||||||
|
execute(node,'bash '+NP+'/install-node.sh --listen-port='+str(n['listen_port'])+' --tls-domain='+n['tls_domain']+' > '+B+'/install-node.log 2>&1',300)
|
||||||
|
execute(node,"echo '55bf71fa9d9f2048d3255ae7c0af929a41897ca7743f6ead34132a6ca4c79043 /opt/dpl-node/xboard-node' | sha256sum -c >/dev/null")
|
||||||
|
domain=n['tls_domain']
|
||||||
|
if read(node,'/etc/letsencrypt/live/'+domain+'/fullchain.pem') is None:
|
||||||
|
execute(node,"certbot certonly --standalone --non-interactive --agree-tos --no-directory-hooks --cert-name "+domain+' -d '+domain+" --deploy-hook '/bin/sh "+NP+"/renew-node.sh' > "+B+'/certbot.log 2>&1',300)
|
||||||
|
execute(node,'systemctl enable --now dpl-node > '+B+'/enable.log 2>&1 && systemctl is-active --quiet dpl-node',60)
|
||||||
|
execute(node,'certbot renew --cert-name '+domain+' --dry-run --no-random-sleep-on-renew --run-deploy-hooks --no-directory-hooks > '+B+'/renew-dry-run.log 2>&1',600)
|
||||||
|
assert_old(node)
|
||||||
|
write(node,B+'/provision-ok.json',json.dumps({'request_sha256':sha,'verified_at':int(time.time())}))
|
||||||
|
return receipt('ready')
|
||||||
|
if a.phase=='relay':
|
||||||
|
try:
|
||||||
|
result=gateway_py(gateway,'import ssl,socket,json\nc=ssl.create_default_context()\nwith socket.create_connection(('+repr(n['relay_host'])+','+str(n['relay_port'])+'),10) as s:\n with c.wrap_socket(s,server_hostname='+repr(n['tls_domain'])+') as t: print(json.dumps({"tls":t.version()}))')
|
||||||
|
return receipt('ready' if json.loads(result).get('tls') else 'waiting')
|
||||||
|
except Exception:return receipt('waiting')
|
||||||
|
if a.phase=='verify':
|
||||||
|
s=state(panel)
|
||||||
|
if s['qa_banned'] and s['groups']==[str(n['formal_group_id'])]:
|
||||||
|
return receipt('ready' if read(panel,B+'/qa-report.json') else 'unknown')
|
||||||
|
existing=read(panel,B+'/qa-report.json')
|
||||||
|
if existing:
|
||||||
|
report=json.loads(existing)
|
||||||
|
if report.get('state')=='live_verified' and report.get('qa_user_id')==s['manifest']['test_user_id'] and 0<=time.time()-report.get('verified_at',0)<=900 and s['node_online'] and s['u']>0 and s['d']>0:
|
||||||
|
self.qa_evidence(n,existing,s)
|
||||||
|
r=receipt('ready');r['valid_until']=min(r['valid_until'],report['verified_at']+900);return r
|
||||||
|
transfer(panel,gateway,node);fetch_qa(gateway)
|
||||||
|
execute(gateway,'python3 '+OPS+'/verify-node.py --qa '+Q+'/qa.json --subscription '+Q+'/subscription.json --binary '+ctx['client_binary']+' --port '+str(n['socks_port'])+' --expected-exit '+shlex.quote(n['expected_exit'])+' --expected-country '+n['country']+' --report '+Q+'/qa-report.json --run > '+Q+'/verify.stdout 2> '+Q+'/verify.stderr',300)
|
||||||
|
data=read(gateway,Q+'/qa-report.json');report=json.loads(data)
|
||||||
|
write(panel,B+'/qa-report.json',data)
|
||||||
|
for _ in range(24):
|
||||||
|
s=state(panel)
|
||||||
|
if s['node_online'] and s['u']>0 and s['d']>0:
|
||||||
|
self.qa_evidence(n,data,s)
|
||||||
|
r=receipt('ready');r['valid_until']=min(r['valid_until'],report['verified_at']+900);return r
|
||||||
|
time.sleep(5)
|
||||||
|
self.qa_evidence(n,data,s)
|
||||||
|
return receipt('waiting')
|
||||||
|
if a.phase=='publish':
|
||||||
|
s=state(panel)
|
||||||
|
report_data=read(panel,B+'/qa-report.json')
|
||||||
|
review_path=Path(a.state_dir)/(key+'-reviewed.json')
|
||||||
|
if not report_data or not review_path.exists():return receipt('waiting')
|
||||||
|
report=json.loads(report_data);review=json.loads(review_path.read_text())
|
||||||
|
valid=review.get('request_sha256')==sha and review.get('report_sha256')==hashlib.sha256(report_data).hexdigest() and review.get('qa_user_id')==s['manifest']['test_user_id'] and review.get('verified_at')==report.get('verified_at')
|
||||||
|
if not valid:return receipt('waiting')
|
||||||
|
if s['qa_banned'] and s['groups']==[str(n['formal_group_id'])]:
|
||||||
|
journal=read(panel,B+'/promotion-journal.json')
|
||||||
|
if not journal:return receipt('conflict')
|
||||||
|
j=json.loads(journal)
|
||||||
|
return receipt('ready' if j.get('manifest')==s['manifest'] and j.get('request',{}).get('mode')=='append' and j.get('request',{}).get('group')==n['formal_group_id'] and j.get('state')=='applied' else 'conflict')
|
||||||
|
if not s['node_online'] or s['u']<=0 or s['d']<=0:return receipt('waiting')
|
||||||
|
if a.action=='probe':return receipt('absent',True)
|
||||||
|
cmd='cd '+APP+' && php '+OPS+'/promote.php --backup='+B+' --mode=append --group='+str(n['formal_group_id'])+' --verification='+B+'/qa-report.json'
|
||||||
|
execute(panel,cmd);execute(panel,cmd+' --apply');return receipt('ready')
|
||||||
|
if a.phase=='audit':
|
||||||
|
s=state(panel);assert_old(node)
|
||||||
|
if not s['qa_banned'] or s['groups']!=[str(n['formal_group_id'])] or not s['node_online']:return receipt('conflict')
|
||||||
|
ok=php(panel,'$b='+payload(B)+r''';$m=json_decode(file_get_contents($b.'/manifest.json'),true);$n=App\Models\Server::findOrFail($m['node_id']);$base=json_decode(file_get_contents('''+payload(ctx['batch_backup_root']+'/baseline.json')+r'''),true);$ok=$base['plans']===App\Models\Plan::orderBy('id')->get()->toArray() && $base['users']===App\Models\User::whereIn('id',array_column($base['users'],'id'))->orderBy('id')->get(['id','group_id','plan_id'])->toArray() && $base['subscribe_url']===admin_setting('subscribe_url');foreach($base['nodes'] as $old){$cur=App\Models\Server::find($old['id'])->toArray();foreach(['name','host','port','server_port','protocol_settings','group_ids','machine_id','enabled','show','cert_config'] as $k)if(($cur[$k]??null)!==($old[$k]??null))$ok=false;}$ids=App\Services\ServerService::getAvailableUsers($n)->pluck('id')->all();$ok=$ok&&!in_array($m['test_user_id'],$ids);echo json_encode($ok);''')
|
||||||
|
if not ok:return receipt('conflict')
|
||||||
|
self.formal(n,panel,gateway,php,execute,read,write)
|
||||||
|
original_gateway=read(panel,ctx['batch_backup_root']+'/gateway-baseline.json')
|
||||||
|
if not original_gateway or containers(gateway)!=json.loads(original_gateway):return receipt('conflict')
|
||||||
|
return receipt('ready')
|
||||||
|
return receipt('unknown')
|
||||||
|
panel=Lazy(self.pool,ctx['panel_vps']);gateway=Lazy(self.pool,ctx['gateway_vps']);node=Lazy(self.pool,n['vps_id'])
|
||||||
|
try:return phase_impl(panel,gateway,node)
|
||||||
|
except Exception as exc:
|
||||||
|
# Never print underlying library/server errors or credentials.
|
||||||
|
safe = str(exc) if type(exc) is RuntimeError and __import__('re').fullmatch('[a-z_0-9]+',str(exc)) else 'inspect_protected_state'
|
||||||
|
self.atomic_json(self.directory/(key+'-'+phase+'-error.json'),{'reason':safe})
|
||||||
|
return receipt('waiting' if safe in ('investigate_preserved_service_restart','formal_group_not_ready') else 'unknown')
|
||||||
|
|
||||||
|
class Lazy:
|
||||||
|
def __init__(self,pool,rid):self.pool,self.rid=pool,rid
|
||||||
|
def __getattr__(self,name):return getattr(self.pool.get(self.rid),name)
|
||||||
99
scripts/node-onboarding/ssh_pool.py
Normal file
99
scripts/node-onboarding/ssh_pool.py
Normal file
@ -0,0 +1,99 @@
|
|||||||
|
"""Thread-local SSH sessions, mandatory explicit jump, strict known-host keys."""
|
||||||
|
import json
|
||||||
|
import hashlib
|
||||||
|
import base64
|
||||||
|
import re
|
||||||
|
import threading
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def private_text(path):
|
||||||
|
p = Path(path)
|
||||||
|
if not p.is_absolute() or p.is_symlink() or not p.is_file() or p.stat().st_mode & 0o077:
|
||||||
|
raise ValueError('Private regular absolute credential/config reference required')
|
||||||
|
return p.read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def private_json(path):
|
||||||
|
return json.loads(private_text(path))
|
||||||
|
|
||||||
|
|
||||||
|
def credentials(spec):
|
||||||
|
if not spec.get('credential_ref'):return {}
|
||||||
|
if spec.get('credential_format','json')=='json':return private_json(spec['credential_ref'])
|
||||||
|
if spec['credential_format']=='codocs-password':
|
||||||
|
matches=re.findall(r'(?m)^[ \t]*(?:[-*][ \t]+)?密码[：:][ \t]*([^\r\n]+)$',private_text(spec['credential_ref']))
|
||||||
|
if len(matches)!=1:raise ValueError('Unambiguous existing CoDocs password field required')
|
||||||
|
return {'password':matches[0].strip()}
|
||||||
|
raise ValueError('Unsupported credential reference format')
|
||||||
|
|
||||||
|
|
||||||
|
class Pool:
|
||||||
|
def __init__(self, config, jump_id, client_factory=None):
|
||||||
|
self.config, self.jump_id = config, jump_id
|
||||||
|
if jump_id not in config['hosts'] or config['hosts'][jump_id].get('role') != 'white_jump':
|
||||||
|
raise ValueError('Explicit registered white jump required')
|
||||||
|
if client_factory is None:
|
||||||
|
known=Path(config['known_hosts'])
|
||||||
|
if not known.is_absolute() or known.is_symlink() or not known.is_file() or known.stat().st_mode & 0o022:raise ValueError('Explicit trusted known-hosts file required')
|
||||||
|
if any(not re.fullmatch('SHA256:[A-Za-z0-9+/]{43}=?',s.get('host_key_sha256','')) for s in config['hosts'].values()):raise ValueError('Approved explicit host fingerprints required')
|
||||||
|
self.factory = client_factory
|
||||||
|
self.local = threading.local()
|
||||||
|
self.clients = []
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
|
||||||
|
def get(self, rid):
|
||||||
|
sessions = getattr(self.local, 'sessions', None)
|
||||||
|
if sessions is None:
|
||||||
|
sessions = self.local.sessions = {}
|
||||||
|
if rid in sessions:
|
||||||
|
c = sessions[rid]
|
||||||
|
if c.get_transport() and c.get_transport().is_active():
|
||||||
|
return c
|
||||||
|
# Do not transparently retry operations after a lost connection.
|
||||||
|
raise RuntimeError('SSH session lost; probe remote outcome on next run')
|
||||||
|
spec = self.config['hosts'][rid]
|
||||||
|
jump = self.get(self.jump_id) if rid != self.jump_id else None
|
||||||
|
if self.factory:
|
||||||
|
client = self.factory(rid, spec, jump)
|
||||||
|
else:
|
||||||
|
import paramiko
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
known=Path(self.config['known_hosts'])
|
||||||
|
if not known.is_absolute() or known.is_symlink() or not known.is_file() or known.stat().st_mode & 0o022:raise ValueError('Trusted non-writable known-hosts reference required')
|
||||||
|
entries=paramiko.HostKeys(str(known))
|
||||||
|
hostname=spec['host'] if spec.get('port',22)==22 else '['+spec['host']+']:'+str(spec['port'])
|
||||||
|
approved=[]
|
||||||
|
for key in (entries.lookup(hostname) or {}).values():
|
||||||
|
actual='SHA256:'+base64.b64encode(hashlib.sha256(key.asbytes()).digest()).decode().rstrip('=')
|
||||||
|
if actual==spec['host_key_sha256'].rstrip('='):approved.append(key)
|
||||||
|
if not approved:raise ValueError('Approved known-host key absent; no credentials transmitted')
|
||||||
|
for key in approved:client.get_host_keys().add(hostname,key.get_name(),key)
|
||||||
|
client.set_missing_host_key_policy(paramiko.RejectPolicy())
|
||||||
|
auth = credentials(spec)
|
||||||
|
if set(auth) - {'username', 'password', 'key_filename', 'passphrase'}:
|
||||||
|
raise ValueError('Unexpected SSH credential fields')
|
||||||
|
if auth.get('username',spec['username'])!=spec['username']:raise ValueError('Credential username differs from registered host')
|
||||||
|
if auth.get('key_filename'):
|
||||||
|
key=Path(auth['key_filename'])
|
||||||
|
if not key.is_absolute() or not key.is_file() or key.stat().st_mode & 0o077:raise ValueError('Private absolute SSH key required')
|
||||||
|
sock = jump.get_transport().open_channel('direct-tcpip', (spec['host'], spec.get('port', 22)), ('127.0.0.1', 0)) if jump else None
|
||||||
|
try:
|
||||||
|
client.connect(spec['host'], port=spec.get('port', 22), username=spec['username'],
|
||||||
|
password=auth.get('password'), key_filename=auth.get('key_filename'),
|
||||||
|
passphrase=auth.get('passphrase'), sock=sock, allow_agent=False,
|
||||||
|
look_for_keys=False, timeout=20, banner_timeout=20, auth_timeout=20)
|
||||||
|
except Exception:
|
||||||
|
client.close()
|
||||||
|
raise RuntimeError('SSH connect failed; inspect registered route/host key/credential reference') from None
|
||||||
|
finally:
|
||||||
|
auth.clear()
|
||||||
|
sessions[rid] = client
|
||||||
|
with self.lock:
|
||||||
|
self.clients.append(client)
|
||||||
|
return client
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
for client in reversed(self.clients):
|
||||||
|
client.close()
|
||||||
|
self.clients.clear()
|
||||||
@ -1,82 +1,109 @@
|
|||||||
<?php
|
<?php
|
||||||
// Run from the production application root. All credential output is private.
|
// Run only in an authorized target environment. --help never boots the app.
|
||||||
require getcwd() . '/vendor/autoload.php';
|
require __DIR__ . '/operations.php';
|
||||||
$app = require getcwd() . '/bootstrap/app.php';
|
use App\Models\{Server,ServerGroup,ServerMachine,User};
|
||||||
$app->make(Illuminate\Contracts\Console\Kernel::class)->bootstrap();
|
|
||||||
|
|
||||||
use App\Models\Server;
|
|
||||||
use App\Models\ServerGroup;
|
|
||||||
use App\Models\ServerMachine;
|
|
||||||
use App\Models\User;
|
|
||||||
use Illuminate\Support\Facades\DB;
|
use Illuminate\Support\Facades\DB;
|
||||||
use Illuminate\Support\Str;
|
use Illuminate\Support\Str;
|
||||||
|
$opts=getopt('', ['key:', 'name:', 'relay-host:', 'relay-port:', 'listen-port:', 'tls-domain:', 'panel-url:', 'backup:', 'help']);
|
||||||
$opts = getopt('', ['key:', 'name:', 'relay-host:', 'relay-port:', 'listen-port:', 'tls-domain:', 'backup:']);
|
if (isset($opts['help'])) {
|
||||||
foreach (['key', 'name', 'relay-host', 'relay-port', 'listen-port', 'tls-domain', 'backup'] as $key) {
|
echo "--key=KEY --name=NAME --relay-host=HOST --relay-port=PORT --listen-port=PORT --tls-domain=DOMAIN --panel-url=HTTPS_URL --backup=DIR\n";
|
||||||
if (empty($opts[$key])) throw new RuntimeException("Missing --$key");
|
|
||||||
}
|
|
||||||
if (!preg_match('/^[a-z0-9-]+$/', $opts['key'])) throw new RuntimeException('Invalid key');
|
|
||||||
foreach (['relay-port', 'listen-port'] as $key) {
|
|
||||||
if (!ctype_digit((string)$opts[$key]) || $opts[$key] < 1024 || $opts[$key] > 65535) throw new RuntimeException('Invalid port');
|
|
||||||
}
|
|
||||||
$dir = rtrim($opts['backup'], '/');
|
|
||||||
umask(0077);
|
|
||||||
if (!is_dir($dir) && !mkdir($dir, 0700, true)) throw new RuntimeException('Backup creation failed');
|
|
||||||
if (is_file("$dir/manifest.json")) {
|
|
||||||
$manifest = json_decode(file_get_contents("$dir/manifest.json"), true, flags: JSON_THROW_ON_ERROR);
|
|
||||||
if ($manifest['key'] !== $opts['key']) throw new RuntimeException('Backup belongs to another node');
|
|
||||||
echo json_encode(['already_staged' => true, 'node_id' => $manifest['node_id']]) . PHP_EOL;
|
|
||||||
exit;
|
exit;
|
||||||
}
|
}
|
||||||
if (ServerMachine::where('name', 'dpl-' . $opts['key'])->exists() || Server::where('name', $opts['name'])->exists()) {
|
try {
|
||||||
throw new RuntimeException('Existing resources found without this backup manifest');
|
foreach (['key','name','relay-host','relay-port','listen-port','tls-domain','panel-url','backup'] as $key) {
|
||||||
|
if (!isset($opts[$key]) || !is_string($opts[$key]) || $opts[$key]==='') throw new RuntimeException('Missing required stage option');
|
||||||
|
}
|
||||||
|
if (!preg_match('/^[a-z0-9-]+$/D',$opts['key'])) throw new RuntimeException('Invalid deployment key');
|
||||||
|
foreach (['relay-port','listen-port'] as $key) {
|
||||||
|
if (!ctype_digit($opts[$key]) || (int)$opts[$key]<1024 || (int)$opts[$key]>65535) throw new RuntimeException('Invalid port');
|
||||||
|
$opts[$key]=(int)$opts[$key];
|
||||||
|
}
|
||||||
|
foreach (['relay-host','tls-domain'] as $key) {
|
||||||
|
if (!preg_match('/^[a-z0-9][a-z0-9.-]*\.[a-z]{2,}$/Di',$opts[$key])) throw new RuntimeException('Explicit hostname required');
|
||||||
|
}
|
||||||
|
$url=parse_url($opts['panel-url']);
|
||||||
|
if (!is_array($url) || ($url['scheme']??'')!=='https' || empty($url['host']) || isset($url['user']) || isset($url['pass']) || isset($url['query']) || isset($url['fragment'])) {
|
||||||
|
throw new RuntimeException('Explicit HTTPS panel URL without credentials/query required');
|
||||||
|
}
|
||||||
|
$dir=rtrim($opts['backup'],'/');
|
||||||
|
if (!str_starts_with($dir,'/') || is_link($dir)) throw new RuntimeException('Absolute private backup directory required');
|
||||||
|
umask(0077);
|
||||||
|
if (!is_dir($dir) && !mkdir($dir,0700,true)) throw new RuntimeException('Cannot create backup directory');
|
||||||
|
if ((fileperms($dir)&0077)!==0) throw new RuntimeException('Backup directory must be private');
|
||||||
|
$parameters=$opts;unset($parameters['backup']);ksort($parameters);
|
||||||
|
DplOnboarding\boot();
|
||||||
|
$m=DplOnboarding\locked($dir,function () use ($dir,$parameters) {
|
||||||
|
$path=$dir.'/manifest.json';
|
||||||
|
if (is_file($path)) {
|
||||||
|
$m=DplOnboarding\manifest($dir);
|
||||||
|
if (($m['parameters']??null)!==$parameters) throw new RuntimeException('Existing manifest has different/legacy parameters; no automatic adoption');
|
||||||
|
$node=Server::findOrFail($m['node_id']);$qa=User::findOrFail($m['test_user_id']);
|
||||||
|
$machine=ServerMachine::findOrFail($m['machine_id']);
|
||||||
|
if ((int)$node->machine_id!==$machine->id || $machine->name!=='dpl-'.$parameters['key'] ||
|
||||||
|
$node->host!==$parameters['relay-host'] || (int)$node->port!==$parameters['relay-port'] ||
|
||||||
|
(int)$node->server_port!==$parameters['listen-port'] || $node->type!=='anytls' ||
|
||||||
|
($node->protocol_settings['tls']['server_name']??null)!==$parameters['tls-domain'] ||
|
||||||
|
(int)$qa->group_id!==$m['group_id']) throw new RuntimeException('Staged resources drifted');
|
||||||
|
if (is_file($dir.'/promotion-journal.json') || is_file($dir.'/promoted.json')) throw new RuntimeException('Node already promoted; staging cannot regenerate credentials');
|
||||||
|
return $m;
|
||||||
|
}
|
||||||
|
return DB::transaction(function () use ($parameters,$path,$dir) {
|
||||||
|
if (ServerMachine::where('name','dpl-'.$parameters['key'])->exists() || Server::where('name',$parameters['name'])->exists() ||
|
||||||
|
Server::where('host',$parameters['relay-host'])->where('port',(string)$parameters['relay-port'])->exists() ||
|
||||||
|
ServerGroup::where('name','dpl 接入验收 '.$parameters['key'])->exists()) {
|
||||||
|
throw new RuntimeException('Existing resource/relay tuple without matching manifest; refusing takeover');
|
||||||
|
}
|
||||||
|
DplOnboarding\writeJson($dir.'/before.json',[
|
||||||
|
'server_ids'=>Server::orderBy('id')->pluck('id')->all(),
|
||||||
|
'machine_ids'=>ServerMachine::orderBy('id')->pluck('id')->all(),
|
||||||
|
'group_ids'=>ServerGroup::orderBy('id')->pluck('id')->all(),
|
||||||
|
]);
|
||||||
|
$group=new ServerGroup();$group->name='dpl 接入验收 '.$parameters['key'];$group->save();
|
||||||
|
$machine=ServerMachine::create(['name'=>'dpl-'.$parameters['key'],
|
||||||
|
'notes'=>'Independent AnyTLS service; existing XrayR preserved','is_active'=>true,'token'=>ServerMachine::generateToken()]);
|
||||||
|
$node=Server::create([
|
||||||
|
'name'=>$parameters['name'],'type'=>'anytls','host'=>$parameters['relay-host'],
|
||||||
|
'port'=>(string)$parameters['relay-port'],'server_port'=>(string)$parameters['listen-port'],
|
||||||
|
'group_ids'=>[(string)$group->id],'route_ids'=>[],'tags'=>[],
|
||||||
|
'rate'=>1,'show'=>true,'enabled'=>true,'machine_id'=>$machine->id,'sort'=>1,
|
||||||
|
'protocol_settings'=>['tls'=>['server_name'=>$parameters['tls-domain'],'allow_insecure'=>false]],
|
||||||
|
'cert_config'=>['cert_mode'=>'file','domain'=>$parameters['tls-domain'],
|
||||||
|
'cert_file'=>'/etc/letsencrypt/live/'.$parameters['tls-domain'].'/fullchain.pem',
|
||||||
|
'key_file'=>'/etc/letsencrypt/live/'.$parameters['tls-domain'].'/privkey.pem'],
|
||||||
|
]);
|
||||||
|
$qa=new User([
|
||||||
|
'email'=>'node-qa-'.$parameters['key'].'-'.Str::lower(Str::random(8)).'@example.invalid',
|
||||||
|
'password'=>password_hash(Str::random(40),PASSWORD_BCRYPT),'uuid'=>(string)Str::uuid(),'token'=>Str::random(32),
|
||||||
|
'group_id'=>$group->id,'transfer_enable'=>104857600,'expired_at'=>time()+86400,'banned'=>0,'is_admin'=>0,
|
||||||
|
'remind_expire'=>0,'remind_traffic'=>0,'device_limit'=>2,
|
||||||
|
]);$qa->saveQuietly();
|
||||||
|
$m=['key'=>$parameters['key'],'node_id'=>$node->id,'machine_id'=>$machine->id,'group_id'=>$group->id,
|
||||||
|
'test_user_id'=>$qa->id,'parameters'=>$parameters];
|
||||||
|
// Save identities before commit. A failed/unknown commit must be inspected,
|
||||||
|
// never repeated into new resources merely because a response was lost.
|
||||||
|
DplOnboarding\writeJson($path,$m);
|
||||||
|
return $m;
|
||||||
|
},1);
|
||||||
|
});
|
||||||
|
// Recovery can regenerate missing transport files from the same staged objects.
|
||||||
|
$machine=ServerMachine::findOrFail($m['machine_id']);$qa=User::findOrFail($m['test_user_id']);
|
||||||
|
$panel=json_encode($parameters['panel-url'],JSON_THROW_ON_ERROR|JSON_UNESCAPED_SLASHES);
|
||||||
|
$config="panel:\n url: $panel\nmachine:\n machine_id: {$machine->id}\n token_env: DPL_MACHINE_TOKEN\nkernel:\n type: singbox\n config_dir: /var/lib/dpl-node/kernel\nlog:\n level: info\nruntime:\n gomemlimit: 384MiB\n gogc: 75\nnode:\n push_interval: 30\n pull_interval: 30\n";
|
||||||
|
foreach (['node-config.yml'=>$config,'node-secret.env'=>'DPL_MACHINE_TOKEN='.$machine->token."\n"] as $file=>$body) {
|
||||||
|
$path=$dir.'/'.$file;
|
||||||
|
if (is_link($path)) throw new RuntimeException('Refusing private transport symlink');
|
||||||
|
$tmp=tempnam($dir,'.private-');if ($tmp===false) throw new RuntimeException('Private file failed');
|
||||||
|
chmod($tmp,0600);
|
||||||
|
if (file_put_contents($tmp,$body)!==strlen($body) || !rename($tmp,$path)) {
|
||||||
|
if (is_file($tmp)) unlink($tmp);
|
||||||
|
throw new RuntimeException('Private file failed');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
DplOnboarding\writeJson($dir.'/qa-private.json',['user_id'=>$qa->id,'uuid'=>$qa->uuid,'subscribe_token'=>$qa->token,
|
||||||
|
'host'=>$parameters['relay-host'],'port'=>$parameters['relay-port'],'sni'=>$parameters['tls-domain']]);
|
||||||
|
echo json_encode(['key'=>$m['key'],'node_id'=>$m['node_id'],'machine_id'=>$m['machine_id'],
|
||||||
|
'group_id'=>$m['group_id'],'test_user_id'=>$m['test_user_id']],JSON_THROW_ON_ERROR).PHP_EOL;
|
||||||
|
} catch (Throwable $e) {
|
||||||
|
$reason=get_class($e)===RuntimeException::class ? $e->getMessage() : 'Inspect manifest/DB state before retrying';
|
||||||
|
fwrite(STDERR,'Staging stopped: '.$reason.PHP_EOL);exit(1);
|
||||||
}
|
}
|
||||||
file_put_contents("$dir/before.json", json_encode([
|
|
||||||
'servers' => DB::table('v2_server')->get(),
|
|
||||||
'groups' => DB::table('v2_server_group')->get(),
|
|
||||||
'machines' => DB::table('v2_server_machine')->get(),
|
|
||||||
], JSON_THROW_ON_ERROR));
|
|
||||||
|
|
||||||
$manifest = DB::transaction(function () use ($opts) {
|
|
||||||
$group = new ServerGroup();
|
|
||||||
$group->name = 'dpl 接入验收 ' . $opts['key'];
|
|
||||||
$group->save();
|
|
||||||
$machine = ServerMachine::create([
|
|
||||||
'name' => 'dpl-' . $opts['key'], 'notes' => 'Independent AnyTLS service; existing XrayR preserved',
|
|
||||||
'is_active' => true, 'token' => ServerMachine::generateToken(),
|
|
||||||
]);
|
|
||||||
$node = Server::create([
|
|
||||||
'name' => $opts['name'], 'type' => 'anytls', 'host' => $opts['relay-host'],
|
|
||||||
'port' => (string)$opts['relay-port'], 'server_port' => (string)$opts['listen-port'],
|
|
||||||
'group_ids' => [(string)$group->id], 'route_ids' => [], 'tags' => [],
|
|
||||||
'rate' => 1, 'show' => true, 'enabled' => true, 'machine_id' => $machine->id, 'sort' => 1,
|
|
||||||
'protocol_settings' => ['tls' => ['server_name' => $opts['tls-domain'], 'allow_insecure' => false]],
|
|
||||||
'cert_config' => [
|
|
||||||
'cert_mode' => 'file', 'domain' => $opts['tls-domain'],
|
|
||||||
'cert_file' => '/etc/letsencrypt/live/' . $opts['tls-domain'] . '/fullchain.pem',
|
|
||||||
'key_file' => '/etc/letsencrypt/live/' . $opts['tls-domain'] . '/privkey.pem',
|
|
||||||
],
|
|
||||||
]);
|
|
||||||
$user = new User([
|
|
||||||
'email' => 'node-qa-' . $opts['key'] . '-' . Str::lower(Str::random(8)) . '@example.invalid',
|
|
||||||
'password' => password_hash(Str::random(40), PASSWORD_BCRYPT),
|
|
||||||
'uuid' => (string)Str::uuid(), 'token' => Str::random(32),
|
|
||||||
'group_id' => $group->id, 'transfer_enable' => 104857600,
|
|
||||||
'expired_at' => time() + 86400, 'banned' => 0, 'is_admin' => 0,
|
|
||||||
'remind_expire' => 0, 'remind_traffic' => 0, 'device_limit' => 2,
|
|
||||||
]);
|
|
||||||
$user->saveQuietly();
|
|
||||||
return ['key' => $opts['key'], 'node_id' => $node->id, 'machine_id' => $machine->id,
|
|
||||||
'group_id' => $group->id, 'test_user_id' => $user->id];
|
|
||||||
});
|
|
||||||
file_put_contents("$dir/manifest.json", json_encode($manifest, JSON_THROW_ON_ERROR));
|
|
||||||
$machine = ServerMachine::findOrFail($manifest['machine_id']);
|
|
||||||
$user = User::findOrFail($manifest['test_user_id']);
|
|
||||||
$config = "panel:\n url: \"https://dp-link.net\"\nmachine:\n machine_id: {$machine->id}\n token_env: DPL_MACHINE_TOKEN\nkernel:\n type: singbox\n config_dir: /var/lib/dpl-node/kernel\nlog:\n level: info\nruntime:\n gomemlimit: 384MiB\n gogc: 75\nnode:\n push_interval: 30\n pull_interval: 30\n";
|
|
||||||
file_put_contents("$dir/node-config.yml", $config);
|
|
||||||
file_put_contents("$dir/node-secret.env", 'DPL_MACHINE_TOKEN=' . $machine->token . "\n");
|
|
||||||
file_put_contents("$dir/qa-private.json", json_encode([
|
|
||||||
'user_id' => $user->id, 'uuid' => $user->uuid, 'subscribe_token' => $user->token,
|
|
||||||
'host' => $opts['relay-host'], 'port' => (int)$opts['relay-port'], 'sni' => $opts['tls-domain'],
|
|
||||||
], JSON_THROW_ON_ERROR));
|
|
||||||
echo json_encode($manifest, JSON_UNESCAPED_UNICODE) . PHP_EOL;
|
|
||||||
|
|||||||
@ -1,72 +1,240 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Run an isolated SOCKS client with strict TLS; never print QA credentials."""
|
"""Validate private subscriptions; network execution requires explicit --run.
|
||||||
|
|
||||||
|
For a formal multi-node subscription use a private --expectations JSON array:
|
||||||
|
[{"tag": "exact outbound tag", "host": "relay.example", "port": 12345,
|
||||||
|
"sni": "node.example", "uuid": "private password", "expected_exit": "192.0.2.1",
|
||||||
|
"expected_country": "JP"}]. Never print credentials or the subscription.
|
||||||
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
|
import copy
|
||||||
|
import ipaddress
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import pathlib
|
from pathlib import Path
|
||||||
import shutil
|
import shutil
|
||||||
import socket
|
import socket
|
||||||
import subprocess
|
import subprocess
|
||||||
import tempfile
|
import tempfile
|
||||||
import time
|
import time
|
||||||
|
|
||||||
p = argparse.ArgumentParser()
|
|
||||||
p.add_argument('--qa', required=True)
|
class VerificationError(Exception):
|
||||||
p.add_argument('--binary', required=True)
|
pass
|
||||||
p.add_argument('--expected-exit', required=True)
|
|
||||||
p.add_argument('--port', type=int, default=19080)
|
|
||||||
p.add_argument('--subscription')
|
def require(condition, message):
|
||||||
a = p.parse_args()
|
if not condition:
|
||||||
q = json.loads(pathlib.Path(a.qa).read_text())
|
raise VerificationError(message)
|
||||||
c = {'log': {'level': 'error'}, 'dns': {'servers': [{'type': 'local', 'tag': 'local'}]},
|
|
||||||
'inbounds': [{'type': 'mixed', 'listen': '127.0.0.1', 'listen_port': a.port}],
|
|
||||||
'outbounds': [{'type': 'anytls', 'tag': 'qa', 'server': q['host'], 'server_port': q['port'],
|
def read_private(path):
|
||||||
'password': q['uuid'], 'tls': {'enabled': True, 'server_name': q['sni'], 'insecure': False}}],
|
p = Path(path)
|
||||||
'route': {'default_domain_resolver': 'local', 'final': 'qa'}}
|
require(p.is_file() and not p.is_symlink(), 'Private input must be a regular file')
|
||||||
if a.subscription:
|
require(p.stat().st_mode & 0o077 == 0, 'Private input must have no group/other permissions')
|
||||||
c = json.loads(pathlib.Path(a.subscription).read_text())
|
return json.loads(p.read_text())
|
||||||
nodes = [x for x in c['outbounds'] if x.get('type') == 'anytls']
|
|
||||||
assert len(nodes) == 1 and nodes[0]['tls'].get('insecure') is False
|
|
||||||
assert nodes[0]['server'] == q['host'] and nodes[0]['server_port'] == q['port']
|
def validate(config, expectations, exact=True):
|
||||||
assert nodes[0]['password'] == q['uuid']
|
require(isinstance(config, dict) and isinstance(config.get('outbounds'), list), 'Missing outbounds')
|
||||||
c['inbounds'] = [{'type': 'mixed', 'listen': '127.0.0.1', 'listen_port': a.port}]
|
all_tags = [o.get('tag') for o in config['outbounds'] if isinstance(o, dict)]
|
||||||
c['log'] = {'level': 'error'}
|
require(len(all_tags) == len(config['outbounds']) and all(isinstance(t, str) and t for t in all_tags), 'Each outbound needs a tag')
|
||||||
os.umask(0o077)
|
require(len(set(all_tags)) == len(all_tags), 'Duplicate outbound tags')
|
||||||
with tempfile.TemporaryDirectory(prefix='dpl-client-') as d:
|
nodes = {o['tag']: o for o in config['outbounds'] if o.get('type') == 'anytls'}
|
||||||
config = pathlib.Path(d) / 'client.json'
|
require(isinstance(expectations, list) and expectations, 'Nonempty expectations required')
|
||||||
config.write_text(json.dumps(c))
|
selected = []
|
||||||
env = dict(os.environ, GOMEMLIMIT='64MiB', GOGC='75')
|
for spec in expectations:
|
||||||
with (pathlib.Path(d) / 'client.log').open('wb') as log:
|
require(isinstance(spec, dict), 'Invalid expectation')
|
||||||
proc = subprocess.Popen([a.binary, 'run', '-c', str(config)], cwd=d, env=env, stdout=log, stderr=log)
|
for key in ['tag', 'host', 'sni', 'uuid', 'expected_exit', 'expected_country']:
|
||||||
|
require(isinstance(spec.get(key), str) and bool(spec[key]), 'Expectation missing required field')
|
||||||
|
require(isinstance(spec.get('port'), int) and not isinstance(spec['port'], bool) and 1 <= spec['port'] <= 65535, 'Invalid relay port')
|
||||||
try:
|
try:
|
||||||
for _ in range(50):
|
ipaddress.ip_address(spec['expected_exit'])
|
||||||
assert proc.poll() is None, 'Client exited during startup'
|
except ValueError as exc:
|
||||||
try:
|
raise VerificationError('Invalid expected exit IP') from exc
|
||||||
with socket.create_connection(('127.0.0.1', a.port), 0.1):
|
require(len(spec['expected_country']) == 2 and spec['expected_country'].isalpha() and spec['expected_country'].isupper(), 'Expected country must be two uppercase letters')
|
||||||
break
|
tag = spec['tag']
|
||||||
except OSError:
|
require(tag in nodes and tag not in selected, 'Missing/duplicate selected node')
|
||||||
time.sleep(0.1)
|
node = nodes[tag]
|
||||||
command = ['curl', '-q', '--fail', '--silent', '--show-error', '--max-time', '30',
|
tls = node.get('tls') or {}
|
||||||
'--socks5-hostname', f'127.0.0.1:{a.port}']
|
require(tls.get('enabled') is True and tls.get('insecure') is False, 'Strict TLS must be explicit')
|
||||||
result = subprocess.run(command + ['https://api.ipify.org'], capture_output=True, text=True, check=True)
|
require(node.get('server') == spec['host'] and node.get('server_port') == spec['port'], 'Relay endpoint mismatch')
|
||||||
ip = result.stdout.strip()
|
require(tls.get('server_name') == spec['sni'], 'SNI mismatch')
|
||||||
assert ip == a.expected_exit, 'Unexpected exit IP'
|
require(node.get('password') == spec['uuid'], 'Subscription password mismatch')
|
||||||
result = subprocess.run(command + ['https://www.cloudflare.com/cdn-cgi/trace'], capture_output=True, text=True, check=True)
|
selected.append(tag)
|
||||||
trace = dict(line.split('=', 1) for line in result.stdout.splitlines() if '=' in line)
|
if exact:
|
||||||
assert trace.get('ip') == a.expected_exit, 'Second exit check mismatch'
|
require(set(selected) == set(nodes), 'Expectations must cover every AnyTLS node exactly')
|
||||||
print(json.dumps({'anytls': True, 'relay': q['host'], 'port': q['port'], 'strict_tls': True,
|
return selected
|
||||||
'exit_ip': ip, 'country': trace.get('loc'), 'https_tls': trace.get('tls')}, ensure_ascii=False))
|
|
||||||
|
|
||||||
|
def probe_config(config, spec, port):
|
||||||
|
result = copy.deepcopy(config)
|
||||||
|
result['inbounds'] = [{'type': 'mixed', 'listen': '127.0.0.1', 'listen_port': port}]
|
||||||
|
result['log'] = {'level': 'error'}
|
||||||
|
# Preserve DNS, remote rulesets and all original outbounds. Only the two
|
||||||
|
# diagnostic destinations are pinned; ordinary subscription routing remains.
|
||||||
|
route = result.setdefault('route', {})
|
||||||
|
rules = route.setdefault('rules', [])
|
||||||
|
require(isinstance(rules, list), 'Invalid route rules')
|
||||||
|
rules.insert(0, {'domain': ['api.ipify.org', 'www.cloudflare.com'],
|
||||||
|
'action': 'route', 'outbound': spec['tag']})
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def stop(proc):
|
||||||
|
if proc is None or proc.poll() is not None:
|
||||||
|
return
|
||||||
|
proc.terminate()
|
||||||
|
try:
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
proc.kill()
|
||||||
|
proc.wait(timeout=5)
|
||||||
|
|
||||||
|
|
||||||
|
def run_node(config, spec, binary, port, diagnostic_dir):
|
||||||
|
require(1024 <= port <= 65535, 'Invalid local SOCKS port')
|
||||||
|
with socket.socket() as free:
|
||||||
|
try:
|
||||||
|
free.bind(('127.0.0.1', port))
|
||||||
|
except OSError as exc:
|
||||||
|
raise VerificationError('Local SOCKS port is occupied') from exc
|
||||||
|
with tempfile.TemporaryDirectory(prefix='dpl-client-', dir=diagnostic_dir) as tmp:
|
||||||
|
temporary = Path(tmp)
|
||||||
|
path = temporary / 'client.json'
|
||||||
|
path.write_text(json.dumps(probe_config(config, spec, port)))
|
||||||
|
path.chmod(0o600)
|
||||||
|
env = dict(os.environ, GOMEMLIMIT='64MiB', GOGC='75')
|
||||||
|
proc = None
|
||||||
|
log_path = temporary / 'client.log'
|
||||||
|
try:
|
||||||
|
with log_path.open('wb') as log:
|
||||||
|
subprocess.run([binary, 'check', '-c', str(path)], env=env,
|
||||||
|
cwd=tmp, stdout=log, stderr=log, check=True, timeout=30)
|
||||||
|
proc = subprocess.Popen([binary, 'run', '-c', str(path)], env=env,
|
||||||
|
cwd=tmp, stdout=log, stderr=log)
|
||||||
|
ready = False
|
||||||
|
for _ in range(100):
|
||||||
|
require(proc.poll() is None, 'Client exited during startup')
|
||||||
|
try:
|
||||||
|
with socket.create_connection(('127.0.0.1', port), 0.1):
|
||||||
|
ready = True
|
||||||
|
break
|
||||||
|
except OSError:
|
||||||
|
time.sleep(0.1)
|
||||||
|
require(ready, 'Client startup timed out')
|
||||||
|
command = ['curl', '-q', '--fail', '--silent', '--show-error', '--max-time', '30',
|
||||||
|
'--socks5-hostname', f'127.0.0.1:{port}', '--noproxy', '']
|
||||||
|
ip = subprocess.run(command + ['https://api.ipify.org'], capture_output=True,
|
||||||
|
text=True, check=True, timeout=35).stdout.strip()
|
||||||
|
require(ipaddress.ip_address(ip) == ipaddress.ip_address(spec['expected_exit']), 'Unexpected exit IP')
|
||||||
|
response = subprocess.run(command + ['https://www.cloudflare.com/cdn-cgi/trace'],
|
||||||
|
capture_output=True, text=True, check=True, timeout=35)
|
||||||
|
trace = dict(line.split('=', 1) for line in response.stdout.splitlines() if '=' in line)
|
||||||
|
require(trace.get('ip') == ip, 'Second exit check mismatch')
|
||||||
|
require(trace.get('loc') == spec['expected_country'], 'Exit country mismatch')
|
||||||
|
require(trace.get('tls', '').startswith('TLSv'), 'Target HTTPS TLS not observed')
|
||||||
|
return {'tag': spec['tag'], 'strict_tls': True, 'exit_ip': ip,
|
||||||
|
'country': trace['loc'], 'https_tls': trace['tls'], 'live': True,
|
||||||
|
'relay': spec['host'], 'relay_port': spec['port'], 'sni': spec['sni']}
|
||||||
except Exception:
|
except Exception:
|
||||||
log.flush()
|
# Logs may contain private config context; retain only in private task directory.
|
||||||
failure = pathlib.Path(a.qa).parent / 'client-failure.log'
|
stop(proc)
|
||||||
shutil.copyfile(pathlib.Path(d) / 'client.log', failure)
|
if log_path.exists():
|
||||||
failure.chmod(0o600)
|
fd, name = tempfile.mkstemp(prefix='client-failure-', suffix='.log', dir=diagnostic_dir)
|
||||||
print(f'Client failed; protected diagnostic log: {failure}')
|
os.close(fd)
|
||||||
|
failure = Path(name)
|
||||||
|
shutil.copyfile(log_path, failure)
|
||||||
|
failure.chmod(0o600)
|
||||||
raise
|
raise
|
||||||
finally:
|
finally:
|
||||||
proc.terminate()
|
stop(proc)
|
||||||
try:
|
|
||||||
proc.wait(timeout=5)
|
|
||||||
except subprocess.TimeoutExpired:
|
def write_report(path, report):
|
||||||
proc.kill()
|
destination = Path(path)
|
||||||
proc.wait()
|
require(destination.parent.is_dir() and destination.parent.stat().st_mode & 0o077 == 0,
|
||||||
|
'Report directory must be private')
|
||||||
|
require(not destination.is_symlink(), 'Refusing report symlink')
|
||||||
|
with tempfile.NamedTemporaryFile(mode='w', dir=destination.parent, delete=False) as stream:
|
||||||
|
temporary = Path(stream.name)
|
||||||
|
try:
|
||||||
|
os.chmod(temporary, 0o600)
|
||||||
|
json.dump(report, stream, ensure_ascii=False)
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
os.replace(temporary, destination)
|
||||||
|
finally:
|
||||||
|
if temporary.exists():
|
||||||
|
temporary.unlink()
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
sources = parser.add_mutually_exclusive_group(required=True)
|
||||||
|
sources.add_argument('--qa')
|
||||||
|
sources.add_argument('--expectations')
|
||||||
|
parser.add_argument('--subscription', required=True)
|
||||||
|
parser.add_argument('--binary')
|
||||||
|
parser.add_argument('--expected-exit')
|
||||||
|
parser.add_argument('--expected-country')
|
||||||
|
parser.add_argument('--target-tag')
|
||||||
|
parser.add_argument('--report', help='Private JSON result file; required for --run')
|
||||||
|
parser.add_argument('--port', type=int, default=19080)
|
||||||
|
execution = parser.add_mutually_exclusive_group()
|
||||||
|
execution.add_argument('--run', action='store_true')
|
||||||
|
execution.add_argument('--validate-only', action='store_true')
|
||||||
|
args = parser.parse_args()
|
||||||
|
os.umask(0o077)
|
||||||
|
try:
|
||||||
|
require(not args.run or args.report, '--run requires a private --report destination')
|
||||||
|
if args.report:
|
||||||
|
destination = Path(args.report)
|
||||||
|
inputs = [args.subscription, args.qa, args.expectations]
|
||||||
|
require(all(destination.resolve() != Path(item).resolve() for item in inputs if item), 'Report must not overwrite a private input')
|
||||||
|
require(destination.parent.is_dir() and destination.parent.stat().st_mode & 0o077 == 0 and not destination.is_symlink(), 'Private regular report destination required')
|
||||||
|
if args.report:
|
||||||
|
write_report(args.report, {'state': 'verification_pending', 'verified_at': None})
|
||||||
|
config = read_private(args.subscription)
|
||||||
|
qa_user_id = None
|
||||||
|
source = args.expectations or args.qa
|
||||||
|
if args.expectations:
|
||||||
|
require(not any([args.expected_exit, args.expected_country, args.target_tag]), 'Put each node expectation in the private expectations file')
|
||||||
|
specs = read_private(args.expectations)
|
||||||
|
else:
|
||||||
|
qa = read_private(args.qa)
|
||||||
|
qa_user_id = qa.get('user_id')
|
||||||
|
require(args.expected_exit and args.expected_country, 'QA requires explicit expected-exit and expected-country')
|
||||||
|
candidates = [n for n in config.get('outbounds', []) if n.get('type') == 'anytls'
|
||||||
|
and n.get('server') == qa.get('host') and n.get('server_port') == qa.get('port')
|
||||||
|
and (not args.target_tag or n.get('tag') == args.target_tag)]
|
||||||
|
require(len(candidates) == 1, 'QA target ambiguous/missing; supply exact target-tag')
|
||||||
|
specs = [{'tag': candidates[0]['tag'], 'host': qa.get('host'), 'port': qa.get('port'),
|
||||||
|
'sni': qa.get('sni'), 'uuid': qa.get('uuid'), 'expected_exit': args.expected_exit,
|
||||||
|
'expected_country': args.expected_country}]
|
||||||
|
tags = validate(config, specs, exact=bool(args.expectations))
|
||||||
|
results = []
|
||||||
|
if args.run:
|
||||||
|
require(args.binary and Path(args.binary).is_file(), 'Explicit existing client binary required')
|
||||||
|
binary = str(Path(args.binary).resolve())
|
||||||
|
folder = Path(source).resolve().parent
|
||||||
|
require(folder.stat().st_mode & 0o077 == 0, 'Live diagnostics directory must be private')
|
||||||
|
for spec in specs:
|
||||||
|
results.append(run_node(config, spec, binary, args.port, str(folder)))
|
||||||
|
report = {'state': 'live_verified' if args.run else 'structure_validated',
|
||||||
|
'verified_at': int(time.time()) if args.run else None,
|
||||||
|
'qa_user_id': qa_user_id, 'selected_nodes': tags, 'results': results,
|
||||||
|
'scope': 'Full subscription retained; only diagnostic destination routes pinned. No traffic/account/GUI acceptance implied.'}
|
||||||
|
if args.report:
|
||||||
|
write_report(args.report, report)
|
||||||
|
print(json.dumps(report, ensure_ascii=False))
|
||||||
|
return 0
|
||||||
|
except (Exception, KeyboardInterrupt) as exc:
|
||||||
|
# No traceback or subprocess stderr: either can contain credentials.
|
||||||
|
message = str(exc) if isinstance(exc, VerificationError) else 'Verification failed; inspect protected diagnostics'
|
||||||
|
print(json.dumps({'state': 'failed', 'reason': message}), file=__import__('sys').stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
raise SystemExit(main())
|
||||||
|
|||||||
71
tests/node-onboarding/AcceptanceUserIsolatedTest.php
Normal file
71
tests/node-onboarding/AcceptanceUserIsolatedTest.php
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
<?php
|
||||||
|
/** Real Eloquent/SQLite and real ServerService. No dotenv, app providers, Redis or HTTP. */
|
||||||
|
require_once dirname(__DIR__,2).'/vendor/autoload.php';
|
||||||
|
require_once dirname(__DIR__,2).'/scripts/node-onboarding/panel-acceptance.php';
|
||||||
|
use App\Models\{User,Server};
|
||||||
|
use DplOnboarding\Acceptance;
|
||||||
|
use Illuminate\Container\Container;
|
||||||
|
use Illuminate\Database\Capsule\Manager as Capsule;
|
||||||
|
use Illuminate\Support\Facades\Facade;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
final class AcceptanceUserIsolatedTest extends TestCase {
|
||||||
|
private Capsule $db; private string $dir;
|
||||||
|
protected function setUp():void {
|
||||||
|
$app=new Container();Container::setInstance($app);Facade::clearResolvedInstances();Facade::setFacadeApplication($app);$app->instance('app',$app);
|
||||||
|
$app->instance('config',new Illuminate\Config\Repository(['cache'=>['default'=>'array','stores'=>['array'=>['driver'=>'array']]],'app'=>['key'=>'offline-test']]));
|
||||||
|
$events=new Illuminate\Events\Dispatcher($app);$app->instance('events',$events);
|
||||||
|
$this->db=new Capsule($app);$this->db->addConnection(['driver'=>'sqlite','database'=>':memory:']);$this->db->setEventDispatcher($events);$this->db->setAsGlobal();$this->db->bootEloquent();$app->instance('db',$this->db->getDatabaseManager());$app->singleton('cache',fn($a)=>new Illuminate\Cache\CacheManager($a));
|
||||||
|
$s=$this->db->schema();
|
||||||
|
$s->create('v2_user',function($t){$t->increments('id');$t->string('email')->unique();foreach(['uuid','token','password'] as $x)$t->string($x);$t->integer('group_id')->nullable();foreach(['u','d'] as $x)$t->integer($x)->default(0);$t->integer('transfer_enable');$t->integer('expired_at')->nullable();foreach(['banned','is_admin','remind_expire','remind_traffic'] as $x)$t->boolean($x)->default(0);foreach(['device_limit','speed_limit'] as $x)$t->integer($x)->nullable();$t->integer('created_at')->nullable();$t->integer('updated_at')->nullable();});
|
||||||
|
$s->create('v2_server_group',function($t){$t->increments('id');$t->string('name');});Capsule::table('v2_server_group')->insert([['id'=>1,'name'=>'formal'],['id'=>2,'name'=>'other']]);
|
||||||
|
$s->create('v2_server',function($t){$t->increments('id');foreach(['name','type','host','port'] as $x)$t->string($x);foreach(['group_ids','protocol_settings'] as $x)$t->text($x);$t->boolean('enabled')->default(1);$t->boolean('show')->default(1);$t->integer('sort')->default(1);foreach(['u','d'] as $x)$t->integer($x)->default(0);$t->integer('transfer_enable')->nullable();$t->float('rate')->default(1);$t->timestamps();});
|
||||||
|
$this->node();$this->dir=sys_get_temp_dir().'/dpl-acceptance-'.bin2hex(random_bytes(8));mkdir($this->dir,0700);
|
||||||
|
}
|
||||||
|
protected function tearDown():void{foreach(glob($this->dir.'/*') as $p)unlink($p);rmdir($this->dir);$this->db->getDatabaseManager()->disconnect();}
|
||||||
|
private function node():void {$i=Server::count()+1;Server::create(['name'=>'node'.$i,'type'=>'anytls','host'=>'relay.example.invalid','port'=>(string)(47000+$i),'group_ids'=>['1'],'protocol_settings'=>['tls'=>['server_name'=>'n'.$i.'.example.invalid','allow_insecure'=>false]]]);}
|
||||||
|
private function user(array $more=[]):User{return User::create(array_replace(['email'=>'existing@example.invalid','uuid'=>'existing-uuid','token'=>'existing-token','password'=>'unused','group_id'=>1,'transfer_enable'=>1000000,'expired_at'=>null],$more));}
|
||||||
|
private function reject(callable $f):void{$caught=false;try{$f();}catch(RuntimeException $e){$caught=true;}self::assertTrue($caught);}
|
||||||
|
public function testPreviewDoesNotCreateUserOrLease():void {
|
||||||
|
self::assertSame('would_create',Acceptance::prepare($this->dir,1,false)['state']);self::assertSame(0,User::count());self::assertFileDoesNotExist($this->dir.'/acceptance-user.json');
|
||||||
|
}
|
||||||
|
public function testCreatesBoundedNonAdminAndUsesDynamicActualNodeSet():void {
|
||||||
|
$r=Acceptance::prepare($this->dir,1,true);$u=User::findOrFail($r['id']);self::assertTrue($r['owned']);self::assertFalse($u->is_admin);self::assertSame(104857600,(int)$u->transfer_enable);self::assertLessThanOrEqual(time()+3600,(int)$u->expired_at);self::assertGreaterThan(time()+3500,(int)$u->expired_at);
|
||||||
|
self::assertCount(1,Acceptance::source($this->dir,1)['nodes']);$this->node();self::assertCount(2,Acceptance::source($this->dir,1)['nodes']);self::assertSame($u->id,Acceptance::prepare($this->dir,1,true)['id']);self::assertSame(1,User::count());
|
||||||
|
Acceptance::cleanup($this->dir,true);self::assertTrue($u->fresh()->banned);self::assertTrue(App\Services\ServerService::getAvailableUsers(Server::first())->isEmpty());self::assertArrayNotHasKey('token',Acceptance::source($this->dir,1,true));$this->reject(fn()=>Acceptance::source($this->dir,1));
|
||||||
|
}
|
||||||
|
public function testExistingEligibleAdminIsUsedWithoutPrivilegeOrPasswordChange():void {
|
||||||
|
$u=$this->user(['is_admin'=>1]);$before=(array)Capsule::table('v2_user')->where('id',$u->id)->first();$r=Acceptance::prepare($this->dir,1,true);self::assertFalse($r['owned']);self::assertSame($u->id,$r['id']);Acceptance::source($this->dir,1);Acceptance::cleanup($this->dir,true);self::assertSame($before,(array)Capsule::table('v2_user')->where('id',$u->id)->first());
|
||||||
|
}
|
||||||
|
public function testExpiredExhaustedBannedWrongGroupAreNotReused():void {
|
||||||
|
foreach([['expired_at'=>time()-1],['u'=>1000000],['banned'=>1],['group_id'=>2]] as $i=>$change)$this->user($change+['email'=>'existing'.$i.'@example.invalid']);
|
||||||
|
$r=Acceptance::prepare($this->dir,1,true);self::assertTrue($r['owned']);self::assertSame(5,User::count());
|
||||||
|
}
|
||||||
|
public function testLostCommitResponseRecoversSameUserAndPreparedRollbackRecoversSameIdentity():void {
|
||||||
|
$r=Acceptance::prepare($this->dir,1,true);$p=$this->dir.'/acceptance-user.json';$j=DplOnboarding\readJson($p);$j['state']='prepared';unset($j['id']);DplOnboarding\writeJson($p,$j);
|
||||||
|
self::assertSame($r['id'],Acceptance::prepare($this->dir,1,true)['id']);self::assertSame(1,User::count());
|
||||||
|
User::find($r['id'])->delete();DplOnboarding\writeJson($p,$j);$again=Acceptance::prepare($this->dir,1,true);self::assertSame(1,User::count());self::assertSame($j['uuid'],User::find($again['id'])->uuid);
|
||||||
|
}
|
||||||
|
public function testClosedOwnedUserNeverReactivatedOnResume():void {
|
||||||
|
$old=Acceptance::prepare($this->dir,1,true)['id'];Acceptance::cleanup($this->dir,true);Acceptance::cleanup($this->dir,true);
|
||||||
|
$new=Acceptance::prepare($this->dir,1,true)['id'];self::assertNotSame($old,$new);self::assertTrue(User::find($old)->banned);self::assertFalse(User::find($new)->banned);self::assertFileExists($this->dir.'/acceptance-closed-'.$old.'.json');Acceptance::cleanup($this->dir,true);
|
||||||
|
}
|
||||||
|
public function testCleanupRefusesDriftAndNeverBansSomeoneElsesAccount():void {
|
||||||
|
$id=Acceptance::prepare($this->dir,1,true)['id'];$u=User::find($id);$u->is_admin=true;$u->saveQuietly();$this->reject(fn()=>Acceptance::cleanup($this->dir,true));self::assertFalse($u->fresh()->banned);
|
||||||
|
}
|
||||||
|
public function testNoEffectivelyEligibleExistingUserCreatesBoundedTestAccount():void {
|
||||||
|
$old=$this->user(['is_admin'=>1]);$id=$old->id;
|
||||||
|
App\Services\Plugin\HookManager::registerFilter('server.users.get',fn($users)=>$users->where('id','!=',$id)->values());
|
||||||
|
$r=Acceptance::prepare($this->dir,1,true);self::assertTrue($r['owned']);self::assertNotSame($id,$r['id']);self::assertFalse(User::find($r['id'])->is_admin);Acceptance::source($this->dir,1);Acceptance::cleanup($this->dir,true);
|
||||||
|
}
|
||||||
|
public function testEffectivePluginExclusionBlocksAcceptanceSource():void {
|
||||||
|
Acceptance::prepare($this->dir,1,true);
|
||||||
|
App\Services\Plugin\HookManager::registerFilter('server.users.get',fn($users)=>collect());
|
||||||
|
$this->reject(fn()=>Acceptance::source($this->dir,1));Acceptance::cleanup($this->dir,true);
|
||||||
|
}
|
||||||
|
public function testFailureFinallyCleanupAndMissingGroup():void {
|
||||||
|
$this->reject(fn()=>Acceptance::prepare($this->dir,99,true));$id=Acceptance::prepare($this->dir,1,true)['id'];
|
||||||
|
try{throw new RuntimeException('Injected client failure');}catch(RuntimeException $e){}finally{Acceptance::cleanup($this->dir,true);}
|
||||||
|
self::assertTrue(User::find($id)->banned);
|
||||||
|
}
|
||||||
|
}
|
||||||
116
tests/node-onboarding/PanelChangeIsolatedTest.php
Normal file
116
tests/node-onboarding/PanelChangeIsolatedTest.php
Normal file
@ -0,0 +1,116 @@
|
|||||||
|
<?php
|
||||||
|
/** Standalone: php vendor/bin/phpunit --no-configuration tests/node-onboarding/PanelChangeIsolatedTest.php
|
||||||
|
* Real Eloquent + SQLite :memory:, no application bootstrap or dotenv.
|
||||||
|
* NodeSync is a test double; no Redis, queue, HTTP or production connections.
|
||||||
|
*/
|
||||||
|
namespace App\Services {
|
||||||
|
final class NodeSyncService {
|
||||||
|
public static array $calls=[];
|
||||||
|
public static bool $fail=false;
|
||||||
|
public static function notifyFullSync(int $id): void {
|
||||||
|
self::$calls[]=['node',$id];
|
||||||
|
if (self::$fail) throw new \RuntimeException('Injected post-commit notification failure');
|
||||||
|
}
|
||||||
|
public static function notifyMachineNodesChanged(int $id): void { self::$calls[]=['machine',$id]; }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
namespace {
|
||||||
|
require_once dirname(__DIR__,2).'/vendor/autoload.php';
|
||||||
|
require_once dirname(__DIR__,2).'/scripts/node-onboarding/operations.php';
|
||||||
|
|
||||||
|
use App\Models\{Server,User};
|
||||||
|
use App\Services\NodeSyncService;
|
||||||
|
use DplOnboarding\PanelChange;
|
||||||
|
use Illuminate\Container\Container;
|
||||||
|
use Illuminate\Database\Capsule\Manager as Capsule;
|
||||||
|
use Illuminate\Events\Dispatcher;
|
||||||
|
use Illuminate\Support\Facades\Facade;
|
||||||
|
use PHPUnit\Framework\TestCase;
|
||||||
|
|
||||||
|
final class PanelChangeIsolatedTest extends TestCase {
|
||||||
|
private Capsule $db;
|
||||||
|
private string $dir;
|
||||||
|
private array $request;
|
||||||
|
protected function setUp(): void {
|
||||||
|
$app=new Container(); Container::setInstance($app); Facade::clearResolvedInstances(); Facade::setFacadeApplication($app);
|
||||||
|
$app->instance('config',new \Illuminate\Config\Repository(['cache'=>['default'=>'array','stores'=>['array'=>['driver'=>'array']]],'app'=>['key'=>'test']]));
|
||||||
|
$events=new Dispatcher($app); $app->instance('events',$events);
|
||||||
|
$this->db=new Capsule($app); $this->db->addConnection(['driver'=>'sqlite','database'=>':memory:','foreign_key_constraints'=>true]);
|
||||||
|
$this->db->setEventDispatcher($events); $this->db->setAsGlobal(); $this->db->bootEloquent();
|
||||||
|
$app->instance('db',$this->db->getDatabaseManager());
|
||||||
|
$app->singleton('cache',fn($app)=>new \Illuminate\Cache\CacheManager($app));
|
||||||
|
$schema=$this->db->schema();
|
||||||
|
$schema->create('v2_server',function($t){$t->increments('id');$t->string('name');$t->string('type');$t->integer('machine_id');$t->text('group_ids');$t->text('protocol_settings');$t->string('host');$t->string('port');$t->boolean('enabled');$t->boolean('show');$t->timestamps();});
|
||||||
|
$schema->create('v2_user',function($t){$t->increments('id');$t->string('email');$t->boolean('is_admin')->default(false);$t->integer('group_id')->nullable();$t->integer('plan_id')->nullable();$t->integer('u')->default(0);$t->integer('d')->default(0);$t->boolean('banned')->default(false);$t->integer('created_at')->nullable();$t->integer('updated_at')->nullable();});
|
||||||
|
$schema->create('v2_server_group',function($t){$t->increments('id');$t->string('name');$t->integer('created_at')->nullable();$t->integer('updated_at')->nullable();});
|
||||||
|
$schema->create('v2_plan',function($t){$t->increments('id');$t->integer('group_id')->nullable();$t->string('name');$t->integer('price');$t->integer('created_at')->nullable();$t->integer('updated_at')->nullable();});
|
||||||
|
$schema->create('v2_settings',function($t){$t->increments('id');$t->string('name')->unique();$t->text('value')->nullable();$t->timestamps();});
|
||||||
|
Capsule::table('v2_server_group')->insert([['id'=>1,'name'=>'formal'],['id'=>2,'name'=>'qa']]);
|
||||||
|
Capsule::table('v2_plan')->insert(['id'=>1,'name'=>'protected-plan','price'=>1234,'group_id'=>1]);
|
||||||
|
Capsule::table('v2_user')->insert([['id'=>1,'email'=>'node-qa-test-key-a@example.invalid','group_id'=>2,'u'=>10,'d'=>20],['id'=>2,'email'=>'formal@example.invalid','group_id'=>1,'u'=>200,'d'=>300]]);
|
||||||
|
Capsule::table('v2_settings')->insert(['name'=>'subscribe_url','value'=>'https://original.example.invalid']);
|
||||||
|
foreach ([1,2] as $id) Server::create(['id'=>$id,'name'=>'node-'.$id,'type'=>'anytls','machine_id'=>$id,'group_ids'=>[(string)$id], 'host'=>'relay.example.invalid','port'=>(string)(47000+$id),'enabled'=>true,'show'=>true,'protocol_settings'=>['tls'=>['server_name'=>'node.example.invalid','allow_insecure'=>false]]]);
|
||||||
|
// Guarded IDs auto-increment to 1/2, matching the manifest.
|
||||||
|
\Illuminate\Support\Facades\Cache::put(\App\Utils\CacheKey::get('SERVER_ANYTLS_LAST_CHECK_AT',2),time());
|
||||||
|
$this->dir=sys_get_temp_dir().'/dpl-isolated-'.bin2hex(random_bytes(8)); mkdir($this->dir,0700);
|
||||||
|
\DplOnboarding\writeJson($this->dir.'/manifest.json',['key'=>'test-key','node_id'=>2,'machine_id'=>2,'group_id'=>2,'test_user_id'=>1]);
|
||||||
|
$this->request=['mode'=>'append','group'=>'1','verification'=>$this->dir.'/report.json'];
|
||||||
|
$this->evidence(); NodeSyncService::$calls=[]; NodeSyncService::$fail=false;
|
||||||
|
}
|
||||||
|
private function evidence(array $overrides=[]): void {
|
||||||
|
\DplOnboarding\writeJson($this->dir.'/report.json',array_replace(['state'=>'live_verified','qa_user_id'=>1,'verified_at'=>time(),'results'=>[['live'=>true,'strict_tls'=>true,'relay'=>'relay.example.invalid','relay_port'=>47002,'sni'=>'node.example.invalid']]],$overrides));
|
||||||
|
}
|
||||||
|
protected function tearDown(): void { foreach(glob($this->dir.'/*') as $p) unlink($p);rmdir($this->dir);$this->db->getDatabaseManager()->disconnect();NodeSyncService::$fail=false; }
|
||||||
|
private function snapshot(): array { $all=[];foreach(['v2_server','v2_user','v2_plan','v2_settings','v2_server_group'] as $t)$all[$t]=Capsule::table($t)->orderBy('id')->get()->toJson();return $all; }
|
||||||
|
private function protectedSnapshot(): array { return [Capsule::table('v2_server')->where('id',1)->first(),Capsule::table('v2_user')->where('id',2)->first(),Capsule::table('v2_plan')->get()->toJson(),Capsule::table('v2_settings')->get()->toJson(),Capsule::table('v2_server_group')->get()->toJson()]; }
|
||||||
|
private function rejects(callable $f): void { $rejected=false;try{$f();}catch(\RuntimeException $e){$rejected=true;}self::assertTrue($rejected,'Expected safe refusal'); }
|
||||||
|
public function testPreviewAppendIdempotenceAndRollback(): void {
|
||||||
|
$before=$this->snapshot();$protected=$this->protectedSnapshot();
|
||||||
|
self::assertSame('preview',PanelChange::promote($this->dir,$this->request,false)['state']);self::assertSame($before,$this->snapshot());self::assertFileDoesNotExist($this->dir.'/promotion-journal.json');
|
||||||
|
self::assertSame('applied',PanelChange::promote($this->dir,$this->request,true)['state']);self::assertSame(['1'],Server::find(2)->group_ids);self::assertTrue((bool)User::find(1)->banned);self::assertEquals($protected,$this->protectedSnapshot());
|
||||||
|
$after=$this->snapshot();PanelChange::promote($this->dir,$this->request,true);self::assertSame($after,$this->snapshot());
|
||||||
|
self::assertSame('preview',PanelChange::rollback($this->dir,false)['state']);self::assertSame($after,$this->snapshot());
|
||||||
|
PanelChange::rollback($this->dir,true);self::assertSame(['2'],Server::find(2)->group_ids);self::assertFalse(Server::find(2)->enabled);self::assertFalse(Server::find(2)->show);self::assertTrue((bool)User::find(1)->banned);self::assertEquals($protected,$this->protectedSnapshot());
|
||||||
|
$rolled=$this->snapshot();PanelChange::rollback($this->dir,true);self::assertSame($rolled,$this->snapshot());
|
||||||
|
}
|
||||||
|
public function testAppendForbidsSharedWritesAndBadEvidence(): void {
|
||||||
|
$before=$this->snapshot();foreach(['plans'=>'1','subscribe-url'=>'https://bad.example.invalid','group-name'=>'bad'] as $key=>$value){$r=$this->request;$r[$key]=$value;$this->rejects(fn()=>PanelChange::promote($this->dir,$r,true));}
|
||||||
|
foreach([['state'=>'structure_validated'],['qa_user_id'=>2],['verified_at'=>time()-901],['verified_at'=>time()+60],['results'=>[]],['results'=>[['live'=>true,'strict_tls'=>true,'relay'=>'wrong.example.invalid','relay_port'=>47002,'sni'=>'node.example.invalid']]],['results'=>[['live'=>true,'strict_tls'=>false,'relay'=>'relay.example.invalid','relay_port'=>47002,'sni'=>'node.example.invalid']]]] as $bad){$this->evidence($bad);$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));self::assertSame($before,$this->snapshot());}
|
||||||
|
}
|
||||||
|
public function testTransactionFailurePreparedBeforeRecovers(): void {
|
||||||
|
$before=$this->snapshot();$once=true;
|
||||||
|
Server::updating(function()use(&$once){if($once){$once=false;throw new \RuntimeException('Injected transaction failure');}});
|
||||||
|
$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));self::assertSame($before,$this->snapshot());self::assertSame('prepared',\DplOnboarding\readJson($this->dir.'/promotion-journal.json')['state']);
|
||||||
|
PanelChange::promote($this->dir,$this->request,true);self::assertTrue((bool)User::find(1)->banned);
|
||||||
|
}
|
||||||
|
public function testCommittedNotificationFailureRecoversWithoutReactivation(): void {
|
||||||
|
NodeSyncService::$fail=true;$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));self::assertTrue((bool)User::find(1)->banned);self::assertSame(['1'],Server::find(2)->group_ids);
|
||||||
|
self::assertSame('prepared',\DplOnboarding\readJson($this->dir.'/promotion-journal.json')['state']);NodeSyncService::$fail=false;
|
||||||
|
$after=$this->snapshot();PanelChange::promote($this->dir,$this->request,true);self::assertSame($after,$this->snapshot());self::assertSame('applied',\DplOnboarding\readJson($this->dir.'/promotion-journal.json')['state']);
|
||||||
|
}
|
||||||
|
public function testRollbackConflictDoesNotOverwriteLaterChange(): void {
|
||||||
|
PanelChange::promote($this->dir,$this->request,true);Capsule::table('v2_server')->where('id',2)->update(['group_ids'=>'["3"]']);$before=$this->snapshot();
|
||||||
|
$this->rejects(fn()=>PanelChange::rollback($this->dir,true));self::assertSame($before,$this->snapshot());self::assertTrue((bool)User::find(1)->banned);
|
||||||
|
}
|
||||||
|
public function testPreparedMixedStateCannotResume(): void {
|
||||||
|
NodeSyncService::$fail=true;$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));NodeSyncService::$fail=false;
|
||||||
|
Capsule::table('v2_user')->where('id',1)->update(['banned'=>0]);$before=$this->snapshot();$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));self::assertSame($before,$this->snapshot());
|
||||||
|
}
|
||||||
|
public function testDifferentRequestAndRollbackPreventRepromotion(): void {
|
||||||
|
PanelChange::promote($this->dir,$this->request,true);$before=$this->snapshot();$r=$this->request;$r['group']='3';
|
||||||
|
$this->rejects(fn()=>PanelChange::promote($this->dir,$r,true));self::assertSame($before,$this->snapshot());
|
||||||
|
PanelChange::rollback($this->dir,true);$rolled=$this->snapshot();$this->rejects(fn()=>PanelChange::promote($this->dir,$this->request,true));self::assertSame($rolled,$this->snapshot());self::assertTrue((bool)User::find(1)->banned);
|
||||||
|
}
|
||||||
|
public function testRollbackNotificationFailureRecoversAfterState(): void {
|
||||||
|
PanelChange::promote($this->dir,$this->request,true);NodeSyncService::$fail=true;
|
||||||
|
$this->rejects(fn()=>PanelChange::rollback($this->dir,true));self::assertSame('prepared',\DplOnboarding\readJson($this->dir.'/rollback-journal.json')['state']);self::assertFalse(Server::find(2)->enabled);self::assertTrue((bool)User::find(1)->banned);
|
||||||
|
$rolled=$this->snapshot();NodeSyncService::$fail=false;PanelChange::rollback($this->dir,true);self::assertSame($rolled,$this->snapshot());self::assertSame('rolled_back',\DplOnboarding\readJson($this->dir.'/rollback-journal.json')['state']);
|
||||||
|
}
|
||||||
|
public function testQaReactivationAndContainmentChangeBlockRollback(): void {
|
||||||
|
PanelChange::promote($this->dir,$this->request,true);Capsule::table('v2_user')->where('id',1)->update(['banned'=>0]);$before=$this->snapshot();
|
||||||
|
$this->rejects(fn()=>PanelChange::rollback($this->dir,true));self::assertSame($before,$this->snapshot());
|
||||||
|
Capsule::table('v2_user')->where('id',1)->update(['banned'=>1]);Capsule::table('v2_server')->where('id',2)->update(['enabled'=>false]);$before=$this->snapshot();
|
||||||
|
$this->rejects(fn()=>PanelChange::rollback($this->dir,true));self::assertSame($before,$this->snapshot());self::assertTrue((bool)User::find(1)->banned);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
202
tests/node-onboarding/test_batch.py
Normal file
202
tests/node-onboarding/test_batch.py
Normal file
@ -0,0 +1,202 @@
|
|||||||
|
"""Offline process-level tests. No Laravel bootstrap, .env, SSH or network."""
|
||||||
|
import copy
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
RUNNER = ROOT / 'scripts/node-onboarding/batch.py'
|
||||||
|
ADAPTER = ROOT / 'scripts/node-onboarding/command-adapter.py'
|
||||||
|
spec = importlib.util.spec_from_file_location('dpl_batch', RUNNER)
|
||||||
|
batch = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(batch)
|
||||||
|
|
||||||
|
FAKE = r'''
|
||||||
|
import argparse, json, os, sys, time
|
||||||
|
from pathlib import Path
|
||||||
|
p=argparse.ArgumentParser()
|
||||||
|
for x in ('plan','node','phase','action','state-dir'): p.add_argument('--'+x,required=True)
|
||||||
|
a=p.parse_args(); n=json.loads(Path(a.node).read_text()); plan=json.loads(Path(a.plan).read_text())
|
||||||
|
root=Path(plan['fixture_root']); root.mkdir(exist_ok=True)
|
||||||
|
identity=dict(key=n['key'], phase=a.phase, request_sha256=n['request_sha256'])
|
||||||
|
def emit(status, **kw): print(json.dumps(dict(identity,status=status,**kw)))
|
||||||
|
if plan.get('bad_receipt'): identity['request_sha256']='wrong'
|
||||||
|
if a.phase=='preflight':
|
||||||
|
emit('waiting' if plan.get('block_preflight') and n['key']=='jp2' else 'ready'); sys.exit()
|
||||||
|
if a.phase in ('verify','audit'): emit('ready'); sys.exit()
|
||||||
|
marker=root/(n['key']+'-'+a.phase)
|
||||||
|
if a.action=='probe':
|
||||||
|
emit('ready' if marker.exists() else 'absent',retry_safe=True); sys.exit()
|
||||||
|
def log(event):
|
||||||
|
fd=os.open(root/'events',os.O_CREAT|os.O_WRONLY|os.O_APPEND,0o600)
|
||||||
|
os.write(fd,(json.dumps(dict(key=n['key'],phase=a.phase,event=event,t=time.monotonic()))+'\n').encode());os.close(fd)
|
||||||
|
log('start'); time.sleep(1.2 if a.phase=='provision' else .12); marker.write_text('owned'); log('end')
|
||||||
|
if plan.get('crash') and a.phase=='provision' and not (root/'crashed').exists():
|
||||||
|
(root/'crashed').write_text('yes'); sys.exit(8)
|
||||||
|
emit('ready')
|
||||||
|
'''
|
||||||
|
|
||||||
|
|
||||||
|
class BatchTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.root = Path(self.tmp.name)
|
||||||
|
self.fake = self.root / 'fake.py'
|
||||||
|
self.fake.write_text(FAKE)
|
||||||
|
self.state = self.root / 'state'
|
||||||
|
self.plan = {
|
||||||
|
'version': 1, 'mode': 'append', 'batch_id': 'offline-test',
|
||||||
|
'context': {'panel_vps': 'VPS-001', 'gateway_vps': 'VPS-002',
|
||||||
|
'jump_vps': 'VPS-003', 'app_root': '/app',
|
||||||
|
'panel_url': 'https://panel.example.invalid',
|
||||||
|
'subscription_url': 'https://sub.example.invalid',
|
||||||
|
'provision_commit': 'a' * 40, 'bundle': '/private/bundle',
|
||||||
|
'client_binary': '/private/sing-box'},
|
||||||
|
'adapter': [sys.executable, str(self.fake)],
|
||||||
|
'fixture_root': str(self.root / 'remote'), 'nodes': [self.node(1)],
|
||||||
|
}
|
||||||
|
self.path = self.root / 'plan.json'
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.tmp.cleanup()
|
||||||
|
|
||||||
|
def node(self, i):
|
||||||
|
return dict(key=f'jp{i}', country='JP', sequence=i, flag='🇯🇵', country_name='日本',
|
||||||
|
name=f'🇯🇵 日本 JP - {i:02d}', vps_id=f'VPS-{10+i:03d}',
|
||||||
|
node_address=f'192.0.2.{i}', listen_port=24443, relay_host='relay.example.invalid',
|
||||||
|
relay_port=47000+i, tls_domain=f'jp{i}.example.invalid', formal_group_id=1,
|
||||||
|
expected_exit=f'192.0.2.{i}', socks_port=19000+i)
|
||||||
|
|
||||||
|
def argv(self, command='run'):
|
||||||
|
self.path.write_text(json.dumps(self.plan, ensure_ascii=False))
|
||||||
|
return [sys.executable, str(RUNNER), command, '--plan', str(self.path),
|
||||||
|
'--state-root', str(self.state), '--jobs', '3'] + (['--execute'] if command == 'run' else [])
|
||||||
|
|
||||||
|
def run_cli(self, command='run', expected=0):
|
||||||
|
p = subprocess.run(self.argv(command), capture_output=True, text=True, timeout=30)
|
||||||
|
self.assertEqual(p.returncode, expected, p.stdout + p.stderr)
|
||||||
|
return json.loads(p.stdout)
|
||||||
|
|
||||||
|
def events(self):
|
||||||
|
path = self.root / 'remote/events'
|
||||||
|
return [json.loads(x) for x in path.read_text().splitlines()] if path.exists() else []
|
||||||
|
|
||||||
|
def test_plan_and_status_are_read_only(self):
|
||||||
|
self.run_cli('plan'); self.run_cli('status')
|
||||||
|
self.assertFalse(self.state.exists())
|
||||||
|
self.assertFalse((self.root/'remote').exists())
|
||||||
|
|
||||||
|
def test_duplicate_parameters_and_naming_rejected(self):
|
||||||
|
for field in ('key','name','vps_id','node_address','tls_domain','relay_port','socks_port','sequence'):
|
||||||
|
with self.subTest(field=field):
|
||||||
|
other = self.node(2); other[field] = self.plan['nodes'][0][field]
|
||||||
|
p = copy.deepcopy(self.plan); p['nodes'].append(other)
|
||||||
|
with self.assertRaises(ValueError): batch.validate(p)
|
||||||
|
p = copy.deepcopy(self.plan); p['nodes'][0]['name'] = '🇯🇵 日本 JP - 11'
|
||||||
|
with self.assertRaises(ValueError): batch.validate(p)
|
||||||
|
|
||||||
|
def test_preflight_barrier_blocks_every_write(self):
|
||||||
|
self.plan['nodes'].append(self.node(2)); self.plan['block_preflight'] = True
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.assertEqual(self.events(), [])
|
||||||
|
|
||||||
|
def test_parallel_nodes_serial_panel_and_resume(self):
|
||||||
|
self.plan['nodes'] = [self.node(i) for i in (1,2,3)]
|
||||||
|
self.run_cli()
|
||||||
|
events = self.events()
|
||||||
|
for phase, expected_peak in [('stage',1), ('publish',1), ('provision',None)]:
|
||||||
|
active = peak = 0
|
||||||
|
for e in sorted((e for e in events if e['phase']==phase), key=lambda e:e['t']):
|
||||||
|
active += 1 if e['event']=='start' else -1; peak=max(peak,active)
|
||||||
|
if expected_peak is None:
|
||||||
|
self.assertGreater(peak,1,'Independent nodes should overlap')
|
||||||
|
self.assertLessEqual(peak,3)
|
||||||
|
else:
|
||||||
|
self.assertEqual(peak, expected_peak, phase)
|
||||||
|
self.run_cli()
|
||||||
|
self.assertEqual(self.events(), events, 'A completed retry must not write again')
|
||||||
|
status = self.run_cli('status')
|
||||||
|
self.assertTrue(all(n['audit']=='done' for n in status.values()))
|
||||||
|
self.assertEqual(self.state.stat().st_mode & 0o777, 0o700)
|
||||||
|
self.assertEqual((self.state/'registry.sqlite3').stat().st_mode & 0o777, 0o600)
|
||||||
|
|
||||||
|
def test_lost_response_recovers_by_probe_without_duplicate(self):
|
||||||
|
self.plan['crash'] = True
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.run_cli()
|
||||||
|
writes = [e for e in self.events() if e['phase']=='provision' and e['event']=='start']
|
||||||
|
self.assertEqual(len(writes),1)
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['audit'],'done')
|
||||||
|
|
||||||
|
def test_cross_batch_and_plan_drift_rejected_before_commands(self):
|
||||||
|
self.run_cli(); events = self.events()
|
||||||
|
self.plan['nodes'][0]['expected_exit'] = '192.0.2.99'
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.plan['batch_id'] = 'other-batch'
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.assertEqual(self.events(), events)
|
||||||
|
|
||||||
|
def test_two_processes_share_locks_and_do_not_duplicate(self):
|
||||||
|
argv = self.argv()
|
||||||
|
a = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||||
|
b = subprocess.Popen(argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||||
|
for process in (a,b):
|
||||||
|
out, err = process.communicate(timeout=30)
|
||||||
|
self.assertEqual(process.returncode,0,out+err)
|
||||||
|
stages = [e for e in self.events() if e['phase']=='stage' and e['event']=='start']
|
||||||
|
self.assertEqual(len(stages),1)
|
||||||
|
|
||||||
|
def test_invalid_receipt_cannot_authorize_write(self):
|
||||||
|
self.plan['bad_receipt'] = True
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.assertEqual(self.events(), [])
|
||||||
|
|
||||||
|
def test_interrupted_apply_records_unknown_and_can_resume(self):
|
||||||
|
p = subprocess.Popen(self.argv(), stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||||
|
try:
|
||||||
|
deadline = time.monotonic() + 10
|
||||||
|
while not any(e['phase']=='provision' and e['event']=='start' for e in self.events()):
|
||||||
|
self.assertIsNone(p.poll())
|
||||||
|
self.assertLess(time.monotonic(), deadline)
|
||||||
|
time.sleep(.02)
|
||||||
|
p.send_signal(signal.SIGTERM)
|
||||||
|
out, err = p.communicate(timeout=10)
|
||||||
|
self.assertEqual(p.returncode,2,out+err)
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['provision'],'unknown')
|
||||||
|
self.run_cli()
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['audit'],'done')
|
||||||
|
finally:
|
||||||
|
if p.poll() is None:
|
||||||
|
p.kill(); p.communicate()
|
||||||
|
|
||||||
|
def test_configured_command_adapter_runs_full_protocol(self):
|
||||||
|
self.plan['adapter'] = [sys.executable,str(ADAPTER)]
|
||||||
|
command = [sys.executable,str(self.fake),'--plan','{plan}','--node','{node}',
|
||||||
|
'--phase','{phase}','--action','{action}','--state-dir','{state_dir}']
|
||||||
|
self.plan['command_profile'] = {
|
||||||
|
'files': {str(self.fake): hashlib.sha256(self.fake.read_bytes()).hexdigest()},
|
||||||
|
'phases': {phase: {'probe': command, 'apply': command}
|
||||||
|
for phase in ('preflight',) + batch.PHASES}}
|
||||||
|
self.run_cli()
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['audit'],'done')
|
||||||
|
|
||||||
|
def test_command_adapter_pins_handlers_and_missing_steps_wait(self):
|
||||||
|
self.plan['adapter'] = [sys.executable,str(ADAPTER)]
|
||||||
|
self.plan['command_profile'] = {'files': {str(self.fake): hashlib.sha256(self.fake.read_bytes()).hexdigest()},
|
||||||
|
'phases': {}}
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['preflight'],'waiting')
|
||||||
|
self.fake.write_text('# changed')
|
||||||
|
self.run_cli(expected=2)
|
||||||
|
self.assertEqual(self.run_cli('status')['jp1']['preflight'],'unknown')
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__': unittest.main()
|
||||||
437
tests/node-onboarding/test_runtime.py
Normal file
437
tests/node-onboarding/test_runtime.py
Normal file
@ -0,0 +1,437 @@
|
|||||||
|
"""Offline adversarial evidence/transport/resume tests; no network or Laravel env."""
|
||||||
|
import copy
|
||||||
|
import hashlib
|
||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
SCRIPTS=Path(__file__).resolve().parents[2]/'scripts/node-onboarding'
|
||||||
|
sys.path.insert(0,str(SCRIPTS))
|
||||||
|
from acceptance import audit,digest,snapshot,service_change,provider_state
|
||||||
|
from ssh_pool import Pool,private_json,credentials
|
||||||
|
import batch
|
||||||
|
|
||||||
|
class EvidenceTests(unittest.TestCase):
|
||||||
|
def fixture(self,count):
|
||||||
|
nodes=[dict(id=i,type='anytls',tag='node-'+str(i),relay='relay.example.invalid',relay_port=47000+i,sni=f'n{i}.example.invalid',expected_exit=f'192.0.2.{i}',country='JP') for i in range(1,count+1)]
|
||||||
|
report={'state':'live_verified','verified_at':1000,'results':[dict(tag=n['tag'],relay=n['relay'],relay_port=n['relay_port'],sni=n['sni'],exit_ip=n['expected_exit'],country=n['country'],live=True,strict_tls=True) for n in nodes]}
|
||||||
|
qas={'captured_at':1000,'plan_sha256':'plan','users':{'new':dict(id=9,http_status=403,banned=True,excluded=True)}}
|
||||||
|
return nodes,report,qas
|
||||||
|
def check(self,nodes,report,qas,after=None,**bindings):
|
||||||
|
raw=json.dumps(report).encode();qraw=json.dumps(qas).encode()
|
||||||
|
binding=dict(plan_sha256='plan',snapshot_sha256=digest(snapshot(nodes)),report_sha256=hashlib.sha256(raw).hexdigest(),qa_rejections_sha256=hashlib.sha256(qraw).hexdigest(),verified_at=1000)
|
||||||
|
binding.update(bindings)
|
||||||
|
return audit(nodes,nodes if after is None else after,raw,qraw,binding,'plan',{'new':9},now=1100)
|
||||||
|
def test_arbitrary_node_counts_and_reordering(self):
|
||||||
|
for count in [1,2,7,23]:
|
||||||
|
nodes,report,qas=self.fixture(count)
|
||||||
|
self.assertEqual(self.check(nodes,report,qas,after=list(reversed(nodes)))['nodes'],count)
|
||||||
|
def test_snapshot_can_be_revalidated_by_formal_audit(self):
|
||||||
|
nodes,report,qas=self.fixture(2)
|
||||||
|
normalized=snapshot(nodes)
|
||||||
|
self.assertEqual(snapshot(normalized),normalized)
|
||||||
|
self.assertTrue(all(n['type']=='anytls' for n in normalized))
|
||||||
|
self.assertEqual(self.check(normalized,report,qas)['nodes'],2)
|
||||||
|
def test_topology_change_including_old_node_is_blocked(self):
|
||||||
|
nodes,r,q=self.fixture(7)
|
||||||
|
for after in [nodes[:-1],nodes+self.fixture(8)[0][-1:],list(reversed(nodes[:-1]))]:
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,r,q,after=after)
|
||||||
|
altered=copy.deepcopy(nodes);altered[0]['expected_exit']='198.51.100.9'
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,r,q,after=altered)
|
||||||
|
def test_missing_extra_duplicate_or_insecure_results_fail(self):
|
||||||
|
nodes,r,q=self.fixture(7)
|
||||||
|
for results in [r['results'][:-1],r['results']+[r['results'][0]],r['results'][1:]+[r['results'][1]]]:
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,dict(r,results=results),q)
|
||||||
|
for field,value in [('exit_ip','198.51.100.1'),('country','US'),('strict_tls',False),('live',False),('sni','wrong.example.invalid')]:
|
||||||
|
wrong=copy.deepcopy(r);wrong['results'][0][field]=value
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,wrong,q)
|
||||||
|
def test_qa_identity_status_revocation_and_hashes_are_bound(self):
|
||||||
|
nodes,r,q=self.fixture(2)
|
||||||
|
for field,value in [('id',10),('http_status',200),('banned',False),('excluded',False)]:
|
||||||
|
wrong=copy.deepcopy(q);wrong['users']['new'][field]=value
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,r,wrong)
|
||||||
|
for field in ['snapshot_sha256','plan_sha256','report_sha256','qa_rejections_sha256']:
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,r,q,**{field:'wrong'})
|
||||||
|
for stamp in [-1,1200,None,'1000']:
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,dict(r,verified_at=stamp),q)
|
||||||
|
with self.assertRaises(ValueError):self.check(nodes,r,dict(q,captured_at=stamp))
|
||||||
|
def test_provider_observations_are_exact_and_not_language_or_substring_dependent(self):
|
||||||
|
node=dict(key='new',tls_domain='a.example.invalid',relay_host='relay.example.invalid',relay_port=47001,node_address='192.0.2.1',listen_port=24443)
|
||||||
|
observed={'captured_at':1000,'dns':[{'type':'A','name':node['tls_domain'],'address':node['node_address'],'proxy':False}],'relays':[{'name':'dpl-new-anytls','host':node['relay_host'],'port':47001,'target':node['node_address'],'listen_port':24443,'proxy_protocol_in':False,'proxy_protocol_out':False}]}
|
||||||
|
self.assertTrue(provider_state(observed,node,1100)['dns'])
|
||||||
|
for section,field,value in [('dns','proxy',True),('dns','address','192.0.2.11'),('relays','proxy_protocol_in',True),('relays','target','192.0.2.11')]:
|
||||||
|
wrong=copy.deepcopy(observed);wrong[section][0][field]=value
|
||||||
|
with self.assertRaises(ValueError):provider_state(wrong,node,1100)
|
||||||
|
self.assertEqual(provider_state(dict(observed,dns=[],relays=[]),node,1100),{'dns':[],'relays':[]})
|
||||||
|
with self.assertRaises(ValueError):provider_state(observed,node,2000)
|
||||||
|
|
||||||
|
def test_registry_and_service_changes_do_not_use_pid_alone(self):
|
||||||
|
for values in [[],[dict(self.fixture(1)[0][0],type='vless')],self.fixture(1)[0]*2]:
|
||||||
|
with self.assertRaises(ValueError):snapshot(values)
|
||||||
|
before=dict(unit_sha256='a',binary_sha256='b',config_sha256='c',listeners=['*:443'],active=True,pid=1,started='yesterday',restarts=0)
|
||||||
|
self.assertEqual(service_change(before,before),'ready')
|
||||||
|
self.assertEqual(service_change(before,dict(before,pid=2)),'investigate_restart')
|
||||||
|
self.assertEqual(service_change(before,dict(before,config_sha256='changed')),'conflict')
|
||||||
|
self.assertEqual(service_change(before,dict(before,active=False)),'conflict')
|
||||||
|
self.assertEqual(service_change(before,dict(before,binary_sha256=None)),'unknown')
|
||||||
|
|
||||||
|
class TransportTests(unittest.TestCase):
|
||||||
|
def test_reuse_and_every_nonwhite_host_has_jump(self):
|
||||||
|
calls=[]
|
||||||
|
class Client:
|
||||||
|
active=True
|
||||||
|
def get_transport(self):return self
|
||||||
|
def is_active(self):return self.active
|
||||||
|
def close(self):self.active=False
|
||||||
|
def factory(rid,spec,jump):calls.append((rid,jump));return Client()
|
||||||
|
config={'hosts':{'white':{'role':'white_jump'},'panel':{},'node':{}}}
|
||||||
|
pool=Pool(config,'white',factory)
|
||||||
|
panel=pool.get('panel');self.assertIs(panel,pool.get('panel'))
|
||||||
|
node=pool.get('node');self.assertIs(calls[1][1],calls[2][1]);self.assertIsNone(calls[0][1]);self.assertEqual(len(calls),3)
|
||||||
|
panel.active=False
|
||||||
|
with self.assertRaises(RuntimeError):pool.get('panel')
|
||||||
|
self.assertEqual(len(calls),3) # Never automatically replay after disconnect.
|
||||||
|
pool.close();self.assertFalse(node.active)
|
||||||
|
def test_sessions_are_not_shared_unsafely_between_workers(self):
|
||||||
|
barrier=threading.Barrier(2);made=[]
|
||||||
|
class Client:
|
||||||
|
def get_transport(self):return self
|
||||||
|
def is_active(self):return True
|
||||||
|
def close(self):pass
|
||||||
|
pool=Pool({'hosts':{'w':{'role':'white_jump'},'n':{}}},'w',lambda *args:Client())
|
||||||
|
def worker():
|
||||||
|
n=pool.get('n');barrier.wait();made.append(n);self.assertIs(n,pool.get('n'))
|
||||||
|
threads=[threading.Thread(target=worker) for _ in range(2)]
|
||||||
|
for t in threads:t.start()
|
||||||
|
for t in threads:t.join(timeout=5)
|
||||||
|
self.assertEqual(len(made),2);self.assertIsNot(made[0],made[1]);pool.close()
|
||||||
|
def test_existing_codocs_credentials_are_read_without_copy_or_rewrite(self):
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
p=Path(d)/'vps.md';body='# Fixture\n密码：test-only-phrase\n';p.write_text(body);p.chmod(0o600)
|
||||||
|
self.assertEqual(credentials({'credential_ref':str(p),'credential_format':'codocs-password'}),{'password':'test-only-phrase'})
|
||||||
|
self.assertEqual(p.read_text(),body);self.assertEqual(len(list(Path(d).iterdir())),1)
|
||||||
|
p.write_text(body+'密码：ambiguous\n')
|
||||||
|
with self.assertRaises(ValueError):credentials({'credential_ref':str(p),'credential_format':'codocs-password'})
|
||||||
|
def test_private_config_and_explicit_jump_enforced(self):
|
||||||
|
with self.assertRaises(ValueError):Pool({'hosts':{'n':{}}},'w')
|
||||||
|
with self.assertRaises(ValueError):Pool({'hosts':{'w':{}}},'w')
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
p=Path(d)/'auth.json';p.write_text('{}');p.chmod(0o644)
|
||||||
|
with self.assertRaises(ValueError):private_json(p)
|
||||||
|
p.chmod(0o600);self.assertEqual(private_json(p),{})
|
||||||
|
link=Path(d)/'link';link.symlink_to(p)
|
||||||
|
with self.assertRaises(ValueError):private_json(link)
|
||||||
|
|
||||||
|
class ResumeTests(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
batch.STOP.clear();self.tmp=tempfile.TemporaryDirectory();self.root=Path(self.tmp.name)
|
||||||
|
spec=importlib.util.spec_from_file_location('batch_fixtures',Path(__file__).with_name('test_batch.py'));m=importlib.util.module_from_spec(spec);spec.loader.exec_module(m)
|
||||||
|
fixture=m.BatchTests();fixture.setUp();self.fixture=fixture;self.plan=fixture.plan
|
||||||
|
self.calls=[];self.generation='a';self.finished=False
|
||||||
|
owner=self
|
||||||
|
class Runner(batch.Coordinator):
|
||||||
|
def call(self,n,phase,action):
|
||||||
|
owner.calls.append((phase,action))
|
||||||
|
r=dict(key=n['key'],phase=phase,request_sha256='irrelevant-in-unit-fixture',status='waiting' if phase=='audit' and not owner.finished else 'ready',valid_until=time.time()+180)
|
||||||
|
if phase=='preflight':r['resume_dependencies']={k:hashlib.sha256((owner.generation if k=='provision' else k).encode()).hexdigest() for k in ['stage','dns','provision','relay','verify']}
|
||||||
|
return r
|
||||||
|
self.Runner=Runner
|
||||||
|
def tearDown(self):self.fixture.tearDown();self.tmp.cleanup();batch.STOP.clear()
|
||||||
|
def run_batch(self):return self.Runner(self.plan,self.root/'ledger',jobs=1).run()
|
||||||
|
def test_fresh_proofs_skip_only_completed_unchanged_phases(self):
|
||||||
|
self.assertFalse(self.run_batch());self.calls.clear();self.finished=True
|
||||||
|
self.assertTrue(self.run_batch());self.assertEqual(self.calls,[('preflight','probe'),('publish','probe'),('audit','probe')])
|
||||||
|
def test_dependency_change_rechecks_downstream_and_not_unrelated_stage(self):
|
||||||
|
self.run_batch();self.calls.clear();self.generation='b';self.finished=True;self.run_batch()
|
||||||
|
self.assertEqual([p for p,a in self.calls],['preflight','provision','relay','verify','publish','audit'])
|
||||||
|
def test_expired_and_unknown_proofs_never_skip(self):
|
||||||
|
self.run_batch();runner=self.Runner(self.plan,self.root/'ledger',jobs=1)
|
||||||
|
with runner.ledger.db() as c:c.execute('UPDATE evidence SET expires=0 WHERE phase="dns"')
|
||||||
|
runner.ledger.mark(self.plan['batch_id'],self.plan['nodes'][0]['key'],'stage','unknown')
|
||||||
|
self.calls.clear();self.finished=True;self.run_batch()
|
||||||
|
self.assertIn(('stage','probe'),self.calls);self.assertIn(('dns','probe'),self.calls)
|
||||||
|
self.assertTrue((self.root/'ledger/events.jsonl').is_file())
|
||||||
|
|
||||||
|
|
||||||
|
class SiteAdapterTests(unittest.TestCase):
|
||||||
|
def test_qa_export_binds_raw_bytes_and_latest_traffic_for_independent_review(self):
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
adapter=object.__new__(SiteAdapter);adapter.directory=Path(d);adapter.atomic_json=batch.atomic_json
|
||||||
|
raw=b'{ "state": "live_verified", "qa_user_id": 9 }'
|
||||||
|
latest=dict(node_online=True,u=12,d=34,manifest={'test_user_id':9})
|
||||||
|
adapter.qa_evidence(dict(key='new',request_sha256='request'),raw,latest)
|
||||||
|
path=Path(d)/'new-evidence.json';value=json.loads(path.read_text())
|
||||||
|
self.assertEqual(value['report_sha256'],hashlib.sha256(raw).hexdigest())
|
||||||
|
self.assertEqual(value['panel'],latest);self.assertEqual(path.stat().st_mode & 0o077,0)
|
||||||
|
|
||||||
|
def test_new_live_qa_exports_new_raw_report_after_transfer_not_prior_report(self):
|
||||||
|
import io
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
adapter=object.__new__(SiteAdapter);adapter.directory=Path(d);adapter.atomic_json=batch.atomic_json
|
||||||
|
ctx=dict(panel_vps='panel',gateway_vps='gateway',node_root='/node',ops_root='/ops',app_root='/app',node_backup_root='/backups',qa_root='/qa',client_binary='/client',subscription_url='https://sub.example.invalid')
|
||||||
|
adapter.plan={'context':ctx};adapter.ctx=ctx
|
||||||
|
raw=json.dumps(dict(state='live_verified',qa_user_id=9,verified_at=int(time.time()),results=[dict(tag='new',live=True,strict_tls=True,exit_ip='192.0.2.11',country='JP')])).encode()
|
||||||
|
class Channel:
|
||||||
|
def shutdown_write(self):pass
|
||||||
|
def recv_exit_status(self):return 0
|
||||||
|
class In:
|
||||||
|
channel=Channel()
|
||||||
|
def write(self,data):pass
|
||||||
|
class Out(io.BytesIO):channel=Channel()
|
||||||
|
class Writable(io.BytesIO):
|
||||||
|
def __init__(self,files,path):super().__init__();self.files,self.path=files,path
|
||||||
|
def close(self):self.files[self.path]=self.getvalue();super().close()
|
||||||
|
class Sftp:
|
||||||
|
def __init__(self,files):self.files=files
|
||||||
|
def __enter__(self):return self
|
||||||
|
def __exit__(self,*args):pass
|
||||||
|
def open(self,path,mode):
|
||||||
|
if mode=='w':return Writable(self.files,path)
|
||||||
|
if path not in self.files:raise FileNotFoundError(path)
|
||||||
|
return io.BytesIO(self.files[path])
|
||||||
|
def chmod(self,*args):pass
|
||||||
|
def posix_rename(self,old,new):self.files[new]=self.files.pop(old)
|
||||||
|
state=dict(manifest={'test_user_id':9},node_online=True,groups=['2'],qa_banned=False,u=12,d=34)
|
||||||
|
qa_programs=[]
|
||||||
|
class Remote:
|
||||||
|
def __init__(self,kind):self.kind=kind;self.files={}
|
||||||
|
def open_sftp(self):return Sftp(self.files)
|
||||||
|
def exec_command(self,cmd,timeout=0):
|
||||||
|
if cmd.endswith(' && php'):return In(),Out(json.dumps(state).encode()),Out()
|
||||||
|
if cmd.startswith("python3 - <<'DPLPY'\n"):
|
||||||
|
program=cmd.split('\n',1)[1].rsplit('\nDPLPY',1)[0]
|
||||||
|
compile(program,'actual-fetch-qa-remote','exec')
|
||||||
|
qa_programs.append(program)
|
||||||
|
if 'verify-node.py' in cmd:self.files['/qa/new/qa-report.json']=raw
|
||||||
|
return In(),Out(b'{}'),Out()
|
||||||
|
remotes={key:Remote(key) for key in ['panel','gateway','node']}
|
||||||
|
remotes['panel'].files.update({'/backups/new/node-config.yml':b'fixture-config','/backups/new/node-secret.env':b'fixture-env','/backups/new/qa-private.json':b'{}'})
|
||||||
|
class Pool:
|
||||||
|
def get(self,key):return remotes[key]
|
||||||
|
adapter.pool=Pool()
|
||||||
|
node=dict(key='new',request_sha256='request',formal_group_id=1,vps_id='node',socks_port=19011,expected_exit='192.0.2.11',country='JP')
|
||||||
|
result=adapter.call(node,'verify','probe');self.assertEqual(result['status'],'ready')
|
||||||
|
self.assertEqual(len(qa_programs),1)
|
||||||
|
self.assertIn('urllib.request.urlopen',qa_programs[0])
|
||||||
|
exported=json.loads((Path(d)/'new-evidence.json').read_text());self.assertEqual(exported['report_sha256'],hashlib.sha256(raw).hexdigest());self.assertEqual(exported['panel']['d'],34)
|
||||||
|
self.assertEqual(remotes['panel'].files['/backups/new/qa-report.json'],raw)
|
||||||
|
|
||||||
|
def test_formal_source_snapshot_audit_and_cached_audit_call_chain(self):
|
||||||
|
import contextlib
|
||||||
|
import io
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
adapter=object.__new__(SiteAdapter)
|
||||||
|
ctx=dict(gateway_vps='gateway',qa_root='/qa',ops_root='/ops',subscription_url='https://sub.example.invalid',client_binary='/client',formal_socks_port=19999,node_backup_root='/backups',batch_backup_root='/batch')
|
||||||
|
candidate=dict(key='new',formal_group_id=7,relay_host='relay.example.invalid',relay_port=47002,tls_domain='n2.example.invalid')
|
||||||
|
adapter.plan={'context':ctx,'nodes':[candidate]};adapter.ctx=ctx
|
||||||
|
adapter.directory=Path('/private/tmp/formal-chain-fixture/ledger/batch')
|
||||||
|
adapter.lock=threading.RLock();adapter.resource_lock=lambda *args:contextlib.nullcontext()
|
||||||
|
adapter.formal_done={};adapter.exit_registry={f'n{i}.example.invalid':dict(expected_exit=f'192.0.2.{i}',country='JP') for i in [1,2]}
|
||||||
|
source={'user_id':1,'owned':False,'token':'fixture-only','nodes':[dict(id=i,type='anytls',tag='node-'+str(i),relay='relay.example.invalid',relay_port=47000+i,sni=f'n{i}.example.invalid',uuid='fixture-only') for i in [1,2]]}
|
||||||
|
accounts=[]
|
||||||
|
def account_command(group,action,apply=False,client=None):
|
||||||
|
accounts.append(action)
|
||||||
|
return copy.deepcopy(source) if action in ['source','snapshot'] else {'state':'closed'}
|
||||||
|
adapter.account_command=account_command
|
||||||
|
now=int(time.time())
|
||||||
|
raw=json.dumps({'state':'live_verified','verified_at':now,'results':[dict(tag='node-'+str(i),relay='relay.example.invalid',relay_port=47000+i,sni=f'n{i}.example.invalid',exit_ip=f'192.0.2.{i}',country='JP',live=True,strict_tls=True) for i in [1,2]]}).encode()
|
||||||
|
files={};persisted=[];programs=[]
|
||||||
|
def execute(client,command,timeout=120):
|
||||||
|
if command.startswith("python3 - <<'DPLFORMAL'\n"):
|
||||||
|
program=command.split('\n',1)[1].rsplit('\nDPLFORMAL',1)[0]
|
||||||
|
compile(program,'actual-formal-remote','exec');programs.append(program)
|
||||||
|
files['/qa/formal-group-7/report.json']=raw
|
||||||
|
return ''
|
||||||
|
def php(client,body):
|
||||||
|
return {} if '$out=[]' in body else {'id':9,'token':'fixture-only','banned':True,'excluded':True}
|
||||||
|
def write(client,path,value):files[path]=value.encode() if isinstance(value,str) else value
|
||||||
|
adapter.atomic_json=lambda path,value:persisted.append(value)
|
||||||
|
class Channel:
|
||||||
|
def shutdown_write(self):pass
|
||||||
|
def recv_exit_status(self):return 0
|
||||||
|
class In:
|
||||||
|
channel=Channel()
|
||||||
|
def write(self,data):pass
|
||||||
|
class Out(io.BytesIO):channel=Channel()
|
||||||
|
class Gateway:
|
||||||
|
def exec_command(self,command,timeout=0):
|
||||||
|
if command!='python3 -':raise AssertionError('Unexpected QA probe command')
|
||||||
|
return In(),Out(b'403\n'),Out()
|
||||||
|
gateway=Gateway()
|
||||||
|
adapter.formal(candidate,None,gateway,php,execute,lambda client,path:files.get(path),write)
|
||||||
|
self.assertEqual(len(programs),1)
|
||||||
|
self.assertEqual(len(persisted),1)
|
||||||
|
self.assertEqual(len(persisted[0]['snapshot']),2,'The old node must remain in complete acceptance')
|
||||||
|
self.assertTrue(all(n['type']=='anytls' for n in persisted[0]['snapshot']))
|
||||||
|
self.assertEqual(persisted[0]['qas']['users']['new']['http_status'],403)
|
||||||
|
self.assertIn('cleanup',accounts)
|
||||||
|
self.assertEqual(adapter.formal(candidate,None,gateway,php,execute,lambda client,path:files.get(path),write)['nodes'],2)
|
||||||
|
self.assertEqual(len(programs),1,'A cached audit rechecks topology without rerunning the client')
|
||||||
|
|
||||||
|
def test_preserved_service_review_binds_original_and_exact_observation(self):
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
adapter=object.__new__(SiteAdapter);adapter.ctx={'preserved_service':dict(unit='XrayR',binary='/bin/xrayr',config='/etc/xrayr.yml',port=443)}
|
||||||
|
current=dict(unit_sha256='u',binary_sha256='b',config_sha256='c',listeners=['*:443'],active=True,pid=1,started='a',restarts=0)
|
||||||
|
files={};read=lambda client,path:files.get(path)
|
||||||
|
def write(client,path,value):files[path]=value.encode() if isinstance(value,str) else value
|
||||||
|
def execute(client,cmd):return json.dumps(current) if cmd.startswith('python3') else ''
|
||||||
|
adapter.guard_service(None,'/backup',read,write,execute)
|
||||||
|
original=copy.deepcopy(current);current.update(pid=2,started='b',restarts=1)
|
||||||
|
with self.assertRaisesRegex(RuntimeError,'investigate_'):adapter.guard_service(None,'/backup',read,write,execute)
|
||||||
|
files['/backup/preserved-service-review.json']=json.dumps(dict(before_sha256=digest(original),after_sha256=digest(current),reason='Independently reviewed lifecycle panic',evidence_sha256='e'*64)).encode()
|
||||||
|
adapter.guard_service(None,'/backup',read,write,execute)
|
||||||
|
current['config_sha256']='changed'
|
||||||
|
with self.assertRaisesRegex(RuntimeError,'conflict'):adapter.guard_service(None,'/backup',read,write,execute)
|
||||||
|
self.assertEqual(json.loads(files['/backup/preserved-service-original.json']),original)
|
||||||
|
def test_generated_remote_python_and_group_independent_php_compile(self):
|
||||||
|
import ast
|
||||||
|
import subprocess
|
||||||
|
import io
|
||||||
|
from site_adapter import SiteAdapter
|
||||||
|
tree=ast.parse((SCRIPTS/'site_adapter.py').read_text())
|
||||||
|
for method in [n for n in ast.walk(tree) if isinstance(n,ast.FunctionDef) and n.name in ('formal','guard_service','certificate_gate')]:
|
||||||
|
for c in ast.walk(method):
|
||||||
|
if isinstance(c,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='script' for t in c.targets):
|
||||||
|
program=eval(compile(ast.Expression(c.value),'script-builder','eval'),{'repr':repr,'spec':{},'ctx':{'subscription_url':'https://sub.example.invalid'},'value':{'token':'test-only'}})
|
||||||
|
compile(program,'remote-'+method.name,'exec')
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
a=object.__new__(SiteAdapter);a.plan={'context':dict(panel_vps='panel',gateway_vps='gateway',node_root='/node',ops_root='/ops',app_root='/app',batch_backup_root='/batch',node_backup_root='/backups',qa_root='/qa',gateway_containers=['gateway-a','gateway-b'])};a.ctx=a.plan['context'];a.directory=Path(d)
|
||||||
|
php_sources=[];calls=[]
|
||||||
|
class Channel:
|
||||||
|
def shutdown_write(self):pass
|
||||||
|
def recv_exit_status(self):return 0
|
||||||
|
class In:
|
||||||
|
channel=Channel()
|
||||||
|
def write(self,body):php_sources.append(body)
|
||||||
|
class Out:
|
||||||
|
channel=Channel()
|
||||||
|
def read(self):
|
||||||
|
body=php_sources[-1]
|
||||||
|
if '$base=json_decode' in body:return b'true'
|
||||||
|
return json.dumps({'manifest':{},'node_online':True,'groups':['7'],'qa_banned':True,'u':1,'d':1}).encode()
|
||||||
|
class DockerOut:
|
||||||
|
channel=Channel()
|
||||||
|
def read(self):return json.dumps([dict(Name='/gateway-a',Id='id-a',State={'StartedAt':'then'},RestartCount=0)]).encode()
|
||||||
|
class Sftp:
|
||||||
|
def __enter__(self):return self
|
||||||
|
def __exit__(self,*args):pass
|
||||||
|
def open(self,path,mode):return io.BytesIO(json.dumps({'/gateway-a':{'id':'id-a','started':'then','restarts':0}}).encode())
|
||||||
|
class Remote:
|
||||||
|
def exec_command(self,cmd,timeout=0):
|
||||||
|
calls.append(cmd)
|
||||||
|
if cmd.startswith('docker inspect'):
|
||||||
|
if cmd!='docker inspect gateway-a gateway-b':raise RuntimeError('Invalid gateway inventory command')
|
||||||
|
return In(),DockerOut(),io.BytesIO()
|
||||||
|
return In(),Out(),io.BytesIO()
|
||||||
|
def open_sftp(self):return Sftp()
|
||||||
|
class Pool:
|
||||||
|
def get(self,rid):return Remote()
|
||||||
|
a.pool=Pool();a.guard_service=lambda *args:None;a.formal=lambda *args:None
|
||||||
|
n=dict(key='sg-new',request_sha256='a'*64,formal_group_id=7,vps_id='node')
|
||||||
|
result=a.call(n,'audit','probe');self.assertEqual(result['status'],'ready');self.assertGreaterEqual(len(php_sources),2)
|
||||||
|
for php in php_sources:
|
||||||
|
result=subprocess.run(['php','-l'],input=php,text=True,capture_output=True)
|
||||||
|
self.assertEqual(result.returncode,0,result.stderr)
|
||||||
|
def test_pool_real_connect_interface_keeps_strict_keys_and_jump_sock(self):
|
||||||
|
import types
|
||||||
|
recorded=[]
|
||||||
|
class FakeClient:
|
||||||
|
def load_host_keys(self,path):recorded.append(('keys',path))
|
||||||
|
def set_missing_host_key_policy(self,p):recorded.append(('reject',p))
|
||||||
|
def get_host_keys(self):return self
|
||||||
|
def add(self,*args):recorded.append(('approved-key',args))
|
||||||
|
def connect(self,host,**kw):recorded.append(('connect',host,kw))
|
||||||
|
def get_transport(self):return self
|
||||||
|
def is_active(self):return True
|
||||||
|
def open_channel(self,kind,target,origin):recorded.append(('jump',kind,target));return 'through-white'
|
||||||
|
def close(self):pass
|
||||||
|
class Key:
|
||||||
|
def asbytes(self):return b'approved-public-key-fixture'
|
||||||
|
def get_name(self):return 'ssh-ed25519'
|
||||||
|
class Keys:
|
||||||
|
def __init__(self,path):pass
|
||||||
|
def lookup(self,name):return {'ssh-ed25519':Key()}
|
||||||
|
module=types.SimpleNamespace(SSHClient=FakeClient,RejectPolicy=lambda:'reject-unknown',HostKeys=Keys)
|
||||||
|
pin='SHA256:'+__import__('base64').b64encode(hashlib.sha256(Key().asbytes()).digest()).decode().rstrip('=')
|
||||||
|
with tempfile.TemporaryDirectory() as d,patch.dict(sys.modules,{'paramiko':module}):
|
||||||
|
keys=Path(d)/'known_hosts';keys.write_text('public fixture');keys.chmod(0o600)
|
||||||
|
credentials=Path(d)/'credentials.json';credentials.write_text(json.dumps({'password':'test-only-fixture'}));credentials.chmod(0o600)
|
||||||
|
hosts={'w':dict(role='white_jump',host='192.0.2.1',username='root',credential_ref=str(credentials)),'n':dict(host='192.0.2.2',username='root',credential_ref=str(credentials))}
|
||||||
|
|
||||||
|
for host in hosts.values():host['host_key_sha256']=pin
|
||||||
|
p=Pool(dict(hosts=hosts,known_hosts=str(keys)),'w');p.get('n');p.get('n');p.close()
|
||||||
|
before_connects=len([x for x in recorded if x[0]=='connect'])
|
||||||
|
hosts['w']['host_key_sha256']='SHA256:'+'b'*43
|
||||||
|
blocked=Pool(dict(hosts=hosts,known_hosts=str(keys)),'w')
|
||||||
|
with self.assertRaises(ValueError):blocked.get('n')
|
||||||
|
self.assertEqual(len([x for x in recorded if x[0]=='connect']),before_connects)
|
||||||
|
blocked.close()
|
||||||
|
connections=[x for x in recorded if x[0]=='connect'];self.assertEqual(len(connections),2);self.assertIsNone(connections[0][2]['sock']);self.assertEqual(connections[1][2]['sock'],'through-white')
|
||||||
|
self.assertEqual(len([x for x in recorded if x[0]=='reject']),2)
|
||||||
|
for _,_,kw in connections:self.assertFalse(kw['allow_agent']);self.assertFalse(kw['look_for_keys'])
|
||||||
|
|
||||||
|
def test_configure_is_local_only_and_pins_code_context_and_interpreter(self):
|
||||||
|
import types
|
||||||
|
spec=importlib.util.spec_from_file_location('configure_site',SCRIPTS/'configure-site.py');module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
root=Path(d);ssh=root/'ssh.json';registry=root/'exits.json';(root/'known_hosts').write_text('public fixture');(root/'known_hosts').chmod(0o600)
|
||||||
|
hosts={'VPS-001':dict(role='white_jump',host='192.0.2.1',username='root'),'VPS-002':dict(host='192.0.2.2',username='root'),'VPS-003':dict(host='192.0.2.3',username='root'),'VPS-011':dict(host='192.0.2.11',username='root')}
|
||||||
|
for host in hosts.values():host['host_key_sha256']='SHA256:'+'a'*43
|
||||||
|
ssh.write_text(json.dumps({'hosts':hosts,'known_hosts':str(root/'known_hosts')}));ssh.chmod(0o600);registry.write_text('{}');registry.chmod(0o600)
|
||||||
|
fixture_spec=importlib.util.spec_from_file_location('base_fixture',Path(__file__).with_name('test_batch.py'));f=importlib.util.module_from_spec(fixture_spec);fixture_spec.loader.exec_module(f)
|
||||||
|
original=f.BatchTests();original.setUp()
|
||||||
|
try:
|
||||||
|
base=original.plan;node=base['nodes'][0];node['node_address']='192.0.2.11';node['expected_exit']='192.0.2.11';base['context'].update(jump_vps='VPS-001',panel_vps='VPS-002',gateway_vps='VPS-003');registry.write_text(json.dumps({node['tls_domain']:{'expected_exit':node['expected_exit'],'country':node['country']}}))
|
||||||
|
site=dict(client_sha256='d'*64,ops_root='/ops',node_root='/node',batch_backup_root='/backup/batch',node_backup_root='/backup/nodes',qa_root='/qa/batch',provider_snapshot=str(root/'provider.json'),exit_registry=str(registry),ssh_config=str(ssh),preserved_service=dict(unit='XrayR',binary='/bin/xrayr',config='/etc/xrayr.yml',port=443),formal_socks_port=19999,gateway_containers=['gateway-a'])
|
||||||
|
with patch.dict(sys.modules,{'paramiko':types.SimpleNamespace(__version__='5.0.0')}):
|
||||||
|
plan=module.configure(base,site);self.assertEqual(plan['runtime']['kind'],'ssh-site-v1');self.assertFalse((root/'ledger').exists())
|
||||||
|
runner=batch.Coordinator(plan,root/'ledger',jobs=1);runner.runtime.close();runner.check_runtime()
|
||||||
|
ssh.write_text(ssh.read_text()+'\n')
|
||||||
|
with self.assertRaises(ValueError):runner.check_runtime()
|
||||||
|
finally:original.tearDown()
|
||||||
|
|
||||||
|
class PackagingTests(unittest.TestCase):
|
||||||
|
def test_committed_flat_snapshot_preserves_worktree_index_and_base(self):
|
||||||
|
import subprocess
|
||||||
|
spec=importlib.util.spec_from_file_location('bundle_ops',SCRIPTS/'bundle-ops.py');module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
|
||||||
|
with tempfile.TemporaryDirectory() as d:
|
||||||
|
root=Path(d);repo=root/'repo';repo.mkdir()
|
||||||
|
def git(*args):return subprocess.check_output(['git','-C',str(repo),*args],stderr=subprocess.DEVNULL)
|
||||||
|
git('init');git('config','user.name','Offline Test');git('config','user.email','test@example.invalid');git('config','commit.gpgsign','false')
|
||||||
|
(repo/'base-only.txt').write_text('preserved operations base');git('add','.');git('commit','-m','base');base=git('rev-parse','HEAD').decode().strip()
|
||||||
|
source_dir=repo/'scripts/node-onboarding';source_dir.mkdir(parents=True)
|
||||||
|
for name in module.FILES:(source_dir/name).write_text('committed '+name)
|
||||||
|
git('add','.');git('commit','-m','source');source=git('rev-parse','HEAD').decode().strip()
|
||||||
|
(repo/'base-only.txt').write_text('unrelated user modification');(repo/'unrelated.txt').write_text('staged unrelated');git('add','unrelated.txt')
|
||||||
|
index_before=git('diff','--cached','--binary');status_before=git('status','--porcelain')
|
||||||
|
output=root/'operations.bundle';result=module.build(repo,source,base,'codex/offline-ops',output)
|
||||||
|
self.assertTrue(output.exists());self.assertEqual(git('diff','--cached','--binary'),index_before);self.assertEqual(git('status','--porcelain'),status_before)
|
||||||
|
self.assertEqual(git('show',result['commit']+':panel-acceptance.php'),b'committed panel-acceptance.php')
|
||||||
|
self.assertEqual(git('show',result['commit']+':base-only.txt'),b'preserved operations base')
|
||||||
|
self.assertEqual(git('rev-parse','HEAD').decode().strip(),source)
|
||||||
|
git('bundle','verify',str(output))
|
||||||
|
with self.assertRaises(ValueError):module.build(repo,source,base,'codex/offline-ops',root/'second.bundle')
|
||||||
|
advanced=module.build(repo,source,result['commit'],'codex/offline-ops',root/'advanced.bundle',advance=True)
|
||||||
|
self.assertEqual(git('rev-parse',advanced['commit']+'^').decode().strip(),result['commit'])
|
||||||
|
with self.assertRaises(ValueError):module.build(repo,source,base,'codex/offline-ops',root/'stale.bundle',advance=True)
|
||||||
|
def test_event_stream_preserves_final_result_and_avoids_status_polling(self):
|
||||||
|
import subprocess
|
||||||
|
spec=importlib.util.spec_from_file_location('event_fixture',Path(__file__).with_name('test_batch.py'));f=importlib.util.module_from_spec(spec);spec.loader.exec_module(f)
|
||||||
|
test=f.BatchTests();test.setUp()
|
||||||
|
try:
|
||||||
|
result=subprocess.run(test.argv()+['--events'],capture_output=True,text=True,timeout=30)
|
||||||
|
self.assertEqual(result.returncode,0,result.stderr)
|
||||||
|
lines=[json.loads(x) for x in result.stdout.splitlines()]
|
||||||
|
self.assertTrue(lines[-1]['complete']);self.assertTrue(any(x.get('phase')=='provision' for x in lines[:-1]))
|
||||||
|
for x in lines[:-1]:self.assertEqual(set(x),{'batch','node','phase','status','time'})
|
||||||
|
finally:test.tearDown()
|
||||||
|
|
||||||
|
if __name__=='__main__':unittest.main()
|
||||||
Loading…
Reference in New Issue
Block a user