fix(env-sync): 同步 provider_config.json(UI 分组/顺序/模型级配置)

问题:只同步 v2/config.json 时,只在 v2/provider_config.json 里存在的
provider 分组到对面就是缺失的 —— 表现为「provider 数量对,但 UI 分组少几个」。
实测:9 个分组里有 4 个(GINKA API / zxcbug / zxcbug-p / d1)纯规则型,
密钥只存在 provider_config.json 的 config.access.apiKey 里,旧实现完全没收集。

修复:
- sync-core:新增 V2_PROVIDER_CONFIG、extract/sanitize/collectProviderConfigSecrets、
  providerConfigKeyManifest;密钥命名空间 PROVIDER_CONFIG/<providerId>/<key>
- export:导出 v2.provider-config.json(脱敏),密钥并入加密束,manifest 记摘要与哈希
- import:按 providerId 合并规则、回填密钥空位、按 (providerId,modelId) 合并模型级配置、
  providerOrder 以快照为准并保留本机特有分组;重复导入幂等
- import:provider_config 路径补「护住本机已有密钥 N 个」提示(原来只在 config.json 侧有)
- 测试:单元 6 项 + E2E 2 项(分组完整同步/强制覆盖),全量 72 项通过
This commit is contained in:
2026-09-21 13:00:10 +08:00
parent 6ed5dda847
commit 06891fa726
6 changed files with 450 additions and 10 deletions
+21 -6
View File
@@ -107,7 +107,7 @@ latest.json # 指向最新快照
机器特有的 host/port/command 可写 `~/.zcode-env-sync/overrides.json`(**不进仓**),
导入时自动深度合并覆盖。
## 模型 provider(默认关)
## 模型 provider 与 UI 分组(默认关)
`~/.zcode/v2/config.json` 里通常是**活的 apiKey**,所以默认完全不碰(快照里无模型)。
需要同步时导出导入两边都显式开:
@@ -117,18 +117,33 @@ ZCODE_SYNC_WITH_MODELS=1 node export.mjs # 导出:脱敏结构明文 + 真密
ZCODE_SYNC_WITH_MODELS=1 node import.mjs # 导入:结构合并 + 密钥回填
```
导出:`v2.providers.json` 脱敏版明文(密钥位置是 `${MODEL_SECRET_REF}`),
真密钥以 `MODELS/<provider>/<key>` 条目并入 `secrets.enc`;
manifest 只记摘要 + 密钥哈希(`modelKeys`),不记值。
### 两份文件都要同步(只同步一份会导致「UI 分组少了」)
导入(`ZCODE_SYNC_WITH_MODELS=1` 时才动 `v2/config.json`,先备份):
| 文件 | 管什么 | 快照里的名字 |
|---|---|---|
| `v2/config.json` 的 `provider` | provider **能不能调**(kind、options、models 定义) | `v2.providers.json` |
| `v2/provider_config.json` | UI **怎么分组显示**(providerOrder、group、access、personalModelIds、modelOrder,以及 contextWindow 等模型级配置) | `v2.provider-config.json` |
- 快照有、本机无的 provider:整体新增
两份各带一份 `access.apiKey`,**不保证重叠**:有些分组只存在于 `provider_config.json`
(在 UI 里建了但没落进 `config.json`)。只同步前者时那些分组到对面就是缺失的 ——
表现为「provider 数量对,但 UI 分组少几个」。所以现在两份都导出、都回填。
导出:两个文件都写**脱敏版明文**(密钥位置是 `${MODEL_SECRET_REF}`),
真密钥分别以 `MODELS/<provider>/<key>` 与 `PROVIDER_CONFIG/<providerId>/<key>`
条目并入 `secrets.enc`;manifest 只记摘要 + 密钥哈希(`modelKeys` / `providerConfigKeys`),不记值。
导入(`ZCODE_SYNC_WITH_MODELS=1` 时才动这两个文件,先备份):
- 快照有、本机无的 provider / 分组:整体新增
- 已有 provider:只补缺的模型定义与非密钥字段,不碰本机已有密钥
- 已有分组:按 `providerId` 合并,补缺失字段(含 `group`/`personalModelIds`/`modelOrder`)
- 模型级配置:按 `(providerId, modelId)` 去重合并
- `providerOrder`:以快照顺序为准,本机特有的分组追加尾部(不丢本机分组)
- 密钥回填:只填**空位**(占位符/空/过短);本机已有密钥默认跳过保护,
确需覆盖再加 `ZCODE_SYNC_OVERWRITE_MODEL_SECRETS=1`
- 选中态(当前用哪个 provider/model)默认不动,
加 `ZCODE_SYNC_APPLY_MODEL_SELECTION=1` 才应用(同样先备份 `v2/setting.json`)
- 重复导入幂等:不会重复追加分组或模型级配置
| 变量 | 说明 |
|---|---|
+44 -3
View File
@@ -7,13 +7,15 @@ import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import {
HOME, USER, HOSTNAME, CLI_CONFIG, PLUGINS_HOME, REPO_DIR, V2_CONFIG,
HOME, USER, HOSTNAME, CLI_CONFIG, PLUGINS_HOME, REPO_DIR, V2_CONFIG, V2_PROVIDER_CONFIG,
nowTag, readJson, writeJson, encodeJsonPaths, encodePaths, fwd,
officialPlugins, enabledPlugins, enabledPluginsDetailed, customPlugins, copyDirFiltered, hashDir,
secretTargets, secretsManifest, buildSecretsBundle, encryptSecrets,
requirePassphrase, scanForSecrets, ensureRepo, gitPush, dirSize, humanSize,
collectModelSecrets, sanitizeProviders, summarizeProviders, modelSelection,
extractModelSecrets, modelKeyManifest, WITH_MODELS, APPLY_MODEL_SELECTION,
extractProviderConfigSecrets, collectProviderConfigSecrets, sanitizeProviderConfig,
providerConfigKeyManifest, PROVIDER_CONFIG_PREFIX,
} from "./sync-core.mjs";
function fail(e) { console.error(`导出失败:${e?.message ?? e}`); process.exitCode = 1; }
@@ -69,6 +71,16 @@ try {
const providerSanitized = v2cfg ? sanitizeProviders(v2cfg) : { providers: {}, withSecrets: [] };
const modelKeys = v2cfg ? extractModelSecrets(v2cfg) : [];
const modelKeysManifest = modelKeyManifest(modelKeys);
/**
* provider_config.json:UI 的分组/顺序/模型级配置。它自带一份 access.apiKey,
* 与 config.json 那份可能重叠但不保证(只在 provider_config 里存在的分组,
* 其密钥也只在它这)。两份都要收,否则那些分组过去就是个空壳。
*/
const pcCfg = WITH_MODELS ? readJson(V2_PROVIDER_CONFIG, null) : null;
const pcSanitized = pcCfg ? sanitizeProviderConfig(pcCfg) : { config: null, withSecrets: [] };
const pcKeys = pcCfg ? extractProviderConfigSecrets(pcCfg) : [];
const pcKeysManifest = providerConfigKeyManifest(pcKeys);
const pcSecrets = pcCfg ? collectProviderConfigSecrets(pcCfg) : [];
let modelKeyCount = 0;
if (v2cfg) {
// 脱敏结果里不允许再有真密钥(helper 漏了就直接停,别把明文推上去)
@@ -82,6 +94,17 @@ try {
modelKeyCount++;
}
}
if (pcCfg) {
const rules = pcSanitized.config?.config?.providerConfigRules?.providerRules ?? [];
const left = rules.flatMap((r) => scanForSecrets(r?.config?.access ?? {}));
if (left.length) throw new Error(`provider_config 脱敏不彻底(${left.slice(0, 5).join(",")}),已中断`);
for (const { provider, key, value } of pcKeys) {
bundle.files.push({
rel: `${PROVIDER_CONFIG_PREFIX}${provider}/${key}`,
contentB64: Buffer.from(String(value), "utf8").toString("base64"),
});
}
}
const enc = encryptSecrets(bundle, pass);
const tag = `${HOSTNAME}-${nowTag()}`;
@@ -112,10 +135,22 @@ try {
modelKeyCount,
});
}
if (pcCfg) {
Object.assign(manifest, {
// UI 分组/顺序/模型级配置的摘要(结构本体在 v2.provider-config.json)
providerConfigOrder: pcSanitized.config?.config?.providerOrder ?? [],
providerConfigRuleCount: (pcSanitized.config?.config?.providerConfigRules?.providerRules ?? []).length,
providerConfigModelRuleCount: (pcSanitized.config?.config?.modelConfigRules?.providerModelRules ?? []).length,
providerConfigWithSecrets: pcSanitized.withSecrets,
providerConfigKeys: pcKeysManifest,
providerConfigKeyCount: pcKeys.length,
});
}
// manifest 与 latest.json 都留到自检通过后再落盘:失败时不留半个快照
writeJson(path.join(snapDir, "mcp.servers.json"), mcpEncoded);
writeJson(path.join(snapDir, "plugins-dirs.json"), dirsEncoded);
if (v2cfg) writeJson(path.join(snapDir, "v2.providers.json"), encodeJsonPaths(providerSanitized.providers));
if (pcCfg) writeJson(path.join(snapDir, "v2.provider-config.json"), encodeJsonPaths(pcSanitized.config));
fs.writeFileSync(path.join(snapDir, "secrets.enc"), enc);
writeJson(path.join(snapDir, "secrets.manifest.json"), manifest.secretFiles);
@@ -138,7 +173,7 @@ try {
// 密钥:全覆盖,命中即硬失败
const plainText = readAll(snapFiles) + "\n" + pendingManifestText;
const leakedSecrets = [...providerSecrets, ...secrets.map((s) => {
const leakedSecrets = [...providerSecrets, ...pcSecrets, ...secrets.map((s) => {
try { return fs.readFileSync(s.abs, "utf8").trim(); } catch { return ""; }
})].filter((v) => v && v.length >= 8 && plainText.includes(v));
if (leakedSecrets.length) {
@@ -214,7 +249,7 @@ try {
});
// 只提交本次快照 + latest.json:不做 add -A,免得把别的会话未推送的快照连带推上去
const pushMsg = gitPush(
`sync-export ${tag} (official:${official.length} custom:${custom.length} mcp:${Object.keys(mcpServers).length} secrets:${secrets.length}${v2cfg ? ` providers:${providerSummary.length}` : ""})`,
`sync-export ${tag} (official:${official.length} custom:${custom.length} mcp:${Object.keys(mcpServers).length} secrets:${secrets.length}${v2cfg ? ` providers:${providerSummary.length}` : ""}${pcCfg ? ` groups:${(pcSanitized.config?.config?.providerOrder ?? []).length}` : ""})`,
{ paths: [path.posix.join("snapshots", tag), "latest.json", ".gitattributes", ".gitignore", "local-overrides.example.json"] });
console.log(`# 导出成功`);
@@ -227,6 +262,12 @@ try {
console.log(`- 模型 provider:${providerSummary.length} 个(脱敏结构明文),密钥 ${modelKeyCount} 个已入加密束`);
console.log(`- 模型选中态:${APPLY_MODEL_SELECTION ? "已随快照应用" : "仅记录,导入时不覆盖本机"}`);
}
if (pcCfg) {
const order = pcSanitized.config?.config?.providerOrder ?? [];
const rules = pcSanitized.config?.config?.providerConfigRules?.providerRules ?? [];
const mRules = pcSanitized.config?.config?.modelConfigRules?.providerModelRules ?? [];
console.log(`- UI 分组:${order.length} 个(providerOrder),规则 ${rules.length} 条,模型级配置 ${mRules.length} 条,密钥 ${pcKeys.length} 个已入加密束`);
}
console.log(`- 明文区自检:无密钥泄漏、配置区无本机路径残留`);
console.log(`- 快照体积:${humanSize(manifest.bytes)}`);
if (pathWarnings.length) {
+92 -1
View File
@@ -10,12 +10,13 @@ import os from "node:os";
import path from "node:path";
import { createHash } from "node:crypto";
import {
HOME, CLI_CONFIG, PLUGINS_HOME, REPO_DIR, BACKUP_ROOT, V2_CONFIG, V2_SETTING,
HOME, CLI_CONFIG, PLUGINS_HOME, REPO_DIR, BACKUP_ROOT, V2_CONFIG, V2_SETTING, V2_PROVIDER_CONFIG,
WITH_MODELS, APPLY_MODEL_SELECTION, OVERWRITE_MODEL_SECRETS, MODEL_SECRET_REF,
nowTag, readJson, writeJson, decodeJsonPaths, encodeJsonPaths,
decryptSecrets, restoreRelPath, isModelEntry, parseModelRel, requirePassphrase,
ensureRepo, readSnapshotManifest, latestSnapshotDir,
deepMerge, loadOverrides,
isProviderConfigEntry, parseProviderConfigRel,
} from "./sync-core.mjs";
function fail(e) { console.error(`导入失败:${e?.message ?? e}`); process.exitCode = 1; }
@@ -101,9 +102,11 @@ try {
const skipSet = skipSecretsSet();
const skipped = [];
const modelKeys = [];
const pcKeys = [];
let n = 0;
for (const f of bundle.files ?? []) {
if (isModelEntry(f.rel)) { modelKeys.push(f); continue; }
if (isProviderConfigEntry(f.rel)) { pcKeys.push(f); continue; }
if (skipSet.has(f.rel)) { skipped.push(f.rel); continue; }
let dst;
try { dst = restoreRelPath(f.rel); }
@@ -190,6 +193,94 @@ try {
report.push(`- 模型:快照未含模型(导出时未开 ZCODE_SYNC_WITH_MODELS=1)`);
}
// 4b) provider_config.json:UI 的分组/顺序/模型级配置
// config.json 决定「provider 能不能调」,这份决定「UI 怎么分组显示」。
// 只同步前者时,仅在 provider_config 里存在的分组到对面就是缺失的。
const pcFile = path.join(snapDir, "v2.provider-config.json");
if (fs.existsSync(pcFile) && (manifest.providerConfigRuleCount ?? 0) > 0) {
const snapPc = decodeJsonPaths(readJson(pcFile, {}));
if (!WITH_MODELS) {
report.push(`- UI 分组:快照含 ${(manifest.providerConfigOrder ?? []).length} 个分组,`
+ `默认不覆盖本机(设 ZCODE_SYNC_WITH_MODELS=1 应用)`);
} else {
const localPc = fs.existsSync(V2_PROVIDER_CONFIG) ? readJson(V2_PROVIDER_CONFIG, {}) : {};
const b3 = backup(V2_PROVIDER_CONFIG); if (b3) report.push(`- 备份 v2/provider_config.json → ${b3}`);
const keyMapPc = new Map();
for (const f of pcKeys) {
const { provider, key } = parseProviderConfigRel(f.rel);
keyMapPc.set(`${provider}\n${key}`, Buffer.from(f.contentB64, "base64").toString("utf8"));
}
const snapCfg = snapPc?.config ?? {};
const localCfg = localPc.config ?? (localPc.config = {});
const snapRules = snapCfg.providerConfigRules?.providerRules ?? [];
const snapModels = snapCfg.modelConfigRules?.providerModelRules ?? [];
const snapManual = snapCfg.modelConfigRules?.manualProviderModelRules ?? [];
localCfg.providerConfigRules = localCfg.providerConfigRules ?? {};
localCfg.modelConfigRules = localCfg.modelConfigRules ?? {};
const localRules = localCfg.providerConfigRules.providerRules ?? (localCfg.providerConfigRules.providerRules = []);
const localModels = localCfg.modelConfigRules.providerModelRules ?? (localCfg.modelConfigRules.providerModelRules = []);
const localManual = localCfg.modelConfigRules.manualProviderModelRules ?? (localCfg.modelConfigRules.manualProviderModelRules = []);
// 按 id 建索引,避免同一 provider 重复追加
const byId = new Map(localRules.map((r, i) => [r?.providerId, i]));
let addRule = 0, mergeRule = 0, filledPcKey = 0, skippedPcKey = 0, skippedPcOverwrite = 0, addModel = 0;
for (const sr of snapRules) {
const id = sr?.providerId;
if (!id) continue;
if (!byId.has(id)) { localRules.push(JSON.parse(JSON.stringify(sr))); byId.set(id, localRules.length - 1); addRule++; }
else {
// 已有:补缺失字段,不动本机已有的值(含密钥)
const lr = localRules[byId.get(id)];
lr.providerName = lr.providerName ?? sr.providerName;
lr.config = lr.config ?? {};
for (const [k, v] of Object.entries(sr.config ?? {})) {
if (k === "access") continue; // 密钥单独处理
if (lr.config[k] === undefined) lr.config[k] = JSON.parse(JSON.stringify(v));
}
lr.config.access = lr.config.access ?? {};
for (const [k, v] of Object.entries(sr.config?.access ?? {})) {
if (v === MODEL_SECRET_REF) continue;
if (lr.config.access[k] === undefined) lr.config.access[k] = v;
}
mergeRule++;
}
// 密钥回填:占位符/空/过短才填
const lr = localRules[byId.get(id)];
lr.config.access = lr.config.access ?? {};
for (const [kk, vv] of keyMapPc) {
const [pid, k] = kk.split("\n");
if (pid !== id) continue;
if (skipSet.has(`PROVIDER_CONFIG/${pid}/${k}`)) { skippedPcKey++; continue; }
const cur = lr.config.access[k];
if (cur === MODEL_SECRET_REF || cur === undefined || cur === "" || (typeof cur === "string" && cur.length < 8)) {
lr.config.access[k] = vv; filledPcKey++;
} else if (OVERWRITE_MODEL_SECRETS) {
lr.config.access[k] = vv; filledPcKey++;
} else {
skippedPcOverwrite++; // 本机已有非空密钥,默认护住
}
}
}
// 模型级配置:按 (providerId, modelId) 去重合并
const mKey = (r) => `${r?.providerId}\n${r?.modelId}`;
const haveModel = new Set(localModels.map(mKey));
for (const mr of snapModels) { if (!haveModel.has(mKey(mr))) { localModels.push(JSON.parse(JSON.stringify(mr))); haveModel.add(mKey(mr)); addModel++; } }
const haveManual = new Set(localManual.map(mKey));
for (const mr of snapManual) { if (!haveManual.has(mKey(mr))) { localManual.push(JSON.parse(JSON.stringify(mr))); haveManual.add(mKey(mr)); } }
// providerOrder:快照的分组顺序(合并本机多出来的,不让本机特有分组消失)
const snapOrder = snapCfg.providerOrder ?? [];
const merged = [...snapOrder];
for (const id of localCfg.providerOrder ?? []) if (!merged.includes(id)) merged.push(id);
localCfg.providerOrder = merged;
if (localPc.schemaVersion === undefined && snapPc.schemaVersion !== undefined) localPc.schemaVersion = snapPc.schemaVersion;
writeJson(V2_PROVIDER_CONFIG, localPc);
report.push(`- UI 分组:${merged.length} 个分组(新增规则 ${addRule},合并 ${mergeRule},密钥回填 ${filledPcKey}`
+ (skippedPcOverwrite ? `,护住本机已有密钥 ${skippedPcOverwrite} 个(需覆盖设 ZCODE_SYNC_OVERWRITE_MODEL_SECRETS=1)` : "")
+ (skippedPcKey ? `,跳过 ${skippedPcKey}(ZCODE_SYNC_SKIP_SECRETS)` : "") + `),模型级配置 +${addModel} 条`);
}
} else if (manifest.providerConfigRuleCount) {
report.push(`- UI 分组:快照仅记录摘要 ${manifest.providerConfigRuleCount} 条(未含配置本体)`);
}
// 5) 官方插件:只报告差异,不自动执行(去插件市场对照启用,重启生效,避免误装)
if (process.env.ZCODE_SYNC_APPLY_OFFICIAL !== "0") {
const { enabledPlugins } = await import("./sync-core.mjs");
+65
View File
@@ -25,6 +25,8 @@ export const REPO_DIR = path.join(WORK, "repo");
export const BACKUP_ROOT = path.join(WORK, "backup");
export const V2_CONFIG = path.join(ZCODE_HOME, "v2", "config.json");
export const V2_SETTING = path.join(ZCODE_HOME, "v2", "setting.json");
/** UI 的分组/排序/模型级配置(providerOrder、group、personalModelIds、contextWindow 等) */
export const V2_PROVIDER_CONFIG = path.join(ZCODE_HOME, "v2", "provider_config.json");
/** 模型同步默认关闭,显式开启才同步(ZCODE_SYNC_WITH_MODELS=1) */
export const WITH_MODELS = process.env.ZCODE_SYNC_WITH_MODELS === "1";
/** 模型选中态(当前用哪个provider/model)默认不覆盖本机,显式开启才应用 */
@@ -318,6 +320,69 @@ export function parseModelRel(rel) {
return { provider: rest.slice(0, i), key: rest.slice(i + 1) };
}
/* ---------- provider_config.json:UI 分组/顺序/模型级配置 ----------
* config.json 只管「provider 能不能调」;UI 上看到的 9 个分组、拖拽顺序、
* 每条 provider 属于哪个 group、模型级参数(contextWindow 等)全在 provider_config.json。
* 早期版本只同步 config.json,于是只在 provider_config 里存在的分组整个丢失。
*/
export const PROVIDER_CONFIG_PREFIX = "PROVIDER_CONFIG/";
export function isProviderConfigEntry(rel) { return String(rel).startsWith(PROVIDER_CONFIG_PREFIX); }
/** PROVIDER_CONFIG/<providerId>/<key> → { provider, key } */
export function parseProviderConfigRel(rel) {
const rest = String(rel).slice(PROVIDER_CONFIG_PREFIX.length);
const i = rest.indexOf("/");
if (i <= 0 || i === rest.length - 1) throw new Error(`非法 provider_config 密钥路径:${rel}`);
return { provider: rest.slice(0, i), key: rest.slice(i + 1) };
}
/** providerRules 里的真密钥(内存中处理,调用方不得打印) */
export function extractProviderConfigSecrets(pcfg) {
const out = [];
for (const r of pcfg?.config?.providerConfigRules?.providerRules ?? []) {
const id = r?.providerId;
if (!id) continue;
for (const [k, v] of Object.entries(r?.config?.access ?? {})) {
if (typeof v === "string" && MODEL_KEY_RE.test(k) && v.length >= 8) out.push({ provider: id, key: k, value: v });
}
}
return out;
}
/** 全量收集,供泄漏扫描比对 */
export function collectProviderConfigSecrets(pcfg) {
const vals = new Set();
const walk = (node) => {
if (Array.isArray(node)) return node.forEach(walk);
if (node && typeof node === "object") {
for (const [k, v] of Object.entries(node)) {
if (typeof v === "string" && MODEL_KEY_RE.test(k) && v.length >= 8) vals.add(v);
else walk(v);
}
}
};
walk(pcfg?.config?.providerConfigRules?.providerRules ?? []);
return [...vals];
}
/** 脱敏:providerRules 里的密钥换成占位符;返回哪些 provider 带密钥(只记有无) */
export function sanitizeProviderConfig(pcfg) {
const clean = JSON.parse(JSON.stringify(pcfg ?? {}));
const withSecrets = [];
for (const r of clean?.config?.providerConfigRules?.providerRules ?? []) {
const id = r?.providerId;
let hit = false;
for (const [k, v] of Object.entries(r?.config?.access ?? {})) {
if (typeof v === "string" && MODEL_KEY_RE.test(k) && v.length >= 8) { r.config.access[k] = MODEL_SECRET_REF; hit = true; }
}
if (hit && id) withSecrets.push(id);
}
return { config: clean, withSecrets };
}
/** provider_config 密钥清单(只记 provider/key/sha) */
export function providerConfigKeyManifest(keys) {
return (keys ?? []).map(({ provider, key, value }) => ({
rel: `${PROVIDER_CONFIG_PREFIX}${provider}/${key}`,
sha256: crypto.createHash("sha256").update(String(value), "utf8").digest("hex"),
}));
}
/* ---------- 脱敏扫描:明文区不得出现密钥值 ---------- */
/** 递归收集 v2 provider 里的敏感值(内存中比对,绝不打印) */
export function collectModelSecrets(v2cfg) {
+81
View File
@@ -16,6 +16,8 @@ import {
isHiddenEntry, copyDirFiltered,
collectModelSecrets, sanitizeProviders, summarizeProviders,
extractModelSecrets, isModelEntry, parseModelRel,
isProviderConfigEntry, parseProviderConfigRel, extractProviderConfigSecrets,
collectProviderConfigSecrets, sanitizeProviderConfig, providerConfigKeyManifest,
} from "../scripts/sync-core.mjs";
/* ---------------- 路径占位符 ---------------- */
@@ -322,3 +324,82 @@ test("HOME/USER 常量与 os 一致", () => {
assert.equal(HOME, os.homedir());
assert.equal(USER, path.basename(os.homedir()));
});
/* ---------------- provider_config.json:UI 分组/顺序/模型级配置 ---------------- */
/** 造一份带 2 个分组的 provider_config(其中一个只在它这里存在) */
function sampleProviderConfig() {
return {
schemaVersion: 1,
config: {
providerOrder: ["p-shared", "new-provider"],
providerConfigRules: { providerRules: [
{ providerId: "p-shared", providerName: "zz",
config: { group: "standard-personal", access: { type: "api-key", apiKey: "sk-sharedkey-1234567890" },
api: { type: "openai-chat-completions", baseUrl: "http://example.invalid/v1" },
personalModelIds: ["m1"], modelOrder: ["m1"] } },
{ providerId: "new-provider", providerName: "GINKA API",
config: { group: "standard-personal", access: { type: "api-key", apiKey: "sk-onlyhere-0987654321" },
api: { type: "openai-chat-completions", baseUrl: "http://example.invalid/v2" },
personalModelIds: ["m2"], modelOrder: ["m2"] } },
] },
modelConfigRules: {
providerModelRules: [{ modelId: "m1", config: { properties: { contextWindow: 1000000 } }, providerId: "p-shared" }],
manualProviderModelRules: [],
},
},
};
}
test("provider_config:密钥路径前缀与解析", () => {
assert.equal(isProviderConfigEntry("PROVIDER_CONFIG/pid/apiKey"), true);
assert.equal(isProviderConfigEntry("MODELS/pid/apiKey"), false);
assert.deepEqual(parseProviderConfigRel("PROVIDER_CONFIG/pid/apiKey"), { provider: "pid", key: "apiKey" });
assert.throws(() => parseProviderConfigRel("PROVIDER_CONFIG/pid"), /非法/);
assert.throws(() => parseProviderConfigRel("PROVIDER_CONFIG//apiKey"), /非法/);
});
test("provider_config:抽取密钥(含只在它这里存在的分组)", () => {
const keys = extractProviderConfigSecrets(sampleProviderConfig());
assert.equal(keys.length, 2, "两个分组的密钥都要抽到");
assert.deepEqual(keys.map((k) => k.provider).sort(), ["new-provider", "p-shared"]);
assert.ok(keys.some((k) => k.value === "sk-onlyhere-0987654321"), "只在 provider_config 的分组密钥不能漏");
});
test("provider_config:脱敏后明文区无真密钥,占位符就位", () => {
const { config, withSecrets } = sanitizeProviderConfig(sampleProviderConfig());
const text = JSON.stringify(config);
assert.ok(!text.includes("sk-sharedkey-1234567890"), "不得残留真密钥");
assert.ok(!text.includes("sk-onlyhere-0987654321"), "不得残留真密钥");
assert.equal(text.match(/\$\{MODEL_SECRET_REF\}/g)?.length, 2, "两处都要换成占位符");
assert.deepEqual(withSecrets.sort(), ["new-provider", "p-shared"]);
// 非密钥字段必须原样保留(分组/顺序/模型级配置是这次修复的重点)
assert.deepEqual(config.config.providerOrder, ["p-shared", "new-provider"], "分组顺序要保留");
assert.equal(config.config.providerConfigRules.providerRules[0].config.group, "standard-personal", "group 要保留");
assert.deepEqual(config.config.providerConfigRules.providerRules[0].config.personalModelIds, ["m1"], "模型清单要保留");
assert.equal(config.config.modelConfigRules.providerModelRules[0].config.properties.contextWindow, 1000000, "模型级配置要保留");
});
test("provider_config:脱敏不破坏原对象(纯函数)", () => {
const src = sampleProviderConfig();
sanitizeProviderConfig(src);
assert.equal(src.config.providerConfigRules.providerRules[0].config.access.apiKey, "sk-sharedkey-1234567890", "原对象不应被改写");
});
test("provider_config:清单只记哈希不记值", () => {
const man = providerConfigKeyManifest(extractProviderConfigSecrets(sampleProviderConfig()));
const text = JSON.stringify(man);
assert.ok(!text.includes("sk-sharedkey"), "清单里不得有明文");
assert.ok(!text.includes("sk-onlyhere"), "清单里不得有明文");
assert.equal(man.length, 2);
for (const m of man) assert.match(m.rel, /^PROVIDER_CONFIG\/.+\/apiKey$/);
});
test("provider_config:空值与缺字段不炸", () => {
assert.deepEqual(extractProviderConfigSecrets(undefined), []);
assert.deepEqual(extractProviderConfigSecrets({}), []);
assert.deepEqual(extractProviderConfigSecrets({ config: { providerConfigRules: { providerRules: [{ providerId: "x" }] } } }), []);
assert.deepEqual(collectProviderConfigSecrets(null), []);
const { config } = sanitizeProviderConfig({ config: { providerOrder: ["a"] } });
assert.deepEqual(config.config.providerOrder, ["a"], "无 rules 时也要原样返回");
});
+147
View File
@@ -640,3 +640,150 @@ test("E2E:无推送模式只落本地,不碰远端", { timeout: 120000 }, () =>
fs.rmSync(root, { recursive: true, force: true });
});
test("E2E:UI 分组(provider_config)完整同步 —— 旧实现会丢只在其中的分组", { timeout: 120000 }, () => {
const root = sandboxRoot();
const homeA = makeMachine(root, "machineL");
seedMachine(homeA, { user: "lena" });
fs.mkdirSync(path.join(homeA, ".zcode/v2"), { recursive: true });
const SHARED = "11111111-1111-1111-1111-111111111111";
const ONLY_PC = "new-provider";
const rule = (id, name, key) => ({ providerId: id, providerName: name, config: {
group: "standard-personal",
access: { type: "api-key", apiKey: key },
api: { type: "openai-chat-completions", baseUrl: "http://example.invalid/v1" },
personalModelIds: [`${name}-m1`], modelOrder: [`${name}-m1`],
} });
// config.json 只有 SHARED;provider_config.json 有两个(ONLY_PC 只存在于这里)
fs.writeFileSync(path.join(homeA, ".zcode/v2/config.json"), JSON.stringify({
provider: { [SHARED]: { name: "shared", kind: "openai-compatible", models: { "shared-m1": {} },
options: { apiKey: "sk-e2e-shared-1234567890" } } },
}, null, 2));
fs.writeFileSync(path.join(homeA, ".zcode/v2/provider_config.json"), JSON.stringify({
schemaVersion: 1,
config: {
providerOrder: [SHARED, ONLY_PC],
providerConfigRules: { providerRules: [rule(SHARED, "shared", "sk-e2e-shared-1234567890"),
rule(ONLY_PC, "only-here", "sk-e2e-onlyhere-0987654321")] },
modelConfigRules: { providerModelRules: [
{ modelId: "shared-m1", config: { properties: { contextWindow: 1000000 } }, providerId: SHARED }],
manualProviderModelRules: [] },
},
}, null, 2));
const bare = path.join(root, "bare.git");
git(["init", "--bare", "-b", "main", bare]);
const w1 = path.join(root, "w1");
const ex = runScript("export.mjs", { home: homeA, env: {
ZCODE_SYNC_PASSPHRASE: PASS, ZCODE_SYNC_WITH_MODELS: "1", ZCODE_SYNC_GIT_REMOTE: bare, ZCODE_SYNC_WORK: w1 } });
assert.equal(ex.code, 0, `导出差错:\n${ex.err}`);
assert.match(ex.out, /UI 分组:2 个/, `导出报告应含分组数 → ${(ex.out.match(/UI 分组:[^\n]*/) || ["—"])[0]}`);
// 快照明文区必须脱敏
const snapName = JSON.parse(fs.readFileSync(path.join(w1, "repo", "latest.json"), "utf8")).snapshot;
const pcSnap = fs.readFileSync(path.join(w1, "repo", snapName, "v2.provider-config.json"), "utf8");
assert.ok(!pcSnap.includes("sk-e2e-shared-"), "明文区不得出现真密钥(共有分组)");
assert.ok(!pcSnap.includes("sk-e2e-onlyhere-"), "明文区不得出现真密钥(独有分组)");
assert.match(pcSnap, /\$\{MODEL_SECRET_REF\}/, "密钥位置应为占位符");
const manSnap = fs.readFileSync(path.join(w1, "repo", snapName, "manifest.json"), "utf8");
assert.ok(!manSnap.includes("sk-e2e-"), "manifest 不得出现密钥值");
// 队友机器:B 完全没有任何 provider_config.json
const homeB = makeMachine(root, "machineM");
seedMachine(homeB, { user: "mike" });
fs.mkdirSync(path.join(homeB, ".zcode/v2"), { recursive: true });
fs.writeFileSync(path.join(homeB, ".zcode/v2/config.json"), JSON.stringify({ provider: {}, setting: {} }, null, 2));
const w2 = path.join(root, "w2");
const im = runScript("import.mjs", { home: homeB, env: {
ZCODE_SYNC_PASSPHRASE: PASS, ZCODE_SYNC_WITH_MODELS: "1", ZCODE_SYNC_GIT_REMOTE: bare, ZCODE_SYNC_WORK: w2 } });
assert.equal(im.code, 0, `导入差错:\n${im.err}`);
assert.match(im.out, /UI 分组:2 个分组/, `导入报告应含分组 → ${(im.out.match(/UI 分组:[^\n]*/) || ["—"])[0]}`);
const got = JSON.parse(fs.readFileSync(path.join(homeB, ".zcode/v2/provider_config.json"), "utf8")).config;
assert.deepEqual(got.providerOrder, [SHARED, ONLY_PC], "分组顺序必须原样还原");
const byId = new Map(got.providerConfigRules.providerRules.map((r) => [r.providerId, r]));
assert.equal(byId.size, 2, "两个分组都要在(旧实现会丢掉只存在于 provider_config 的那个)");
assert.equal(byId.get(SHARED).config.access.apiKey, "sk-e2e-shared-1234567890", "共有分组密钥回填");
assert.equal(byId.get(ONLY_PC).config.access.apiKey, "sk-e2e-onlyhere-0987654321",
"只在 provider_config 里存在的分组,密钥也必须回填");
assert.equal(byId.get(ONLY_PC).config.group, "standard-personal", "group 字段要还原");
assert.deepEqual(byId.get(ONLY_PC).config.personalModelIds, ["only-here-m1"], "模型清单要还原");
assert.equal(got.modelConfigRules.providerModelRules.length, 1, "模型级配置(如 contextWindow)要带过来");
assert.equal(got.modelConfigRules.providerModelRules[0].config.properties.contextWindow, 1000000);
// 幂等:再导一次不应重复追加
const w3 = path.join(root, "w3");
const im2 = runScript("import.mjs", { home: homeB, env: {
ZCODE_SYNC_PASSPHRASE: PASS, ZCODE_SYNC_WITH_MODELS: "1", ZCODE_SYNC_GIT_REMOTE: bare, ZCODE_SYNC_WORK: w3 } });
assert.equal(im2.code, 0, `二次导入应成功\n${im2.err}`);
const got2 = JSON.parse(fs.readFileSync(path.join(homeB, ".zcode/v2/provider_config.json"), "utf8")).config;
assert.equal(got2.providerConfigRules.providerRules.length, 2, "重复导入不得追加重复分组");
assert.equal(got2.modelConfigRules.providerModelRules.length, 1, "重复导入不得追加重复模型级配置");
assert.equal(got2.providerOrder.length, 2, "重复导入不得把 providerOrder 撑大");
fs.rmSync(root, { recursive: true, force: true });
});
test("E2E:OVERWRITE_MODEL_SECRETS —— 默认护住本机密钥,显式开启才强制覆盖(两份文件)", { timeout: 120000 }, () => {
const root = sandboxRoot();
const PID = "aaaaaaaa-1111-2222-3333-444444444444";
const A_KEY = "sk-snapshot-source-1111111111"; // 快照里的值
const B_KEY = "sk-stale-local-9999999999"; // 队友机器上已有的陈旧值
const writeV2 = (home, key) => {
fs.mkdirSync(path.join(home, ".zcode/v2"), { recursive: true });
fs.writeFileSync(path.join(home, ".zcode/v2/config.json"), JSON.stringify({
provider: { [PID]: { name: "p", kind: "openai-compatible", models: { m: {} }, options: { apiKey: key } } },
}, null, 2));
fs.writeFileSync(path.join(home, ".zcode/v2/provider_config.json"), JSON.stringify({
schemaVersion: 1,
config: {
providerOrder: [PID],
providerConfigRules: { providerRules: [{ providerId: PID, providerName: "p",
config: { group: "standard-personal", access: { type: "api-key", apiKey: key } } }] },
modelConfigRules: { providerModelRules: [], manualProviderModelRules: [] },
},
}, null, 2));
};
const readKeys = (home) => ({
cfg: JSON.parse(fs.readFileSync(path.join(home, ".zcode/v2/config.json"), "utf8")).provider[PID].options.apiKey,
pc: JSON.parse(fs.readFileSync(path.join(home, ".zcode/v2/provider_config.json"), "utf8"))
.config.providerConfigRules.providerRules[0].config.access.apiKey,
});
const homeA = makeMachine(root, "machineN");
seedMachine(homeA, { user: "nina" });
writeV2(homeA, A_KEY);
const bare = path.join(root, "bare.git");
git(["init", "--bare", "-b", "main", bare]);
const ex = runScript("export.mjs", { home: homeA, env: {
ZCODE_SYNC_PASSPHRASE: PASS, ZCODE_SYNC_WITH_MODELS: "1", ZCODE_SYNC_GIT_REMOTE: bare,
ZCODE_SYNC_WORK: path.join(root, "w1") } });
assert.equal(ex.code, 0, `导出差错:\n${ex.err}`);
// 队友机器:两份文件里都已有一个"陈旧的、非空的"密钥
const homeB = makeMachine(root, "machineO");
seedMachine(homeB, { user: "omar" });
const importInto = (work, extra = {}) => runScript("import.mjs", { home: homeB, env: {
ZCODE_SYNC_PASSPHRASE: PASS, ZCODE_SYNC_WITH_MODELS: "1", ZCODE_SYNC_GIT_REMOTE: bare,
ZCODE_SYNC_WORK: path.join(root, work), ...extra } });
// --- 默认行为:护住本机密钥,不覆盖 ---
writeV2(homeB, B_KEY);
const r1 = importInto("w2");
assert.equal(r1.code, 0, `导入应成功\n${r1.err}`);
assert.match(r1.out, /跳过覆盖本机已有密钥|护住本机已有密钥/, `默认应报告被护住的密钥 → ${(r1.out.match(/模型 provider:[^\n]*/) || ["—"])[0]}`);
let got = readKeys(homeB);
assert.equal(got.cfg, B_KEY, "默认不得覆盖 config.json 里的本机密钥");
assert.equal(got.pc, B_KEY, "默认不得覆盖 provider_config.json 里的本机密钥");
// --- 显式开启:强制覆盖为快照值(两份文件都要换) ---
writeV2(homeB, B_KEY);
const r2 = importInto("w3", { ZCODE_SYNC_OVERWRITE_MODEL_SECRETS: "1" });
assert.equal(r2.code, 0, `强制覆盖导入应成功\n${r2.err}`);
got = readKeys(homeB);
assert.equal(got.cfg, A_KEY, "强制覆盖后 config.json 应等于快照值");
assert.equal(got.pc, A_KEY, "强制覆盖后 provider_config.json 应等于快照值");
fs.rmSync(root, { recursive: true, force: true });
});