Appearance
Shadow Harness — v0.1 → v1.0 持续工程主线
Shadow Harness 是 13 周路线的工程验证载体。每周在同一个仓库、同一条 Run lifecycle 上增量演进;禁止另起独立 demo 替代上一周版本。 版本号表达“当前已经建立哪些 invariant”,不是产品发布语义。
版本演进
text
v0.1 Model → Final / Action Intent
↓
v0.2 Model → validated ToolCall → ToolResult
↓
v0.3 parallel Tool + timeout/cancel + Observation
↓
v0.4 ContextManager + budget + compaction + snapshot
↓
v0.5 Retriever source + context injection
↓
v0.6 AgentLoop + RunStateMachine + termination
↓
v0.7 Planner → validated Graph → Scheduler
↓
v0.8 retry + idempotency + policy + approval
↓
v0.9 checkpoint + DB transaction + outbox + resume
↓
v0.10 Run API + SSE + Queue/Worker + Lease/Fencing
↓
v0.11 MCP Adapter + Workspace + Capability/Security boundary
↓
v0.12 Trace + deterministic faults + Eval + Benchmark
↓
v1.0 deploy + graceful drain + crash/resume + full regression最终依赖图
text
API / Event Stream
↓
Run Runtime / State Machine
├─ ContextManager ← Retriever / Memory
├─ ModelAdapter → Planner → GraphCompiler / Scheduler
├─ ToolExecutor ← ToolRegistry ← MCP Adapter / Workspace
├─ Policy / Approval / Capability
├─ StateStore / CheckpointStore / EventLog / Outbox
├─ Queue / Worker / Lease / Fencing
└─ Trace / Test / Eval / Budget / Benchmark
↓
Deployment / Graceful Drain / Resume每周增量、接口与 invariant
| 版本 | 新增核心接口/状态 | 本版本必须成立的 invariant | 典型 failure injection |
|---|---|---|---|
| v0.1 | ModelClient, Run, Turn, ActionIntent | Model output 只是候选决策;Runtime 才提交事实 | unknown action type |
| v0.2 | ToolSpec, ToolRegistry, ToolCall, ToolResult | ToolCall 执行前完成 schema/domain validation | malformed ToolCall |
| v0.3 | ToolExecutor, ToolExecutionId, Observation | 每个 execution 有独立 lifecycle;timeout/cancel 可区分 | success + timeout + cancel 并发 |
| v0.4 | ContextManager, TokenBudget, ContextSnapshot | Model Context 是 Runtime state 的受控投影 | overflow + protected state compaction |
| v0.5 | Retriever, evidence/provenance | retrieval 是 Context source,不是 execution truth | high-similarity wrong evidence |
| v0.6 | RunStateMachine, TurnSettlement | terminal state 由可验证 evidence 驱动 | model falsely claims done |
| v0.7 | StructuredPlan, PlanValidator, GraphCompiler, Scheduler | Plan 不直接执行;Graph 先 deterministic validation | cycle / missing dependency |
| v0.8 | RetryPolicy, ExecutionRecord, IdempotencyKey, ApprovalGate | unknown side effect 不能被 LLM 自行重试 | timeout after possible side effect |
| v0.9 | StateStore, CheckpointStore, Outbox, resume state | crash 恢复不丢 execution facts、不重复已提交副作用 | crash-before-checkpoint / commit-before-dispatch |
| v0.10 | Run API, SSE cursor, dispatch envelope, lease/fencing token | disconnect≠cancel;redelivery/双 claim 不产生重复提交 | stream disconnect / stale worker commit |
| v0.11 | MCP Adapter, workspace capability, permission boundary | 协议/ToolResult 不能提升自身权限 | prompt/tool-result injection / capability escape |
| v0.12 | trace correlation, fault suite, eval, benchmark | 每个关键 failure 可复盘;容量边界可测 | overload / latency / retry storm |
| v1.0 | RunManifest, drain protocol, deployment config | rolling deploy 中 active run 可安全 drain/resume | SIGTERM during active run |
兼容性规则
每次升级都必须复用上一版本 domain identity 与 regression tests:
run_id、turn_id、tool_execution_id一旦成为 execution identity,不因新版本改名而丢失语义;- 新增 state 字段必须有默认值或 migration,旧 checkpoint fixture 仍可恢复;
- 不能为了接入 MCP/Queue/HTTP 而让 Provider/MCP SDK object 穿透 Runtime domain;
- 新增 capability 必须通过 Adapter / Policy 接入,不绕过既有 validation、execution record 与 trace;
- 上一周 failure injection 必须继续通过,本周只新增新的失败维度。
Golden Run:同一个任务贯穿 13 周
固定案例:修复 auth token expiration bug,并运行测试。
| 阶段 | 观察点 |
|---|---|
| W1 | User → Model → Action → Observation 的最小边界 |
| W2 | search_code/read_file/apply_patch Tool Schema |
| W3 | parallel search、timeout/cancel、ToolResult→Observation |
| W4 | 哪些文件/Tool/state 被投影到 Context |
| W5 | retrieval evidence 怎样进入 Context |
| W6 | Observation 如何推动 Run/Turn state |
| W7 | 修复计划怎样变为 validated Graph |
| W8 | patch/test 是否安全 retry;何时 approval |
| W9 | patch 成功后 checkpoint 前 crash 怎样恢复 |
| W10 | API 启动、stream 断开、Queue redelivery、worker fencing |
| W11 | workspace/shell/MCP capability 与安全边界 |
| W12 | 同一 run_id 的 model/tool/retry/queue trace + eval + benchmark |
| W13 | rolling deploy 中 active run 如何 drain/resume |
每周页面必须回答三项
- 本周以前系统做不到什么? — 作为学习动机;
- 本周只新增哪些抽象? — 防止提前过度设计;
- 本周新增的 failure injection 是什么? — 证明能力不是 happy path。