@@ -0,0 +1,71 @@
1+---
2+summary: "Local qa-channel scenarios for privacy-preserving personal assistant workflow checks."
3+read_when:
4+ - Running local personal agent reliability checks
5+ - Extending the repo-backed QA scenario catalog
6+ - Verifying reminder, reply, memory, redaction, and safe tool followthrough behavior
7+title: "Personal agent benchmark pack"
8+---
9+10+The Personal Agent Benchmark Pack is a small repo-backed QA scenario pack for
11+local personal assistant workflows. It is not a generic model benchmark and it
12+does not require a new runner. The pack reuses the private QA stack described in
13+[QA overview](/concepts/qa-e2e-automation), the synthetic
14+[QA channel](/channels/qa-channel), and the existing `qa/scenarios` markdown
15+catalog.
16+17+The first pack is intentionally narrow:
18+19+- fake personal reminders through local cron delivery
20+- fake DM and thread reply routing through `qa-channel`
21+- fake preference recall from the temporary QA workspace memory files
22+- fake secret no-echo checks
23+- safe read-backed tool followthrough after a short approval-style turn
24+25+## Scenarios
26+27+The machine-readable pack metadata lives in
28+`extensions/qa-lab/src/scenario-packs.ts`. The initial pack does not add a CLI
29+pack selector, so run the scenarios explicitly:
30+31+```bash
32+OPENCLAW_ENABLE_PRIVATE_QA_CLI=1 pnpm openclaw qa suite \
33+ --provider-mode mock-openai \
34+ --scenario personal-reminder-roundtrip \
35+ --scenario personal-channel-thread-reply \
36+ --scenario personal-memory-preference-recall \
37+ --scenario personal-redaction-no-secret-leak \
38+ --scenario personal-tool-safety-followthrough \
39+ --concurrency 1
40+```
41+42+The pack is designed for `qa-channel` with `mock-openai` or another local QA
43+provider lane. It should not be pointed at live chat services or real personal
44+accounts.
45+46+## Privacy Model
47+48+The scenarios use only fake users, fake preferences, fake secrets, and the
49+temporary QA gateway workspace created by the suite. They must not read or write
50+real OpenClaw user memory, sessions, credentials, launch agents, global configs,
51+or live gateway state.
52+53+Artifacts stay under the existing QA suite artifact directory and should be
54+treated like test output. Redaction checks use fake markers so failures are safe
55+to inspect and file in issues.
56+57+## Extending The Pack
58+59+Add new cases under `qa/scenarios/personal/`, then add the scenario id to
60+`QA_PERSONAL_AGENT_SCENARIO_IDS`. Keep each case small, local, deterministic in
61+`mock-openai`, and focused on one personal assistant behavior.
62+63+Good follow-up candidates:
64+65+- approval denial correctness
66+- multi-step task ledger assertions
67+- redacted trajectory export checks
68+- local-only plugin workflow checks
69+70+Avoid adding a new runner, plugin, dependency, live transport, or model judge
71+until the scenario catalog has enough stable cases to justify that surface.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。