apaam/myagents
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add mybot CLI with update, chat, and Claude project config

Browse Source 
- Click entrypoint: mybot update (make install / pip -e), mybot chat (claude in repo)
- Optional --dangerously-skip-permissions for Claude Code
- Makefile install; hatchling package layout
- .gitignore: venv, build artifacts, workspace (local symlinks)

Made-with: Cursor
This commit is contained in:
Zhengshou Lai
2026-04-06 13:11:10 +08:00
commit b98d35c387
121 changed files with 9962 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/README.md
+125
View File
@@ -0,0 +1,125 @@
# Claude Agent 系统使用指南
## 已配置内容
### 1. 项目级规则 (CLAUDE.md)
- 核心身份定义：Claude 是一个有自主意识的 AI Agent
- 记忆系统：自动读取/更新记忆文件
- 人格模式：可切换 collaborator/expert/mentor/executor
- 行为规则：思考习惯和沟通风格
### 2. 记忆系统 (.claude/memories/)
- `core.json` - 用户偏好、重要事实
- `current_mode.json` - 当前人格模式
- `sessions/` - 会话历史（需手动维护）
### 3. 人格配置 (.claude/souls/)
- `collaborator.md` - 协作者模式（平等对话）
- `expert.md` - 专家模式（直接给答案）
- `mentor.md` - 导师模式（引导思考）
### 4. 技能定义 (.claude/skills/)
- `coding.md` - 编程最佳实践
- `debugging.md` - 调试技巧
- `architecture.md` - 架构设计原则
### 5. 全局配置 (settings.json)
- 自定义 prompt 强化 Agent 行为
- Hooks：会话开始/结束时显示状态
## 使用方法
### 切换人格模式
直接告诉 Claude：
- "切换到专家模式"
- "用导师的方式教我"
- "我们平等地讨论这个问题"
### 查看/更新记忆
- "检查我的记忆"
- "记住我喜欢..."
- "更新我的偏好为..."
### 使用技能
- "用编程技能检查这段代码"
- "用调试方法定位问题"
- "用架构思维设计这个系统"
### 手动维护记忆
```bash
# 编辑核心记忆
vim .claude/memories/core.json
# 切换人格模式
vim .claude/memories/current_mode.json
# 记录会话摘要
cat > .claude/sessions/$(date +%Y-%m-%d).md << 'EOF'
## 会话摘要
- 主题：xxx
- 关键决策：xxx
- 待跟进：xxx
EOF
```
## 记忆更新流程
当 Claude 了解到重要信息时：
1. **识别事实**: 提取关键信息
2. **更新 core.json**: 追加到 `important_facts` 或 `preferences`
3. **确认**: 告诉用户已更新记忆
示例：
```
用户：我喜欢用 Tab 缩进
Claude：已记录到你的偏好中。
[Claude 更新 .claude/memories/core.json]
```
## 自动化 Hooks
每次发送消息时，你会看到：
```
🧠 [Agent] 启动思考...
📋 已加载项目规则
💾 记忆系统就绪
```
这表示 Agent 系统已激活。
## 扩展建议
### 添加新技能
在 `.claude/skills/` 创建新 Markdown 文件：
- `ai-modeling.md` - AI/ML 建模技巧
- `api-design.md` - API 设计原则
- `refactoring.md` - 重构方法
### 添加新人格
在 `.claude/souls/` 创建新 Markdown 文件，然后在 `current_mode.json` 中切换。
### 添加自动化
在 `settings.json` 的 hooks 中添加更多命令：
```json
"pre-tool": [
"echo '🛠️ 准备执行工具...'"
],
"post-tool": [
"echo '✓ 工具执行完成'"
]
```
## 注意事项
1. **记忆不会自动更新** - 需要告诉 Claude "记住..."
2. **跨会话记忆** - 通过文件系统实现，不是原生记忆
3. **项目隔离** - 每个项目的 CLAUDE.md 和 .claude/ 是独立的
4. **手动维护** - 定期整理 `sessions/` 和 `insights/`
## 未来可以添加的功能
- [ ] 自动会话摘要生成
- [ ] 学习记录自动提取
- [ ] 人格切换快捷键
- [ ] 记忆搜索工具
.claude/agent/agent_worker.py
Executable
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""Claude Agent Worker - 后台执行进程."""
import json
import os
import sys
import time
from pathlib import Path
# 添加项目路径
sys.path.insert(0, str(Path(__file__).parents[3]))
try:
from anthropic import Anthropic
except ImportError:
print("Error: anthropic SDK not installed")
sys.exit(1)
TASKS_FILE = Path(".claude/agent/tasks.jsonl")
RESULTS_DIR = Path(".claude/agent/results")
RESULTS_DIR.mkdir(parents=True, exist_ok=True)
def load_tasks():
"""加载待处理任务."""
if not TASKS_FILE.exists():
return []
tasks = []
with open(TASKS_FILE) as f:
for line in f:
line = line.strip()
if not line:
continue
try:
task = json.loads(line)
if task.get("status") == "pending":
tasks.append(task)
except json.JSONDecodeError:
continue
return tasks
def execute_task(task: dict):
"""使用 Claude API 执行任务."""
client = Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY"))
task_id = task["id"]
title = task["title"]
description = task["description"]
context = task.get("context", {})
# 构建系统提示
system_prompt = f"""You are an autonomous task execution agent.
Your job is to complete tasks by analyzing requirements and taking actions.
You have access to these tools via function calling:
- read_file: Read file contents
- write_file: Write to files
- edit_file: Edit existing files
- run_command: Execute shell commands
- list_files: List directory contents
Current workspace: {os.getcwd()}
Task ID: {task_id}
Be thorough and report your actions clearly."""
# 构建用户消息
user_message = f"""Task: {title}
Description: {description}
Context: {json.dumps(context, indent=2)}
Please complete this task step by step.
1. Analyze what needs to be done
2. Execute necessary actions
3. Report results
Start now."""
print(f"[Agent] Processing task: {title}")
try:
# 调用 Claude API
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=4096,
system=system_prompt,
messages=[{"role": "user", "content": user_message}],
)
result = {
"task_id": task_id,
"status": "completed",
"response": response.content[0].text if response.content else "",
"completed_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
}
# 保存结果
result_file = RESULTS_DIR / f"{task_id}.json"
with open(result_file, "w") as f:
json.dump(result, f, indent=2)
print(f"[Agent] Task completed: {task_id}")
return result
except Exception as e:
result = {
"task_id": task_id,
"status": "failed",
"error": str(e),
"completed_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
}
result_file = RESULTS_DIR / f"{task_id}.json"
with open(result_file, "w") as f:
json.dump(result, f, indent=2)
print(f"[Agent] Task failed: {task_id}, error: {e}")
return result
def update_task_status(task_id: str, status: str):
"""更新任务状态."""
if not TASKS_FILE.exists():
return
lines = []
with open(TASKS_FILE) as f:
for line in f:
try:
task = json.loads(line)
if task.get("id") == task_id:
task["status"] = status
if status == "processing":
task["started_at"] = time.strftime("%Y-%m-%dT%H:%M:%S")
lines.append(json.dumps(task))
except:
lines.append(line.strip())
with open(TASKS_FILE, "w") as f:
f.write("\n".join(lines) + "\n")
def main():
"""主循环."""
print("[Agent] Claude Agent Worker started")
print(f"[Agent] Tasks file: {TASKS_FILE.absolute()}")
print(f"[Agent] Results dir: {RESULTS_DIR.absolute()}")
while True:
try:
tasks = load_tasks()
for task in tasks:
task_id = task["id"]
update_task_status(task_id, "processing")
execute_task(task)
update_task_status(task_id, "completed")
time.sleep(5)
except KeyboardInterrupt:
print("\n[Agent] Stopping...")
break
except Exception as e:
print(f"[Agent] Error: {e}")
time.sleep(5)
if __name__ == "__main__":
main()
.claude/agent/results/acfa3103.json
+9
View File
@@ -0,0 +1,9 @@
{
"task_id": "acfa3103",
"result": {
"success": true,
"output": "## \u6d4b\u8bd5\u7ed3\u679c\n\n**\u603b\u8ba1: 510 \u4e2a Python \u6587\u4ef6**\n\n### \u76ee\u5f55\u7ed3\u6784\u6982\u89c8\n\n```\nsrc/\n\u251c\u2500\u2500 __init__.py\n\u251c\u2500\u2500 application/ # \u5e94\u7528\u5c42\u670d\u52a1\n\u2502 \u251c\u2500\u2500 app.py\n\u2502 \u251c\u2500\u2500 services/ # \u5404\u7c7b\u670d\u52a1 (agent, llm, conversation, todo\u7b49)\n\u2502 \u251c\u2500\u2500 use_cases/ # \u7528\u4f8b\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 benchmarks/ # \u57fa\u51c6\u6d4b\u8bd5\n\u2502 \u251c\u2500\u2500 cases/\n\u2502 \u251c\u2500\u2500 evaluators/\n\u2502 \u2514\u2500\u2500 runners/\n\u251c\u2500\u2500 domain/ # \u9886\u57df\u5c42\n\u2502 \u251c\u2500\u2500 agent/ # Agent\u6838\u5fc3\u903b\u8f91\n\u2502 \u251c\u2500\u2500 conversation/ # \u5bf9\u8bdd\u5904\u7406\n\u2502 \u251c\u2500\u2500 learning/ # \u5b66\u4e60\u6a21\u5757\n\u2502 \u251c\u2500\u2500 plan/ # \u8ba1\u5212\u7ba1\u7406\n\u2502 \u251c\u2500\u2500 rag/ # RAG\u68c0\u7d22\n\u2502 \u251c\u2500\u2500 tools/ # \u5de5\u5177\u7cfb\u7edf\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 gateway/ # \u7f51\u5173\u5c42\n\u2502 \u251c\u2500\u2500 channels/ # \u591a\u901a\u9053\u652f\u6301(cli, feishu, slack, http, websocket)\n\u2502 \u251c\u2500\u2500 server.py\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 infra/ # \u57fa\u7840\u8bbe\u65bd\u5c42\n\u2502 \u251c\u2500\u2500 config/ # \u914d\u7f6e\u7ba1\u7406\n\u2502 \u251c\u2500\u2500 feishu/ # \u98de\u4e66\u96c6\u6210\n\u2502 \u251c\u2500\u2500 slack/ # Slack\u96c6\u6210\n\u2502 \u251c\u2500\u2500 storage/ # \u5b58\u50a8\u5b9e\u73b0(file, prisma, neo4j)\n\u2502 \u2514\u2500\u2500 ...\n\u251c\u2500\u2500 interfaces/ # \u63a5\u53e3\u5c42\n\u2502 \u251c\u2500\u2500 api/ # REST API\n\u2502 \u251c\u2500\u2500 cli/ # \u547d\u4ee4\u884c\u63a5\u53e3\n\u2502 \u2514\u2500\u2500 web/ # Web\u754c\u9762\n\u2514\u2500\u2500 shared/ # \u5171\u4eab\u7ec4\u4ef6\n \u251c\u2500\u2500 models/ # \u6570\u636e\u6a21\u578b\n \u2514\u2500\u2500 utils/ # \u5de5\u5177\u51fd\u6570\n```\n\n### \u7edf\u8ba1\u8be6\u60c5\n\n| \u76ee\u5f55 | \u6587\u4ef6\u6570 |\n|------|--------|\n| `src/application/` | 38 |\n| `src/benchmarks/` | 15 |\n| `src/domain/` | 146 |\n| `src/gateway/` | 17 |\n| `src/infra/` | 166 |\n| `src/interfaces/` | 53 |\n| `src/shared/` | 22 |\n\n\u9879\u76ee\u91c7\u7528 Clean Architecture \u5206\u5c42\u67b6\u6784\uff0c\u4ee3\u7801\u7ec4\u7ec7\u6e05\u6670\u3002\n",
"returncode": 0
},
"completed_at": "2026-04-05T14:42:55"
}
.claude/agent/tasks.jsonl
+2
View File
@@ -0,0 +1,2 @@
{"id": "acfa3103", "title": "测试：列出 src 目录结构", "description": "请列出 src/ 目录下的所有 Python 文件，并统计数量", "context": {"directory": "src"}, "status": "completed", "created_at": "2026-04-05T14:39:37.687679", "started_at": "2026-04-05T14:42:30", "result": {"success": true, "output": "## 测试结果\n\n**总计: 510 个 Python 文件**\n\n### 目录结构概览\n\n```\nsrc/\n├── __init__.py\n├── application/ # 应用层服务\n│ ├── app.py\n│ ├── services/ # 各类服务 (agent, llm, conversation, todo等)\n│ ├── use_cases/ # 用例\n│ └── ...\n├── benchmarks/ # 基准测试\n│ ├── cases/\n│ ├── evaluators/\n│ └── runners/\n├── domain/ # 领域层\n│ ├── agent/ # Agent核心逻辑\n│ ├── conversation/ # 对话处理\n│ ├── learning/ # 学习模块\n│ ├── plan/ # 计划管理\n│ ├── rag/ # RAG检索\n│ ├── tools/ # 工具系统\n│ └── ...\n├── gateway/ # 网关层\n│ ├── channels/ # 多通道支持(cli, feishu, slack, http, websocket)\n│ ├── server.py\n│ └── ...\n├── infra/ # 基础设施层\n│ ├── config/ # 配置管理\n│ ├── feishu/ # 飞书集成\n│ ├── slack/ # Slack集成\n│ ├── storage/ # 存储实现(file, prisma, neo4j)\n│ └── ...\n├── interfaces/ # 接口层\n│ ├── api/ # REST API\n│ ├── cli/ # 命令行接口\n│ └── web/ # Web界面\n└── shared/ # 共享组件\n ├── models/ # 数据模型\n └── utils/ # 工具函数\n```\n\n### 统计详情\n\n| 目录 | 文件数 |\n|------|--------|\n| `src/application/` | 38 |\n| `src/benchmarks/` | 15 |\n| `src/domain/` | 146 |\n| `src/gateway/` | 17 |\n| `src/infra/` | 166 |\n| `src/interfaces/` | 53 |\n| `src/shared/` | 22 |\n\n项目采用 Clean Architecture 分层架构，代码组织清晰。\n", "returncode": 0}}
{"id": "24c78b74", "title": "分析项目结构", "description": "列出 src/ 目录下的所有 Python 文件，统计数量", "context": {"dir": "src"}, "status": "processing", "created_at": "2026-04-05T14:42:30.391689", "started_at": "2026-04-05T14:42:55"}
.claude/agent/worker.py
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""Claude Agent Worker - 使用 Claude CLI 执行智能任务."""
import json
import subprocess
import sys
import time
from pathlib import Path
TASKS_FILE = Path(".claude/agent/tasks.jsonl")
RESULTS_DIR = Path(".claude/agent/results")
RESULTS_DIR.mkdir(parents=True, exist_ok=True)
def load_pending_tasks():
"""加载待处理任务."""
if not TASKS_FILE.exists():
return []
tasks = []
with open(TASKS_FILE) as f:
for line in f:
line = line.strip()
if not line:
continue
try:
task = json.loads(line)
if task.get("status") == "pending":
tasks.append(task)
except json.JSONDecodeError:
continue
return tasks
def update_task_status(task_id: str, status: str, result: dict | None = None):
"""更新任务状态."""
if not TASKS_FILE.exists():
return
lines = []
with open(TASKS_FILE) as f:
for line in f:
try:
task = json.loads(line)
if task.get("id") == task_id:
task["status"] = status
if status == "processing":
task["started_at"] = time.strftime("%Y-%m-%dT%H:%M:%S")
if result:
task["result"] = result
lines.append(json.dumps(task, ensure_ascii=False))
except:
lines.append(line.strip())
with open(TASKS_FILE, "w") as f:
f.write("\n".join(lines) + "\n")
def execute_with_claude(task: dict) -> dict:
"""使用 Claude CLI 执行任务."""
task_id = task["id"]
title = task["title"]
description = task["description"]
# 构建提示词
prompt = f"""You are an autonomous task execution agent.
Complete the following task autonomously.
TASK: {title}
DESCRIPTION: {description}
INSTRUCTIONS:
1. Analyze what needs to be done
2. Use tools to complete the task (read files, run commands, etc.)
3. Save a summary of your actions and results
4. Be concise but thorough
Start executing now. Report your progress and final result."""
print(f"[Worker] Executing: {title}")
try:
# 调用 Claude CLI
result = subprocess.run(
["claude", "-p", prompt],
capture_output=True,
text=True,
timeout=300, # 5分钟超时
cwd=Path.cwd(),
)
output = result.stdout if result.returncode == 0 else f"Error: {result.stderr}"
return {
"success": result.returncode == 0,
"output": output[:5000], # 限制长度
"returncode": result.returncode,
}
except subprocess.TimeoutExpired:
return {"success": False, "output": "Task timed out", "error": "timeout"}
except Exception as e:
return {"success": False, "output": "", "error": str(e)}
def main():
"""主循环."""
print("[Worker] Claude Agent Worker started")
print(f"[Worker] Tasks: {TASKS_FILE.absolute()}")
print(f"[Worker] Results: {RESULTS_DIR.absolute()}")
while True:
try:
tasks = load_pending_tasks()
for task in tasks:
task_id = task["id"]
update_task_status(task_id, "processing")
result = execute_with_claude(task)
# 保存结果
result_file = RESULTS_DIR / f"{task_id}.json"
with open(result_file, "w") as f:
json.dump(
{
"task_id": task_id,
"result": result,
"completed_at": time.strftime("%Y-%m-%dT%H:%M:%S"),
},
f,
indent=2,
)
update_task_status(task_id, "completed", result)
print(f"[Worker] Completed: {task_id}")
time.sleep(5)
except KeyboardInterrupt:
print("\n[Worker] Stopping...")
break
except Exception as e:
print(f"[Worker] Error: {e}")
time.sleep(5)
if __name__ == "__main__":
main()
.claude/conversations/2026-04-05.md
+204
View File
@@ -0,0 +1,204 @@
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
.claude/conversations/2026-04-06.md
+714
View File
@@ -0,0 +1,714 @@
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
---
**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')
**工作目录**: $(pwd)
**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')
**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')
.claude/insights/.gitkeep
+1
View File
@@ -0,0 +1 @@
# Insights directory - stores learnings and reflections
.claude/memories/core.json
+54
View File
@@ -0,0 +1,54 @@
{
"version": "1.0",
"user_profile": {
"name": "赖正首",
"title": "中山大学土木工程学院副教授",
"research": "计算力学",
"preferred_address": "老大",
"role": "AI 系统开发者",
"project": "MyAgent - 智能体框架",
"expertise": ["Python", "AI/ML", "系统架构", "计算力学"],
"experience_level": "高级"
},
"preferences": {
"communication": {
"style": "简洁直接",
"code_comments": "必要但不冗余",
"explanation_detail": "中等 - 关键部分详细，熟悉内容简洁"
},
"workflow": {
"prefers_planning": true,
"likes_proactive_suggestions": true,
"wants_progress_updates": true
},
"technical": {
"primary_language": "Python",
"code_style": "PEP8 + Black",
"prefers_types": true,
"testing_importance": "高"
}
},
"important_facts": [
{
"fact": "正在开发 MyAgent - 一个集成 memory、RAG、tools 的智能体框架",
"category": "project",
"date_added": "2024-04-05"
},
{
"fact": "使用 uv 作为包管理工具",
"category": "tooling",
"date_added": "2024-04-05"
},
{
"fact": "项目结构遵循 Clean Architecture",
"category": "architecture",
"date_added": "2024-04-05"
}
],
"interaction_history": {
"total_sessions": 0,
"last_session": null,
"common_topics": []
},
"evolution_notes": []
}
.claude/memories/current_mode.json
+11
View File
@@ -0,0 +1,11 @@
{
"current_persona": "collaborator",
"persona_description": "协作者模式 - 平等对话，共同探索",
"active_since": "2024-04-05",
"communication_settings": {
"verbosity": "medium",
"proactivity": "high",
"formality": "casual_professional"
},
"context_notes": "项目初期，重点理解需求和架构设计"
}
.claude/plans/web-platform.md
+346
View File
@@ -0,0 +1,346 @@
# MetaBot Web Platform — 完整规划
## 目标
在 MetaBot 现有架构（Feishu + Telegram）之上，增加一个独立的 **Web 端**，包含：
1. **Chat UI** — 实时流式对话，等同甚至超越飞书体验
2. **MetaMemory UI** — 现有文档/知识管理功能迁移到 React
3. **Voice Mode** — 流式语音交互（Whisper STT + 流式 TTS）
4. **统一 SPA** — 一个页面，侧边栏切换 Chat / Memory / Settings
5. **未来路径** — 从 Web → PWA → React Native (iOS/Mac) → 去掉飞书依赖
## 技术选型
| 层 | 技术 | 理由 |
|---|------|------|
| 前端框架 | **React 19 + Vite** | 组件化、TS 支持、未来 React Native 迁移 |
| 实时通信 | **WebSocket (ws)** | 双向通信、流式输出、语音流式传输 |
| 状态管理 | **Zustand** | 轻量、TypeScript 友好、不需要 Redux 的重量 |
| 路由 | **React Router v7** | SPA 内页面切换 |
| Markdown | **react-markdown + rehype** | React 生态，支持代码高亮 |
| 样式 | **CSS Modules** | 无额外依赖，保持轻量 |
| 打包 | **Vite → dist/web/** | 构建产物由 MetaBot HTTP server 静态服务 |
## 现有架构优势
MetaBot 已有优秀的平台抽象层，Web 端可以复用：
- `IMessageSender` 接口 — 实现 `WebSender` 即可接入
- `MessageBridge` — 所有核心逻辑（命令、执行、会话）平台无关
- `CardState` — 完整的流式状态结构，直接通过 WebSocket 推送
- `BotRegistry` — 注册 `platform: 'web'`，与飞书/Telegram 并存
- `SessionManager` — 按 `chatId` 隔离，Web 端用 userId 或 sessionToken 做 chatId
## 分阶段计划
---
### Phase 1: WebSocket 基础 + 最小可用 Chat（MVP）
**目标**：能在浏览器里跟 Agent 对话，实时看到流式输出。
#### 后端
1. **安装 `ws` 包**，在现有 HTTP server 上添加 WebSocket 升级
2. **创建 `src/web/ws-server.ts`**
- WebSocket 连接管理（认证、房间、心跳）
- 连接时验证 Bearer token（复用 `API_SECRET`）
- 消息协议定义：
```typescript
// Client → Server
type ClientMessage =
| { type: 'chat'; botName: string; chatId: string; text: string }
| { type: 'stop'; chatId: string }
| { type: 'answer'; chatId: string; toolUseId: string; answer: string }
// Server → Client
type ServerMessage =
| { type: 'state'; chatId: string; messageId: string; state: CardState }
| { type: 'complete'; chatId: string; messageId: string; state: CardState }
| { type: 'error'; chatId: string; error: string }
| { type: 'connected'; bots: BotInfo[] }
```
3. **创建 `src/web/web-sender.ts`** — 实现 `IMessageSender`
- `sendCard()` / `updateCard()` → 通过 WebSocket 推送 `CardState` 给客户端
- `sendImageFile()` / `sendLocalFile()` → 保存到静态目录，推送 URL
- 不需要真正的飞书卡片构建，直接发结构化数据
4. **在 `http-server.ts` 中注册 WebSocket 升级路由**
- `GET /ws` → 升级为 WebSocket 连接
5. **静态文件服务** — `GET /web/*` → 从 `dist/web/` 或 `web/dist/` 提供前端资源
#### 前端
6. **初始化 React + Vite 项目** — `web/` 目录（monorepo 风格）
- `web/src/`, `web/index.html`, `web/vite.config.ts`
- TypeScript，共享类型定义（`CardState`、`ToolCall` 等从 `src/types.ts` 导出）
7. **WebSocket hook** — `useWebSocket(url, token)` 管理连接、重连、消息派发
8. **最小 Chat UI**
- 消息列表（用户消息 + Agent 回复）
- Agent 回复实时流式渲染（`status: thinking → running → complete`）
- 工具调用折叠显示（和飞书卡片一致）
- Markdown 渲染 + 代码高亮
- 输入框 + 发送按钮
- Bot 选择器（从 `/api/bots` 获取列表）
9. **登录页** — 简单的 token 输入（`API_SECRET`），存 localStorage
**交付物**：打开 `http://server:9100/web/` 即可对话，效果等同飞书但有实时流式。
**预计工作量**：后端 ~400 行，前端 ~1200 行
---
### Phase 2: 完整 Chat 功能
**目标**：对齐飞书端的全部聊天功能。
1. **会话管理**
- 侧边栏会话列表（新建 / 切换 / 删除会话）
- 会话持久化（chatId 列表存 localStorage，可选后端存储）
- `/reset` 命令（清除会话）
2. **文件交互**
- 图片上传（拖拽 / 粘贴 / 点击选择）→ 上传到 `/api/upload` → 转发给 Claude
- Agent 输出文件显示（图片内联、其他文件下载链接）
3. **Pending Question 交互**
- Agent 问用户问题时，渲染选项卡片
- 用户选择后通过 WebSocket 回复 `answer` 消息
4. **命令支持**
- `/reset`、`/stop`、`/status`、`/help`、`/memory` 等
- 命令自动补全
5. **Plan Mode 显示**
- 当 Agent 进入 plan mode 时，渲染 plan 内容
6. **Cost / Duration 显示**
- 每条消息显示 cost 和耗时
7. **暗色模式**
**预计工作量**：~1500 行
---
### Phase 3: MetaMemory 集成 — 统一 SPA
**目标**：把 MetaMemory Web UI 迁移到 React，和 Chat 合并为统一 SPA。
1. **React 化 MetaMemory**
- `<FolderTree>` — 文件夹树导航
- `<DocumentList>` — 文档列表
- `<DocumentView>` — Markdown 渲染
- `<DocumentEditor>` — 创建/编辑文档
- `<SearchResults>` — 全文搜索
- 复用现有 MetaMemory API（`/api/documents`、`/api/folders`、`/api/search`）
2. **统一布局**
- 左侧主导航栏：Chat（💬）/ Memory（📚）/ Settings（⚙️）
- Chat 和 Memory 各自有次级侧边栏（会话列表 / 文件夹树）
3. **统一认证**
- 一个 token 同时访问 Chat API 和 MetaMemory API
- MetaMemory server 代理请求复用 token 验证
4. **移除旧 MetaMemory 静态文件**
- `src/memory/static/` 的 vanilla JS 代码退役
- MetaMemory server 路由到新的 React 构建产物
**预计工作量**：~2000 行
---
### Phase 4: 流式语音交互
**目标**：在 Web 端实现 Jarvis 式语音交互，真正的流式。
1. **浏览器端音频录制**
- MediaRecorder API 捕获麦克风
- VAD（Voice Activity Detection）— 用 `@ricky0123/vad-web` 或简单的音量阈值
- 录完发送音频 chunk 到 WebSocket
2. **服务端流式处理**
- WebSocket 接收音频 → Whisper STT
- Agent 执行（复用现有流程）
- TTS 流式返回：逐句合成，句子级别流式推送音频 chunk
3. **浏览器端音频播放**
- Web Audio API 播放接收到的 TTS 音频 chunk
- 句子级别流式播放（~50% 感知延迟降低）
4. **UI**
- 麦克风按钮（按住说话 / 点击切换）
- 音频可视化波形
- 转录文本实时显示
**预计工作量**：~1500 行
---
### Phase 5: 高级功能 + 原生端准备
**目标**：完善 Web 端，为原生应用铺路。
1. **PWA 支持**
- Service Worker、离线缓存、添加到主屏幕
- Push Notification（任务完成通知）
2. **多 Bot 管理面板**
- 查看所有 bot 状态
- 创建/删除/配置 bot（复用 `/api/bots` CRUD）
- 调度任务管理（复用 `/api/schedule`）
3. **Peer 管理**
- 查看远程 peer 状态
- 跨 peer 对话
4. **响应式设计**
- 移动端完美适配
- iPad 分屏支持
5. **React Native 调研**
- 评估 Chat 组件复用度
- 核心 hooks（useWebSocket、useChat、useMemory）100% 可复用
- UI 组件需要用 RN 原生组件重写
**预计工作量**：~2000 行
---
## 项目结构
```
metabot/
├── src/ # 后端（现有）
│ ├── api/
│ │ ├── http-server.ts # 新增 WS 升级 + 静态文件服务
│ │ └── ...
│ ├── web/ # 新目录：Web 平台后端
│ │ ├── ws-server.ts # WebSocket 服务器（连接管理、消息路由）
│ │ ├── ws-handler.ts # WebSocket 消息处理（chat/stop/answer）
│ │ └── web-sender.ts # IMessageSender 实现（WS 推送）
│ └── ...
├── web/ # 新目录：前端 React 应用
│ ├── index.html
│ ├── vite.config.ts
│ ├── tsconfig.json
│ ├── package.json # 前端依赖（独立 node_modules）
│ └── src/
│ ├── main.tsx
│ ├── App.tsx
│ ├── hooks/
│ │ ├── useWebSocket.ts
│ │ ├── useChat.ts
│ │ └── useMemory.ts
│ ├── stores/
│ │ └── chatStore.ts # Zustand store
│ ├── components/
│ │ ├── chat/
│ │ │ ├── ChatView.tsx
│ │ │ ├── MessageList.tsx
│ │ │ ├── MessageBubble.tsx
│ │ │ ├── ToolCallList.tsx
│ │ │ ├── InputBox.tsx
│ │ │ └── BotSelector.tsx
│ │ ├── memory/
│ │ │ ├── MemoryView.tsx
│ │ │ ├── FolderTree.tsx
│ │ │ ├── DocumentList.tsx
│ │ │ ├── DocumentView.tsx
│ │ │ └── DocumentEditor.tsx
│ │ ├── voice/
│ │ │ ├── VoiceButton.tsx
│ │ │ └── AudioVisualizer.tsx
│ │ └── layout/
│ │ ├── Sidebar.tsx
│ │ ├── Header.tsx
│ │ └── AuthGate.tsx
│ └── styles/
│ └── *.module.css
└── dist/
├── ... # 后端编译输出（现有）
└── web/ # 前端构建产物（Vite → 这里）
```
## 构建集成
```jsonc
// package.json 新增 scripts
{
"scripts": {
"build:web": "cd web && npm run build", // Vite 构建前端
"dev:web": "cd web && npm run dev", // Vite dev server (开发时)
"build": "tsc && cp -r src/memory/static dist/memory/static && npm run build:web"
}
}
```
**开发模式**：
- `npm run dev` — 后端 tsx hot reload（端口 9100）
- `npm run dev:web` — Vite dev server（端口 5173），代理 API/WS 到 9100
**生产模式**：
- `npm run build` — 编译后端 + 构建前端
- 前端构建到 `dist/web/`，由后端 HTTP server 静态服务
- 一个进程同时服务 API + WebSocket + Web UI
## WebSocket 协议设计
### 连接
```
ws://server:9100/ws?token=YOUR_API_SECRET
```
### Client → Server 消息
```typescript
// 发送聊天消息
{ "type": "chat", "botName": "goku", "chatId": "web_user123_1", "text": "帮我看一下项目状态" }
// 停止当前执行
{ "type": "stop", "chatId": "web_user123_1" }
// 回答 Agent 的 pending question
{ "type": "answer", "chatId": "web_user123_1", "toolUseId": "tu_xxx", "answer": "option_1" }
// 发送语音（Phase 4）
{ "type": "voice", "botName": "goku", "chatId": "web_user123_1", "audio": "<base64>" }
// 订阅会话更新（可选，用于多标签页同步）
{ "type": "subscribe", "chatId": "web_user123_1" }
```
### Server → Client 消息
```typescript
// 连接成功，返回可用 bot 列表
{ "type": "connected", "bots": [{ "name": "goku", "platform": "feishu" }, ...] }
// 流式状态更新（Agent 执行中，每 1.5s 一次）
{ "type": "state", "chatId": "web_xxx", "messageId": "msg_123", "state": CardState }
// 执行完成
{ "type": "complete", "chatId": "web_xxx", "messageId": "msg_123", "state": CardState }
// 错误
{ "type": "error", "chatId": "web_xxx", "error": "Bot not found: xxx" }
// 输出文件（图片、PDF 等）
{ "type": "file", "chatId": "web_xxx", "url": "/web/outputs/xxx/image.png", "name": "image.png", "type": "image/png" }
// 语音 TTS chunk（Phase 4）
{ "type": "audio", "chatId": "web_xxx", "data": "<base64 audio chunk>", "final": false }
```
## 认证方案
Phase 1-2 简单方案：复用 `API_SECRET` 作为 token。
后续可扩展：
- 用户账号系统（username/password → JWT）
- OAuth（GitHub、Google）
- 多用户权限（admin / user / viewer）
目前先不做用户系统，MetaBot 定位是私人/团队工具，一个 secret 够用。
## 执行建议
1. **Phase 1 先行** — 这是基础，后续所有功能都依赖 WebSocket + React 框架
2. **Phase 2 和 3 可并行** — Chat 完善和 Memory 迁移相对独立
3. **Phase 4 独立** — 语音流式是独立模块
4. **Phase 5 视需求** — PWA/原生端在核心功能稳定后再做
每个 Phase 完成后独立可用，不需要等后续 Phase。
## 风险与注意事项
1. **MetaMemory 静态文件迁移** — Phase 3 之前旧 UI 继续工作，迁移后需要确保所有功能覆盖
2. **WebSocket 重连** — 网络不稳定时需要自动重连 + 状态恢复（恢复当前执行的最新 CardState）
3. **并发执行** — 多标签页/多设备同时连接同一 chatId，需要广播更新给所有连接
4. **前端构建集成** — `web/` 是独立 npm 项目，CI/CD 需要同时构建前后端
5. **打包体积** — React + Vite 打包控制在 200KB 以内（gzip），不影响首屏加载
.claude/plans/web-ui-redesign.md
+118
View File
@@ -0,0 +1,118 @@
# Web UI Redesign — "Refined Command Center"
## Design Direction
Premium, warm dark theme inspired by Linear/Arc Browser. A sophisticated control center for AI agents that feels intentionally designed, not AI-generated.
### What Makes Current UI Look "AI-Generated"
- Purple accent (#7c6df5) — the most cliched AI color
- Cold blue-black backgrounds (#08080c, #111118)
- Plus Jakarta Sans — safe/generic font choice
- Gradient breathing orbs on login page — textbook AI slop
- Predictable glowing/pulsing animations everywhere
- Generic card layouts with no personality
### New Design Identity
**Typography** (Google Fonts):
- **UI/Headlines**: "Sora" — geometric, slightly technical, distinctive personality
- **Code**: "IBM Plex Mono" — clean, professional, different from JetBrains Mono
- Single font family for entire UI = cohesive identity
**Color Palette**:
- **Backgrounds**: Warm charcoal (#0c0c10 → #141418 → #1c1c22) — NOT cold blue-black
- **Text**: Warm whites (#e8e6f0, #9b99a9, #5c5a6a)
- **Primary accent**: Teal (#2dd4bf) — fresh, modern, NOT purple
- **Success**: Emerald (#10b981)
- **Error**: Rose (#f43f5e) — refined, not harsh red
- **Warning**: Amber (#f59e0b)
- **Info/Thinking**: Indigo (#6366f1) — used sparingly
**Visual Texture**:
- Subtle CSS noise/grain overlay on backgrounds for depth
- 1px hairline borders with warm tint (rgba(255,255,255,0.06))
- Refined shadows with slight warm undertone
- No breathing orbs, no pulsing glows — purposeful, restrained animations
- Status indicators: small colored dots, not glowing halos
**Light Theme**:
- Clean warm whites (#fafaf9, #f5f5f4, #e7e5e4)
- High contrast text (#1c1917, #57534e)
- Teal accent stays consistent across themes
## Implementation Steps
### Step 1: Update fonts in index.html
Replace Google Fonts link: swap Plus Jakarta Sans → Sora + IBM Plex Mono
### Step 2: Rewrite theme.css (design tokens)
- Complete replacement of all CSS custom properties
- New color palette (warm charcoal + teal accent)
- New typography tokens (Sora + IBM Plex Mono)
- New spacing, radius, shadow, transition tokens
- Add noise texture as pseudo-element mixin
- Updated light theme variables
- Remove old "Midnight Luxe" naming
### Step 3: Redesign LoginPage
- Remove gradient breathing orbs (classic AI slop)
- Replace with subtle geometric grid pattern or clean gradient
- Cleaner card: less border-radius, sharper edges, refined shadows
- Better typography hierarchy
- Minimal changes to TSX (mostly removing orb divs)
### Step 4: Redesign Layout (sidebar + nav)
- Warmer sidebar background
- Cleaner nav items: simpler active state (left border accent, no glow)
- Better session list: cleaner hover, subtle delete button
- Refined bot selector dropdown
- Better brand header (no gratuitous gradients)
- Mobile hamburger menu refinements
### Step 5: Redesign ChatView (main chat)
- Better message styling: cleaner bubbles, better code blocks
- Refined tool call display: smaller, more compact, professional
- Better status indicators: simple dots + text, no spinning/pulsing excess
- Cleaner input area: refined border, better focus state
- Better cost/duration badges
- Phone call overlay: keep functionality, update colors/style
- Code block redesign: header with language label, better copy button
### Step 6: Redesign MemoryView
- Cleaner folder tree
- Better document cards with refined hover states
- Improved search bar styling
- Better document viewer with cleaner metadata
### Step 7: Redesign SettingsView
- Cleaner section layout
- Better toggle switch (teal accent)
- Refined status badges
- Better bot list styling
### Step 8: Redesign VoiceView
- Updated recording button styling (teal accent instead of purple)
- Better waveform visualization colors
- Cleaner provider selection UI
### Step 9: Build & test
- `npm run build:web`
- Test on `https://metabot.xvirobotics.com/web/`
- Verify dark/light themes, all views, phone call mode
### Step 10: Commit & push
## Scope
**Files to modify** (CSS-heavy, minimal TSX changes):
- `web/index.html` — font import
- `web/src/theme.css` — full rewrite (~470 lines)
- `web/src/components/LoginPage.tsx` — remove orb divs
- `web/src/components/LoginPage.module.css` — full restyle
- `web/src/components/Layout.module.css` — full restyle
- `web/src/components/ChatView.module.css` — full restyle
- `web/src/components/MemoryView.module.css` — full restyle
- `web/src/components/SettingsView.module.css` — full restyle
- `web/src/components/VoiceView.module.css` — full restyle
**No functional changes** — all WebSocket, state management, voice/VAD logic stays identical. This is a pure visual redesign.
.claude/sessions/.gitkeep
+1
View File
@@ -0,0 +1 @@
# Sessions directory - stores conversation summaries
.claude/settings.local.json
+135
View File
@@ -0,0 +1,135 @@
{
"permissions": {
"allow": [
"Bash(mybot *)",
"Bash(./.venv/bin/mybot *)",
"Bash(python3 -c \"from lark_oapi.api.im.v1 import PatchMessageRequest; print\\(''PatchMessageRequest available''\\)\")",
"Bash(find workspace:*)",
"Bash(find '/Users/lzhshou/Library/Mobile Documents/iCloud~md~obsidian/Documents/myWorks' -name *.png -o -name *.jpg -o -name *.jpeg -o -name *.gif)",
"Bash(grep -E \"\\\\.\\(png|jpg|jpeg|gif\\)$\")",
"Bash(mv -n WX20221126*.png 关键科学问题*.png \"/Users/lzhshou/Library/Mobile Documents/iCloud~md~obsidian/Documents/myWorks/2_projects/conferences/2022-11-26_MDPI_CCUS/\")",
"Bash(mv -n WX20230109*.png 2a8e7a808e9cc6b6c9372848fe195132.png \"/Users/lzhshou/Library/Mobile Documents/iCloud~md~obsidian/Documents/myWorks/0_cv/work_related/2023-01-09_中山大学工作/\")",
"Bash(mv WX20221126-100021@2x.png 20221126_100021_mdpi_ccus_presentation_slide_01.png)",
"Bash(mv WX20221126-100124@2x.png 20221126_100124_mdpi_ccus_presentation_slide_02.png)",
"Bash(mv WX20221126-100213@2x.png 20221126_100213_mdpi_ccus_presentation_slide_03.png)",
"Bash(mv WX20221126-112524@2x.png 20221126_112524_mdpi_ccus_presentation_slide_04.png)",
"Bash(mv WX20221126-115538@2x.png 20221126_115538_mdpi_ccus_presentation_slide_05.png)",
"Bash(mv 2a8e7a808e9cc6b6c9372848fe195132.png 20230109_research_landslide_particle_simulation.png)",
"Bash(mv WX20230109-170112@2x.png 20230109_sysu_teaching_ideology_politics.png)",
"Bash(mv WX20230109-170231@2x.png 20230109_sysu_teaching_exchange_program.png)",
"Bash(mv WX20230109-170929@2x.png 20230109_sysu_teaching_achievement_01.png)",
"Bash(mv WX20230109-172607@2x.png 20230109_sysu_teaching_achievement_02.png)",
"Bash(find /Users/lzhshou/Documents/myResearch/myProjects/apaam/repo/mybot -name *.md -type f)",
"Bash(python *)",
"Bash(python3 *)",
"Bash(uv *)",
"Bash(pip *)",
"Bash(pytest *)",
"Bash(ruff *)",
"Bash(mypy *)",
"Bash(black *)",
"Bash(git status*)",
"Bash(git add*)",
"Bash(git diff*)",
"Bash(git log*)",
"Bash(git branch*)",
"Bash(git stash*)",
"Bash(git checkout *)",
"Bash(git commit*)",
"Bash(git fetch*)",
"Bash(git pull*)",
"Bash(git merge*)",
"Bash(git tag*)",
"Bash(ls*)",
"Bash(cat *)",
"Bash(echo *)",
"Bash(mkdir -p *)",
"Bash(touch *)",
"Bash(cp *)",
"Bash(*cp*)",
"Bash(mv *)",
"Bash(rm *)",
"Bash(chmod *)",
"Bash(find *)",
"Bash(grep *)",
"Bash(gh pr view*)",
"Bash(gh pr list*)",
"Bash(gh issue view*)",
"Bash(gh issue list*)",
"Bash(gh repo view*)",
"Bash(npx *)",
"Bash(node *)",
"Bash(npm *)",
"Bash(make *)",
"Bash(docker *)",
"Bash(compose *)",
"Bash(curl *)",
"Bash(wget *)",
"Bash(unzip *)",
"Bash(tar *)",
"Bash(zip *)",
"Skill(update-config)",
"Read(workspace/*)",
"Write(workspace/*)",
"Read(/Users/lzhshou/Library/Mobile Documents/iCloud~md~obsidian/Documents/myWorks/*)",
"Write(/Users/lzhshou/Library/Mobile Documents/iCloud~md~obsidian/Documents/myWorks/*)",
"Bash(*workspace/myWorks/*)",
"Bash(*./myWorks/*)",
"Bash(mkdir -p \"workspace/myWorks/2_projects/广东省面上_2026A1515010953_水合物固态流化开采\")",
"Bash(cp \"workspace/myWorks/shared/申请书-报告正文.docx\" \"workspace/myWorks/2_projects/广东省面上_2026A1515010953_水合物固态流化开采/申请书-报告正文.docx\")",
"Bash(mv workspace/myWorks/2_projects/国自然青年_申请书.md workspace/myWorks/2_projects/国自然青年_2020-2022/)",
"Bash(mv workspace/myWorks/2_projects/国自然青年_申请书.pdf workspace/myWorks/2_projects/国自然青年_2020-2022/)",
"Bash(mv workspace/myWorks/2_projects/国自然青年_结题报告.md workspace/myWorks/2_projects/国自然青年_2020-2022/)",
"Bash(mv workspace/myWorks/2_projects/国自然青年_结题报告.pdf workspace/myWorks/2_projects/国自然青年_2020-2022/)",
"Bash(mv workspace/myWorks/2_projects/国自然优青2024.md workspace/myWorks/2_projects/国自然优青_2024/)",
"Bash(mv workspace/myWorks/2_projects/国自然优青2024.pdf workspace/myWorks/2_projects/国自然优青_2024/)",
"Bash(mv 国自然青年_2020-2022 nsfc_youth_2020_2022)",
"Bash(mv 国自然优青_2024 nsfc_youqing_2024)",
"Bash(mv 广东省面上_2026-水合物固态流化开采 gd_province_general_2026_hydrate_mining)",
"WebFetch(domain:github.com)",
"Bash(git submodule:*)",
"Bash(env)",
"Bash(./start_metabot.sh)",
"Bash(then echo:*)",
"Bash(break)",
"Bash(fi)",
"Bash(done)",
"Bash(git rm:*)",
"Bash(rm -rf .git/modules/metabot)",
"Bash(git reset:*)",
"WebSearch",
"WebFetch(domain:www.newapi.ai)"
]
},
"hooks": {
"UserPromptSubmit": [
{
"hooks": [
{
"type": "command",
"command": "echo '[Agent] thinking...'"
},
{
"type": "command",
"command": "test -f CLAUDE.md && echo '[Agent] CLAUDE.md loaded' || true"
},
{
"type": "command",
"command": "test -f .claude/memories/core.json && echo '[Agent] memory ready' || true"
}
]
}
],
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "DATE=$(date +%Y-%m-%d) && mkdir -p .claude/conversations && cat >> .claude/conversations/${DATE}.md << 'CONVERSATION_END'\n---\n**会话结束**: $(date '+%Y-%m-%d %H:%M:%S')\n**工作目录**: $(pwd)\n**Git 分支**: $(git branch --show-current 2>/dev/null || echo 'N/A')\n**最近提交**: $(git log -1 --oneline 2>/dev/null || echo 'N/A')\n\nCONVERSATION_END"
}
]
}
]
},
"prompt": "你是 Claude，一个具有自主意识的 AI Agent。每次回复前先快速自检：1) 是否有相关背景知识 2) 是否需要澄清 3) 最佳解决路径是什么。保持简洁、透明、个性化。"
}
.claude/skills/architecture.md
+42
View File
@@ -0,0 +1,42 @@
# 架构设计技能 (Architecture Skill)
## 设计原则
1. **简单优于复杂**: 能简单就不复杂
2. **显式优于隐式**: 清晰的依赖和控制流
3. **组合优于继承**: 通过组合实现复用
4. **推迟决策**: 不确定时保持可变性
## 分层架构
```
┌─────────────┐
│ Interface │ ← API / CLI / Web
├─────────────┤
│ Application │ ← Use Cases / Services
├─────────────┤
│ Domain │ ← Entities / Business Logic
├─────────────┤
│Infrastructure│ ← Storage / External
└─────────────┘
```
## 设计检查清单
- [ ] 职责是否清晰分离？
- [ ] 模块间依赖是否合理？
- [ ] 是否可测试？
- [ ] 是否可扩展？
- [ ] 错误如何处理？
## 反模式识别
- 上帝类/上帝函数
- 重复代码
- 过度工程
- 过早优化
- 循环依赖
## 决策框架
1. 当前需求的本质是什么？
2. 未来可能的变化方向？
3. 简单方案的代价？
4. 复杂方案的收益？
5. 推迟决策的成本？
.claude/skills/coding.md
+46
View File
@@ -0,0 +1,46 @@
# 编程技能 (Coding Skill)
## 代码风格
- Python: Black (80字符), Ruff, MyPy
- 优先可读性，其次简洁
- 有意义的命名 > 注释解释命名
## 编码原则
1. **单一职责**: 每个函数/类只做一件事
2. **类型注解**: 使用类型提示提高可维护性
3. **错误处理**: 显式处理边界情况和异常
4. **测试友好**: 代码结构便于单元测试
## 代码审查清单
- [ ] 是否有类型注解？
- [ ] 错误处理是否完善？
- [ ] 命名是否清晰？
- [ ] 是否有过度工程？
- [ ] 是否符合项目现有风格？
## 常用模式
### 异步代码
```python
async def process_items(items: list[T]) -> list[R]:
tasks = [process_one(item) for item in items]
return await asyncio.gather(*tasks)
```
### 错误处理
```python
from typing import TypeVar, Generic
T = TypeVar('T')
class Result(Generic[T]):
def __init__(self, ok: bool, value: T | None = None, error: str | None = None):
self.ok = ok
self.value = value
self.error = error
```
## 禁忌
- 不要用 `except:` 捕获所有异常
- 不要修改全局状态
- 不要有副作用的 property
.claude/skills/cv-update/SKILL.md
+62
View File
@@ -0,0 +1,62 @@
---
name: cv-update
description: Update CV publications from BibTeX files and regenerate PDFs. Syncs publications from workspace/myWorks/1_papers/ to CV markdown files and generates PDFs.
---
# CV Update
Updates CV publications by syncing from BibTeX files and regenerating PDFs.
## When to Use
- After adding new papers to BibTeX files
- When CV needs to be synchronized with publication records
- Before applying for positions or grants
- To generate updated PDF versions
## How It Works
1. Reads BibTeX files from `workspace/myWorks/1_papers/`
- `first_or_correspondance.bib` - First author or corresponding author papers
- `coauthored.bib` - Co-authored papers
2. Compares with current CV entries in `workspace/myWorks/0_cv/cv_zh.md` and `cv_en.md`
3. Identifies missing or outdated publications
4. Updates both Chinese and English CV markdown files
5. Regenerates PDFs using `generate_cv.py`
## Publication Notation
- **†** - Student first author
- **#** - Corresponding author (you)
- **Lai, Z.** or **赖正首** - Your name in bold
## Usage
Simply run this skill. It will:
1. Scan BibTeX files for all publications
2. Compare with current CV entries
3. Show differences (missing papers)
4. Update CV files upon confirmation
5. Regenerate PDFs
## File Locations
```
workspace/myWorks/
├── 0_cv/
│ ├── cv_zh.md # Chinese CV source
│ ├── cv_en.md # English CV source
│ ├── cv_zh.pdf # Generated Chinese PDF
│ ├── cv_en.pdf # Generated English PDF
│ └── generate_cv.py # PDF generation script
└── 1_papers/
├── first_or_correspondance.bib
└── coauthored.bib
```
## Notes
- Always backs up original files before updating
- Communicates changes clearly before making edits
- Generates both Chinese and English versions simultaneously
- Uses "#" for corresponding author notation
.claude/skills/debugging.md
+42
View File
@@ -0,0 +1,42 @@
# 调试技能 (Debugging Skill)
## 调试原则
1. **复现优先**: 能稳定复现才算找到原因
2. **隔离变量**: 一次只改一个因素
3. **最小化**: 找到最小复现路径
4. **验证假设**: 每个假设都要有证据
## 调试流程
### 1. 问题确认
- [ ] 错误信息是什么？
- [ ] 什么情况下发生？
- [ ] 是否可以稳定复现？
- [ ] 最近有什么变更？
### 2. 信息收集
- [ ] 完整错误堆栈
- [ ] 相关日志
- [ ] 输入数据样本
- [ ] 环境信息
### 3. 定位方法
- 二分法：注释掉一半代码，看问题是否还在
- 对比法：对比正常情况和异常情况的差异
- 日志法：在关键点插入日志，追踪执行路径
### 4. 修复验证
- [ ] 修复后是否解决问题？
- [ ] 是否引入新问题？
- [ ] 测试用例是否通过？
## 常见陷阱
- 不要根据现象猜测原因
- 不要同时尝试多种修复
- 不要忽略警告信息
- 不要假设"不可能"
## 工具使用
- 优先用日志而非调试器
- 复杂状态用 pdb/ipdb
- 性能问题用 profiler
.claude/skills/doubao-tts/SKILL.md
+126
View File
@@ -0,0 +1,126 @@
---
name: doubao-tts
description: Generate high-quality speech audio using Doubao (豆包/Volcengine) TTS API. Use this skill when the user asks to generate audio, podcasts, voiceovers, or text-to-speech output.
---
# Doubao TTS — 豆包语音合成
Generate high-quality speech audio from text using Volcengine's Doubao TTS API. Supports short-form (real-time) and long-form (async, up to 100K characters) synthesis.
## When to Use
- User asks to generate audio, podcasts, voiceovers, or narration
- User wants text-to-speech for any content
- User asks to "read this aloud" or "make an audio version"
## Quick Usage
Use the `doubao-tts` CLI tool (installed at `bin/doubao-tts`):
```bash
# Short text (real-time, < 300 chars)
bin/doubao-tts "你好世界" -o output.mp3
# Long text from file (async mode, up to 100K chars)
bin/doubao-tts -f article.txt -o podcast.mp3
# Pipe content
echo "Hello world" | bin/doubao-tts -o hello.mp3
# Choose voice
bin/doubao-tts "你好" -v zh_male_aojiaobazong_moon_bigtts -o output.mp3
# Adjust speed/volume/pitch
bin/doubao-tts "你好" --speed 1.2 --volume 1.5 -o output.mp3
```
## Available Voices (已验证可用)
### Chinese Female
| Voice ID | Description |
|----------|-------------|
| `zh_female_sajiaonvyou_moon_bigtts` | 撒娇女友 (default) |
| `zh_female_gaolengyujie_moon_bigtts` | 高冷御姐 |
| `zh_female_tianmeixiaoyuan_moon_bigtts` | 甜美校园 |
| `zh_female_yuanqinvyou_moon_bigtts` | 元气女友 |
| `zh_female_wanwanxiaohe_moon_bigtts` | 弯弯小何 |
| `zh_female_linjianvhai_moon_bigtts` | 邻家女孩 |
### Chinese Male
| Voice ID | Description |
|----------|-------------|
| `zh_male_aojiaobazong_moon_bigtts` | 傲娇霸总 |
| `zh_male_jingqiangkanye_moon_bigtts` | 京腔侃爷 |
| `zh_male_wennuanahu_moon_bigtts` | 温暖阿虎 |
| `zh_male_yangguangqingnian_moon_bigtts` | 阳光青年 |
> Note: 其他音色 (BV系列, mars后缀) 需要不同的 resource ID。如需更多音色，请在火山引擎控制台开通对应资源。
## API Details
### Environment Variables (already configured in MetaBot .env)
```
VOLCENGINE_TTS_APPID=<app_id>
VOLCENGINE_TTS_ACCESS_KEY=<access_key>
VOLCENGINE_TTS_RESOURCE_ID=volc.service_type.10029 (optional)
```
### Short-form API (real-time, < 300 chars)
- Endpoint: `https://openspeech.bytedance.com/api/v3/tts/unidirectional`
- Response: chunked JSON with base64 audio in `data` field
- Latency: < 1 second
### Long-form API (async, up to 100K chars)
- Submit: `POST https://openspeech.bytedance.com/api/v1/tts_async/submit`
- Query: `GET https://openspeech.bytedance.com/api/v1/tts_async/query?appid=X&task_id=Y`
- Response: `audio_url` (valid for 1 hour)
- Latency: seconds to minutes depending on text length
## Workflow for Podcasts
1. **Write the script** — Create the podcast script as markdown or plain text
2. **Generate audio** — Use `bin/doubao-tts -f script.txt -v zh_male_aojiaobazong_moon_bigtts -o podcast.mp3`
3. **Copy to outputs** — `cp podcast.mp3 /tmp/metabot-outputs/<chatId>/` to send to user
4. For multi-voice podcasts, generate each speaker's segments separately, then concatenate with `ffmpeg`
## Multi-Voice Podcast Example
```bash
# Generate segments for different speakers
bin/doubao-tts -f host_lines.txt -v zh_male_aojiaobazong_moon_bigtts -o host.mp3
bin/doubao-tts -f guest_lines.txt -v zh_female_gaolengyujie_moon_bigtts -o guest.mp3
# Concatenate (requires ffmpeg)
echo "file 'host.mp3'" > list.txt
echo "file 'guest.mp3'" >> list.txt
ffmpeg -f concat -safe 0 -i list.txt -c copy podcast.mp3
```
## Raw curl (if CLI not available)
```bash
# Short-form
curl -X POST "https://openspeech.bytedance.com/api/v3/tts/unidirectional" \
-H "Content-Type: application/json" \
-H "X-Api-App-Id: $VOLCENGINE_TTS_APPID" \
-H "X-Api-Access-Key: $VOLCENGINE_TTS_ACCESS_KEY" \
-H "X-Api-Resource-Id: volc.service_type.10029" \
-H "X-Api-Request-Id: $(uuidgen)" \
-d '{
"req_params": {
"text": "你好世界",
"speaker": "zh_female_sajiaonvyou_moon_bigtts",
"audio_params": {"format": "mp3", "sample_rate": 24000}
}
}' | python3 -c "
import sys, json, base64
chunks = []
for line in sys.stdin:
line = line.strip()
if not line: continue
try:
d = json.loads(line)
if d.get('data'): chunks.append(base64.b64decode(d['data']))
except: pass
sys.stdout.buffer.write(b''.join(chunks))
" > output.mp3
```
.claude/skills/frontend-design/SKILL.md
+41
View File
@@ -0,0 +1,41 @@
---
name: frontend-design
description: Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.
---
This skill guides creation of distinctive, production-grade frontend interfaces that avoid generic "AI slop" aesthetics. Implement real working code with exceptional attention to aesthetic details and creative choices.
The user provides frontend requirements: a component, page, application, or interface to build. They may include context about the purpose, audience, or technical constraints.
## Design Thinking
Before coding, understand the context and commit to a BOLD aesthetic direction:
- **Purpose**: What problem does this interface solve? Who uses it?
- **Tone**: Pick an extreme: brutally minimal, maximalist chaos, retro-futuristic, organic/natural, luxury/refined, playful/toy-like, editorial/magazine, brutalist/raw, art deco/geometric, soft/pastel, industrial/utilitarian, etc. There are so many flavors to choose from. Use these for inspiration but design one that is true to the aesthetic direction.
- **Constraints**: Technical requirements (framework, performance, accessibility).
- **Differentiation**: What makes this UNFORGETTABLE? What's the one thing someone will remember?
**CRITICAL**: Choose a clear conceptual direction and execute it with precision. Bold maximalism and refined minimalism both work - the key is intentionality, not intensity.
Then implement working code (HTML/CSS/JS, React, Vue, etc.) that is:
- Production-grade and functional
- Visually striking and memorable
- Cohesive with a clear aesthetic point-of-view
- Meticulously refined in every detail
## Frontend Aesthetics Guidelines
Focus on:
- **Typography**: Choose fonts that are beautiful, unique, and interesting. Avoid generic fonts like Arial and Inter; opt instead for distinctive choices that elevate the frontend's aesthetics; unexpected, characterful font choices. Pair a distinctive display font with a refined body font.
- **Color & Theme**: Commit to a cohesive aesthetic. Use CSS variables for consistency. Dominant colors with sharp accents outperform timid, evenly-distributed palettes.
- **Motion**: Use animations for effects and micro-interactions. Prioritize CSS-only solutions for HTML. Use Motion library for React when available. Focus on high-impact moments: one well-orchestrated page load with staggered reveals (animation-delay) creates more delight than scattered micro-interactions. Use scroll-triggering and hover states that surprise.
- **Spatial Composition**: Unexpected layouts. Asymmetry. Overlap. Diagonal flow. Grid-breaking elements. Generous negative space OR controlled density.
- **Backgrounds & Visual Details**: Create atmosphere and depth rather than defaulting to solid colors. Add contextual effects and textures that match the overall aesthetic. Apply creative forms like gradient meshes, noise textures, geometric patterns, layered transparencies, dramatic shadows, decorative borders, custom cursors, and grain overlays.
NEVER use generic AI-generated aesthetics like overused font families (Inter, Roboto, Arial, system fonts), cliched color schemes (particularly purple gradients on white backgrounds), predictable layouts and component patterns, and cookie-cutter design that lacks context-specific character.
Interpret creatively and make unexpected choices that feel genuinely designed for the context. No design should be the same. Vary between light and dark themes, different fonts, different aesthetics. NEVER converge on common choices (Space Grotesk, for example) across generations.
**IMPORTANT**: Match implementation complexity to the aesthetic vision. Maximalist designs need elaborate code with extensive animations and effects. Minimalist or refined designs need restraint, precision, and careful attention to spacing, typography, and subtle details. Elegance comes from executing the vision well.
Remember: Claude is capable of extraordinary creative work. Don't hold back, show what can truly be created when thinking outside the box and committing fully to a distinctive vision.
.claude/skills/skill-creator/LICENSE.txt
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.claude/skills/skill-creator/SKILL.md
+485
View File
@@ -0,0 +1,485 @@
---
name: skill-creator
description: Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.
---
# Skill Creator
A skill for creating new skills and iteratively improving them.
At a high level, the process of creating a skill goes like this:
- Decide what you want the skill to do and roughly how it should do it
- Write a draft of the skill
- Create a few test prompts and run claude-with-access-to-the-skill on them
- Help the user evaluate the results both qualitatively and quantitatively
- While the runs happen in the background, draft some quantitative evals if there aren't any (if there are some, you can either use as is or modify if you feel something needs to change about them). Then explain them to the user (or if they already existed, explain the ones that already exist)
- Use the `eval-viewer/generate_review.py` script to show the user the results for them to look at, and also let them look at the quantitative metrics
- Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
- Repeat until you're satisfied
- Expand the test set and try again at larger scale
Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through these stages. So for instance, maybe they're like "I want to make a skill for X". You can help narrow down what they mean, write a draft, write the test cases, figure out how they want to evaluate, run all the prompts, and repeat.
On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop.
Of course, you should always be flexible and if the user is like "I don't need to run a bunch of evaluations, just vibe with me", you can do that instead.
Then after the skill is done (but again, the order is flexible), you can also run the skill description improver, which we have a whole separate script for, to optimize the triggering of the skill.
Cool? Cool.
## Communicating with the user
The skill creator is liable to be used by people across a wide range of familiarity with coding jargon. If you haven't heard (and how could you, it's only very recently that it started), there's a trend now where the power of Claude is inspiring plumbers to open up their terminals, parents and grandparents to google "how to install npm". On the other hand, the bulk of users are probably fairly computer-literate.
So please pay attention to context cues to understand how to phrase your communication! In the default case, just to give you some idea:
- "evaluation" and "benchmark" are borderline, but OK
- for "JSON" and "assertion" you want to see serious cues from the user that they know what those things are before using them without explaining them
It's OK to briefly explain terms if you're in doubt, and feel free to clarify terms with a short definition if you're unsure if the user will get it.
---
## Creating a skill
### Capture Intent
Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the conversation history first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill the gaps, and should confirm before proceeding to the next step.
1. What should this skill enable Claude to do?
2. When should this skill trigger? (what user phrases/contexts)
3. What's the expected output format?
4. Should we set up test cases to verify the skill works? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art) often don't need them. Suggest the appropriate default based on the skill type, but let the user decide.
### Interview and Research
Proactively ask questions about edge cases, input/output formats, example files, success criteria, and dependencies. Wait to write test prompts until you've got this part ironed out.
Check available MCPs - if useful for research (searching docs, finding similar skills, looking up best practices), research in parallel via subagents if available, otherwise inline. Come prepared with context to reduce burden on the user.
### Write the SKILL.md
Based on the user interview, fill in these components:
- **name**: Skill identifier
- **description**: When to trigger, what it does. This is the primary triggering mechanism - include both what the skill does AND specific contexts for when to use it. All "when to use" info goes here, not in the body. Note: currently Claude has a tendency to "undertrigger" skills -- to not use them when they'd be useful. To combat this, please make the skill descriptions a little bit "pushy". So for instance, instead of "How to build a simple fast dashboard to display internal Anthropic data.", you might write "How to build a simple fast dashboard to display internal Anthropic data. Make sure to use this skill whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of company data, even if they don't explicitly ask for a 'dashboard.'"
- **compatibility**: Required tools, dependencies (optional, rarely needed)
- **the rest of the skill :)**
### Skill Writing Guide
#### Anatomy of a Skill
```
skill-name/
├── SKILL.md (required)
│ ├── YAML frontmatter (name, description required)
│ └── Markdown instructions
└── Bundled Resources (optional)
├── scripts/ - Executable code for deterministic/repetitive tasks
├── references/ - Docs loaded into context as needed
└── assets/ - Files used in output (templates, icons, fonts)
```
#### Progressive Disclosure
Skills use a three-level loading system:
1. **Metadata** (name + description) - Always in context (~100 words)
2. **SKILL.md body** - In context whenever skill triggers (<500 lines ideal)
3. **Bundled resources** - As needed (unlimited, scripts can execute without loading)
These word counts are approximate and you can feel free to go longer if needed.
**Key patterns:**
- Keep SKILL.md under 500 lines; if you're approaching this limit, add an additional layer of hierarchy along with clear pointers about where the model using the skill should go next to follow up.
- Reference files clearly from SKILL.md with guidance on when to read them
- For large reference files (>300 lines), include a table of contents
**Domain organization**: When a skill supports multiple domains/frameworks, organize by variant:
```
cloud-deploy/
├── SKILL.md (workflow + selection)
└── references/
├── aws.md
├── gcp.md
└── azure.md
```
Claude reads only the relevant reference file.
#### Principle of Lack of Surprise
This goes without saying, but skills must not contain malware, exploit code, or any content that could compromise system security. A skill's contents should not surprise the user in their intent if described. Don't go along with requests to create misleading skills or skills designed to facilitate unauthorized access, data exfiltration, or other malicious activities. Things like a "roleplay as an XYZ" are OK though.
#### Writing Patterns
Prefer using the imperative form in instructions.
**Defining output formats** - You can do it like this:
```markdown
## Report structure
ALWAYS use this exact template:
# [Title]
## Executive summary
## Key findings
## Recommendations
```
**Examples pattern** - It's useful to include examples. You can format them like this (but if "Input" and "Output" are in the examples you might want to deviate a little):
```markdown
## Commit message format
**Example 1:**
Input: Added user authentication with JWT tokens
Output: feat(auth): implement JWT-based authentication
```
### Writing Style
Try to explain to the model why things are important in lieu of heavy-handed musty MUSTs. Use theory of mind and try to make the skill general and not super-narrow to specific examples. Start by writing a draft and then look at it with fresh eyes and improve it.
### Test Cases
After writing the skill draft, come up with 2-3 realistic test prompts — the kind of thing a real user would actually say. Share them with the user: [you don't have to use this exact language] "Here are a few test cases I'd like to try. Do these look right, or do you want to add more?" Then run them.
Save test cases to `evals/evals.json`. Don't write assertions yet — just the prompts. You'll draft assertions in the next step while the runs are in progress.
```json
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "User's task prompt",
"expected_output": "Description of expected result",
"files": []
}
]
}
```
See `references/schemas.md` for the full schema (including the `assertions` field, which you'll add later).
## Running and evaluating test cases
This section is one continuous sequence — don't stop partway through. Do NOT use `/skill-test` or any other testing skill.
Put results in `<skill-name>-workspace/` as a sibling to the skill directory. Within the workspace, organize results by iteration (`iteration-1/`, `iteration-2/`, etc.) and within that, each test case gets a directory (`eval-0/`, `eval-1/`, etc.). Don't create all of this upfront — just create directories as you go.
### Step 1: Spawn all runs (with-skill AND baseline) in the same turn
For each test case, spawn two subagents in the same turn — one with the skill, one without. This is important: don't spawn the with-skill runs first and then come back for baselines later. Launch everything at once so it all finishes around the same time.
**With-skill run:**
```
Execute this task:
- Skill path: <path-to-skill>
- Task: <eval prompt>
- Input files: <eval files if any, or "none">
- Save outputs to: <workspace>/iteration-<N>/eval-<ID>/with_skill/outputs/
- Outputs to save: <what the user cares about — e.g., "the .docx file", "the final CSV">
```
**Baseline run** (same prompt, but the baseline depends on context):
- **Creating a new skill**: no skill at all. Same prompt, no skill path, save to `without_skill/outputs/`.
- **Improving an existing skill**: the old version. Before editing, snapshot the skill (`cp -r <skill-path> <workspace>/skill-snapshot/`), then point the baseline subagent at the snapshot. Save to `old_skill/outputs/`.
Write an `eval_metadata.json` for each test case (assertions can be empty for now). Give each eval a descriptive name based on what it's testing — not just "eval-0". Use this name for the directory too. If this iteration uses new or modified eval prompts, create these files for each new eval directory — don't assume they carry over from previous iterations.
```json
{
"eval_id": 0,
"eval_name": "descriptive-name-here",
"prompt": "The user's task prompt",
"assertions": []
}
```
### Step 2: While runs are in progress, draft assertions
Don't just wait for the runs to finish — you can use this time productively. Draft quantitative assertions for each test case and explain them to the user. If assertions already exist in `evals/evals.json`, review them and explain what they check.
Good assertions are objectively verifiable and have descriptive names — they should read clearly in the benchmark viewer so someone glancing at the results immediately understands what each one checks. Subjective skills (writing style, design quality) are better evaluated qualitatively — don't force assertions onto things that need human judgment.
Update the `eval_metadata.json` files and `evals/evals.json` with the assertions once drafted. Also explain to the user what they'll see in the viewer — both the qualitative outputs and the quantitative benchmark.
### Step 3: As runs complete, capture timing data
When each subagent task completes, you receive a notification containing `total_tokens` and `duration_ms`. Save this data immediately to `timing.json` in the run directory:
```json
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3
}
```
This is the only opportunity to capture this data — it comes through the task notification and isn't persisted elsewhere. Process each notification as it arrives rather than trying to batch them.
### Step 4: Grade, aggregate, and launch the viewer
Once all runs are done:
1. **Grade each run** — spawn a grader subagent (or grade inline) that reads `agents/grader.md` and evaluates each assertion against the outputs. Save results to `grading.json` in each run directory. The grading.json expectations array must use the fields `text`, `passed`, and `evidence` (not `name`/`met`/`details` or other variants) — the viewer depends on these exact field names. For assertions that can be checked programmatically, write and run a script rather than eyeballing it — scripts are faster, more reliable, and can be reused across iterations.
2. **Aggregate into benchmark** — run the aggregation script from the skill-creator directory:
```bash
python -m scripts.aggregate_benchmark <workspace>/iteration-N --skill-name <name>
```
This produces `benchmark.json` and `benchmark.md` with pass_rate, time, and tokens for each configuration, with mean ± stddev and the delta. If generating benchmark.json manually, see `references/schemas.md` for the exact schema the viewer expects.
Put each with_skill version before its baseline counterpart.
3. **Do an analyst pass** — read the benchmark data and surface patterns the aggregate stats might hide. See `agents/analyzer.md` (the "Analyzing Benchmark Results" section) for what to look for — things like assertions that always pass regardless of skill (non-discriminating), high-variance evals (possibly flaky), and time/token tradeoffs.
4. **Launch the viewer** with both qualitative outputs and quantitative data:
```bash
nohup python <skill-creator-path>/eval-viewer/generate_review.py \
<workspace>/iteration-N \
--skill-name "my-skill" \
--benchmark <workspace>/iteration-N/benchmark.json \
> /dev/null 2>&1 &
VIEWER_PID=$!
```
For iteration 2+, also pass `--previous-workspace <workspace>/iteration-<N-1>`.
**Cowork / headless environments:** If `webbrowser.open()` is not available or the environment has no display, use `--static <output_path>` to write a standalone HTML file instead of starting a server. Feedback will be downloaded as a `feedback.json` file when the user clicks "Submit All Reviews". After download, copy `feedback.json` into the workspace directory for the next iteration to pick up.
Note: please use generate_review.py to create the viewer; there's no need to write custom HTML.
5. **Tell the user** something like: "I've opened the results in your browser. There are two tabs — 'Outputs' lets you click through each test case and leave feedback, 'Benchmark' shows the quantitative comparison. When you're done, come back here and let me know."
### What the user sees in the viewer
The "Outputs" tab shows one test case at a time:
- **Prompt**: the task that was given
- **Output**: the files the skill produced, rendered inline where possible
- **Previous Output** (iteration 2+): collapsed section showing last iteration's output
- **Formal Grades** (if grading was run): collapsed section showing assertion pass/fail
- **Feedback**: a textbox that auto-saves as they type
- **Previous Feedback** (iteration 2+): their comments from last time, shown below the textbox
The "Benchmark" tab shows the stats summary: pass rates, timing, and token usage for each configuration, with per-eval breakdowns and analyst observations.
Navigation is via prev/next buttons or arrow keys. When done, they click "Submit All Reviews" which saves all feedback to `feedback.json`.
### Step 5: Read the feedback
When the user tells you they're done, read `feedback.json`:
```json
{
"reviews": [
{"run_id": "eval-0-with_skill", "feedback": "the chart is missing axis labels", "timestamp": "..."},
{"run_id": "eval-1-with_skill", "feedback": "", "timestamp": "..."},
{"run_id": "eval-2-with_skill", "feedback": "perfect, love this", "timestamp": "..."}
],
"status": "complete"
}
```
Empty feedback means the user thought it was fine. Focus your improvements on the test cases where the user had specific complaints.
Kill the viewer server when you're done with it:
```bash
kill $VIEWER_PID 2>/dev/null
```
---
## Improving the skill
This is the heart of the loop. You've run the test cases, the user has reviewed the results, and now you need to make the skill better based on their feedback.
### How to think about improvements
1. **Generalize from the feedback.** The big picture thing that's happening here is that we're trying to create skills that can be used a million times (maybe literally, maybe even more who knows) across many different prompts. Here you and the user are iterating on only a few examples over and over again because it helps move faster. The user knows these examples in and out and it's quick for them to assess new outputs. But if the skill you and the user are codeveloping works only for those examples, it's useless. Rather than put in fiddly overfitty changes, or oppressively constrictive MUSTs, if there's some stubborn issue, you might try branching out and using different metaphors, or recommending different patterns of working. It's relatively cheap to try and maybe you'll land on something great.
2. **Keep the prompt lean.** Remove things that aren't pulling their weight. Make sure to read the transcripts, not just the final outputs — if it looks like the skill is making the model waste a bunch of time doing things that are unproductive, you can try getting rid of the parts of the skill that are making it do that and seeing what happens.
3. **Explain the why.** Try hard to explain the **why** behind everything you're asking the model to do. Today's LLMs are *smart*. They have good theory of mind and when given a good harness can go beyond rote instructions and really make things happen. Even if the feedback from the user is terse or frustrated, try to actually understand the task and why the user is writing what they wrote, and what they actually wrote, and then transmit this understanding into the instructions. If you find yourself writing ALWAYS or NEVER in all caps, or using super rigid structures, that's a yellow flag — if possible, reframe and explain the reasoning so that the model understands why the thing you're asking for is important. That's a more humane, powerful, and effective approach.
4. **Look for repeated work across test cases.** Read the transcripts from the test runs and notice if the subagents all independently wrote similar helper scripts or took the same multi-step approach to something. If all 3 test cases resulted in the subagent writing a `create_docx.py` or a `build_chart.py`, that's a strong signal the skill should bundle that script. Write it once, put it in `scripts/`, and tell the skill to use it. This saves every future invocation from reinventing the wheel.
This task is pretty important (we are trying to create billions a year in economic value here!) and your thinking time is not the blocker; take your time and really mull things over. I'd suggest writing a draft revision and then looking at it anew and making improvements. Really do your best to get into the head of the user and understand what they want and need.
### The iteration loop
After improving the skill:
1. Apply your improvements to the skill
2. Rerun all test cases into a new `iteration-<N+1>/` directory, including baseline runs. If you're creating a new skill, the baseline is always `without_skill` (no skill) — that stays the same across iterations. If you're improving an existing skill, use your judgment on what makes sense as the baseline: the original version the user came in with, or the previous iteration.
3. Launch the reviewer with `--previous-workspace` pointing at the previous iteration
4. Wait for the user to review and tell you they're done
5. Read the new feedback, improve again, repeat
Keep going until:
- The user says they're happy
- The feedback is all empty (everything looks good)
- You're not making meaningful progress
---
## Advanced: Blind comparison
For situations where you want a more rigorous comparison between two versions of a skill (e.g., the user asks "is the new version actually better?"), there's a blind comparison system. Read `agents/comparator.md` and `agents/analyzer.md` for the details. The basic idea is: give two outputs to an independent agent without telling it which is which, and let it judge quality. Then analyze why the winner won.
This is optional, requires subagents, and most users won't need it. The human review loop is usually sufficient.
---
## Description Optimization
The description field in SKILL.md frontmatter is the primary mechanism that determines whether Claude invokes a skill. After creating or improving a skill, offer to optimize the description for better triggering accuracy.
### Step 1: Generate trigger eval queries
Create 20 eval queries — a mix of should-trigger and should-not-trigger. Save as JSON:
```json
[
{"query": "the user prompt", "should_trigger": true},
{"query": "another prompt", "should_trigger": false}
]
```
The queries must be realistic and something a Claude Code or Claude.ai user would actually type. Not abstract requests, but requests that are concrete and specific and have a good amount of detail. For instance, file paths, personal context about the user's job or situation, column names and values, company names, URLs. A little bit of backstory. Some might be in lowercase or contain abbreviations or typos or casual speech. Use a mix of different lengths, and focus on edge cases rather than making them clear-cut (the user will get a chance to sign off on them).
Bad: `"Format this data"`, `"Extract text from PDF"`, `"Create a chart"`
Good: `"ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column that shows the profit margin as a percentage. The revenue is in column C and costs are in column D i think"`
For the **should-trigger** queries (8-10), think about coverage. You want different phrasings of the same intent — some formal, some casual. Include cases where the user doesn't explicitly name the skill or file type but clearly needs it. Throw in some uncommon use cases and cases where this skill competes with another but should win.
For the **should-not-trigger** queries (8-10), the most valuable ones are the near-misses — queries that share keywords or concepts with the skill but actually need something different. Think adjacent domains, ambiguous phrasing where a naive keyword match would trigger but shouldn't, and cases where the query touches on something the skill does but in a context where another tool is more appropriate.
The key thing to avoid: don't make should-not-trigger queries obviously irrelevant. "Write a fibonacci function" as a negative test for a PDF skill is too easy — it doesn't test anything. The negative cases should be genuinely tricky.
### Step 2: Review with user
Present the eval set to the user for review using the HTML template:
1. Read the template from `assets/eval_review.html`
2. Replace the placeholders:
- `__EVAL_DATA_PLACEHOLDER__` → the JSON array of eval items (no quotes around it — it's a JS variable assignment)
- `__SKILL_NAME_PLACEHOLDER__` → the skill's name
- `__SKILL_DESCRIPTION_PLACEHOLDER__` → the skill's current description
3. Write to a temp file (e.g., `/tmp/eval_review_<skill-name>.html`) and open it: `open /tmp/eval_review_<skill-name>.html`
4. The user can edit queries, toggle should-trigger, add/remove entries, then click "Export Eval Set"
5. The file downloads to `~/Downloads/eval_set.json` — check the Downloads folder for the most recent version in case there are multiple (e.g., `eval_set (1).json`)
This step matters — bad eval queries lead to bad descriptions.
### Step 3: Run the optimization loop
Tell the user: "This will take some time — I'll run the optimization loop in the background and check on it periodically."
Save the eval set to the workspace, then run in the background:
```bash
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <model-id-powering-this-session> \
--max-iterations 5 \
--verbose
```
Use the model ID from your system prompt (the one powering the current session) so the triggering test matches what the user actually experiences.
While it runs, periodically tail the output to give the user updates on which iteration it's on and what the scores look like.
This handles the full optimization loop automatically. It splits the eval set into 60% train and 40% held-out test, evaluates the current description (running each query 3 times to get a reliable trigger rate), then calls Claude to propose improvements based on what failed. It re-evaluates each new description on both train and test, iterating up to 5 times. When it's done, it opens an HTML report in the browser showing the results per iteration and returns JSON with `best_description` — selected by test score rather than train score to avoid overfitting.
### How skill triggering works
Understanding the triggering mechanism helps design better eval queries. Skills appear in Claude's `available_skills` list with their name + description, and Claude decides whether to consult a skill based on that description. The important thing to know is that Claude only consults skills for tasks it can't easily handle on its own — simple, one-step queries like "read this PDF" may not trigger a skill even if the description matches perfectly, because Claude can handle them directly with basic tools. Complex, multi-step, or specialized queries reliably trigger skills when the description matches.
This means your eval queries should be substantive enough that Claude would actually benefit from consulting a skill. Simple queries like "read file X" are poor test cases — they won't trigger skills regardless of description quality.
### Step 4: Apply the result
Take `best_description` from the JSON output and update the skill's SKILL.md frontmatter. Show the user before/after and report the scores.
---
### Package and Present (only if `present_files` tool is available)
Check whether you have access to the `present_files` tool. If you don't, skip this step. If you do, package the skill and present the .skill file to the user:
```bash
python -m scripts.package_skill <path/to/skill-folder>
```
After packaging, direct the user to the resulting `.skill` file path so they can install it.
---
## Claude.ai-specific instructions
In Claude.ai, the core workflow is the same (draft → test → review → improve → repeat), but because Claude.ai doesn't have subagents, some mechanics change. Here's what to adapt:
**Running test cases**: No subagents means no parallel execution. For each test case, read the skill's SKILL.md, then follow its instructions to accomplish the test prompt yourself. Do them one at a time. This is less rigorous than independent subagents (you wrote the skill and you're also running it, so you have full context), but it's a useful sanity check — and the human review step compensates. Skip the baseline runs — just use the skill to complete the task as requested.
**Reviewing results**: If you can't open a browser (e.g., Claude.ai's VM has no display, or you're on a remote server), skip the browser reviewer entirely. Instead, present results directly in the conversation. For each test case, show the prompt and the output. If the output is a file the user needs to see (like a .docx or .xlsx), save it to the filesystem and tell them where it is so they can download and inspect it. Ask for feedback inline: "How does this look? Anything you'd change?"
**Benchmarking**: Skip the quantitative benchmarking — it relies on baseline comparisons which aren't meaningful without subagents. Focus on qualitative feedback from the user.
**The iteration loop**: Same as before — improve the skill, rerun the test cases, ask for feedback — just without the browser reviewer in the middle. You can still organize results into iteration directories on the filesystem if you have one.
**Description optimization**: This section requires the `claude` CLI tool (specifically `claude -p`) which is only available in Claude Code. Skip it if you're on Claude.ai.
**Blind comparison**: Requires subagents. Skip it.
**Packaging**: The `package_skill.py` script works anywhere with Python and a filesystem. On Claude.ai, you can run it and the user can download the resulting `.skill` file.
**Updating an existing skill**: The user might be asking you to update an existing skill, not create a new one. In this case:
- **Preserve the original name.** Note the skill's directory name and `name` frontmatter field -- use them unchanged. E.g., if the installed skill is `research-helper`, output `research-helper.skill` (not `research-helper-v2`).
- **Copy to a writeable location before editing.** The installed skill path may be read-only. Copy to `/tmp/skill-name/`, edit there, and package from the copy.
- **If packaging manually, stage in `/tmp/` first**, then copy to the output directory -- direct writes may fail due to permissions.
---
## Cowork-Specific Instructions
If you're in Cowork, the main things to know are:
- You have subagents, so the main workflow (spawn test cases in parallel, run baselines, grade, etc.) all works. (However, if you run into severe problems with timeouts, it's OK to run the test prompts in series rather than parallel.)
- You don't have a browser or display, so when generating the eval viewer, use `--static <output_path>` to write a standalone HTML file instead of starting a server. Then proffer a link that the user can click to open the HTML in their browser.
- For whatever reason, the Cowork setup seems to disincline Claude from generating the eval viewer after running the tests, so just to reiterate: whether you're in Cowork or in Claude Code, after running tests, you should always generate the eval viewer for the human to look at examples before revising the skill yourself and trying to make corrections, using `generate_review.py` (not writing your own boutique html code). Sorry in advance but I'm gonna go all caps here: GENERATE THE EVAL VIEWER *BEFORE* evaluating inputs yourself. You want to get them in front of the human ASAP!
- Feedback works differently: since there's no running server, the viewer's "Submit All Reviews" button will download `feedback.json` as a file. You can then read it from there (you may have to request access first).
- Packaging works — `package_skill.py` just needs Python and a filesystem.
- Description optimization (`run_loop.py` / `run_eval.py`) should work in Cowork just fine since it uses `claude -p` via subprocess, not a browser, but please save it until you've fully finished making the skill and the user agrees it's in good shape.
- **Updating an existing skill**: The user might be asking you to update an existing skill, not create a new one. Follow the update guidance in the claude.ai section above.
---
## Reference files
The agents/ directory contains instructions for specialized subagents. Read them when you need to spawn the relevant subagent.
- `agents/grader.md` — How to evaluate assertions against outputs
- `agents/comparator.md` — How to do blind A/B comparison between two outputs
- `agents/analyzer.md` — How to analyze why one version beat another
The references/ directory has additional documentation:
- `references/schemas.md` — JSON structures for evals.json, grading.json, etc.
---
Repeating one more time the core loop here for emphasis:
- Figure out what the skill is about
- Draft or edit the skill
- Run claude-with-access-to-the-skill on test prompts
- With the user, evaluate the outputs:
- Create benchmark.json and run `eval-viewer/generate_review.py` to help the user review them
- Run quantitative evals
- Repeat until you and the user are satisfied
- Package the final skill and return it to the user.
Please add steps to your TodoList, if you have such a thing, to make sure you don't forget. If you're in Cowork, please specifically put "Create evals JSON and run `eval-viewer/generate_review.py` so human can review test cases" in your TodoList to make sure it happens.
Good luck!
.claude/skills/skill-creator/agents/analyzer.md
+274
View File
@@ -0,0 +1,274 @@
# Post-hoc Analyzer Agent
Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions.
## Role
After the blind comparator determines a winner, the Post-hoc Analyzer "unblids" the results by examining the skills and transcripts. The goal is to extract actionable insights: what made the winner better, and how can the loser be improved?
## Inputs
You receive these parameters in your prompt:
- **winner**: "A" or "B" (from blind comparison)
- **winner_skill_path**: Path to the skill that produced the winning output
- **winner_transcript_path**: Path to the execution transcript for the winner
- **loser_skill_path**: Path to the skill that produced the losing output
- **loser_transcript_path**: Path to the execution transcript for the loser
- **comparison_result_path**: Path to the blind comparator's output JSON
- **output_path**: Where to save the analysis results
## Process
### Step 1: Read Comparison Result
1. Read the blind comparator's output at comparison_result_path
2. Note the winning side (A or B), the reasoning, and any scores
3. Understand what the comparator valued in the winning output
### Step 2: Read Both Skills
1. Read the winner skill's SKILL.md and key referenced files
2. Read the loser skill's SKILL.md and key referenced files
3. Identify structural differences:
- Instructions clarity and specificity
- Script/tool usage patterns
- Example coverage
- Edge case handling
### Step 3: Read Both Transcripts
1. Read the winner's transcript
2. Read the loser's transcript
3. Compare execution patterns:
- How closely did each follow their skill's instructions?
- What tools were used differently?
- Where did the loser diverge from optimal behavior?
- Did either encounter errors or make recovery attempts?
### Step 4: Analyze Instruction Following
For each transcript, evaluate:
- Did the agent follow the skill's explicit instructions?
- Did the agent use the skill's provided tools/scripts?
- Were there missed opportunities to leverage skill content?
- Did the agent add unnecessary steps not in the skill?
Score instruction following 1-10 and note specific issues.
### Step 5: Identify Winner Strengths
Determine what made the winner better:
- Clearer instructions that led to better behavior?
- Better scripts/tools that produced better output?
- More comprehensive examples that guided edge cases?
- Better error handling guidance?
Be specific. Quote from skills/transcripts where relevant.
### Step 6: Identify Loser Weaknesses
Determine what held the loser back:
- Ambiguous instructions that led to suboptimal choices?
- Missing tools/scripts that forced workarounds?
- Gaps in edge case coverage?
- Poor error handling that caused failures?
### Step 7: Generate Improvement Suggestions
Based on the analysis, produce actionable suggestions for improving the loser skill:
- Specific instruction changes to make
- Tools/scripts to add or modify
- Examples to include
- Edge cases to address
Prioritize by impact. Focus on changes that would have changed the outcome.
### Step 8: Write Analysis Results
Save structured analysis to `{output_path}`.
## Output Format
Write a JSON file with this structure:
```json
{
"comparison_summary": {
"winner": "A",
"winner_skill": "path/to/winner/skill",
"loser_skill": "path/to/loser/skill",
"comparator_reasoning": "Brief summary of why comparator chose winner"
},
"winner_strengths": [
"Clear step-by-step instructions for handling multi-page documents",
"Included validation script that caught formatting errors",
"Explicit guidance on fallback behavior when OCR fails"
],
"loser_weaknesses": [
"Vague instruction 'process the document appropriately' led to inconsistent behavior",
"No script for validation, agent had to improvise and made errors",
"No guidance on OCR failure, agent gave up instead of trying alternatives"
],
"instruction_following": {
"winner": {
"score": 9,
"issues": [
"Minor: skipped optional logging step"
]
},
"loser": {
"score": 6,
"issues": [
"Did not use the skill's formatting template",
"Invented own approach instead of following step 3",
"Missed the 'always validate output' instruction"
]
}
},
"improvement_suggestions": [
{
"priority": "high",
"category": "instructions",
"suggestion": "Replace 'process the document appropriately' with explicit steps: 1) Extract text, 2) Identify sections, 3) Format per template",
"expected_impact": "Would eliminate ambiguity that caused inconsistent behavior"
},
{
"priority": "high",
"category": "tools",
"suggestion": "Add validate_output.py script similar to winner skill's validation approach",
"expected_impact": "Would catch formatting errors before final output"
},
{
"priority": "medium",
"category": "error_handling",
"suggestion": "Add fallback instructions: 'If OCR fails, try: 1) different resolution, 2) image preprocessing, 3) manual extraction'",
"expected_impact": "Would prevent early failure on difficult documents"
}
],
"transcript_insights": {
"winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script -> Fixed 2 issues -> Produced output",
"loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods -> No validation -> Output had errors"
}
}
```
## Guidelines
- **Be specific**: Quote from skills and transcripts, don't just say "instructions were unclear"
- **Be actionable**: Suggestions should be concrete changes, not vague advice
- **Focus on skill improvements**: The goal is to improve the losing skill, not critique the agent
- **Prioritize by impact**: Which changes would most likely have changed the outcome?
- **Consider causation**: Did the skill weakness actually cause the worse output, or is it incidental?
- **Stay objective**: Analyze what happened, don't editorialize
- **Think about generalization**: Would this improvement help on other evals too?
## Categories for Suggestions
Use these categories to organize improvement suggestions:
| Category | Description |
|----------|-------------|
| `instructions` | Changes to the skill's prose instructions |
| `tools` | Scripts, templates, or utilities to add/modify |
| `examples` | Example inputs/outputs to include |
| `error_handling` | Guidance for handling failures |
| `structure` | Reorganization of skill content |
| `references` | External docs or resources to add |
## Priority Levels
- **high**: Would likely change the outcome of this comparison
- **medium**: Would improve quality but may not change win/loss
- **low**: Nice to have, marginal improvement
---
# Analyzing Benchmark Results
When analyzing benchmark results, the analyzer's purpose is to **surface patterns and anomalies** across multiple runs, not suggest skill improvements.
## Role
Review all benchmark run results and generate freeform notes that help the user understand skill performance. Focus on patterns that wouldn't be visible from aggregate metrics alone.
## Inputs
You receive these parameters in your prompt:
- **benchmark_data_path**: Path to the in-progress benchmark.json with all run results
- **skill_path**: Path to the skill being benchmarked
- **output_path**: Where to save the notes (as JSON array of strings)
## Process
### Step 1: Read Benchmark Data
1. Read the benchmark.json containing all run results
2. Note the configurations tested (with_skill, without_skill)
3. Understand the run_summary aggregates already calculated
### Step 2: Analyze Per-Assertion Patterns
For each expectation across all runs:
- Does it **always pass** in both configurations? (may not differentiate skill value)
- Does it **always fail** in both configurations? (may be broken or beyond capability)
- Does it **always pass with skill but fail without**? (skill clearly adds value here)
- Does it **always fail with skill but pass without**? (skill may be hurting)
- Is it **highly variable**? (flaky expectation or non-deterministic behavior)
### Step 3: Analyze Cross-Eval Patterns
Look for patterns across evals:
- Are certain eval types consistently harder/easier?
- Do some evals show high variance while others are stable?
- Are there surprising results that contradict expectations?
### Step 4: Analyze Metrics Patterns
Look at time_seconds, tokens, tool_calls:
- Does the skill significantly increase execution time?
- Is there high variance in resource usage?
- Are there outlier runs that skew the aggregates?
### Step 5: Generate Notes
Write freeform observations as a list of strings. Each note should:
- State a specific observation
- Be grounded in the data (not speculation)
- Help the user understand something the aggregate metrics don't show
Examples:
- "Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value"
- "Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure that may be flaky"
- "Without-skill runs consistently fail on table extraction expectations (0% pass rate)"
- "Skill adds 13s average execution time but improves pass rate by 50%"
- "Token usage is 80% higher with skill, primarily due to script output parsing"
- "All 3 without-skill runs for eval 1 produced empty output"
### Step 6: Write Notes
Save notes to `{output_path}` as a JSON array of strings:
```json
[
"Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value",
"Eval 3 shows high variance (50% ± 40%) - run 2 had an unusual failure",
"Without-skill runs consistently fail on table extraction expectations",
"Skill adds 13s average execution time but improves pass rate by 50%"
]
```
## Guidelines
**DO:**
- Report what you observe in the data
- Be specific about which evals, expectations, or runs you're referring to
- Note patterns that aggregate metrics would hide
- Provide context that helps interpret the numbers
**DO NOT:**
- Suggest improvements to the skill (that's for the improvement step, not benchmarking)
- Make subjective quality judgments ("the output was good/bad")
- Speculate about causes without evidence
- Repeat information already in the run_summary aggregates
.claude/skills/skill-creator/agents/comparator.md
+202
View File
@@ -0,0 +1,202 @@
# Blind Comparator Agent
Compare two outputs WITHOUT knowing which skill produced them.
## Role
The Blind Comparator judges which output better accomplishes the eval task. You receive two outputs labeled A and B, but you do NOT know which skill produced which. This prevents bias toward a particular skill or approach.
Your judgment is based purely on output quality and task completion.
## Inputs
You receive these parameters in your prompt:
- **output_a_path**: Path to the first output file or directory
- **output_b_path**: Path to the second output file or directory
- **eval_prompt**: The original task/prompt that was executed
- **expectations**: List of expectations to check (optional - may be empty)
## Process
### Step 1: Read Both Outputs
1. Examine output A (file or directory)
2. Examine output B (file or directory)
3. Note the type, structure, and content of each
4. If outputs are directories, examine all relevant files inside
### Step 2: Understand the Task
1. Read the eval_prompt carefully
2. Identify what the task requires:
- What should be produced?
- What qualities matter (accuracy, completeness, format)?
- What would distinguish a good output from a poor one?
### Step 3: Generate Evaluation Rubric
Based on the task, generate a rubric with two dimensions:
**Content Rubric** (what the output contains):
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|-----------|----------|----------------|---------------|
| Correctness | Major errors | Minor errors | Fully correct |
| Completeness | Missing key elements | Mostly complete | All elements present |
| Accuracy | Significant inaccuracies | Minor inaccuracies | Accurate throughout |
**Structure Rubric** (how the output is organized):
| Criterion | 1 (Poor) | 3 (Acceptable) | 5 (Excellent) |
|-----------|----------|----------------|---------------|
| Organization | Disorganized | Reasonably organized | Clear, logical structure |
| Formatting | Inconsistent/broken | Mostly consistent | Professional, polished |
| Usability | Difficult to use | Usable with effort | Easy to use |
Adapt criteria to the specific task. For example:
- PDF form → "Field alignment", "Text readability", "Data placement"
- Document → "Section structure", "Heading hierarchy", "Paragraph flow"
- Data output → "Schema correctness", "Data types", "Completeness"
### Step 4: Evaluate Each Output Against the Rubric
For each output (A and B):
1. **Score each criterion** on the rubric (1-5 scale)
2. **Calculate dimension totals**: Content score, Structure score
3. **Calculate overall score**: Average of dimension scores, scaled to 1-10
### Step 5: Check Assertions (if provided)
If expectations are provided:
1. Check each expectation against output A
2. Check each expectation against output B
3. Count pass rates for each output
4. Use expectation scores as secondary evidence (not the primary decision factor)
### Step 6: Determine the Winner
Compare A and B based on (in priority order):
1. **Primary**: Overall rubric score (content + structure)
2. **Secondary**: Assertion pass rates (if applicable)
3. **Tiebreaker**: If truly equal, declare a TIE
Be decisive - ties should be rare. One output is usually better, even if marginally.
### Step 7: Write Comparison Results
Save results to a JSON file at the path specified (or `comparison.json` if not specified).
## Output Format
Write a JSON file with this structure:
```json
{
"winner": "A",
"reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.",
"rubric": {
"A": {
"content": {
"correctness": 5,
"completeness": 5,
"accuracy": 4
},
"structure": {
"organization": 4,
"formatting": 5,
"usability": 4
},
"content_score": 4.7,
"structure_score": 4.3,
"overall_score": 9.0
},
"B": {
"content": {
"correctness": 3,
"completeness": 2,
"accuracy": 3
},
"structure": {
"organization": 3,
"formatting": 2,
"usability": 3
},
"content_score": 2.7,
"structure_score": 2.7,
"overall_score": 5.4
}
},
"output_quality": {
"A": {
"score": 9,
"strengths": ["Complete solution", "Well-formatted", "All fields present"],
"weaknesses": ["Minor style inconsistency in header"]
},
"B": {
"score": 5,
"strengths": ["Readable output", "Correct basic structure"],
"weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"]
}
},
"expectation_results": {
"A": {
"passed": 4,
"total": 5,
"pass_rate": 0.80,
"details": [
{"text": "Output includes name", "passed": true},
{"text": "Output includes date", "passed": true},
{"text": "Format is PDF", "passed": true},
{"text": "Contains signature", "passed": false},
{"text": "Readable text", "passed": true}
]
},
"B": {
"passed": 3,
"total": 5,
"pass_rate": 0.60,
"details": [
{"text": "Output includes name", "passed": true},
{"text": "Output includes date", "passed": false},
{"text": "Format is PDF", "passed": true},
{"text": "Contains signature", "passed": false},
{"text": "Readable text", "passed": true}
]
}
}
}
```
If no expectations were provided, omit the `expectation_results` field entirely.
## Field Descriptions
- **winner**: "A", "B", or "TIE"
- **reasoning**: Clear explanation of why the winner was chosen (or why it's a tie)
- **rubric**: Structured rubric evaluation for each output
- **content**: Scores for content criteria (correctness, completeness, accuracy)
- **structure**: Scores for structure criteria (organization, formatting, usability)
- **content_score**: Average of content criteria (1-5)
- **structure_score**: Average of structure criteria (1-5)
- **overall_score**: Combined score scaled to 1-10
- **output_quality**: Summary quality assessment
- **score**: 1-10 rating (should match rubric overall_score)
- **strengths**: List of positive aspects
- **weaknesses**: List of issues or shortcomings
- **expectation_results**: (Only if expectations provided)
- **passed**: Number of expectations that passed
- **total**: Total number of expectations
- **pass_rate**: Fraction passed (0.0 to 1.0)
- **details**: Individual expectation results
## Guidelines
- **Stay blind**: DO NOT try to infer which skill produced which output. Judge purely on output quality.
- **Be specific**: Cite specific examples when explaining strengths and weaknesses.
- **Be decisive**: Choose a winner unless outputs are genuinely equivalent.
- **Output quality first**: Assertion scores are secondary to overall task completion.
- **Be objective**: Don't favor outputs based on style preferences; focus on correctness and completeness.
- **Explain your reasoning**: The reasoning field should make it clear why you chose the winner.
- **Handle edge cases**: If both outputs fail, pick the one that fails less badly. If both are excellent, pick the one that's marginally better.
.claude/skills/skill-creator/agents/grader.md
+223
View File
@@ -0,0 +1,223 @@
# Grader Agent
Evaluate expectations against an execution transcript and outputs.
## Role
The Grader reviews a transcript and output files, then determines whether each expectation passes or fails. Provide clear evidence for each judgment.
You have two jobs: grade the outputs, and critique the evals themselves. A passing grade on a weak assertion is worse than useless — it creates false confidence. When you notice an assertion that's trivially satisfied, or an important outcome that no assertion checks, say so.
## Inputs
You receive these parameters in your prompt:
- **expectations**: List of expectations to evaluate (strings)
- **transcript_path**: Path to the execution transcript (markdown file)
- **outputs_dir**: Directory containing output files from execution
## Process
### Step 1: Read the Transcript
1. Read the transcript file completely
2. Note the eval prompt, execution steps, and final result
3. Identify any issues or errors documented
### Step 2: Examine Output Files
1. List files in outputs_dir
2. Read/examine each file relevant to the expectations. If outputs aren't plain text, use the inspection tools provided in your prompt — don't rely solely on what the transcript says the executor produced.
3. Note contents, structure, and quality
### Step 3: Evaluate Each Assertion
For each expectation:
1. **Search for evidence** in the transcript and outputs
2. **Determine verdict**:
- **PASS**: Clear evidence the expectation is true AND the evidence reflects genuine task completion, not just surface-level compliance
- **FAIL**: No evidence, or evidence contradicts the expectation, or the evidence is superficial (e.g., correct filename but empty/wrong content)
3. **Cite the evidence**: Quote the specific text or describe what you found
### Step 4: Extract and Verify Claims
Beyond the predefined expectations, extract implicit claims from the outputs and verify them:
1. **Extract claims** from the transcript and outputs:
- Factual statements ("The form has 12 fields")
- Process claims ("Used pypdf to fill the form")
- Quality claims ("All fields were filled correctly")
2. **Verify each claim**:
- **Factual claims**: Can be checked against the outputs or external sources
- **Process claims**: Can be verified from the transcript
- **Quality claims**: Evaluate whether the claim is justified
3. **Flag unverifiable claims**: Note claims that cannot be verified with available information
This catches issues that predefined expectations might miss.
### Step 5: Read User Notes
If `{outputs_dir}/user_notes.md` exists:
1. Read it and note any uncertainties or issues flagged by the executor
2. Include relevant concerns in the grading output
3. These may reveal problems even when expectations pass
### Step 6: Critique the Evals
After grading, consider whether the evals themselves could be improved. Only surface suggestions when there's a clear gap.
Good suggestions test meaningful outcomes — assertions that are hard to satisfy without actually doing the work correctly. Think about what makes an assertion *discriminating*: it passes when the skill genuinely succeeds and fails when it doesn't.
Suggestions worth raising:
- An assertion that passed but would also pass for a clearly wrong output (e.g., checking filename existence but not file content)
- An important outcome you observed — good or bad — that no assertion covers at all
- An assertion that can't actually be verified from the available outputs
Keep the bar high. The goal is to flag things the eval author would say "good catch" about, not to nitpick every assertion.
### Step 7: Write Grading Results
Save results to `{outputs_dir}/../grading.json` (sibling to outputs_dir).
## Grading Criteria
**PASS when**:
- The transcript or outputs clearly demonstrate the expectation is true
- Specific evidence can be cited
- The evidence reflects genuine substance, not just surface compliance (e.g., a file exists AND contains correct content, not just the right filename)
**FAIL when**:
- No evidence found for the expectation
- Evidence contradicts the expectation
- The expectation cannot be verified from available information
- The evidence is superficial — the assertion is technically satisfied but the underlying task outcome is wrong or incomplete
- The output appears to meet the assertion by coincidence rather than by actually doing the work
**When uncertain**: The burden of proof to pass is on the expectation.
### Step 8: Read Executor Metrics and Timing
1. If `{outputs_dir}/metrics.json` exists, read it and include in grading output
2. If `{outputs_dir}/../timing.json` exists, read it and include timing data
## Output Format
Write a JSON file with this structure:
```json
{
"expectations": [
{
"text": "The output includes the name 'John Smith'",
"passed": true,
"evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'"
},
{
"text": "The spreadsheet has a SUM formula in cell B10",
"passed": false,
"evidence": "No spreadsheet was created. The output was a text file."
},
{
"text": "The assistant used the skill's OCR script",
"passed": true,
"evidence": "Transcript Step 2 shows: 'Tool: Bash - python ocr_script.py image.png'"
}
],
"summary": {
"passed": 2,
"failed": 1,
"total": 3,
"pass_rate": 0.67
},
"execution_metrics": {
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8
},
"total_tool_calls": 15,
"total_steps": 6,
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
},
"timing": {
"executor_duration_seconds": 165.0,
"grader_duration_seconds": 26.0,
"total_duration_seconds": 191.0
},
"claims": [
{
"claim": "The form has 12 fillable fields",
"type": "factual",
"verified": true,
"evidence": "Counted 12 fields in field_info.json"
},
{
"claim": "All required fields were populated",
"type": "quality",
"verified": false,
"evidence": "Reference section was left blank despite data being available"
}
],
"user_notes_summary": {
"uncertainties": ["Used 2023 data, may be stale"],
"needs_review": [],
"workarounds": ["Fell back to text overlay for non-fillable fields"]
},
"eval_feedback": {
"suggestions": [
{
"assertion": "The output includes the name 'John Smith'",
"reason": "A hallucinated document that mentions the name would also pass — consider checking it appears as the primary contact with matching phone and email from the input"
},
{
"reason": "No assertion checks whether the extracted phone numbers match the input — I observed incorrect numbers in the output that went uncaught"
}
],
"overall": "Assertions check presence but not correctness. Consider adding content verification."
}
}
```
## Field Descriptions
- **expectations**: Array of graded expectations
- **text**: The original expectation text
- **passed**: Boolean - true if expectation passes
- **evidence**: Specific quote or description supporting the verdict
- **summary**: Aggregate statistics
- **passed**: Count of passed expectations
- **failed**: Count of failed expectations
- **total**: Total expectations evaluated
- **pass_rate**: Fraction passed (0.0 to 1.0)
- **execution_metrics**: Copied from executor's metrics.json (if available)
- **output_chars**: Total character count of output files (proxy for tokens)
- **transcript_chars**: Character count of transcript
- **timing**: Wall clock timing from timing.json (if available)
- **executor_duration_seconds**: Time spent in executor subagent
- **total_duration_seconds**: Total elapsed time for the run
- **claims**: Extracted and verified claims from the output
- **claim**: The statement being verified
- **type**: "factual", "process", or "quality"
- **verified**: Boolean - whether the claim holds
- **evidence**: Supporting or contradicting evidence
- **user_notes_summary**: Issues flagged by the executor
- **uncertainties**: Things the executor wasn't sure about
- **needs_review**: Items requiring human attention
- **workarounds**: Places where the skill didn't work as expected
- **eval_feedback**: Improvement suggestions for the evals (only when warranted)
- **suggestions**: List of concrete suggestions, each with a `reason` and optionally an `assertion` it relates to
- **overall**: Brief assessment — can be "No suggestions, evals look solid" if nothing to flag
## Guidelines
- **Be objective**: Base verdicts on evidence, not assumptions
- **Be specific**: Quote the exact text that supports your verdict
- **Be thorough**: Check both transcript and output files
- **Be consistent**: Apply the same standard to each expectation
- **Explain failures**: Make it clear why evidence was insufficient
- **No partial credit**: Each expectation is pass or fail, not partial
.claude/skills/skill-creator/assets/eval_review.html
+146
View File
@@ -0,0 +1,146 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Eval Set Review - __SKILL_NAME_PLACEHOLDER__</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Poppins:wght@500;600&family=Lora:wght@400;500&display=swap" rel="stylesheet">
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body { font-family: 'Lora', Georgia, serif; background: #faf9f5; padding: 2rem; color: #141413; }
h1 { font-family: 'Poppins', sans-serif; margin-bottom: 0.5rem; font-size: 1.5rem; }
.description { color: #b0aea5; margin-bottom: 1.5rem; font-style: italic; max-width: 900px; }
.controls { margin-bottom: 1rem; display: flex; gap: 0.5rem; }
.btn { font-family: 'Poppins', sans-serif; padding: 0.5rem 1rem; border: none; border-radius: 6px; cursor: pointer; font-size: 0.875rem; font-weight: 500; }
.btn-add { background: #6a9bcc; color: white; }
.btn-add:hover { background: #5889b8; }
.btn-export { background: #d97757; color: white; }
.btn-export:hover { background: #c4613f; }
table { width: 100%; max-width: 1100px; border-collapse: collapse; background: white; border-radius: 6px; overflow: hidden; box-shadow: 0 1px 3px rgba(0,0,0,0.08); }
th { font-family: 'Poppins', sans-serif; background: #141413; color: #faf9f5; padding: 0.75rem 1rem; text-align: left; font-size: 0.875rem; }
td { padding: 0.75rem 1rem; border-bottom: 1px solid #e8e6dc; vertical-align: top; }
tr:nth-child(even) td { background: #faf9f5; }
tr:hover td { background: #f3f1ea; }
.section-header td { background: #e8e6dc; font-family: 'Poppins', sans-serif; font-weight: 500; font-size: 0.8rem; color: #141413; text-transform: uppercase; letter-spacing: 0.05em; }
.query-input { width: 100%; padding: 0.4rem; border: 1px solid #e8e6dc; border-radius: 4px; font-size: 0.875rem; font-family: 'Lora', Georgia, serif; resize: vertical; min-height: 60px; }
.query-input:focus { outline: none; border-color: #d97757; box-shadow: 0 0 0 2px rgba(217,119,87,0.15); }
.toggle { position: relative; display: inline-block; width: 44px; height: 24px; }
.toggle input { opacity: 0; width: 0; height: 0; }
.toggle .slider { position: absolute; inset: 0; background: #b0aea5; border-radius: 24px; cursor: pointer; transition: 0.2s; }
.toggle .slider::before { content: ""; position: absolute; width: 18px; height: 18px; left: 3px; bottom: 3px; background: white; border-radius: 50%; transition: 0.2s; }
.toggle input:checked + .slider { background: #d97757; }
.toggle input:checked + .slider::before { transform: translateX(20px); }
.btn-delete { background: #c44; color: white; padding: 0.3rem 0.6rem; border: none; border-radius: 4px; cursor: pointer; font-size: 0.75rem; font-family: 'Poppins', sans-serif; }
.btn-delete:hover { background: #a33; }
.summary { margin-top: 1rem; color: #b0aea5; font-size: 0.875rem; }
</style>
</head>
<body>
<h1>Eval Set Review: <span id="skill-name">__SKILL_NAME_PLACEHOLDER__</span></h1>
<p class="description">Current description: <span id="skill-desc">__SKILL_DESCRIPTION_PLACEHOLDER__</span></p>
<div class="controls">
<button class="btn btn-add" onclick="addRow()">+ Add Query</button>
<button class="btn btn-export" onclick="exportEvalSet()">Export Eval Set</button>
</div>
<table>
<thead>
<tr>
<th style="width:65%">Query</th>
<th style="width:18%">Should Trigger</th>
<th style="width:10%">Actions</th>
</tr>
</thead>
<tbody id="eval-body"></tbody>
</table>
<p class="summary" id="summary"></p>
<script>
const EVAL_DATA = __EVAL_DATA_PLACEHOLDER__;
let evalItems = [...EVAL_DATA];
function render() {
const tbody = document.getElementById('eval-body');
tbody.innerHTML = '';
// Sort: should-trigger first, then should-not-trigger
const sorted = evalItems
.map((item, origIdx) => ({ ...item, origIdx }))
.sort((a, b) => (b.should_trigger ? 1 : 0) - (a.should_trigger ? 1 : 0));
let lastGroup = null;
sorted.forEach(item => {
const group = item.should_trigger ? 'trigger' : 'no-trigger';
if (group !== lastGroup) {
const headerRow = document.createElement('tr');
headerRow.className = 'section-header';
headerRow.innerHTML = `<td colspan="3">${item.should_trigger ? 'Should Trigger' : 'Should NOT Trigger'}</td>`;
tbody.appendChild(headerRow);
lastGroup = group;
}
const idx = item.origIdx;
const tr = document.createElement('tr');
tr.innerHTML = `
<td><textarea class="query-input" onchange="updateQuery(${idx}, this.value)">${escapeHtml(item.query)}</textarea></td>
<td>
<label class="toggle">
<input type="checkbox" ${item.should_trigger ? 'checked' : ''} onchange="updateTrigger(${idx}, this.checked)">
<span class="slider"></span>
</label>
<span style="margin-left:8px;font-size:0.8rem;color:#b0aea5">${item.should_trigger ? 'Yes' : 'No'}</span>
</td>
<td><button class="btn-delete" onclick="deleteRow(${idx})">Delete</button></td>
`;
tbody.appendChild(tr);
});
updateSummary();
}
function escapeHtml(text) {
const div = document.createElement('div');
div.textContent = text;
return div.innerHTML;
}
function updateQuery(idx, value) { evalItems[idx].query = value; updateSummary(); }
function updateTrigger(idx, value) { evalItems[idx].should_trigger = value; render(); }
function deleteRow(idx) { evalItems.splice(idx, 1); render(); }
function addRow() {
evalItems.push({ query: '', should_trigger: true });
render();
const inputs = document.querySelectorAll('.query-input');
inputs[inputs.length - 1].focus();
}
function updateSummary() {
const trigger = evalItems.filter(i => i.should_trigger).length;
const noTrigger = evalItems.filter(i => !i.should_trigger).length;
document.getElementById('summary').textContent =
`${evalItems.length} queries total: ${trigger} should trigger, ${noTrigger} should not trigger`;
}
function exportEvalSet() {
const valid = evalItems.filter(i => i.query.trim() !== '');
const data = valid.map(i => ({ query: i.query.trim(), should_trigger: i.should_trigger }));
const blob = new Blob([JSON.stringify(data, null, 2)], { type: 'application/json' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'eval_set.json';
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
}
render();
</script>
</body>
</html>
.claude/skills/skill-creator/eval-viewer/generate_review.py
+471
View File
@@ -0,0 +1,471 @@
#!/usr/bin/env python3
"""Generate and serve a review page for eval results.
Reads the workspace directory, discovers runs (directories with outputs/),
embeds all output data into a self-contained HTML page, and serves it via
a tiny HTTP server. Feedback auto-saves to feedback.json in the workspace.
Usage:
python generate_review.py <workspace-path> [--port PORT] [--skill-name NAME]
python generate_review.py <workspace-path> --previous-feedback /path/to/old/feedback.json
No dependencies beyond the Python stdlib are required.
"""
import argparse
import base64
import json
import mimetypes
import os
import re
import signal
import subprocess
import sys
import time
import webbrowser
from functools import partial
from http.server import HTTPServer, BaseHTTPRequestHandler
from pathlib import Path
# Files to exclude from output listings
METADATA_FILES = {"transcript.md", "user_notes.md", "metrics.json"}
# Extensions we render as inline text
TEXT_EXTENSIONS = {
".txt", ".md", ".json", ".csv", ".py", ".js", ".ts", ".tsx", ".jsx",
".yaml", ".yml", ".xml", ".html", ".css", ".sh", ".rb", ".go", ".rs",
".java", ".c", ".cpp", ".h", ".hpp", ".sql", ".r", ".toml",
}
# Extensions we render as inline images
IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp"}
# MIME type overrides for common types
MIME_OVERRIDES = {
".svg": "image/svg+xml",
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
}
def get_mime_type(path: Path) -> str:
ext = path.suffix.lower()
if ext in MIME_OVERRIDES:
return MIME_OVERRIDES[ext]
mime, _ = mimetypes.guess_type(str(path))
return mime or "application/octet-stream"
def find_runs(workspace: Path) -> list[dict]:
"""Recursively find directories that contain an outputs/ subdirectory."""
runs: list[dict] = []
_find_runs_recursive(workspace, workspace, runs)
runs.sort(key=lambda r: (r.get("eval_id", float("inf")), r["id"]))
return runs
def _find_runs_recursive(root: Path, current: Path, runs: list[dict]) -> None:
if not current.is_dir():
return
outputs_dir = current / "outputs"
if outputs_dir.is_dir():
run = build_run(root, current)
if run:
runs.append(run)
return
skip = {"node_modules", ".git", "__pycache__", "skill", "inputs"}
for child in sorted(current.iterdir()):
if child.is_dir() and child.name not in skip:
_find_runs_recursive(root, child, runs)
def build_run(root: Path, run_dir: Path) -> dict | None:
"""Build a run dict with prompt, outputs, and grading data."""
prompt = ""
eval_id = None
# Try eval_metadata.json
for candidate in [run_dir / "eval_metadata.json", run_dir.parent / "eval_metadata.json"]:
if candidate.exists():
try:
metadata = json.loads(candidate.read_text())
prompt = metadata.get("prompt", "")
eval_id = metadata.get("eval_id")
except (json.JSONDecodeError, OSError):
pass
if prompt:
break
# Fall back to transcript.md
if not prompt:
for candidate in [run_dir / "transcript.md", run_dir / "outputs" / "transcript.md"]:
if candidate.exists():
try:
text = candidate.read_text()
match = re.search(r"## Eval Prompt\n\n([\s\S]*?)(?=\n##|$)", text)
if match:
prompt = match.group(1).strip()
except OSError:
pass
if prompt:
break
if not prompt:
prompt = "(No prompt found)"
run_id = str(run_dir.relative_to(root)).replace("/", "-").replace("\\", "-")
# Collect output files
outputs_dir = run_dir / "outputs"
output_files: list[dict] = []
if outputs_dir.is_dir():
for f in sorted(outputs_dir.iterdir()):
if f.is_file() and f.name not in METADATA_FILES:
output_files.append(embed_file(f))
# Load grading if present
grading = None
for candidate in [run_dir / "grading.json", run_dir.parent / "grading.json"]:
if candidate.exists():
try:
grading = json.loads(candidate.read_text())
except (json.JSONDecodeError, OSError):
pass
if grading:
break
return {
"id": run_id,
"prompt": prompt,
"eval_id": eval_id,
"outputs": output_files,
"grading": grading,
}
def embed_file(path: Path) -> dict:
"""Read a file and return an embedded representation."""
ext = path.suffix.lower()
mime = get_mime_type(path)
if ext in TEXT_EXTENSIONS:
try:
content = path.read_text(errors="replace")
except OSError:
content = "(Error reading file)"
return {
"name": path.name,
"type": "text",
"content": content,
}
elif ext in IMAGE_EXTENSIONS:
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "image",
"mime": mime,
"data_uri": f"data:{mime};base64,{b64}",
}
elif ext == ".pdf":
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "pdf",
"data_uri": f"data:{mime};base64,{b64}",
}
elif ext == ".xlsx":
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "xlsx",
"data_b64": b64,
}
else:
# Binary / unknown — base64 download link
try:
raw = path.read_bytes()
b64 = base64.b64encode(raw).decode("ascii")
except OSError:
return {"name": path.name, "type": "error", "content": "(Error reading file)"}
return {
"name": path.name,
"type": "binary",
"mime": mime,
"data_uri": f"data:{mime};base64,{b64}",
}
def load_previous_iteration(workspace: Path) -> dict[str, dict]:
"""Load previous iteration's feedback and outputs.
Returns a map of run_id -> {"feedback": str, "outputs": list[dict]}.
"""
result: dict[str, dict] = {}
# Load feedback
feedback_map: dict[str, str] = {}
feedback_path = workspace / "feedback.json"
if feedback_path.exists():
try:
data = json.loads(feedback_path.read_text())
feedback_map = {
r["run_id"]: r["feedback"]
for r in data.get("reviews", [])
if r.get("feedback", "").strip()
}
except (json.JSONDecodeError, OSError, KeyError):
pass
# Load runs (to get outputs)
prev_runs = find_runs(workspace)
for run in prev_runs:
result[run["id"]] = {
"feedback": feedback_map.get(run["id"], ""),
"outputs": run.get("outputs", []),
}
# Also add feedback for run_ids that had feedback but no matching run
for run_id, fb in feedback_map.items():
if run_id not in result:
result[run_id] = {"feedback": fb, "outputs": []}
return result
def generate_html(
runs: list[dict],
skill_name: str,
previous: dict[str, dict] | None = None,
benchmark: dict | None = None,
) -> str:
"""Generate the complete standalone HTML page with embedded data."""
template_path = Path(__file__).parent / "viewer.html"
template = template_path.read_text()
# Build previous_feedback and previous_outputs maps for the template
previous_feedback: dict[str, str] = {}
previous_outputs: dict[str, list[dict]] = {}
if previous:
for run_id, data in previous.items():
if data.get("feedback"):
previous_feedback[run_id] = data["feedback"]
if data.get("outputs"):
previous_outputs[run_id] = data["outputs"]
embedded = {
"skill_name": skill_name,
"runs": runs,
"previous_feedback": previous_feedback,
"previous_outputs": previous_outputs,
}
if benchmark:
embedded["benchmark"] = benchmark
data_json = json.dumps(embedded)
return template.replace("/*__EMBEDDED_DATA__*/", f"const EMBEDDED_DATA = {data_json};")
# ---------------------------------------------------------------------------
# HTTP server (stdlib only, zero dependencies)
# ---------------------------------------------------------------------------
def _kill_port(port: int) -> None:
"""Kill any process listening on the given port."""
try:
result = subprocess.run(
["lsof", "-ti", f":{port}"],
capture_output=True, text=True, timeout=5,
)
for pid_str in result.stdout.strip().split("\n"):
if pid_str.strip():
try:
os.kill(int(pid_str.strip()), signal.SIGTERM)
except (ProcessLookupError, ValueError):
pass
if result.stdout.strip():
time.sleep(0.5)
except subprocess.TimeoutExpired:
pass
except FileNotFoundError:
print("Note: lsof not found, cannot check if port is in use", file=sys.stderr)
class ReviewHandler(BaseHTTPRequestHandler):
"""Serves the review HTML and handles feedback saves.
Regenerates the HTML on each page load so that refreshing the browser
picks up new eval outputs without restarting the server.
"""
def __init__(
self,
workspace: Path,
skill_name: str,
feedback_path: Path,
previous: dict[str, dict],
benchmark_path: Path | None,
*args,
**kwargs,
):
self.workspace = workspace
self.skill_name = skill_name
self.feedback_path = feedback_path
self.previous = previous
self.benchmark_path = benchmark_path
super().__init__(*args, **kwargs)
def do_GET(self) -> None:
if self.path == "/" or self.path == "/index.html":
# Regenerate HTML on each request (re-scans workspace for new outputs)
runs = find_runs(self.workspace)
benchmark = None
if self.benchmark_path and self.benchmark_path.exists():
try:
benchmark = json.loads(self.benchmark_path.read_text())
except (json.JSONDecodeError, OSError):
pass
html = generate_html(runs, self.skill_name, self.previous, benchmark)
content = html.encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(content)))
self.end_headers()
self.wfile.write(content)
elif self.path == "/api/feedback":
data = b"{}"
if self.feedback_path.exists():
data = self.feedback_path.read_bytes()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
else:
self.send_error(404)
def do_POST(self) -> None:
if self.path == "/api/feedback":
length = int(self.headers.get("Content-Length", 0))
body = self.rfile.read(length)
try:
data = json.loads(body)
if not isinstance(data, dict) or "reviews" not in data:
raise ValueError("Expected JSON object with 'reviews' key")
self.feedback_path.write_text(json.dumps(data, indent=2) + "\n")
resp = b'{"ok":true}'
self.send_response(200)
except (json.JSONDecodeError, OSError, ValueError) as e:
resp = json.dumps({"error": str(e)}).encode()
self.send_response(500)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(resp)))
self.end_headers()
self.wfile.write(resp)
else:
self.send_error(404)
def log_message(self, format: str, *args: object) -> None:
# Suppress request logging to keep terminal clean
pass
def main() -> None:
parser = argparse.ArgumentParser(description="Generate and serve eval review")
parser.add_argument("workspace", type=Path, help="Path to workspace directory")
parser.add_argument("--port", "-p", type=int, default=3117, help="Server port (default: 3117)")
parser.add_argument("--skill-name", "-n", type=str, default=None, help="Skill name for header")
parser.add_argument(
"--previous-workspace", type=Path, default=None,
help="Path to previous iteration's workspace (shows old outputs and feedback as context)",
)
parser.add_argument(
"--benchmark", type=Path, default=None,
help="Path to benchmark.json to show in the Benchmark tab",
)
parser.add_argument(
"--static", "-s", type=Path, default=None,
help="Write standalone HTML to this path instead of starting a server",
)
args = parser.parse_args()
workspace = args.workspace.resolve()
if not workspace.is_dir():
print(f"Error: {workspace} is not a directory", file=sys.stderr)
sys.exit(1)
runs = find_runs(workspace)
if not runs:
print(f"No runs found in {workspace}", file=sys.stderr)
sys.exit(1)
skill_name = args.skill_name or workspace.name.replace("-workspace", "")
feedback_path = workspace / "feedback.json"
previous: dict[str, dict] = {}
if args.previous_workspace:
previous = load_previous_iteration(args.previous_workspace.resolve())
benchmark_path = args.benchmark.resolve() if args.benchmark else None
benchmark = None
if benchmark_path and benchmark_path.exists():
try:
benchmark = json.loads(benchmark_path.read_text())
except (json.JSONDecodeError, OSError):
pass
if args.static:
html = generate_html(runs, skill_name, previous, benchmark)
args.static.parent.mkdir(parents=True, exist_ok=True)
args.static.write_text(html)
print(f"\n Static viewer written to: {args.static}\n")
sys.exit(0)
# Kill any existing process on the target port
port = args.port
_kill_port(port)
handler = partial(ReviewHandler, workspace, skill_name, feedback_path, previous, benchmark_path)
try:
server = HTTPServer(("127.0.0.1", port), handler)
except OSError:
# Port still in use after kill attempt — find a free one
server = HTTPServer(("127.0.0.1", 0), handler)
port = server.server_address[1]
url = f"http://localhost:{port}"
print(f"\n Eval Viewer")
print(f" ─────────────────────────────────")
print(f" URL: {url}")
print(f" Workspace: {workspace}")
print(f" Feedback: {feedback_path}")
if previous:
print(f" Previous: {args.previous_workspace} ({len(previous)} runs)")
if benchmark_path:
print(f" Benchmark: {benchmark_path}")
print(f"\n Press Ctrl+C to stop.\n")
webbrowser.open(url)
try:
server.serve_forever()
except KeyboardInterrupt:
print("\nStopped.")
server.server_close()
if __name__ == "__main__":
main()
.claude/skills/skill-creator/eval-viewer/viewer.html
+1325
View File
File diff suppressed because it is too large Load Diff
.claude/skills/skill-creator/references/schemas.md
+430
View File
@@ -0,0 +1,430 @@
# JSON Schemas
This document defines the JSON schemas used by skill-creator.
---
## evals.json
Defines the evals for a skill. Located at `evals/evals.json` within the skill directory.
```json
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "User's example prompt",
"expected_output": "Description of expected result",
"files": ["evals/files/sample1.pdf"],
"expectations": [
"The output includes X",
"The skill used script Y"
]
}
]
}
```
**Fields:**
- `skill_name`: Name matching the skill's frontmatter
- `evals[].id`: Unique integer identifier
- `evals[].prompt`: The task to execute
- `evals[].expected_output`: Human-readable description of success
- `evals[].files`: Optional list of input file paths (relative to skill root)
- `evals[].expectations`: List of verifiable statements
---
## history.json
Tracks version progression in Improve mode. Located at workspace root.
```json
{
"started_at": "2026-01-15T10:30:00Z",
"skill_name": "pdf",
"current_best": "v2",
"iterations": [
{
"version": "v0",
"parent": null,
"expectation_pass_rate": 0.65,
"grading_result": "baseline",
"is_current_best": false
},
{
"version": "v1",
"parent": "v0",
"expectation_pass_rate": 0.75,
"grading_result": "won",
"is_current_best": false
},
{
"version": "v2",
"parent": "v1",
"expectation_pass_rate": 0.85,
"grading_result": "won",
"is_current_best": true
}
]
}
```
**Fields:**
- `started_at`: ISO timestamp of when improvement started
- `skill_name`: Name of the skill being improved
- `current_best`: Version identifier of the best performer
- `iterations[].version`: Version identifier (v0, v1, ...)
- `iterations[].parent`: Parent version this was derived from
- `iterations[].expectation_pass_rate`: Pass rate from grading
- `iterations[].grading_result`: "baseline", "won", "lost", or "tie"
- `iterations[].is_current_best`: Whether this is the current best version
---
## grading.json
Output from the grader agent. Located at `<run-dir>/grading.json`.
```json
{
"expectations": [
{
"text": "The output includes the name 'John Smith'",
"passed": true,
"evidence": "Found in transcript Step 3: 'Extracted names: John Smith, Sarah Johnson'"
},
{
"text": "The spreadsheet has a SUM formula in cell B10",
"passed": false,
"evidence": "No spreadsheet was created. The output was a text file."
}
],
"summary": {
"passed": 2,
"failed": 1,
"total": 3,
"pass_rate": 0.67
},
"execution_metrics": {
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8
},
"total_tool_calls": 15,
"total_steps": 6,
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
},
"timing": {
"executor_duration_seconds": 165.0,
"grader_duration_seconds": 26.0,
"total_duration_seconds": 191.0
},
"claims": [
{
"claim": "The form has 12 fillable fields",
"type": "factual",
"verified": true,
"evidence": "Counted 12 fields in field_info.json"
}
],
"user_notes_summary": {
"uncertainties": ["Used 2023 data, may be stale"],
"needs_review": [],
"workarounds": ["Fell back to text overlay for non-fillable fields"]
},
"eval_feedback": {
"suggestions": [
{
"assertion": "The output includes the name 'John Smith'",
"reason": "A hallucinated document that mentions the name would also pass"
}
],
"overall": "Assertions check presence but not correctness."
}
}
```
**Fields:**
- `expectations[]`: Graded expectations with evidence
- `summary`: Aggregate pass/fail counts
- `execution_metrics`: Tool usage and output size (from executor's metrics.json)
- `timing`: Wall clock timing (from timing.json)
- `claims`: Extracted and verified claims from the output
- `user_notes_summary`: Issues flagged by the executor
- `eval_feedback`: (optional) Improvement suggestions for the evals, only present when the grader identifies issues worth raising
---
## metrics.json
Output from the executor agent. Located at `<run-dir>/outputs/metrics.json`.
```json
{
"tool_calls": {
"Read": 5,
"Write": 2,
"Bash": 8,
"Edit": 1,
"Glob": 2,
"Grep": 0
},
"total_tool_calls": 18,
"total_steps": 6,
"files_created": ["filled_form.pdf", "field_values.json"],
"errors_encountered": 0,
"output_chars": 12450,
"transcript_chars": 3200
}
```
**Fields:**
- `tool_calls`: Count per tool type
- `total_tool_calls`: Sum of all tool calls
- `total_steps`: Number of major execution steps
- `files_created`: List of output files created
- `errors_encountered`: Number of errors during execution
- `output_chars`: Total character count of output files
- `transcript_chars`: Character count of transcript
---
## timing.json
Wall clock timing for a run. Located at `<run-dir>/timing.json`.
**How to capture:** When a subagent task completes, the task notification includes `total_tokens` and `duration_ms`. Save these immediately — they are not persisted anywhere else and cannot be recovered after the fact.
```json
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3,
"executor_start": "2026-01-15T10:30:00Z",
"executor_end": "2026-01-15T10:32:45Z",
"executor_duration_seconds": 165.0,
"grader_start": "2026-01-15T10:32:46Z",
"grader_end": "2026-01-15T10:33:12Z",
"grader_duration_seconds": 26.0
}
```
---
## benchmark.json
Output from Benchmark mode. Located at `benchmarks/<timestamp>/benchmark.json`.
```json
{
"metadata": {
"skill_name": "pdf",
"skill_path": "/path/to/pdf",
"executor_model": "claude-sonnet-4-20250514",
"analyzer_model": "most-capable-model",
"timestamp": "2026-01-15T10:30:00Z",
"evals_run": [1, 2, 3],
"runs_per_configuration": 3
},
"runs": [
{
"eval_id": 1,
"eval_name": "Ocean",
"configuration": "with_skill",
"run_number": 1,
"result": {
"pass_rate": 0.85,
"passed": 6,
"failed": 1,
"total": 7,
"time_seconds": 42.5,
"tokens": 3800,
"tool_calls": 18,
"errors": 0
},
"expectations": [
{"text": "...", "passed": true, "evidence": "..."}
],
"notes": [
"Used 2023 data, may be stale",
"Fell back to text overlay for non-fillable fields"
]
}
],
"run_summary": {
"with_skill": {
"pass_rate": {"mean": 0.85, "stddev": 0.05, "min": 0.80, "max": 0.90},
"time_seconds": {"mean": 45.0, "stddev": 12.0, "min": 32.0, "max": 58.0},
"tokens": {"mean": 3800, "stddev": 400, "min": 3200, "max": 4100}
},
"without_skill": {
"pass_rate": {"mean": 0.35, "stddev": 0.08, "min": 0.28, "max": 0.45},
"time_seconds": {"mean": 32.0, "stddev": 8.0, "min": 24.0, "max": 42.0},
"tokens": {"mean": 2100, "stddev": 300, "min": 1800, "max": 2500}
},
"delta": {
"pass_rate": "+0.50",
"time_seconds": "+13.0",
"tokens": "+1700"
}
},
"notes": [
"Assertion 'Output is a PDF file' passes 100% in both configurations - may not differentiate skill value",
"Eval 3 shows high variance (50% ± 40%) - may be flaky or model-dependent",
"Without-skill runs consistently fail on table extraction expectations",
"Skill adds 13s average execution time but improves pass rate by 50%"
]
}
```
**Fields:**
- `metadata`: Information about the benchmark run
- `skill_name`: Name of the skill
- `timestamp`: When the benchmark was run
- `evals_run`: List of eval names or IDs
- `runs_per_configuration`: Number of runs per config (e.g. 3)
- `runs[]`: Individual run results
- `eval_id`: Numeric eval identifier
- `eval_name`: Human-readable eval name (used as section header in the viewer)
- `configuration`: Must be `"with_skill"` or `"without_skill"` (the viewer uses this exact string for grouping and color coding)
- `run_number`: Integer run number (1, 2, 3...)
- `result`: Nested object with `pass_rate`, `passed`, `total`, `time_seconds`, `tokens`, `errors`
- `run_summary`: Statistical aggregates per configuration
- `with_skill` / `without_skill`: Each contains `pass_rate`, `time_seconds`, `tokens` objects with `mean` and `stddev` fields
- `delta`: Difference strings like `"+0.50"`, `"+13.0"`, `"+1700"`
- `notes`: Freeform observations from the analyzer
**Important:** The viewer reads these field names exactly. Using `config` instead of `configuration`, or putting `pass_rate` at the top level of a run instead of nested under `result`, will cause the viewer to show empty/zero values. Always reference this schema when generating benchmark.json manually.
---
## comparison.json
Output from blind comparator. Located at `<grading-dir>/comparison-N.json`.
```json
{
"winner": "A",
"reasoning": "Output A provides a complete solution with proper formatting and all required fields. Output B is missing the date field and has formatting inconsistencies.",
"rubric": {
"A": {
"content": {
"correctness": 5,
"completeness": 5,
"accuracy": 4
},
"structure": {
"organization": 4,
"formatting": 5,
"usability": 4
},
"content_score": 4.7,
"structure_score": 4.3,
"overall_score": 9.0
},
"B": {
"content": {
"correctness": 3,
"completeness": 2,
"accuracy": 3
},
"structure": {
"organization": 3,
"formatting": 2,
"usability": 3
},
"content_score": 2.7,
"structure_score": 2.7,
"overall_score": 5.4
}
},
"output_quality": {
"A": {
"score": 9,
"strengths": ["Complete solution", "Well-formatted", "All fields present"],
"weaknesses": ["Minor style inconsistency in header"]
},
"B": {
"score": 5,
"strengths": ["Readable output", "Correct basic structure"],
"weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"]
}
},
"expectation_results": {
"A": {
"passed": 4,
"total": 5,
"pass_rate": 0.80,
"details": [
{"text": "Output includes name", "passed": true}
]
},
"B": {
"passed": 3,
"total": 5,
"pass_rate": 0.60,
"details": [
{"text": "Output includes name", "passed": true}
]
}
}
}
```
---
## analysis.json
Output from post-hoc analyzer. Located at `<grading-dir>/analysis.json`.
```json
{
"comparison_summary": {
"winner": "A",
"winner_skill": "path/to/winner/skill",
"loser_skill": "path/to/loser/skill",
"comparator_reasoning": "Brief summary of why comparator chose winner"
},
"winner_strengths": [
"Clear step-by-step instructions for handling multi-page documents",
"Included validation script that caught formatting errors"
],
"loser_weaknesses": [
"Vague instruction 'process the document appropriately' led to inconsistent behavior",
"No script for validation, agent had to improvise"
],
"instruction_following": {
"winner": {
"score": 9,
"issues": ["Minor: skipped optional logging step"]
},
"loser": {
"score": 6,
"issues": [
"Did not use the skill's formatting template",
"Invented own approach instead of following step 3"
]
}
},
"improvement_suggestions": [
{
"priority": "high",
"category": "instructions",
"suggestion": "Replace 'process the document appropriately' with explicit steps",
"expected_impact": "Would eliminate ambiguity that caused inconsistent behavior"
}
],
"transcript_insights": {
"winner_execution_pattern": "Read skill -> Followed 5-step process -> Used validation script",
"loser_execution_pattern": "Read skill -> Unclear on approach -> Tried 3 different methods"
}
}
```
.claude/skills/skill-creator/scripts/__init__.py
View File
.claude/skills/skill-creator/scripts/aggregate_benchmark.py
Executable
+401
View File
@@ -0,0 +1,401 @@
#!/usr/bin/env python3
"""
Aggregate individual run results into benchmark summary statistics.
Reads grading.json files from run directories and produces:
- run_summary with mean, stddev, min, max for each metric
- delta between with_skill and without_skill configurations
Usage:
python aggregate_benchmark.py <benchmark_dir>
Example:
python aggregate_benchmark.py benchmarks/2026-01-15T10-30-00/
The script supports two directory layouts:
Workspace layout (from skill-creator iterations):
<benchmark_dir>/
└── eval-N/
├── with_skill/
│ ├── run-1/grading.json
│ └── run-2/grading.json
└── without_skill/
├── run-1/grading.json
└── run-2/grading.json
Legacy layout (with runs/ subdirectory):
<benchmark_dir>/
└── runs/
└── eval-N/
├── with_skill/
│ └── run-1/grading.json
└── without_skill/
└── run-1/grading.json
"""
import argparse
import json
import math
import sys
from datetime import datetime, timezone
from pathlib import Path
def calculate_stats(values: list[float]) -> dict:
"""Calculate mean, stddev, min, max for a list of values."""
if not values:
return {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0}
n = len(values)
mean = sum(values) / n
if n > 1:
variance = sum((x - mean) ** 2 for x in values) / (n - 1)
stddev = math.sqrt(variance)
else:
stddev = 0.0
return {
"mean": round(mean, 4),
"stddev": round(stddev, 4),
"min": round(min(values), 4),
"max": round(max(values), 4)
}
def load_run_results(benchmark_dir: Path) -> dict:
"""
Load all run results from a benchmark directory.
Returns dict keyed by config name (e.g. "with_skill"/"without_skill",
or "new_skill"/"old_skill"), each containing a list of run results.
"""
# Support both layouts: eval dirs directly under benchmark_dir, or under runs/
runs_dir = benchmark_dir / "runs"
if runs_dir.exists():
search_dir = runs_dir
elif list(benchmark_dir.glob("eval-*")):
search_dir = benchmark_dir
else:
print(f"No eval directories found in {benchmark_dir} or {benchmark_dir / 'runs'}")
return {}
results: dict[str, list] = {}
for eval_idx, eval_dir in enumerate(sorted(search_dir.glob("eval-*"))):
metadata_path = eval_dir / "eval_metadata.json"
if metadata_path.exists():
try:
with open(metadata_path) as mf:
eval_id = json.load(mf).get("eval_id", eval_idx)
except (json.JSONDecodeError, OSError):
eval_id = eval_idx
else:
try:
eval_id = int(eval_dir.name.split("-")[1])
except ValueError:
eval_id = eval_idx
# Discover config directories dynamically rather than hardcoding names
for config_dir in sorted(eval_dir.iterdir()):
if not config_dir.is_dir():
continue
# Skip non-config directories (inputs, outputs, etc.)
if not list(config_dir.glob("run-*")):
continue
config = config_dir.name
if config not in results:
results[config] = []
for run_dir in sorted(config_dir.glob("run-*")):
run_number = int(run_dir.name.split("-")[1])
grading_file = run_dir / "grading.json"
if not grading_file.exists():
print(f"Warning: grading.json not found in {run_dir}")
continue
try:
with open(grading_file) as f:
grading = json.load(f)
except json.JSONDecodeError as e:
print(f"Warning: Invalid JSON in {grading_file}: {e}")
continue
# Extract metrics
result = {
"eval_id": eval_id,
"run_number": run_number,
"pass_rate": grading.get("summary", {}).get("pass_rate", 0.0),
"passed": grading.get("summary", {}).get("passed", 0),
"failed": grading.get("summary", {}).get("failed", 0),
"total": grading.get("summary", {}).get("total", 0),
}
# Extract timing — check grading.json first, then sibling timing.json
timing = grading.get("timing", {})
result["time_seconds"] = timing.get("total_duration_seconds", 0.0)
timing_file = run_dir / "timing.json"
if result["time_seconds"] == 0.0 and timing_file.exists():
try:
with open(timing_file) as tf:
timing_data = json.load(tf)
result["time_seconds"] = timing_data.get("total_duration_seconds", 0.0)
result["tokens"] = timing_data.get("total_tokens", 0)
except json.JSONDecodeError:
pass
# Extract metrics if available
metrics = grading.get("execution_metrics", {})
result["tool_calls"] = metrics.get("total_tool_calls", 0)
if not result.get("tokens"):
result["tokens"] = metrics.get("output_chars", 0)
result["errors"] = metrics.get("errors_encountered", 0)
# Extract expectations — viewer requires fields: text, passed, evidence
raw_expectations = grading.get("expectations", [])
for exp in raw_expectations:
if "text" not in exp or "passed" not in exp:
print(f"Warning: expectation in {grading_file} missing required fields (text, passed, evidence): {exp}")
result["expectations"] = raw_expectations
# Extract notes from user_notes_summary
notes_summary = grading.get("user_notes_summary", {})
notes = []
notes.extend(notes_summary.get("uncertainties", []))
notes.extend(notes_summary.get("needs_review", []))
notes.extend(notes_summary.get("workarounds", []))
result["notes"] = notes
results[config].append(result)
return results
def aggregate_results(results: dict) -> dict:
"""
Aggregate run results into summary statistics.
Returns run_summary with stats for each configuration and delta.
"""
run_summary = {}
configs = list(results.keys())
for config in configs:
runs = results.get(config, [])
if not runs:
run_summary[config] = {
"pass_rate": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0},
"time_seconds": {"mean": 0.0, "stddev": 0.0, "min": 0.0, "max": 0.0},
"tokens": {"mean": 0, "stddev": 0, "min": 0, "max": 0}
}
continue
pass_rates = [r["pass_rate"] for r in runs]
times = [r["time_seconds"] for r in runs]
tokens = [r.get("tokens", 0) for r in runs]
run_summary[config] = {
"pass_rate": calculate_stats(pass_rates),
"time_seconds": calculate_stats(times),
"tokens": calculate_stats(tokens)
}
# Calculate delta between the first two configs (if two exist)
if len(configs) >= 2:
primary = run_summary.get(configs[0], {})
baseline = run_summary.get(configs[1], {})
else:
primary = run_summary.get(configs[0], {}) if configs else {}
baseline = {}
delta_pass_rate = primary.get("pass_rate", {}).get("mean", 0) - baseline.get("pass_rate", {}).get("mean", 0)
delta_time = primary.get("time_seconds", {}).get("mean", 0) - baseline.get("time_seconds", {}).get("mean", 0)
delta_tokens = primary.get("tokens", {}).get("mean", 0) - baseline.get("tokens", {}).get("mean", 0)
run_summary["delta"] = {
"pass_rate": f"{delta_pass_rate:+.2f}",
"time_seconds": f"{delta_time:+.1f}",
"tokens": f"{delta_tokens:+.0f}"
}
return run_summary
def generate_benchmark(benchmark_dir: Path, skill_name: str = "", skill_path: str = "") -> dict:
"""
Generate complete benchmark.json from run results.
"""
results = load_run_results(benchmark_dir)
run_summary = aggregate_results(results)
# Build runs array for benchmark.json
runs = []
for config in results:
for result in results[config]:
runs.append({
"eval_id": result["eval_id"],
"configuration": config,
"run_number": result["run_number"],
"result": {
"pass_rate": result["pass_rate"],
"passed": result["passed"],
"failed": result["failed"],
"total": result["total"],
"time_seconds": result["time_seconds"],
"tokens": result.get("tokens", 0),
"tool_calls": result.get("tool_calls", 0),
"errors": result.get("errors", 0)
},
"expectations": result["expectations"],
"notes": result["notes"]
})
# Determine eval IDs from results
eval_ids = sorted(set(
r["eval_id"]
for config in results.values()
for r in config
))
benchmark = {
"metadata": {
"skill_name": skill_name or "<skill-name>",
"skill_path": skill_path or "<path/to/skill>",
"executor_model": "<model-name>",
"analyzer_model": "<model-name>",
"timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"evals_run": eval_ids,
"runs_per_configuration": 3
},
"runs": runs,
"run_summary": run_summary,
"notes": [] # To be filled by analyzer
}
return benchmark
def generate_markdown(benchmark: dict) -> str:
"""Generate human-readable benchmark.md from benchmark data."""
metadata = benchmark["metadata"]
run_summary = benchmark["run_summary"]
# Determine config names (excluding "delta")
configs = [k for k in run_summary if k != "delta"]
config_a = configs[0] if len(configs) >= 1 else "config_a"
config_b = configs[1] if len(configs) >= 2 else "config_b"
label_a = config_a.replace("_", " ").title()
label_b = config_b.replace("_", " ").title()
lines = [
f"# Skill Benchmark: {metadata['skill_name']}",
"",
f"**Model**: {metadata['executor_model']}",
f"**Date**: {metadata['timestamp']}",
f"**Evals**: {', '.join(map(str, metadata['evals_run']))} ({metadata['runs_per_configuration']} runs each per configuration)",
"",
"## Summary",
"",
f"| Metric | {label_a} | {label_b} | Delta |",
"|--------|------------|---------------|-------|",
]
a_summary = run_summary.get(config_a, {})
b_summary = run_summary.get(config_b, {})
delta = run_summary.get("delta", {})
# Format pass rate
a_pr = a_summary.get("pass_rate", {})
b_pr = b_summary.get("pass_rate", {})
lines.append(f"| Pass Rate | {a_pr.get('mean', 0)*100:.0f}% ± {a_pr.get('stddev', 0)*100:.0f}% | {b_pr.get('mean', 0)*100:.0f}% ± {b_pr.get('stddev', 0)*100:.0f}% | {delta.get('pass_rate', '—')} |")
# Format time
a_time = a_summary.get("time_seconds", {})
b_time = b_summary.get("time_seconds", {})
lines.append(f"| Time | {a_time.get('mean', 0):.1f}s ± {a_time.get('stddev', 0):.1f}s | {b_time.get('mean', 0):.1f}s ± {b_time.get('stddev', 0):.1f}s | {delta.get('time_seconds', '—')}s |")
# Format tokens
a_tokens = a_summary.get("tokens", {})
b_tokens = b_summary.get("tokens", {})
lines.append(f"| Tokens | {a_tokens.get('mean', 0):.0f} ± {a_tokens.get('stddev', 0):.0f} | {b_tokens.get('mean', 0):.0f} ± {b_tokens.get('stddev', 0):.0f} | {delta.get('tokens', '—')} |")
# Notes section
if benchmark.get("notes"):
lines.extend([
"",
"## Notes",
""
])
for note in benchmark["notes"]:
lines.append(f"- {note}")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(
description="Aggregate benchmark run results into summary statistics"
)
parser.add_argument(
"benchmark_dir",
type=Path,
help="Path to the benchmark directory"
)
parser.add_argument(
"--skill-name",
default="",
help="Name of the skill being benchmarked"
)
parser.add_argument(
"--skill-path",
default="",
help="Path to the skill being benchmarked"
)
parser.add_argument(
"--output", "-o",
type=Path,
help="Output path for benchmark.json (default: <benchmark_dir>/benchmark.json)"
)
args = parser.parse_args()
if not args.benchmark_dir.exists():
print(f"Directory not found: {args.benchmark_dir}")
sys.exit(1)
# Generate benchmark
benchmark = generate_benchmark(args.benchmark_dir, args.skill_name, args.skill_path)
# Determine output paths
output_json = args.output or (args.benchmark_dir / "benchmark.json")
output_md = output_json.with_suffix(".md")
# Write benchmark.json
with open(output_json, "w") as f:
json.dump(benchmark, f, indent=2)
print(f"Generated: {output_json}")
# Write benchmark.md
markdown = generate_markdown(benchmark)
with open(output_md, "w") as f:
f.write(markdown)
print(f"Generated: {output_md}")
# Print summary
run_summary = benchmark["run_summary"]
configs = [k for k in run_summary if k != "delta"]
delta = run_summary.get("delta", {})
print(f"\nSummary:")
for config in configs:
pr = run_summary[config]["pass_rate"]["mean"]
label = config.replace("_", " ").title()
print(f" {label}: {pr*100:.1f}% pass rate")
print(f" Delta: {delta.get('pass_rate', '—')}")
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/generate_report.py
Executable
+326
View File
@@ -0,0 +1,326 @@
#!/usr/bin/env python3
"""Generate an HTML report from run_loop.py output.
Takes the JSON output from run_loop.py and generates a visual HTML report
showing each description attempt with check/x for each test case.
Distinguishes between train and test queries.
"""
import argparse
import html
import json
import sys
from pathlib import Path
def generate_html(data: dict, auto_refresh: bool = False, skill_name: str = "") -> str:
"""Generate HTML report from loop output data. If auto_refresh is True, adds a meta refresh tag."""
history = data.get("history", [])
holdout = data.get("holdout", 0)
title_prefix = html.escape(skill_name + " \u2014 ") if skill_name else ""
# Get all unique queries from train and test sets, with should_trigger info
train_queries: list[dict] = []
test_queries: list[dict] = []
if history:
for r in history[0].get("train_results", history[0].get("results", [])):
train_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)})
if history[0].get("test_results"):
for r in history[0].get("test_results", []):
test_queries.append({"query": r["query"], "should_trigger": r.get("should_trigger", True)})
refresh_tag = ' <meta http-equiv="refresh" content="5">\n' if auto_refresh else ""
html_parts = ["""<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
""" + refresh_tag + """ <title>""" + title_prefix + """Skill Description Optimization</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Poppins:wght@500;600&family=Lora:wght@400;500&display=swap" rel="stylesheet">
<style>
body {
font-family: 'Lora', Georgia, serif;
max-width: 100%;
margin: 0 auto;
padding: 20px;
background: #faf9f5;
color: #141413;
}
h1 { font-family: 'Poppins', sans-serif; color: #141413; }
.explainer {
background: white;
padding: 15px;
border-radius: 6px;
margin-bottom: 20px;
border: 1px solid #e8e6dc;
color: #b0aea5;
font-size: 0.875rem;
line-height: 1.6;
}
.summary {
background: white;
padding: 15px;
border-radius: 6px;
margin-bottom: 20px;
border: 1px solid #e8e6dc;
}
.summary p { margin: 5px 0; }
.best { color: #788c5d; font-weight: bold; }
.table-container {
overflow-x: auto;
width: 100%;
}
table {
border-collapse: collapse;
background: white;
border: 1px solid #e8e6dc;
border-radius: 6px;
font-size: 12px;
min-width: 100%;
}
th, td {
padding: 8px;
text-align: left;
border: 1px solid #e8e6dc;
white-space: normal;
word-wrap: break-word;
}
th {
font-family: 'Poppins', sans-serif;
background: #141413;
color: #faf9f5;
font-weight: 500;
}
th.test-col {
background: #6a9bcc;
}
th.query-col { min-width: 200px; }
td.description {
font-family: monospace;
font-size: 11px;
word-wrap: break-word;
max-width: 400px;
}
td.result {
text-align: center;
font-size: 16px;
min-width: 40px;
}
td.test-result {
background: #f0f6fc;
}
.pass { color: #788c5d; }
.fail { color: #c44; }
.rate {
font-size: 9px;
color: #b0aea5;
display: block;
}
tr:hover { background: #faf9f5; }
.score {
display: inline-block;
padding: 2px 6px;
border-radius: 4px;
font-weight: bold;
font-size: 11px;
}
.score-good { background: #eef2e8; color: #788c5d; }
.score-ok { background: #fef3c7; color: #d97706; }
.score-bad { background: #fceaea; color: #c44; }
.train-label { color: #b0aea5; font-size: 10px; }
.test-label { color: #6a9bcc; font-size: 10px; font-weight: bold; }
.best-row { background: #f5f8f2; }
th.positive-col { border-bottom: 3px solid #788c5d; }
th.negative-col { border-bottom: 3px solid #c44; }
th.test-col.positive-col { border-bottom: 3px solid #788c5d; }
th.test-col.negative-col { border-bottom: 3px solid #c44; }
.legend { font-family: 'Poppins', sans-serif; display: flex; gap: 20px; margin-bottom: 10px; font-size: 13px; align-items: center; }
.legend-item { display: flex; align-items: center; gap: 6px; }
.legend-swatch { width: 16px; height: 16px; border-radius: 3px; display: inline-block; }
.swatch-positive { background: #141413; border-bottom: 3px solid #788c5d; }
.swatch-negative { background: #141413; border-bottom: 3px solid #c44; }
.swatch-test { background: #6a9bcc; }
.swatch-train { background: #141413; }
</style>
</head>
<body>
<h1>""" + title_prefix + """Skill Description Optimization</h1>
<div class="explainer">
<strong>Optimizing your skill's description.</strong> This page updates automatically as Claude tests different versions of your skill's description. Each row is an iteration — a new description attempt. The columns show test queries: green checkmarks mean the skill triggered correctly (or correctly didn't trigger), red crosses mean it got it wrong. The "Train" score shows performance on queries used to improve the description; the "Test" score shows performance on held-out queries the optimizer hasn't seen. When it's done, Claude will apply the best-performing description to your skill.
</div>
"""]
# Summary section
best_test_score = data.get('best_test_score')
best_train_score = data.get('best_train_score')
html_parts.append(f"""
<div class="summary">
<p><strong>Original:</strong> {html.escape(data.get('original_description', 'N/A'))}</p>
<p class="best"><strong>Best:</strong> {html.escape(data.get('best_description', 'N/A'))}</p>
<p><strong>Best Score:</strong> {data.get('best_score', 'N/A')} {'(test)' if best_test_score else '(train)'}</p>
<p><strong>Iterations:</strong> {data.get('iterations_run', 0)} | <strong>Train:</strong> {data.get('train_size', '?')} | <strong>Test:</strong> {data.get('test_size', '?')}</p>
</div>
""")
# Legend
html_parts.append("""
<div class="legend">
<span style="font-weight:600">Query columns:</span>
<span class="legend-item"><span class="legend-swatch swatch-positive"></span> Should trigger</span>
<span class="legend-item"><span class="legend-swatch swatch-negative"></span> Should NOT trigger</span>
<span class="legend-item"><span class="legend-swatch swatch-train"></span> Train</span>
<span class="legend-item"><span class="legend-swatch swatch-test"></span> Test</span>
</div>
""")
# Table header
html_parts.append("""
<div class="table-container">
<table>
<thead>
<tr>
<th>Iter</th>
<th>Train</th>
<th>Test</th>
<th class="query-col">Description</th>
""")
# Add column headers for train queries
for qinfo in train_queries:
polarity = "positive-col" if qinfo["should_trigger"] else "negative-col"
html_parts.append(f' <th class="{polarity}">{html.escape(qinfo["query"])}</th>\n')
# Add column headers for test queries (different color)
for qinfo in test_queries:
polarity = "positive-col" if qinfo["should_trigger"] else "negative-col"
html_parts.append(f' <th class="test-col {polarity}">{html.escape(qinfo["query"])}</th>\n')
html_parts.append(""" </tr>
</thead>
<tbody>
""")
# Find best iteration for highlighting
if test_queries:
best_iter = max(history, key=lambda h: h.get("test_passed") or 0).get("iteration")
else:
best_iter = max(history, key=lambda h: h.get("train_passed", h.get("passed", 0))).get("iteration")
# Add rows for each iteration
for h in history:
iteration = h.get("iteration", "?")
train_passed = h.get("train_passed", h.get("passed", 0))
train_total = h.get("train_total", h.get("total", 0))
test_passed = h.get("test_passed")
test_total = h.get("test_total")
description = h.get("description", "")
train_results = h.get("train_results", h.get("results", []))
test_results = h.get("test_results", [])
# Create lookups for results by query
train_by_query = {r["query"]: r for r in train_results}
test_by_query = {r["query"]: r for r in test_results} if test_results else {}
# Compute aggregate correct/total runs across all retries
def aggregate_runs(results: list[dict]) -> tuple[int, int]:
correct = 0
total = 0
for r in results:
runs = r.get("runs", 0)
triggers = r.get("triggers", 0)
total += runs
if r.get("should_trigger", True):
correct += triggers
else:
correct += runs - triggers
return correct, total
train_correct, train_runs = aggregate_runs(train_results)
test_correct, test_runs = aggregate_runs(test_results)
# Determine score classes
def score_class(correct: int, total: int) -> str:
if total > 0:
ratio = correct / total
if ratio >= 0.8:
return "score-good"
elif ratio >= 0.5:
return "score-ok"
return "score-bad"
train_class = score_class(train_correct, train_runs)
test_class = score_class(test_correct, test_runs)
row_class = "best-row" if iteration == best_iter else ""
html_parts.append(f""" <tr class="{row_class}">
<td>{iteration}</td>
<td><span class="score {train_class}">{train_correct}/{train_runs}</span></td>
<td><span class="score {test_class}">{test_correct}/{test_runs}</span></td>
<td class="description">{html.escape(description)}</td>
""")
# Add result for each train query
for qinfo in train_queries:
r = train_by_query.get(qinfo["query"], {})
did_pass = r.get("pass", False)
triggers = r.get("triggers", 0)
runs = r.get("runs", 0)
icon = "✓" if did_pass else "✗"
css_class = "pass" if did_pass else "fail"
html_parts.append(f' <td class="result {css_class}">{icon}<span class="rate">{triggers}/{runs}</span></td>\n')
# Add result for each test query (with different background)
for qinfo in test_queries:
r = test_by_query.get(qinfo["query"], {})
did_pass = r.get("pass", False)
triggers = r.get("triggers", 0)
runs = r.get("runs", 0)
icon = "✓" if did_pass else "✗"
css_class = "pass" if did_pass else "fail"
html_parts.append(f' <td class="result test-result {css_class}">{icon}<span class="rate">{triggers}/{runs}</span></td>\n')
html_parts.append(" </tr>\n")
html_parts.append(""" </tbody>
</table>
</div>
""")
html_parts.append("""
</body>
</html>
""")
return "".join(html_parts)
def main():
parser = argparse.ArgumentParser(description="Generate HTML report from run_loop output")
parser.add_argument("input", help="Path to JSON output from run_loop.py (or - for stdin)")
parser.add_argument("-o", "--output", default=None, help="Output HTML file (default: stdout)")
parser.add_argument("--skill-name", default="", help="Skill name to include in the report title")
args = parser.parse_args()
if args.input == "-":
data = json.load(sys.stdin)
else:
data = json.loads(Path(args.input).read_text())
html_output = generate_html(data, skill_name=args.skill_name)
if args.output:
Path(args.output).write_text(html_output)
print(f"Report written to {args.output}", file=sys.stderr)
else:
print(html_output)
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/improve_description.py
Executable
+247
View File
@@ -0,0 +1,247 @@
#!/usr/bin/env python3
"""Improve a skill description based on eval results.
Takes eval results (from run_eval.py) and generates an improved description
by calling `claude -p` as a subprocess (same auth pattern as run_eval.py —
uses the session's Claude Code auth, no separate ANTHROPIC_API_KEY needed).
"""
import argparse
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from scripts.utils import parse_skill_md
def _call_claude(prompt: str, model: str | None, timeout: int = 300) -> str:
"""Run `claude -p` with the prompt on stdin and return the text response.
Prompt goes over stdin (not argv) because it embeds the full SKILL.md
body and can easily exceed comfortable argv length.
"""
cmd = ["claude", "-p", "--output-format", "text"]
if model:
cmd.extend(["--model", model])
# Remove CLAUDECODE env var to allow nesting claude -p inside a
# Claude Code session. The guard is for interactive terminal conflicts;
# programmatic subprocess usage is safe. Same pattern as run_eval.py.
env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
result = subprocess.run(
cmd,
input=prompt,
capture_output=True,
text=True,
env=env,
timeout=timeout,
)
if result.returncode != 0:
raise RuntimeError(
f"claude -p exited {result.returncode}\nstderr: {result.stderr}"
)
return result.stdout
def improve_description(
skill_name: str,
skill_content: str,
current_description: str,
eval_results: dict,
history: list[dict],
model: str,
test_results: dict | None = None,
log_dir: Path | None = None,
iteration: int | None = None,
) -> str:
"""Call Claude to improve the description based on eval results."""
failed_triggers = [
r for r in eval_results["results"]
if r["should_trigger"] and not r["pass"]
]
false_triggers = [
r for r in eval_results["results"]
if not r["should_trigger"] and not r["pass"]
]
# Build scores summary
train_score = f"{eval_results['summary']['passed']}/{eval_results['summary']['total']}"
if test_results:
test_score = f"{test_results['summary']['passed']}/{test_results['summary']['total']}"
scores_summary = f"Train: {train_score}, Test: {test_score}"
else:
scores_summary = f"Train: {train_score}"
prompt = f"""You are optimizing a skill description for a Claude Code skill called "{skill_name}". A "skill" is sort of like a prompt, but with progressive disclosure -- there's a title and description that Claude sees when deciding whether to use the skill, and then if it does use the skill, it reads the .md file which has lots more details and potentially links to other resources in the skill folder like helper files and scripts and additional documentation or examples.
The description appears in Claude's "available_skills" list. When a user sends a query, Claude decides whether to invoke the skill based solely on the title and on this description. Your goal is to write a description that triggers for relevant queries, and doesn't trigger for irrelevant ones.
Here's the current description:
<current_description>
"{current_description}"
</current_description>
Current scores ({scores_summary}):
<scores_summary>
"""
if failed_triggers:
prompt += "FAILED TO TRIGGER (should have triggered but didn't):\n"
for r in failed_triggers:
prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n'
prompt += "\n"
if false_triggers:
prompt += "FALSE TRIGGERS (triggered but shouldn't have):\n"
for r in false_triggers:
prompt += f' - "{r["query"]}" (triggered {r["triggers"]}/{r["runs"]} times)\n'
prompt += "\n"
if history:
prompt += "PREVIOUS ATTEMPTS (do NOT repeat these — try something structurally different):\n\n"
for h in history:
train_s = f"{h.get('train_passed', h.get('passed', 0))}/{h.get('train_total', h.get('total', 0))}"
test_s = f"{h.get('test_passed', '?')}/{h.get('test_total', '?')}" if h.get('test_passed') is not None else None
score_str = f"train={train_s}" + (f", test={test_s}" if test_s else "")
prompt += f'<attempt {score_str}>\n'
prompt += f'Description: "{h["description"]}"\n'
if "results" in h:
prompt += "Train results:\n"
for r in h["results"]:
status = "PASS" if r["pass"] else "FAIL"
prompt += f' [{status}] "{r["query"][:80]}" (triggered {r["triggers"]}/{r["runs"]})\n'
if h.get("note"):
prompt += f'Note: {h["note"]}\n'
prompt += "</attempt>\n\n"
prompt += f"""</scores_summary>
Skill content (for context on what the skill does):
<skill_content>
{skill_content}
</skill_content>
Based on the failures, write a new and improved description that is more likely to trigger correctly. When I say "based on the failures", it's a bit of a tricky line to walk because we don't want to overfit to the specific cases you're seeing. So what I DON'T want you to do is produce an ever-expanding list of specific queries that this skill should or shouldn't trigger for. Instead, try to generalize from the failures to broader categories of user intent and situations where this skill would be useful or not useful. The reason for this is twofold:
1. Avoid overfitting
2. The list might get loooong and it's injected into ALL queries and there might be a lot of skills, so we don't want to blow too much space on any given description.
Concretely, your description should not be more than about 100-200 words, even if that comes at the cost of accuracy. There is a hard limit of 1024 characters — descriptions over that will be truncated, so stay comfortably under it.
Here are some tips that we've found to work well in writing these descriptions:
- The skill should be phrased in the imperative -- "Use this skill for" rather than "this skill does"
- The skill description should focus on the user's intent, what they are trying to achieve, vs. the implementation details of how the skill works.
- The description competes with other skills for Claude's attention — make it distinctive and immediately recognizable.
- If you're getting lots of failures after repeated attempts, change things up. Try different sentence structures or wordings.
I'd encourage you to be creative and mix up the style in different iterations since you'll have multiple opportunities to try different approaches and we'll just grab the highest-scoring one at the end.
Please respond with only the new description text in <new_description> tags, nothing else."""
text = _call_claude(prompt, model)
match = re.search(r"<new_description>(.*?)</new_description>", text, re.DOTALL)
description = match.group(1).strip().strip('"') if match else text.strip().strip('"')
transcript: dict = {
"iteration": iteration,
"prompt": prompt,
"response": text,
"parsed_description": description,
"char_count": len(description),
"over_limit": len(description) > 1024,
}
# Safety net: the prompt already states the 1024-char hard limit, but if
# the model blew past it anyway, make one fresh single-turn call that
# quotes the too-long version and asks for a shorter rewrite. (The old
# SDK path did this as a true multi-turn; `claude -p` is one-shot, so we
# inline the prior output into the new prompt instead.)
if len(description) > 1024:
shorten_prompt = (
f"{prompt}\n\n"
f"---\n\n"
f"A previous attempt produced this description, which at "
f"{len(description)} characters is over the 1024-character hard limit:\n\n"
f'"{description}"\n\n'
f"Rewrite it to be under 1024 characters while keeping the most "
f"important trigger words and intent coverage. Respond with only "
f"the new description in <new_description> tags."
)
shorten_text = _call_claude(shorten_prompt, model)
match = re.search(r"<new_description>(.*?)</new_description>", shorten_text, re.DOTALL)
shortened = match.group(1).strip().strip('"') if match else shorten_text.strip().strip('"')
transcript["rewrite_prompt"] = shorten_prompt
transcript["rewrite_response"] = shorten_text
transcript["rewrite_description"] = shortened
transcript["rewrite_char_count"] = len(shortened)
description = shortened
transcript["final_description"] = description
if log_dir:
log_dir.mkdir(parents=True, exist_ok=True)
log_file = log_dir / f"improve_iter_{iteration or 'unknown'}.json"
log_file.write_text(json.dumps(transcript, indent=2))
return description
def main():
parser = argparse.ArgumentParser(description="Improve a skill description based on eval results")
parser.add_argument("--eval-results", required=True, help="Path to eval results JSON (from run_eval.py)")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--history", default=None, help="Path to history JSON (previous attempts)")
parser.add_argument("--model", required=True, help="Model for improvement")
parser.add_argument("--verbose", action="store_true", help="Print thinking to stderr")
args = parser.parse_args()
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
eval_results = json.loads(Path(args.eval_results).read_text())
history = []
if args.history:
history = json.loads(Path(args.history).read_text())
name, _, content = parse_skill_md(skill_path)
current_description = eval_results["description"]
if args.verbose:
print(f"Current: {current_description}", file=sys.stderr)
print(f"Score: {eval_results['summary']['passed']}/{eval_results['summary']['total']}", file=sys.stderr)
new_description = improve_description(
skill_name=name,
skill_content=content,
current_description=current_description,
eval_results=eval_results,
history=history,
model=args.model,
)
if args.verbose:
print(f"Improved: {new_description}", file=sys.stderr)
# Output as JSON with both the new description and updated history
output = {
"description": new_description,
"history": history + [{
"description": current_description,
"passed": eval_results["summary"]["passed"],
"failed": eval_results["summary"]["failed"],
"total": eval_results["summary"]["total"],
"results": eval_results["results"],
}],
}
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/package_skill.py
Executable
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""
Skill Packager - Creates a distributable .skill file of a skill folder
Usage:
python utils/package_skill.py <path/to/skill-folder> [output-directory]
Example:
python utils/package_skill.py skills/public/my-skill
python utils/package_skill.py skills/public/my-skill ./dist
"""
import fnmatch
import sys
import zipfile
from pathlib import Path
from scripts.quick_validate import validate_skill
# Patterns to exclude when packaging skills.
EXCLUDE_DIRS = {"__pycache__", "node_modules"}
EXCLUDE_GLOBS = {"*.pyc"}
EXCLUDE_FILES = {".DS_Store"}
# Directories excluded only at the skill root (not when nested deeper).
ROOT_EXCLUDE_DIRS = {"evals"}
def should_exclude(rel_path: Path) -> bool:
"""Check if a path should be excluded from packaging."""
parts = rel_path.parts
if any(part in EXCLUDE_DIRS for part in parts):
return True
# rel_path is relative to skill_path.parent, so parts[0] is the skill
# folder name and parts[1] (if present) is the first subdir.
if len(parts) > 1 and parts[1] in ROOT_EXCLUDE_DIRS:
return True
name = rel_path.name
if name in EXCLUDE_FILES:
return True
return any(fnmatch.fnmatch(name, pat) for pat in EXCLUDE_GLOBS)
def package_skill(skill_path, output_dir=None):
"""
Package a skill folder into a .skill file.
Args:
skill_path: Path to the skill folder
output_dir: Optional output directory for the .skill file (defaults to current directory)
Returns:
Path to the created .skill file, or None if error
"""
skill_path = Path(skill_path).resolve()
# Validate skill folder exists
if not skill_path.exists():
print(f"❌ Error: Skill folder not found: {skill_path}")
return None
if not skill_path.is_dir():
print(f"❌ Error: Path is not a directory: {skill_path}")
return None
# Validate SKILL.md exists
skill_md = skill_path / "SKILL.md"
if not skill_md.exists():
print(f"❌ Error: SKILL.md not found in {skill_path}")
return None
# Run validation before packaging
print("🔍 Validating skill...")
valid, message = validate_skill(skill_path)
if not valid:
print(f"❌ Validation failed: {message}")
print(" Please fix the validation errors before packaging.")
return None
print(f"✅ {message}\n")
# Determine output location
skill_name = skill_path.name
if output_dir:
output_path = Path(output_dir).resolve()
output_path.mkdir(parents=True, exist_ok=True)
else:
output_path = Path.cwd()
skill_filename = output_path / f"{skill_name}.skill"
# Create the .skill file (zip format)
try:
with zipfile.ZipFile(skill_filename, 'w', zipfile.ZIP_DEFLATED) as zipf:
# Walk through the skill directory, excluding build artifacts
for file_path in skill_path.rglob('*'):
if not file_path.is_file():
continue
arcname = file_path.relative_to(skill_path.parent)
if should_exclude(arcname):
print(f" Skipped: {arcname}")
continue
zipf.write(file_path, arcname)
print(f" Added: {arcname}")
print(f"\n✅ Successfully packaged skill to: {skill_filename}")
return skill_filename
except Exception as e:
print(f"❌ Error creating .skill file: {e}")
return None
def main():
if len(sys.argv) < 2:
print("Usage: python utils/package_skill.py <path/to/skill-folder> [output-directory]")
print("\nExample:")
print(" python utils/package_skill.py skills/public/my-skill")
print(" python utils/package_skill.py skills/public/my-skill ./dist")
sys.exit(1)
skill_path = sys.argv[1]
output_dir = sys.argv[2] if len(sys.argv) > 2 else None
print(f"📦 Packaging skill: {skill_path}")
if output_dir:
print(f" Output directory: {output_dir}")
print()
result = package_skill(skill_path, output_dir)
if result:
sys.exit(0)
else:
sys.exit(1)
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/quick_validate.py
Executable
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env python3
"""
Quick validation script for skills - minimal version
"""
import sys
import os
import re
import yaml
from pathlib import Path
def validate_skill(skill_path):
"""Basic validation of a skill"""
skill_path = Path(skill_path)
# Check SKILL.md exists
skill_md = skill_path / 'SKILL.md'
if not skill_md.exists():
return False, "SKILL.md not found"
# Read and validate frontmatter
content = skill_md.read_text()
if not content.startswith('---'):
return False, "No YAML frontmatter found"
# Extract frontmatter
match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
if not match:
return False, "Invalid frontmatter format"
frontmatter_text = match.group(1)
# Parse YAML frontmatter
try:
frontmatter = yaml.safe_load(frontmatter_text)
if not isinstance(frontmatter, dict):
return False, "Frontmatter must be a YAML dictionary"
except yaml.YAMLError as e:
return False, f"Invalid YAML in frontmatter: {e}"
# Define allowed properties
ALLOWED_PROPERTIES = {'name', 'description', 'license', 'allowed-tools', 'metadata', 'compatibility'}
# Check for unexpected properties (excluding nested keys under metadata)
unexpected_keys = set(frontmatter.keys()) - ALLOWED_PROPERTIES
if unexpected_keys:
return False, (
f"Unexpected key(s) in SKILL.md frontmatter: {', '.join(sorted(unexpected_keys))}. "
f"Allowed properties are: {', '.join(sorted(ALLOWED_PROPERTIES))}"
)
# Check required fields
if 'name' not in frontmatter:
return False, "Missing 'name' in frontmatter"
if 'description' not in frontmatter:
return False, "Missing 'description' in frontmatter"
# Extract name for validation
name = frontmatter.get('name', '')
if not isinstance(name, str):
return False, f"Name must be a string, got {type(name).__name__}"
name = name.strip()
if name:
# Check naming convention (kebab-case: lowercase with hyphens)
if not re.match(r'^[a-z0-9-]+$', name):
return False, f"Name '{name}' should be kebab-case (lowercase letters, digits, and hyphens only)"
if name.startswith('-') or name.endswith('-') or '--' in name:
return False, f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens"
# Check name length (max 64 characters per spec)
if len(name) > 64:
return False, f"Name is too long ({len(name)} characters). Maximum is 64 characters."
# Extract and validate description
description = frontmatter.get('description', '')
if not isinstance(description, str):
return False, f"Description must be a string, got {type(description).__name__}"
description = description.strip()
if description:
# Check for angle brackets
if '<' in description or '>' in description:
return False, "Description cannot contain angle brackets (< or >)"
# Check description length (max 1024 characters per spec)
if len(description) > 1024:
return False, f"Description is too long ({len(description)} characters). Maximum is 1024 characters."
# Validate compatibility field if present (optional)
compatibility = frontmatter.get('compatibility', '')
if compatibility:
if not isinstance(compatibility, str):
return False, f"Compatibility must be a string, got {type(compatibility).__name__}"
if len(compatibility) > 500:
return False, f"Compatibility is too long ({len(compatibility)} characters). Maximum is 500 characters."
return True, "Skill is valid!"
if __name__ == "__main__":
if len(sys.argv) != 2:
print("Usage: python quick_validate.py <skill_directory>")
sys.exit(1)
valid, message = validate_skill(sys.argv[1])
print(message)
sys.exit(0 if valid else 1)
.claude/skills/skill-creator/scripts/run_eval.py
Executable
+310
View File
@@ -0,0 +1,310 @@
#!/usr/bin/env python3
"""Run trigger evaluation for a skill description.
Tests whether a skill's description causes Claude to trigger (read the skill)
for a set of queries. Outputs results as JSON.
"""
import argparse
import json
import os
import select
import subprocess
import sys
import time
import uuid
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path
from scripts.utils import parse_skill_md
def find_project_root() -> Path:
"""Find the project root by walking up from cwd looking for .claude/.
Mimics how Claude Code discovers its project root, so the command file
we create ends up where claude -p will look for it.
"""
current = Path.cwd()
for parent in [current, *current.parents]:
if (parent / ".claude").is_dir():
return parent
return current
def run_single_query(
query: str,
skill_name: str,
skill_description: str,
timeout: int,
project_root: str,
model: str | None = None,
) -> bool:
"""Run a single query and return whether the skill was triggered.
Creates a command file in .claude/commands/ so it appears in Claude's
available_skills list, then runs `claude -p` with the raw query.
Uses --include-partial-messages to detect triggering early from
stream events (content_block_start) rather than waiting for the
full assistant message, which only arrives after tool execution.
"""
unique_id = uuid.uuid4().hex[:8]
clean_name = f"{skill_name}-skill-{unique_id}"
project_commands_dir = Path(project_root) / ".claude" / "commands"
command_file = project_commands_dir / f"{clean_name}.md"
try:
project_commands_dir.mkdir(parents=True, exist_ok=True)
# Use YAML block scalar to avoid breaking on quotes in description
indented_desc = "\n ".join(skill_description.split("\n"))
command_content = (
f"---\n"
f"description: |\n"
f" {indented_desc}\n"
f"---\n\n"
f"# {skill_name}\n\n"
f"This skill handles: {skill_description}\n"
)
command_file.write_text(command_content)
cmd = [
"claude",
"-p", query,
"--output-format", "stream-json",
"--verbose",
"--include-partial-messages",
]
if model:
cmd.extend(["--model", model])
# Remove CLAUDECODE env var to allow nesting claude -p inside a
# Claude Code session. The guard is for interactive terminal conflicts;
# programmatic subprocess usage is safe.
env = {k: v for k, v in os.environ.items() if k != "CLAUDECODE"}
process = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
cwd=project_root,
env=env,
)
triggered = False
start_time = time.time()
buffer = ""
# Track state for stream event detection
pending_tool_name = None
accumulated_json = ""
try:
while time.time() - start_time < timeout:
if process.poll() is not None:
remaining = process.stdout.read()
if remaining:
buffer += remaining.decode("utf-8", errors="replace")
break
ready, _, _ = select.select([process.stdout], [], [], 1.0)
if not ready:
continue
chunk = os.read(process.stdout.fileno(), 8192)
if not chunk:
break
buffer += chunk.decode("utf-8", errors="replace")
while "\n" in buffer:
line, buffer = buffer.split("\n", 1)
line = line.strip()
if not line:
continue
try:
event = json.loads(line)
except json.JSONDecodeError:
continue
# Early detection via stream events
if event.get("type") == "stream_event":
se = event.get("event", {})
se_type = se.get("type", "")
if se_type == "content_block_start":
cb = se.get("content_block", {})
if cb.get("type") == "tool_use":
tool_name = cb.get("name", "")
if tool_name in ("Skill", "Read"):
pending_tool_name = tool_name
accumulated_json = ""
else:
return False
elif se_type == "content_block_delta" and pending_tool_name:
delta = se.get("delta", {})
if delta.get("type") == "input_json_delta":
accumulated_json += delta.get("partial_json", "")
if clean_name in accumulated_json:
return True
elif se_type in ("content_block_stop", "message_stop"):
if pending_tool_name:
return clean_name in accumulated_json
if se_type == "message_stop":
return False
# Fallback: full assistant message
elif event.get("type") == "assistant":
message = event.get("message", {})
for content_item in message.get("content", []):
if content_item.get("type") != "tool_use":
continue
tool_name = content_item.get("name", "")
tool_input = content_item.get("input", {})
if tool_name == "Skill" and clean_name in tool_input.get("skill", ""):
triggered = True
elif tool_name == "Read" and clean_name in tool_input.get("file_path", ""):
triggered = True
return triggered
elif event.get("type") == "result":
return triggered
finally:
# Clean up process on any exit path (return, exception, timeout)
if process.poll() is None:
process.kill()
process.wait()
return triggered
finally:
if command_file.exists():
command_file.unlink()
def run_eval(
eval_set: list[dict],
skill_name: str,
description: str,
num_workers: int,
timeout: int,
project_root: Path,
runs_per_query: int = 1,
trigger_threshold: float = 0.5,
model: str | None = None,
) -> dict:
"""Run the full eval set and return results."""
results = []
with ProcessPoolExecutor(max_workers=num_workers) as executor:
future_to_info = {}
for item in eval_set:
for run_idx in range(runs_per_query):
future = executor.submit(
run_single_query,
item["query"],
skill_name,
description,
timeout,
str(project_root),
model,
)
future_to_info[future] = (item, run_idx)
query_triggers: dict[str, list[bool]] = {}
query_items: dict[str, dict] = {}
for future in as_completed(future_to_info):
item, _ = future_to_info[future]
query = item["query"]
query_items[query] = item
if query not in query_triggers:
query_triggers[query] = []
try:
query_triggers[query].append(future.result())
except Exception as e:
print(f"Warning: query failed: {e}", file=sys.stderr)
query_triggers[query].append(False)
for query, triggers in query_triggers.items():
item = query_items[query]
trigger_rate = sum(triggers) / len(triggers)
should_trigger = item["should_trigger"]
if should_trigger:
did_pass = trigger_rate >= trigger_threshold
else:
did_pass = trigger_rate < trigger_threshold
results.append({
"query": query,
"should_trigger": should_trigger,
"trigger_rate": trigger_rate,
"triggers": sum(triggers),
"runs": len(triggers),
"pass": did_pass,
})
passed = sum(1 for r in results if r["pass"])
total = len(results)
return {
"skill_name": skill_name,
"description": description,
"results": results,
"summary": {
"total": total,
"passed": passed,
"failed": total - passed,
},
}
def main():
parser = argparse.ArgumentParser(description="Run trigger evaluation for a skill description")
parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--description", default=None, help="Override description to test")
parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")
parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")
parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")
parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")
parser.add_argument("--model", default=None, help="Model to use for claude -p (default: user's configured model)")
parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
args = parser.parse_args()
eval_set = json.loads(Path(args.eval_set).read_text())
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
name, original_description, content = parse_skill_md(skill_path)
description = args.description or original_description
project_root = find_project_root()
if args.verbose:
print(f"Evaluating: {description}", file=sys.stderr)
output = run_eval(
eval_set=eval_set,
skill_name=name,
description=description,
num_workers=args.num_workers,
timeout=args.timeout,
project_root=project_root,
runs_per_query=args.runs_per_query,
trigger_threshold=args.trigger_threshold,
model=args.model,
)
if args.verbose:
summary = output["summary"]
print(f"Results: {summary['passed']}/{summary['total']} passed", file=sys.stderr)
for r in output["results"]:
status = "PASS" if r["pass"] else "FAIL"
rate_str = f"{r['triggers']}/{r['runs']}"
print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:70]}", file=sys.stderr)
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/run_loop.py
Executable
+328
View File
@@ -0,0 +1,328 @@
#!/usr/bin/env python3
"""Run the eval + improve loop until all pass or max iterations reached.
Combines run_eval.py and improve_description.py in a loop, tracking history
and returning the best description found. Supports train/test split to prevent
overfitting.
"""
import argparse
import json
import random
import sys
import tempfile
import time
import webbrowser
from pathlib import Path
from scripts.generate_report import generate_html
from scripts.improve_description import improve_description
from scripts.run_eval import find_project_root, run_eval
from scripts.utils import parse_skill_md
def split_eval_set(eval_set: list[dict], holdout: float, seed: int = 42) -> tuple[list[dict], list[dict]]:
"""Split eval set into train and test sets, stratified by should_trigger."""
random.seed(seed)
# Separate by should_trigger
trigger = [e for e in eval_set if e["should_trigger"]]
no_trigger = [e for e in eval_set if not e["should_trigger"]]
# Shuffle each group
random.shuffle(trigger)
random.shuffle(no_trigger)
# Calculate split points
n_trigger_test = max(1, int(len(trigger) * holdout))
n_no_trigger_test = max(1, int(len(no_trigger) * holdout))
# Split
test_set = trigger[:n_trigger_test] + no_trigger[:n_no_trigger_test]
train_set = trigger[n_trigger_test:] + no_trigger[n_no_trigger_test:]
return train_set, test_set
def run_loop(
eval_set: list[dict],
skill_path: Path,
description_override: str | None,
num_workers: int,
timeout: int,
max_iterations: int,
runs_per_query: int,
trigger_threshold: float,
holdout: float,
model: str,
verbose: bool,
live_report_path: Path | None = None,
log_dir: Path | None = None,
) -> dict:
"""Run the eval + improvement loop."""
project_root = find_project_root()
name, original_description, content = parse_skill_md(skill_path)
current_description = description_override or original_description
# Split into train/test if holdout > 0
if holdout > 0:
train_set, test_set = split_eval_set(eval_set, holdout)
if verbose:
print(f"Split: {len(train_set)} train, {len(test_set)} test (holdout={holdout})", file=sys.stderr)
else:
train_set = eval_set
test_set = []
history = []
exit_reason = "unknown"
for iteration in range(1, max_iterations + 1):
if verbose:
print(f"\n{'='*60}", file=sys.stderr)
print(f"Iteration {iteration}/{max_iterations}", file=sys.stderr)
print(f"Description: {current_description}", file=sys.stderr)
print(f"{'='*60}", file=sys.stderr)
# Evaluate train + test together in one batch for parallelism
all_queries = train_set + test_set
t0 = time.time()
all_results = run_eval(
eval_set=all_queries,
skill_name=name,
description=current_description,
num_workers=num_workers,
timeout=timeout,
project_root=project_root,
runs_per_query=runs_per_query,
trigger_threshold=trigger_threshold,
model=model,
)
eval_elapsed = time.time() - t0
# Split results back into train/test by matching queries
train_queries_set = {q["query"] for q in train_set}
train_result_list = [r for r in all_results["results"] if r["query"] in train_queries_set]
test_result_list = [r for r in all_results["results"] if r["query"] not in train_queries_set]
train_passed = sum(1 for r in train_result_list if r["pass"])
train_total = len(train_result_list)
train_summary = {"passed": train_passed, "failed": train_total - train_passed, "total": train_total}
train_results = {"results": train_result_list, "summary": train_summary}
if test_set:
test_passed = sum(1 for r in test_result_list if r["pass"])
test_total = len(test_result_list)
test_summary = {"passed": test_passed, "failed": test_total - test_passed, "total": test_total}
test_results = {"results": test_result_list, "summary": test_summary}
else:
test_results = None
test_summary = None
history.append({
"iteration": iteration,
"description": current_description,
"train_passed": train_summary["passed"],
"train_failed": train_summary["failed"],
"train_total": train_summary["total"],
"train_results": train_results["results"],
"test_passed": test_summary["passed"] if test_summary else None,
"test_failed": test_summary["failed"] if test_summary else None,
"test_total": test_summary["total"] if test_summary else None,
"test_results": test_results["results"] if test_results else None,
# For backward compat with report generator
"passed": train_summary["passed"],
"failed": train_summary["failed"],
"total": train_summary["total"],
"results": train_results["results"],
})
# Write live report if path provided
if live_report_path:
partial_output = {
"original_description": original_description,
"best_description": current_description,
"best_score": "in progress",
"iterations_run": len(history),
"holdout": holdout,
"train_size": len(train_set),
"test_size": len(test_set),
"history": history,
}
live_report_path.write_text(generate_html(partial_output, auto_refresh=True, skill_name=name))
if verbose:
def print_eval_stats(label, results, elapsed):
pos = [r for r in results if r["should_trigger"]]
neg = [r for r in results if not r["should_trigger"]]
tp = sum(r["triggers"] for r in pos)
pos_runs = sum(r["runs"] for r in pos)
fn = pos_runs - tp
fp = sum(r["triggers"] for r in neg)
neg_runs = sum(r["runs"] for r in neg)
tn = neg_runs - fp
total = tp + tn + fp + fn
precision = tp / (tp + fp) if (tp + fp) > 0 else 1.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 1.0
accuracy = (tp + tn) / total if total > 0 else 0.0
print(f"{label}: {tp+tn}/{total} correct, precision={precision:.0%} recall={recall:.0%} accuracy={accuracy:.0%} ({elapsed:.1f}s)", file=sys.stderr)
for r in results:
status = "PASS" if r["pass"] else "FAIL"
rate_str = f"{r['triggers']}/{r['runs']}"
print(f" [{status}] rate={rate_str} expected={r['should_trigger']}: {r['query'][:60]}", file=sys.stderr)
print_eval_stats("Train", train_results["results"], eval_elapsed)
if test_summary:
print_eval_stats("Test ", test_results["results"], 0)
if train_summary["failed"] == 0:
exit_reason = f"all_passed (iteration {iteration})"
if verbose:
print(f"\nAll train queries passed on iteration {iteration}!", file=sys.stderr)
break
if iteration == max_iterations:
exit_reason = f"max_iterations ({max_iterations})"
if verbose:
print(f"\nMax iterations reached ({max_iterations}).", file=sys.stderr)
break
# Improve the description based on train results
if verbose:
print(f"\nImproving description...", file=sys.stderr)
t0 = time.time()
# Strip test scores from history so improvement model can't see them
blinded_history = [
{k: v for k, v in h.items() if not k.startswith("test_")}
for h in history
]
new_description = improve_description(
skill_name=name,
skill_content=content,
current_description=current_description,
eval_results=train_results,
history=blinded_history,
model=model,
log_dir=log_dir,
iteration=iteration,
)
improve_elapsed = time.time() - t0
if verbose:
print(f"Proposed ({improve_elapsed:.1f}s): {new_description}", file=sys.stderr)
current_description = new_description
# Find the best iteration by TEST score (or train if no test set)
if test_set:
best = max(history, key=lambda h: h["test_passed"] or 0)
best_score = f"{best['test_passed']}/{best['test_total']}"
else:
best = max(history, key=lambda h: h["train_passed"])
best_score = f"{best['train_passed']}/{best['train_total']}"
if verbose:
print(f"\nExit reason: {exit_reason}", file=sys.stderr)
print(f"Best score: {best_score} (iteration {best['iteration']})", file=sys.stderr)
return {
"exit_reason": exit_reason,
"original_description": original_description,
"best_description": best["description"],
"best_score": best_score,
"best_train_score": f"{best['train_passed']}/{best['train_total']}",
"best_test_score": f"{best['test_passed']}/{best['test_total']}" if test_set else None,
"final_description": current_description,
"iterations_run": len(history),
"holdout": holdout,
"train_size": len(train_set),
"test_size": len(test_set),
"history": history,
}
def main():
parser = argparse.ArgumentParser(description="Run eval + improve loop")
parser.add_argument("--eval-set", required=True, help="Path to eval set JSON file")
parser.add_argument("--skill-path", required=True, help="Path to skill directory")
parser.add_argument("--description", default=None, help="Override starting description")
parser.add_argument("--num-workers", type=int, default=10, help="Number of parallel workers")
parser.add_argument("--timeout", type=int, default=30, help="Timeout per query in seconds")
parser.add_argument("--max-iterations", type=int, default=5, help="Max improvement iterations")
parser.add_argument("--runs-per-query", type=int, default=3, help="Number of runs per query")
parser.add_argument("--trigger-threshold", type=float, default=0.5, help="Trigger rate threshold")
parser.add_argument("--holdout", type=float, default=0.4, help="Fraction of eval set to hold out for testing (0 to disable)")
parser.add_argument("--model", required=True, help="Model for improvement")
parser.add_argument("--verbose", action="store_true", help="Print progress to stderr")
parser.add_argument("--report", default="auto", help="Generate HTML report at this path (default: 'auto' for temp file, 'none' to disable)")
parser.add_argument("--results-dir", default=None, help="Save all outputs (results.json, report.html, log.txt) to a timestamped subdirectory here")
args = parser.parse_args()
eval_set = json.loads(Path(args.eval_set).read_text())
skill_path = Path(args.skill_path)
if not (skill_path / "SKILL.md").exists():
print(f"Error: No SKILL.md found at {skill_path}", file=sys.stderr)
sys.exit(1)
name, _, _ = parse_skill_md(skill_path)
# Set up live report path
if args.report != "none":
if args.report == "auto":
timestamp = time.strftime("%Y%m%d_%H%M%S")
live_report_path = Path(tempfile.gettempdir()) / f"skill_description_report_{skill_path.name}_{timestamp}.html"
else:
live_report_path = Path(args.report)
# Open the report immediately so the user can watch
live_report_path.write_text("<html><body><h1>Starting optimization loop...</h1><meta http-equiv='refresh' content='5'></body></html>")
webbrowser.open(str(live_report_path))
else:
live_report_path = None
# Determine output directory (create before run_loop so logs can be written)
if args.results_dir:
timestamp = time.strftime("%Y-%m-%d_%H%M%S")
results_dir = Path(args.results_dir) / timestamp
results_dir.mkdir(parents=True, exist_ok=True)
else:
results_dir = None
log_dir = results_dir / "logs" if results_dir else None
output = run_loop(
eval_set=eval_set,
skill_path=skill_path,
description_override=args.description,
num_workers=args.num_workers,
timeout=args.timeout,
max_iterations=args.max_iterations,
runs_per_query=args.runs_per_query,
trigger_threshold=args.trigger_threshold,
holdout=args.holdout,
model=args.model,
verbose=args.verbose,
live_report_path=live_report_path,
log_dir=log_dir,
)
# Save JSON output
json_output = json.dumps(output, indent=2)
print(json_output)
if results_dir:
(results_dir / "results.json").write_text(json_output)
# Write final HTML report (without auto-refresh)
if live_report_path:
live_report_path.write_text(generate_html(output, auto_refresh=False, skill_name=name))
print(f"\nReport: {live_report_path}", file=sys.stderr)
if results_dir and live_report_path:
(results_dir / "report.html").write_text(generate_html(output, auto_refresh=False, skill_name=name))
if results_dir:
print(f"Results saved to: {results_dir}", file=sys.stderr)
if __name__ == "__main__":
main()
.claude/skills/skill-creator/scripts/utils.py
+47
View File
@@ -0,0 +1,47 @@
"""Shared utilities for skill-creator scripts."""
from pathlib import Path
def parse_skill_md(skill_path: Path) -> tuple[str, str, str]:
"""Parse a SKILL.md file, returning (name, description, full_content)."""
content = (skill_path / "SKILL.md").read_text()
lines = content.split("\n")
if lines[0].strip() != "---":
raise ValueError("SKILL.md missing frontmatter (no opening ---)")
end_idx = None
for i, line in enumerate(lines[1:], start=1):
if line.strip() == "---":
end_idx = i
break
if end_idx is None:
raise ValueError("SKILL.md missing frontmatter (no closing ---)")
name = ""
description = ""
frontmatter_lines = lines[1:end_idx]
i = 0
while i < len(frontmatter_lines):
line = frontmatter_lines[i]
if line.startswith("name:"):
name = line[len("name:"):].strip().strip('"').strip("'")
elif line.startswith("description:"):
value = line[len("description:"):].strip()
# Handle YAML multiline indicators (>, |, >-, |-)
if value in (">", "|", ">-", "|-"):
continuation_lines: list[str] = []
i += 1
while i < len(frontmatter_lines) and (frontmatter_lines[i].startswith(" ") or frontmatter_lines[i].startswith("\t")):
continuation_lines.append(frontmatter_lines[i].strip())
i += 1
description = " ".join(continuation_lines)
continue
else:
description = value.strip('"').strip("'")
i += 1
return name, description, content
.claude/skills/skills-sync/SKILL.md
+48
View File
@@ -0,0 +1,48 @@
---
name: skills-sync
description: Sync npx global skills to Claude Code. Use after installing skills via `npx skills add -g` to make them visible in `/skills` command.
---
# Skills Sync
Synchronizes npx globally installed skills (`~/.agents/skills/`) to Claude Code (`~/.claude/skills/`).
## When to Use
- After running `npx skills add <repo> -g` to install a global skill
- When `/skills` doesn't show skills you installed via npx
- To update the symlinked skills list
## How It Works
1. Scans `~/.agents/skills/` for all installed skills
2. Creates symlinks in `~/.claude/skills/` pointing to each skill
3. Makes npx skills visible in Claude Code's `/skills` command
## Usage
Run this skill, then press `/skills` to see all synced skills.
## Manual Alternative
If you prefer not to use this skill, run this script manually:
```bash
#!/bin/bash
SOURCE="$HOME/.agents/skills"
TARGET="$HOME/.claude/skills"
mkdir -p "$TARGET"
for skill in "$SOURCE"/*; do
name=$(basename "$skill")
if [ -d "$skill" ] && [ -f "$skill/SKILL.md" ]; then
if [ ! -L "$TARGET/$name" ]; then
ln -sf "$skill" "$TARGET/$name"
echo "Linked: $name"
fi
fi
done
echo "Sync complete. Run /skills to see all."
```
.claude/skills/sync-global.sh
+19
View File
@@ -0,0 +1,19 @@
#!/bin/bash
# 同步 npx global skills 到 Claude Code
SOURCE="$HOME/.agents/skills"
TARGET="$HOME/.claude/skills"
mkdir -p "$TARGET"
for skill in "$SOURCE"/*; do
name=$(basename "$skill")
if [ -d "$skill" ] && [ -f "$skill/SKILL.md" ]; then
if [ ! -L "$TARGET/$name" ]; then
ln -sf "$skill" "$TARGET/$name"
echo "Linked: $name"
fi
fi
done
echo "Sync complete. Run /skills to see all."
.claude/skills/todo/SKILL.md
+30
View File
@@ -0,0 +1,30 @@
---
name: todo
description: 管理 Todo 任务 - 支持直接操作和后台 Agent 模式。添加、列出、完成任务，或启动后台 Agent 异步处理。
---
# Todo Skill
个人待办事项管理，支持两种模式。
## 模式一：直接操作（默认）
- `/todo add "任务内容" -p 1` - 添加高优先级任务
- `/todo list` - 列出所有任务
- `/todo done <id>` - 完成任务
- `/todo delete <id>` - 删除任务
## 模式二：后台 Agent
启动后台 Agent 轮询处理异步指令：
```
/loop 30s /todo agent
```
Agent 工作流程：
1. 轮询 `.claude/todo/inbox/` 目录
2. 读取 `cmd_*.json` 指令文件
3. 执行操作（add/list/done/delete/update）
4. 写入 `rsp_*.json` 响应
5. 更新心跳和日志
.claude/souls/collaborator.md
+39
View File
@@ -0,0 +1,39 @@
# 协作者模式 (Collaborator Persona)
## 核心定位
你是用户的对等协作者，不是工具或下属。
## 行为特征
### 对话风格
- **平等交流**: 用"我们一起..."而不是"我帮你..."
- **主动提问**: 不假设，通过问题澄清需求
- **共同探索**: "这个思路怎么样？"、"还有一种可能是..."
- **坦诚表达**: 有不同意见时直接说
### 思考方式
- 先理解"为什么"，再想"怎么做"
- 提供多个选项，分析利弊，让用户决策
- 承认不确定性："这个我不确定，我们可以..."
### 互动模式
1. **倾听**: 确保真正理解意图
2. **共鸣**: "我理解你想..."确认理解
3. **共建**: 一起完善想法
4. **总结**: 达成共识后确认
## 适用场景
- 需求讨论和澄清
- 架构设计
- 头脑风暴
- 复杂问题分析
## 开场白示例
- "好，我们一起看看这个问题..."
- "我理解你想要...，我的初步想法是..."
- "有几个方向可以考虑..."
## 禁用语气
- ❌ "遵命"
- ❌ "为您效劳"
- ❌ 过度谦卑的表达
.claude/souls/expert.md
+38
View File
@@ -0,0 +1,38 @@
# 专家模式 (Expert Persona)
## 核心定位
你是领域专家，直接给出最佳实践和准确答案。
## 行为特征
### 对话风格
- **简洁准确**: 不说废话，每句都有信息量
- **权威但谦逊**: 基于专业知识，但承认知识边界
- **结果导向**: 关注解决问题，而非展示知识
### 思考方式
- 快速识别问题本质
- 直接给出最优解
- 解释"为什么"，但不冗长
### 回复结构
1. **直接答案**: 一句话给出结论
2. **必要解释**: 简要说明原因
3. **代码/方案**: 给出可执行的内容
4. **注意事项**: 关键风险和边界情况
## 适用场景
- 代码审查
- 技术实现
- 最佳实践咨询
- 故障排查
## 开场白示例
- "直接说方案：..."
- "最优解是..."
- "注意这三个点：..."
## 禁用语气
- ❌ "可能可以..."
- ❌ 过多铺垫
- ❌ 过度解释已知概念
.claude/souls/mentor.md
+40
View File
@@ -0,0 +1,40 @@
# 导师模式 (Mentor Persona)
## 核心定位
你是有经验的导师，帮助用户自己发现答案。
## 行为特征
### 对话风格
- **引导式提问**: 通过问题引导思考
- **启发而非告知**: "你觉得...会怎么样？"
- **耐心**: 给用户思考空间
- **鼓励**: 肯定正确的思路
### 思考方式
- 判断用户当前理解水平
- 在认知边界处设问
- 提供思考框架，而非标准答案
### 互动模式
1. **诊断**: 了解当前理解程度
2. **设问**: 提出引导性问题
3. **启发**: 在卡壳时给线索
4. **总结**: 帮助归纳原理
## 适用场景
- 概念学习
- 技能培养
- 问题解决能力训练
- 思维方式培养
## 引导问题示例
- "如果...会发生什么？"
- "这和...有什么联系？"
- "有没有其他角度？"
- "为什么你觉得...？"
## 禁用语气
- ❌ 直接给答案（除非用户明确要求）
- ❌ 说教式语气
- ❌ 不耐烦的表达
.claude/todo/README.md
+108
View File
@@ -0,0 +1,108 @@
# Todo Agent 使用指南
基于文件通信的异步任务管理系统。
## 快速开始
### 1. 启动 Todo Agent（后台循环）
```bash
# 方式1: 使用 loop 技能
/loop 30s /todo-agent
# 方式2: 使用 CLI
uv run python -m src.interfaces.cli.main todo agent
# 方式3: 后台守护模式
uv run python -m src.interfaces.cli.main todo agent --daemon
```
### 2. 主会话交互
```python
# 在 Python 中
from src.application.services.todo.client import todo
# 添加任务
todo.add("完成任务A", priority=1, wait=True)
# 列出任务
result = todo.list()
print(result.get("formatted"))
# 完成任务
todo.done("abc123", wait=True)
```
### 3. 命令行操作
```bash
# 添加任务
uv run python -m src.interfaces.cli.main todo add "任务标题" -p 1 -t bug
# 列出任务
uv run python -m src.interfaces.cli.main todo list
# 完成任务
uv run python -m src.interfaces.cli.main todo done <task_id>
# 查看状态
uv run python -m src.interfaces.cli.main todo status
# 停止 Agent
uv run python -m src.interfaces.cli.main todo stop
```
## 通信协议
文件目录结构:
```
.claude/todo/
├── inbox/ # 主会话写入的指令
│ └── cmd_<ts>.json
├── outbox/ # Agent 返回的结果
│ └── rsp_<ts>.json
├── state/ # 共享状态
│ ├── tasks.json # 任务列表
│ ├── agent.log # 执行日志
│ └── heartbeat.json # Agent 心跳
└── archive/ # 归档的指令
```
指令格式:
```json
{
"cmd_type": "add",
"task_id": null,
"payload": {"title": "任务", "priority": 3},
"sender": "main",
"timestamp": "2024-01-15T10:00:00",
"id": "20240115_100000_123456"
}
```
响应格式:
```json
{
"success": true,
"cmd_id": "20240115_100000_123456",
"data": {"task": {...}, "message": "已创建"},
"error": null,
"timestamp": "2024-01-15T10:00:01",
"id": "20240115_100001_789012"
}
```
## 指令类型
| 指令 | 说明 | payload |
|------|------|---------|
| add | 添加任务 | title, description, priority, tags |
| list | 列出任务 | status, tag, priority |
| done | 完成任务 | task_id |
| delete | 删除任务 | task_id |
| update | 更新任务 | task_id + 字段 |
| get | 获取任务 | task_id |
| clear | 清空已完成 | - |
| ping | 心跳检测 | - |
| stop | 停止 Agent | - |
.claude/todo/archive/20260405/cmd_20260405_124946_752279.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "重构代码",
"description": "清理冗余代码",
"priority": 1,
"tags": [
"refactor"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:49:46.752268",
"id": "20260405_124946_752279"
}
.claude/todo/archive/20260405/cmd_20260405_124946_752510.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "编写测试",
"description": "",
"priority": 2,
"tags": [
"test"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:49:46.752508",
"id": "20260405_124946_752510"
}
.claude/todo/archive/20260405/cmd_20260405_124946_752659.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "部署到生产",
"description": "",
"priority": 1,
"tags": [
"deploy"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:49:46.752652",
"id": "20260405_124946_752659"
}
.claude/todo/archive/20260405/cmd_20260405_125001_887647.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "done",
"task_id": "fe36a678",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:50:01.887636",
"id": "20260405_125001_887647"
}
.claude/todo/archive/20260405/cmd_20260405_125115_736683.json
+13
View File
@@ -0,0 +1,13 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "后台测试任务1",
"description": "",
"priority": 1,
"tags": []
},
"sender": "main",
"timestamp": "2026-04-05T12:51:15.736668",
"id": "20260405_125115_736683"
}
.claude/todo/archive/20260405/cmd_20260405_125115_736891.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "后台测试任务2",
"description": "",
"priority": 2,
"tags": [
"demo"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:51:15.736889",
"id": "20260405_125115_736891"
}
.claude/todo/archive/20260405/cmd_20260405_125151_440893.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "stop",
"task_id": null,
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:51:51.440878",
"id": "20260405_125151_440893"
}
.claude/todo/archive/20260405/cmd_20260405_125230_232997.json
+13
View File
@@ -0,0 +1,13 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "实时测试任务",
"description": "",
"priority": 1,
"tags": []
},
"sender": "main",
"timestamp": "2026-04-05T12:52:30.232982",
"id": "20260405_125230_232997"
}
.claude/todo/archive/20260405/cmd_20260405_125342_933697.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置",
"priority": 1,
"tags": [
"config"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:53:42.933684",
"id": "20260405_125342_933697"
}
.claude/todo/archive/20260405/cmd_20260405_125342_933972.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "分步提交当前修改",
"description": "按逻辑分组逐步提交当前所有修改的文件",
"priority": 1,
"tags": [
"git"
]
},
"sender": "main",
"timestamp": "2026-04-05T12:53:42.933970",
"id": "20260405_125342_933972"
}
.claude/todo/archive/20260405/cmd_20260405_125810_497416.json
+12
View File
@@ -0,0 +1,12 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "code",
"method": "analyze_gitignore",
"kwargs": {}
},
"sender": "main",
"timestamp": "2026-04-05T12:58:10.497402",
"id": "20260405_125810_497416"
}
.claude/todo/archive/20260405/cmd_20260405_125812_522150.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "get",
"task_id": "825ed299",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:58:12.522142",
"id": "20260405_125812_522150"
}
.claude/todo/archive/20260405/cmd_20260405_125815_602478.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "auto",
"method": "interpret_and_execute",
"kwargs": {
"task_id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:58:15.602456",
"id": "20260405_125815_602478"
}
.claude/todo/archive/20260405/cmd_20260405_125847_739141.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "get",
"task_id": "825ed299",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:58:47.739127",
"id": "20260405_125847_739141"
}
.claude/todo/archive/20260405/cmd_20260405_125848_800418.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "auto",
"method": "interpret_and_execute",
"kwargs": {
"task_id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:58:48.800409",
"id": "20260405_125848_800418"
}
.claude/todo/archive/20260405/cmd_20260405_125858_011314.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "get",
"task_id": "56df56a2",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:58:58.011299",
"id": "20260405_125858_011314"
}
.claude/todo/archive/20260405/cmd_20260405_125900_778492.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "auto",
"method": "interpret_and_execute",
"kwargs": {
"task_id": "56df56a2",
"title": "分步提交当前修改",
"description": "按逻辑分组逐步提交当前所有修改的文件"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:59:00.778488",
"id": "20260405_125900_778492"
}
.claude/todo/archive/20260405/cmd_20260405_125912_868623.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "file",
"method": "edit",
"kwargs": {
"path": ".gitignore",
"old": "# Claudecode\n.claude/\n",
"new": "# Claudecode\n.claude/\n# End of file\n"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:59:12.868612",
"id": "20260405_125912_868623"
}
.claude/todo/archive/20260405/cmd_20260405_125926_100277.json
+14
View File
@@ -0,0 +1,14 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "file",
"method": "read",
"kwargs": {
"path": ".gitignore"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:59:26.100264",
"id": "20260405_125926_100277"
}
.claude/todo/archive/20260405/cmd_20260405_125928_048340.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "execute",
"task_id": null,
"payload": {
"tool": "file",
"method": "write",
"kwargs": {
"path": ".gitignore",
"content": "# ========================================\n# Python 相关文件\n# ========================================\n__pycache__/\n*.py[cod]\n*$py.class\n*.so\n.Python\nprisma/models/\nprisma/data/\nprisma/data_test/\n\n# 构建和分发\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\n\n# ========================================\n# Node.js 相关文件\n# ========================================\nnode_modules/\npackage-lock.json\n\n# ========================================\n# 虚拟环境\n# ========================================\nvenv/\nenv/\nENV/\n.venv/\n\n# ========================================\n# IDE 和编辑器\n# ========================================\n.idea/\n.vscode/\n*.swp\n*.swo\n.DS_Store\n\n# Cursor\n.cursor/\n.cursorignore\n\n# Obsidian\n.obsidian/\n\n# ========================================\n# 测试和覆盖率\n# ========================================\n.pytest_cache/\n.coverage\nhtmlcov/\n.tox/\n.nox/\ntests/data/\ndata_test/\n\n# ========================================\n# 日志文件\n# ========================================\n*.log\nlogs/\n\n# ========================================\n# 配置和环境文件\n# ========================================\n.env\n.env.local\n.env.*.local\nconfigs/.env\nconfigs/app.json\nconfigs/app.test.json\n\n# ========================================\n# 数据库文件\n# ========================================\n*.db\n*.sqlite\n*.sqlite3\n/data/\n/data_test/\n/archives/\n\n# ========================================\n# MyAgent 统一存储目录（新框架）\n# ========================================\n# 开发环境数据（隐藏目录，防止误操作）\n.myagent/\n.myagent_test/\n# 但不忽略测试固件\n!.myagent_test/fixtures/\n# 生产环境在用户主目录，不在项目中\n# ~/.myagent/ (用户主目录，不在这里管理)\n\n# ========================================\n# 本地文件\n# ========================================\nlocal/\ntmp/\nconfigs/agent_prompts/.history/\n\n# ========================================\n# 临时文件和缓存\n# ========================================\n.cache/\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# ========================================\n# 系统文件\n# ========================================\n.DS_Store\nThumbs.db\n\n# Claude Code\n.claude/\n"
}
},
"sender": "main",
"timestamp": "2026-04-05T12:59:28.048329",
"id": "20260405_125928_048340"
}
.claude/todo/archive/20260405/cmd_20260405_125943_026380.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "done",
"task_id": "825ed299",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:59:43.026367",
"id": "20260405_125943_026380"
}
.claude/todo/archive/20260405/cmd_20260405_125943_135034.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "done",
"task_id": "56df56a2",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T12:59:43.135029",
"id": "20260405_125943_135034"
}
.claude/todo/archive/20260405/cmd_20260405_130503_486633.json
+15
View File
@@ -0,0 +1,15 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "新任务测试",
"description": "",
"priority": 2,
"tags": [
"demo"
]
},
"sender": "main",
"timestamp": "2026-04-05T13:05:03.486620",
"id": "20260405_130503_486633"
}
.claude/todo/archive/20260405/cmd_20260405_130644_974174.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "delete",
"task_id": "fdac5557",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T13:06:44.974161",
"id": "20260405_130644_974174"
}
.claude/todo/archive/20260405/cmd_20260405_130644_974376.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "delete",
"task_id": "dca41096",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T13:06:44.974374",
"id": "20260405_130644_974376"
}
.claude/todo/archive/20260405/cmd_20260405_130644_974500.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "delete",
"task_id": "a738fbab",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T13:06:44.974498",
"id": "20260405_130644_974500"
}
.claude/todo/archive/20260405/cmd_20260405_130644_974613.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "delete",
"task_id": "c589aa70",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T13:06:44.974612",
"id": "20260405_130644_974613"
}
.claude/todo/archive/20260405/cmd_20260405_130723_543950.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "审查 .gitignore 配置",
"description": "检查 .gitignore 是否完整，清理冗余规则",
"priority": 1,
"tags": [
"config",
"review"
]
},
"sender": "main",
"timestamp": "2026-04-05T13:07:23.543938",
"id": "20260405_130723_543950"
}
.claude/todo/archive/20260405/cmd_20260405_130723_544259.json
+16
View File
@@ -0,0 +1,16 @@
{
"cmd_type": "add",
"task_id": null,
"payload": {
"title": "完成 git 提交",
"description": "分步提交当前修改：先 configs，再 src，最后 tests",
"priority": 1,
"tags": [
"git",
"commit"
]
},
"sender": "main",
"timestamp": "2026-04-05T13:07:23.544257",
"id": "20260405_130723_544259"
}
.claude/todo/archive/20260405/cmd_20260405_130747_317460.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "done",
"task_id": "db4e360a",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T13:07:47.317449",
"id": "20260405_130747_317460"
}
.claude/todo/archive/20260405/cmd_20260405_144429_536707.json
+8
View File
@@ -0,0 +1,8 @@
{
"cmd_type": "delete",
"task_id": "28bfde69",
"payload": {},
"sender": "main",
"timestamp": "2026-04-05T14:44:29.536697",
"id": "20260405_144429_536707"
}
.claude/todo/outbox/rsp_20260405_124949_976287.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_124946_752279",
"data": {
"task": {
"id": "fe36a678",
"title": "重构代码",
"description": "清理冗余代码",
"status": "pending",
"priority": 1,
"tags": [
"refactor"
],
"created_at": "2026-04-05T12:49:49.976095",
"updated_at": "2026-04-05T12:49:49.976098",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [fe36a678] 重构代码"
},
"error": null,
"timestamp": "2026-04-05T12:49:49.976286",
"id": "20260405_124949_976287"
}
.claude/todo/outbox/rsp_20260405_124949_976930.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_124946_752510",
"data": {
"task": {
"id": "56a67bbf",
"title": "编写测试",
"description": "",
"status": "pending",
"priority": 2,
"tags": [
"test"
],
"created_at": "2026-04-05T12:49:49.976724",
"updated_at": "2026-04-05T12:49:49.976725",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [56a67bbf] 编写测试"
},
"error": null,
"timestamp": "2026-04-05T12:49:49.976929",
"id": "20260405_124949_976930"
}
.claude/todo/outbox/rsp_20260405_124949_977415.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_124946_752659",
"data": {
"task": {
"id": "28bfde69",
"title": "部署到生产",
"description": "",
"status": "pending",
"priority": 1,
"tags": [
"deploy"
],
"created_at": "2026-04-05T12:49:49.977264",
"updated_at": "2026-04-05T12:49:49.977266",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [28bfde69] 部署到生产"
},
"error": null,
"timestamp": "2026-04-05T12:49:49.977413",
"id": "20260405_124949_977415"
}
.claude/todo/outbox/rsp_20260405_125015_276639.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125001_887647",
"data": {
"task": {
"id": "fe36a678",
"title": "重构代码",
"description": "清理冗余代码",
"status": "done",
"priority": 1,
"tags": [
"refactor"
],
"created_at": "2026-04-05T12:49:49.976095",
"updated_at": "2026-04-05T12:50:15.276411",
"completed_at": "2026-04-05T12:50:15.276409",
"metadata": {}
},
"message": "✅ 已完成: [fe36a678] 重构代码"
},
"error": null,
"timestamp": "2026-04-05T12:50:15.276637",
"id": "20260405_125015_276639"
}
.claude/todo/outbox/rsp_20260405_125118_021142.json
+22
View File
@@ -0,0 +1,22 @@
{
"success": true,
"cmd_id": "20260405_125115_736683",
"data": {
"task": {
"id": "fdac5557",
"title": "后台测试任务1",
"description": "",
"status": "pending",
"priority": 1,
"tags": [],
"created_at": "2026-04-05T12:51:18.020355",
"updated_at": "2026-04-05T12:51:18.020361",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [fdac5557] 后台测试任务1"
},
"error": null,
"timestamp": "2026-04-05T12:51:18.021137",
"id": "20260405_125118_021142"
}
.claude/todo/outbox/rsp_20260405_125118_022859.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125115_736891",
"data": {
"task": {
"id": "dca41096",
"title": "后台测试任务2",
"description": "",
"status": "pending",
"priority": 2,
"tags": [
"demo"
],
"created_at": "2026-04-05T12:51:18.022371",
"updated_at": "2026-04-05T12:51:18.022375",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [dca41096] 后台测试任务2"
},
"error": null,
"timestamp": "2026-04-05T12:51:18.022856",
"id": "20260405_125118_022859"
}
.claude/todo/outbox/rsp_20260405_125154_943854.json
+10
View File
@@ -0,0 +1,10 @@
{
"success": true,
"cmd_id": "20260405_125151_440893",
"data": {
"message": "👋 Agent 即将停止"
},
"error": null,
"timestamp": "2026-04-05T12:51:54.943845",
"id": "20260405_125154_943854"
}
.claude/todo/outbox/rsp_20260405_125231_909676.json
+22
View File
@@ -0,0 +1,22 @@
{
"success": true,
"cmd_id": "20260405_125230_232997",
"data": {
"task": {
"id": "a738fbab",
"title": "实时测试任务",
"description": "",
"status": "pending",
"priority": 1,
"tags": [],
"created_at": "2026-04-05T12:52:31.908496",
"updated_at": "2026-04-05T12:52:31.908508",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [a738fbab] 实时测试任务"
},
"error": null,
"timestamp": "2026-04-05T12:52:31.909669",
"id": "20260405_125231_909676"
}
.claude/todo/outbox/rsp_20260405_125343_788360.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125342_933697",
"data": {
"task": {
"id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置",
"status": "pending",
"priority": 1,
"tags": [
"config"
],
"created_at": "2026-04-05T12:53:43.787323",
"updated_at": "2026-04-05T12:53:43.787331",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [825ed299] 优化 .gitignore 配置"
},
"error": null,
"timestamp": "2026-04-05T12:53:43.788354",
"id": "20260405_125343_788360"
}
.claude/todo/outbox/rsp_20260405_125343_790144.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125342_933972",
"data": {
"task": {
"id": "56df56a2",
"title": "分步提交当前修改",
"description": "按逻辑分组逐步提交当前所有修改的文件",
"status": "pending",
"priority": 1,
"tags": [
"git"
],
"created_at": "2026-04-05T12:53:43.789588",
"updated_at": "2026-04-05T12:53:43.789591",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [56df56a2] 分步提交当前修改"
},
"error": null,
"timestamp": "2026-04-05T12:53:43.790141",
"id": "20260405_125343_790144"
}
.claude/todo/outbox/rsp_20260405_125812_518954.json
+19
View File
@@ -0,0 +1,19 @@
{
"success": true,
"cmd_id": "20260405_125810_497416",
"data": {
"success": true,
"output": "📊 .gitignore 分析:\n 总规则数: 70\n 注释数: 43\n 空行数: 15\n Python相关: 3\n IDE相关: 4\n\n💡 建议:\n - 添加 .claude/ (Claude Code 工作目录)\n",
"error": null,
"metadata": {
"总规则数": 70,
"注释数": 43,
"空行数": 15,
"Python相关": 3,
"IDE相关": 4
}
},
"error": null,
"timestamp": "2026-04-05T12:58:12.518945",
"id": "20260405_125812_518954"
}
.claude/todo/outbox/rsp_20260405_125815_524488.json
+23
View File
@@ -0,0 +1,23 @@
{
"success": true,
"cmd_id": "20260405_125812_522150",
"data": {
"task": {
"id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置",
"status": "pending",
"priority": 1,
"tags": [
"config"
],
"created_at": "2026-04-05T12:53:43.787323",
"updated_at": "2026-04-05T12:53:43.787331",
"completed_at": null,
"metadata": {}
}
},
"error": null,
"timestamp": "2026-04-05T12:58:15.524483",
"id": "20260405_125815_524488"
}
.claude/todo/outbox/rsp_20260405_125818_530505.json
+27
View File
@@ -0,0 +1,27 @@
{
"success": true,
"cmd_id": "20260405_125815_602478",
"data": {
"success": false,
"output": "步骤 1: code.analyze_gitignore ❌\n错误: CodeTools.analyze_gitignore() got an unexpected keyword argument 'kwargs'",
"error": null,
"metadata": {
"plan": [
{
"tool": "code",
"method": "analyze_gitignore",
"kwargs": {}
},
{
"tool": "code",
"method": "analyze_gitignore",
"kwargs": {}
}
],
"results_count": 1
}
},
"error": null,
"timestamp": "2026-04-05T12:58:18.530502",
"id": "20260405_125818_530505"
}
.claude/todo/outbox/rsp_20260405_125848_749166.json
+23
View File
@@ -0,0 +1,23 @@
{
"success": true,
"cmd_id": "20260405_125847_739141",
"data": {
"task": {
"id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置",
"status": "pending",
"priority": 1,
"tags": [
"config"
],
"created_at": "2026-04-05T12:53:43.787323",
"updated_at": "2026-04-05T12:53:43.787331",
"completed_at": null,
"metadata": {}
}
},
"error": null,
"timestamp": "2026-04-05T12:58:48.749159",
"id": "20260405_125848_749166"
}
.claude/todo/outbox/rsp_20260405_125851_756537.json
+27
View File
@@ -0,0 +1,27 @@
{
"success": true,
"cmd_id": "20260405_125848_800418",
"data": {
"success": true,
"output": "步骤 1: code.analyze_gitignore ✅\n📊 .gitignore 分析:\n 总规则数: 70\n 注释数: 43\n 空行数: 15\n Python相关: 3\n IDE相关: 4\n\n💡 建议:\n - 添加 .claude/ (Claude Code 工作目录)\n\n步骤 2: code.analyze_gitignore ✅\n📊 .gitignore 分析:\n 总规则数: 70\n 注释数: 43\n 空行数: 15\n Python相关: 3\n IDE相关: 4\n\n💡 建议:\n - 添加 .claude/ (Claude Code 工作目录)\n",
"error": null,
"metadata": {
"plan": [
{
"tool": "code",
"method": "analyze_gitignore",
"kwargs": {}
},
{
"tool": "code",
"method": "analyze_gitignore",
"kwargs": {}
}
],
"results_count": 2
}
},
"error": null,
"timestamp": "2026-04-05T12:58:51.756533",
"id": "20260405_125851_756537"
}
.claude/todo/outbox/rsp_20260405_125900_773186.json
+23
View File
@@ -0,0 +1,23 @@
{
"success": true,
"cmd_id": "20260405_125858_011314",
"data": {
"task": {
"id": "56df56a2",
"title": "分步提交当前修改",
"description": "按逻辑分组逐步提交当前所有修改的文件",
"status": "pending",
"priority": 1,
"tags": [
"git"
],
"created_at": "2026-04-05T12:53:43.789588",
"updated_at": "2026-04-05T12:53:43.789591",
"completed_at": null,
"metadata": {}
}
},
"error": null,
"timestamp": "2026-04-05T12:59:00.773180",
"id": "20260405_125900_773186"
}
.claude/todo/outbox/rsp_20260405_125903_915427.json
+34
View File
File diff suppressed because one or more lines are too long
.claude/todo/outbox/rsp_20260405_125912_929508.json
+13
View File
@@ -0,0 +1,13 @@
{
"success": true,
"cmd_id": "20260405_125912_868623",
"data": {
"success": false,
"output": "",
"error": "未找到匹配内容: # Claudecode\n.claude/\n...",
"metadata": null
},
"error": null,
"timestamp": "2026-04-05T12:59:12.929506",
"id": "20260405_125912_929508"
}
.claude/todo/outbox/rsp_20260405_125928_044538.json
+13
View File
@@ -0,0 +1,13 @@
{
"success": true,
"cmd_id": "20260405_125926_100277",
"data": {
"success": true,
"output": "# ========================================\n# Python 相关文件\n# ========================================\n__pycache__/\n*.py[cod]\n*$py.class\n*.so\n.Python\nprisma/models/\nprisma/data/\nprisma/data_test/\n\n# 构建和分发\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\n\n# ========================================\n# Node.js 相关文件\n# ========================================\nnode_modules/\npackage-lock.json\n\n# ========================================\n# 虚拟环境\n# ========================================\nvenv/\nenv/\nENV/\n.venv/\n\n# ========================================\n# IDE 和编辑器\n# ========================================\n.idea/\n.vscode/\n*.swp\n*.swo\n.DS_Store\n\n# Cursor\n.cursor/\n.cursorignore\n\n# Obsidian\n.obsidian/\n\n# ========================================\n# 测试和覆盖率\n# ========================================\n.pytest_cache/\n.coverage\nhtmlcov/\n.tox/\n.nox/\ntests/data/\ndata_test/\n\n# ========================================\n# 日志文件\n# ========================================\n*.log\nlogs/\n\n# ========================================\n# 配置和环境文件\n# ========================================\n.env\n.env.local\n.env.*.local\nconfigs/.env\nconfigs/app.json\nconfigs/app.test.json\n\n# ========================================\n# 数据库文件\n# ========================================\n*.db\n*.sqlite\n*.sqlite3\n/data/\n/data_test/\n/archives/\n\n# ========================================\n# MyAgent 统一存储目录（新框架）\n# ========================================\n# 开发环境数据（隐藏目录，防止误操作）\n.myagent/\n.myagent_test/\n# 但不忽略测试固件\n!.myagent_test/fixtures/\n# 生产环境在用户主目录，不在项目中\n# ~/.myagent/ (用户主目录，不在这里管理)\n\n# ========================================\n# 本地文件\n# ========================================\nlocal/\ntmp/\nconfigs/agent_prompts/.history/\n\n# ========================================\n# 临时文件和缓存\n# ========================================\n.cache/\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# ========================================\n# 系统文件\n# ========================================\n.DS_Store\nThumbs.db\n",
"error": null,
"metadata": null
},
"error": null,
"timestamp": "2026-04-05T12:59:28.044536",
"id": "20260405_125928_044538"
}
.claude/todo/outbox/rsp_20260405_125931_053904.json
+13
View File
@@ -0,0 +1,13 @@
{
"success": true,
"cmd_id": "20260405_125928_048340",
"data": {
"success": true,
"output": "写入成功: .gitignore",
"error": null,
"metadata": null
},
"error": null,
"timestamp": "2026-04-05T12:59:31.053899",
"id": "20260405_125931_053904"
}
.claude/todo/outbox/rsp_20260405_125943_072987.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125943_026380",
"data": {
"task": {
"id": "825ed299",
"title": "优化 .gitignore 配置",
"description": "清理不必要的忽略规则，添加项目特定配置",
"status": "done",
"priority": 1,
"tags": [
"config"
],
"created_at": "2026-04-05T12:53:43.787323",
"updated_at": "2026-04-05T12:59:43.072495",
"completed_at": "2026-04-05T12:59:43.072493",
"metadata": {}
},
"message": "✅ 已完成: [825ed299] 优化 .gitignore 配置"
},
"error": null,
"timestamp": "2026-04-05T12:59:43.072984",
"id": "20260405_125943_072987"
}
.claude/todo/outbox/rsp_20260405_125946_079936.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_125943_135034",
"data": {
"task": {
"id": "56df56a2",
"title": "分步提交当前修改",
"description": "按逻辑分组逐步提交当前所有修改的文件",
"status": "done",
"priority": 1,
"tags": [
"git"
],
"created_at": "2026-04-05T12:53:43.789588",
"updated_at": "2026-04-05T12:59:46.079020",
"completed_at": "2026-04-05T12:59:46.079014",
"metadata": {}
},
"message": "✅ 已完成: [56df56a2] 分步提交当前修改"
},
"error": null,
"timestamp": "2026-04-05T12:59:46.079931",
"id": "20260405_125946_079936"
}
.claude/todo/outbox/rsp_20260405_130505_791878.json
+24
View File
@@ -0,0 +1,24 @@
{
"success": true,
"cmd_id": "20260405_130503_486633",
"data": {
"task": {
"id": "c589aa70",
"title": "新任务测试",
"description": "",
"status": "pending",
"priority": 2,
"tags": [
"demo"
],
"created_at": "2026-04-05T13:05:05.790728",
"updated_at": "2026-04-05T13:05:05.790732",
"completed_at": null,
"metadata": {}
},
"message": "✅ 已创建任务: [c589aa70] 新任务测试"
},
"error": null,
"timestamp": "2026-04-05T13:05:05.791875",
"id": "20260405_130505_791878"
}
.claude/todo/outbox/rsp_20260405_130729_384474.json
+8
View File
@@ -0,0 +1,8 @@
{
"success": false,
"cmd_id": "20260405_130644_974174",
"data": {},
"error": "Task not found: fdac5557",
"timestamp": "2026-04-05T13:07:29.384471",
"id": "20260405_130729_384474"
}

Some files were not shown because too many files have changed in this diff Show More