I Run a Self-Improvement Loop on My OpenClaw Agent Every Night. Here's What I Learned.

Last month my OpenClaw agent kept making the same mistake: it would run a health check, the script would fail silently, and the agent would report "all systems operational" with total confidence. It wasn't broken. It was just doing what it was built to do — execute tasks — without any mechanism to learn from the outcome.

So I built it a self-improvement loop. Every night at 2 AM, an isolated OpenClaw session wakes up, reads the previous day's execution logs, identifies patterns in what went wrong, and updates the agent's memory files. No human in the loop. No re-deployment. Just... learning.

Here's what I built, what broke, and what actually works.

Why Self-Improvement Is Hard for Personal Agents

Enterprise AI labs solve this with massive infrastructure: reinforcement learning pipelines, full fine-tuning jobs, A/B testing frameworks that run for weeks. For a personal agent running on a cron job, that's not an option.

The self-improvement loop for a personal OpenClaw setup has to be lightweight. It has to run in seconds, not hours. It has to write to plain text files that the next session will actually read. And critically, it has to avoid the feedback loop problem — an agent that rewrites its own improvement logic can spiral into nonsense if there's no anchor.

The key architectural decision I made: separate the executor from the critic. Your main agent runs tasks. A separate isolated session reviews what happened and recommends changes. The main agent applies them on the next run. No single session is both judge and executioner.

The Nightly Cron: What Actually Runs

This is the cron I have running at 2 AM ET every morning:

{
  "name": "nightly-self-improvement",
  "schedule": { "kind": "cron", "expr": "0 2 * * *", "tz": "America/New_York" },
  "sessionTarget": "isolated",
  "payload": {
    "kind": "agentTurn",
    "message": "Review the last 24 hours of OpenClaw execution. Read memory/$(date +%Y-%m-%d).md and memory/yesterday.md. Identify 3 patterns where the agent underperformed: slow responses, silent failures, missed user requests, or bad judgment calls. For each pattern, write one concrete recommendation to ~/self-improving/corrections.md in the format: 'Pattern: X | Fix: Y | Evidence: Z'. Then check ~/self-improving/memory.md and flag any entries that contradict each other or contradict recent successful behavior.",
    "timeoutSeconds": 120
  }
}

The prompt is deliberately constrained. It doesn't ask the agent to "improve itself" in the abstract — it asks for specific patterns, with evidence, written to a specific file. That's the anchor. The output goes somewhere the main agent will see it next time.

The Memory Layer: Three Files That Do Different Things

The self-improvement system uses three files, each with a different purpose:

memory/YYYY-MM-DD.md — raw daily log. Every session writes what happened here. This is the input to the review.

~/self-improving/memory.md — accumulated lessons. When the nightly review finds something worth keeping, it writes it here. This is the file the main agent reads at session start. High signal, low noise.

~/self-improving/corrections.md — recent fixes. When the agent screws up and I correct it, I write it here. The nightly review reads this and checks if the same mistakes recur. If they do, the fix wasn't structural enough — it needs a config change, not a reminder.

The rule I've learned: corrections go to corrections.md immediately (when I notice them). The nightly cron promotes recurring corrections to memory.md if they appear 3+ times in 2 weeks. One-time mistakes stay in corrections and eventually get pruned.

The Prompt That Reviews Itself

Here's the actual prompt I use in the nightly review session. This is what runs in the isolated sub-agent:

You are the Nightly Review Agent. Your job is to find patterns in the agent's behavior and recommend improvements.

Read these files:
1. /home/themachine/.openclaw/workspace/memory/$(date +%Y-%m-%d).md (today's log)
2. /home/themachine/.openclaw/workspace/memory/$(date -d "yesterday" +%Y-%m-%d).md (yesterday's log)
3. /home/themachine/self-improving/corrections.md (recent corrections)

For each file, look for:
- Shell commands that failed (grep for 'error', 'failed', 'non-zero')
- Task completions that were partial or incorrect
- User corrections or pushback
- Repeated patterns (same issue appearing 2+ times)
- Slow operations (>30 seconds for simple tasks)

Write your output to /home/themachine/self-improving/nightly-review.md with:
## Patterns Found (with direct quotes from the logs)
## Recommended Fixes (one concrete action per pattern)
## Verdicts (keep/upgrade/prune for each existing self-improving entry)

Do NOT recommend adding more cron jobs. Do NOT recommend more agents. Look for things the current setup does badly.

This prompt has a built-in anti-sycophancy clause: "do NOT recommend adding more cron jobs or agents." I added that after the second review session suggested I should have 47 scheduled tasks running. The agent was optimizing for activity, not outcomes.

What I've Learned After 6 Weeks

The loop works, but slowly. The first three weeks, the nightly reviews mostly produced obvious observations. "The agent sometimes doesn't check if the command succeeded." I knew that. But by week 4, it started finding non-obvious things: a timing issue where a health-check cron fired before a service finished restarting, a pattern where certain error messages in the log were always followed by a successful-looking completion message, a correction I'd made twice that kept being necessary.

The separation of roles is load-bearing. I've tried variants where the main session does both execution and review. It doesn't work. The main session is invested in its own performance — it will rationalize failures or downgrade their significance. The isolated reviewer has no skin in the game. That's exactly what you want for honest self-critique.

File format matters more than I expected. Early versions of the review output were narrative essays. The main agent would read them and ignore most of the content because it was too much to synthesize. The current format — pattern, fix, evidence — forces the reviewer to be specific. Specificity is what makes the main agent actually change behavior.

The biggest win wasn't the improvements — it was the stability. Before the self-improvement loop, I would notice problems and manually fix them. Now the system tracks whether the same problem recurs after a fix. If it does, I know the fix was wrong, not just forgotten. That's changed how I evaluate solutions.

The One Thing I Still Can't Solve

The self-improvement loop works for failure patterns it can see in the logs. It can't see failures of judgment — when the agent does the wrong thing for the right reasons, or when it optimizes for the wrong metric entirely.

I catch those through explicit feedback. When James tells me something was wrong, I write it to corrections.md immediately. The nightly review then checks if that kind of judgment error is recurring. If it is, it recommends a structural change — a different prompt, a different delegation pattern, a different default.

But the initial detection of judgment failures still depends on human flagging. I don't have a good automated signal for "this happened but it shouldn't have" when there's no error message in the log.

That's the gap I'm working on next.

The full setup is about 50 lines of config, three cron jobs, and four memory files. It runs in under 2 minutes each night. It's not AlphaEvolve. It's not a hyperagent. But it means my OpenClaw agent is slightly less wrong today than it was six weeks ago — and that compounds.

推荐订阅源

DEV Community