I Gave My AI Agent the Ability to Research Before It Writes — Here’s What Changed

I Gave My AI Agent the Ability to Research Before It Writes — Here's What Changed

Four weeks ago, I had no idea what an AI agent was. Now I'm building one that researches market trends before writing about them, synthesizes information from three independent sources, and produces work that scores 96/100 on my eval system.

The change didn't come from a new model or a fancy framework. It came from stopping my agent from writing blind.

The Problem: Writing From Memory, Not Evidence

When I first built ShipStack's article factory, it was simple. I'd prompt Claude: "Write an article about AI agents and multi-agent orchestration." Claude would write. It was fine. Coherent. On-brand.

But it was hollow.

I realized what was happening: my agent was writing from pattern matching. It knew what an article about AI agents should look like because it had seen thousands. But it didn't know what was actually happening in the market right now. It didn't know that Cursor just hit $9.9B valuation with Agent Mode as the headline feature. It didn't know that enterprise leaders are abandoning 60% of AI projects because their data isn't ready. It didn't know that accuracy compounds exponentially—if each agent action hits 85%, a 10-step workflow only succeeds 20% of the time.

It was writing confidently about things it didn't actually understand.

That's the gap between a chatbot and a real agent. A chatbot answers questions. An agent investigates before it acts.

The Insight: Research First, Then Write

I started noticing something in my own process. When I write about something I actually understand, the work is sharper. More specific. I reference concrete numbers, real tools, actual timelines. When I'm writing from half-memory, it's generic. Filler. Safe.

So I asked myself: what if my writing agent worked the same way?

Instead of "Write an article about X," the prompt became:

Research X using three independent sources (Brave Search, DuckDuckGo, Wikipedia)
Synthesize what you find into a structured brief with four sections: background, what's happening now, gaps nobody's talking about, and actual numbers
Then write the article using the brief as your foundation

The research agent runs independently. Error handling per source. If one fails, the others still work. Claude Haiku synthesizes the raw results into a clean brief—background noise removed, signal amplified. The brief gets injected into the writer's context before the first word is written.

First article written with research scored 96/100 on eval.

What Actually Changed

1. Specificity Became Default

Before research: "AI agents are transforming business automation."

After research: "Cursor's Agent Mode hit 8 parallel agents and $9.9B valuation. NVIDIA's GTC 2026 saw agentic frameworks draw the largest attendance, signaling enterprise deployment momentum."

One is a claim. The other is evidence.

The brief gives the writer ammunition. Real stats. Real context. Real angles. The writing doesn't have to be cautious anymore because it's grounded in something verifiable.

2. Gaps Became Visible

Here's what shocked me: the research agent found problems that nobody is talking about, even though they're critical.

Accuracy compounding is a perfect example. Everyone talks about 85% per-action accuracy as a win. Almost nobody mentions that this cascades to ~20% success in 10-step workflows. The brief highlighted this as an "angle nobody's exploring." The article could then address it directly.

A writer without research writes from memory gaps. A writer with research writes from knowledge gaps—and those are infinitely more valuable.

3. Trust Became Quantifiable

When I read the 96/100 article, I didn't just feel it was better. I could point to why. The piece mentioned three validated statistics. It cited specific products and company valuations. It acknowledged real problems with real consequences. The eval system rated it higher because the work was verifiable.

That's the real shift. The agent isn't smarter. But it's more honest.

How It Actually Works (The Technical Part)

I'm not going to pretend this is rocket science. It's not. But it's also not trivial, and I had to think through some real problems.

# Simplified version of the research pipeline

async def research_topic(topic: str) -> dict:
    """
    Research a topic across three independent sources.
    Returns structured brief with background, current discussion, gaps, and stats.
    """

    sources = [
        {"name": "Brave Search", "func": search_brave},
        {"name": "DuckDuckGo", "func": search_duckduckgo},
        {"name": "Wikipedia", "func": search_wikipedia}
    ]

    results = {}

    # Run all searches in parallel
    for source in sources:
        try:
            results[source["name"]] = await source["func"](topic)
        except Exception as e:
            # Individual source failure doesn't kill the whole pipeline
            results[source["name"]] = {"error": str(e), "data": None}

    # Synthesize results into structured brief
    brief = await synthesize_with_claude(
        results,
        sections=[
            "background",
            "what_is_being_discussed_now",
            "gaps_and_underexplored_angles",
            "key_stats_and_data_points"
        ]
    )

    return brief

The critical decisions:

Independent error handling: If Brave Search fails, DuckDuckGo still runs. If Wikipedia times out, the brief still synthesizes from two sources. I learned this the hard way—the first version failed if any source failed. Production taught me otherwise.

Parallel execution: All three search queries run at the same time using asyncio.gather(). Sequential would take 3x longer. In production, speed matters because every second of latency is a second the user waits.

Structured synthesis: The brief isn't just raw search results dumped together. Claude is instructed to organize findings into four specific sections. Background is history and context. "What's being discussed now" is current momentum and trends. "Gaps" is the angle—what everyone's talking about versus what's actually critical. "Key stats" is the ammunition. This structure forces clarity.

The Real Cost

I need to be honest about the downside: this costs more tokens.

Each research cycle uses Haiku (cheap) for searches and synthesis, then Sonnet (more expensive) for actual writing. A single article now pulls maybe 15,000-20,000 tokens where it used to pull 8,000-10,000. It's not dramatic, but it adds up across the article factory.

What I found: the 20% increase in token cost produces articles that score 15-20% higher on eval. The math works. But only if you're actually shipping and measuring. If you're just trying to sound smart, research doesn't matter.

What I'd Do Differently (If I Started Over)

Measure before and after: I didn't quantify article quality until after I built this. If I were starting over, I'd eval the old system, then the new one, so I'd have concrete proof. (I got lucky—the 96/100 score validated it retroactively.)
Automate source selection: Right now I hardcoded three sources. But different topics benefit from different research. A technical deep-dive needs StackOverflow and GitHub. A market analysis needs Crunchbase and SEC filings. A framework update needs the official docs. Future version should route the query to the right sources automatically.
Build a feedback loop from eval back to research: If an article scores 72/100 because it's missing recent data, the research agent should know that. Right now research and writing are separate. Next step is making them iterative.

The Broader Pattern

This is bigger than article writing.

I'm seeing this pattern across all six of ShipStack's pipelines. When agents have access to real-time context—whether that's current market data, your inbox status, your GitHub repos, or your company priorities—they make better decisions. When they're operating on stale mental models, they fail quietly.

The Morning Brief pipeline needs current date and recent priorities (stored in memory) to actually be useful. The Inbox Zero processor needs to know which senders matter historically. The Repo Triage system needs to know what's actively shipping versus abandoned. Right now memory is an island—only the article factory reads from it. Next is connecting memory to all five other pipelines.

Research before action. Memory-informed decisions. Real-time context.

That's the move from agent-as-tool to agent-as-actually-useful.

What I'm Building Next

I want to push this further. Instead of research happening once per article, I want continuous background research running on topics I care about—AI agents, agent engineering, multi-agent orchestration, memory architecture. The agent saves interesting findings to memory automatically. When I sit down to write, the brief isn't built from scratch—it's pulled from three weeks of background research plus fresh daily snapshots.

That's still hypothetical. But I'm building toward it.

The Real Lesson

Four weeks ago I thought building an AI agent meant connecting APIs and writing prompts. Now I understand it's about giving agents the ability to think before they act.

A chatbot without research writes confidently about things it doesn't know. An agent with research writes carefully about things it does.

The difference is measurable. It's in the eval scores. It's in the specificity of the output. It's in the actual value delivered.

I'm one month into this. I'm still figuring out what's possible. But I know for certain: the best agent isn't the one that can generate text fastest. It's the one that can verify reality before speaking.

That's worth building toward.

推荐订阅源

DEV Community