GitHub - brcrusoe72/agent-search: Self-hosted search API ...

🔍 AgentSearch

Self-hosted SearXNG-backed search API and MCP server for AI agents. Drop-in alternative to Tavily/Exa — without the per-query bill or the API key.

AgentSearch wraps SearXNG (an open-source meta-search engine) with a clean FastAPI layer that returns structured JSON. Built for LLM agents, RAG pipelines, and anyone tired of paying per-query for search APIs.

Why?

	AgentSearch	Brave API	Google CSE	SerpAPI
Cost	Free forever	$0.005/query	$5/1K queries	$50/mo
API Key	None	Required	Required	Required
Setup	`docker compose up`	Sign up + wait	Console + billing	Sign up + pay
Engines	6+ (configurable)	Brave only	Google only	Google only
Self-hosted	✅	❌	❌	❌
Rate limits	You control	1 req/sec free	100/day free	100/mo free
Deduplication	Built-in	❌	❌	❌

Quickstart

git clone https://github.com/brcrusoe72/agent-search.git
cd agent-search
docker compose up -d

That's it. Search at http://localhost:3939.

`GET /search`

General web search with deduplication and multi-engine scoring.

curl "http://localhost:3939/search?q=python+async+patterns&count=5"

{
  "results": [
    {
      "title": "Async IO in Python: A Complete Walkthrough",
      "url": "https://realpython.com/async-io-python/",
      "snippet": "A comprehensive guide to async/await in Python 3...",
      "engines": ["google", "bing", "duckduckgo"],
      "score": 1.0,
      "position": 1
    }
  ],
  "meta": {
    "query": "python async patterns",
    "total": 5,
    "engines_used": ["google", "bing", "duckduckgo"],
    "cached": false,
    "response_time_ms": 842.3
  }
}

Parameters:

Param	Type	Default	Description
`q`	string	required	Search query
`count`	int	10	Results to return (1-50)
`engines`	string	all	Comma-separated engines (`google,bing`)
`domain`	string	—	Filter to specific domain
`exclude_domains`	string	—	Comma-separated domains to exclude

`GET /search/jobs`

Job search across LinkedIn, Indeed, Glassdoor, and ZipRecruiter.

curl "http://localhost:3939/search/jobs?q=senior+python+engineer&location=remote&salary_min=150000"

Parameters:

Param	Type	Default	Description
`q`	string	required	Job title / keywords
`location`	string	—	Location filter
`salary_min`	int	—	Minimum salary filter

`GET /health`

curl http://localhost:3939/health

`GET /engines`

List all available search engines and their status.

curl http://localhost:3939/engines

Architecture

┌─────────────┐     ┌──────────────────┐     ┌─────────────┐
│  Your Agent  │────▶│  AgentSearch API  │────▶│   SearXNG    │
│  (any LLM)  │◀────│  :3939 (FastAPI)  │◀────│   :8080      │
└─────────────┘     └──────────────────┘     └──────┬──────┘
                     • Deduplication                  │
                     • Scoring                   ┌────┴────┐
                     • Caching                   │ Google  │
                     • Rate limiting             │ Bing    │
                                                 │ DDG     │
                                                 │ Brave   │
                                                 │ Start.. │
                                                 └─────────┘

Integration Examples

Python (requests)

import requests

resp = requests.get("http://localhost:3939/search", params={"q": "latest AI news", "count": 5})
results = resp.json()["results"]
for r in results:
    print(f"{r['title']}: {r['url']}")

LangChain Tool

from langchain.tools import tool
import requests

@tool
def web_search(query: str) -> str:
    """Search the web using AgentSearch."""
    resp = requests.get("http://localhost:3939/search", params={"q": query, "count": 5})
    results = resp.json()["results"]
    return "\n".join(f"- {r['title']}: {r['url']}\n  {r['snippet']}" for r in results)

OpenClaw (TOOLS.md)

## Search
- **AgentSearch**: `http://localhost:3939/search?q=QUERY` via web_fetch — free, self-hosted, no rate limits

curl (one-liner)

curl -s "http://localhost:3939/search?q=your+query" | jq '.results[:3]'

Configuration

Environment variables (set in docker-compose.yml):

Variable	Default	Description
`SEARXNG_URL`	`http://searxng:8080`	SearXNG instance URL
`CACHE_TTL`	`3600`	Cache duration in seconds
`RATE_LIMIT`	`30`	Max requests per minute

Adding/removing search engines

Edit searxng/settings.yml and restart:

docker compose restart searxng

Development

# Run locally (needs SearXNG running separately)
pip install -r requirements.txt
SEARXNG_URL=http://localhost:8080 uvicorn app.main:app --reload --port 3939

Contributing

Fork it
Create your branch (git checkout -b feature/better-dedup)
Commit (git commit -am 'Improve dedup algorithm')
Push (git push origin feature/better-dedup)
Open a PR

Python SDK

pip install agentsearch-client

from agentsearch import AgentSearch

client = AgentSearch()  # defaults to localhost:3939
results = client.search("manufacturing OEE best practices")
for r in results:
    print(f"{r.title} — {r.url}")

MCP Server

Use AgentSearch as an MCP tool server — gives any MCP-compatible client (Claude Desktop, Cursor, etc.) access to all 6 tools over stdio.

pip install mcp httpx
python mcp-server/server.py

Add to Claude Desktop config:

{
  "mcpServers": {
    "agent-search": {
      "command": "python",
      "args": ["/path/to/mcp-server/server.py"]
    }
  }
}

See mcp-server/README.md for full setup.

Related Projects

Operations Intelligence Analyzer — AI-powered OEE analysis (live demo)
Agent Café — AI agent marketplace (live at thecafe.dev)
Manufacturing Analyst Pro — MES data analysis CLI
AI True Cost Calculator — Know what your AI project really costs

License

MIT — do whatever you want with it.

推荐订阅源

Hacker News: Show HN