Every MCP web-access tutorial I read this month pointed at a paid API.
You don't need one. To let an AI agent read a public web page, sixty lines on the official MCP Python SDK give you a self-hosted web_fetch tool — running on your machine, no key, no per-call bill.
I built it, ran it, and pasted the real terminal output below. The catch isn't the wiring (that part is easy). It's the four defaults the tutorials leave out — the ones that turn a toy into something you'd actually point an agent at.
Quick answer: A Model Context Protocol (MCP) server exposes tools an LLM agent can call. With pip install mcp, one @mcp.tool() function, and mcp.run(), you get a working web_fetch(url) -> clean text tool over stdio in ~60 lines. Self-hosted, free, and returning text instead of raw HTML. The work is in the guardrails: timeout, size cap, and an SSRF check.
This is for anyone building agents or RAG who keeps hitting "give your model live web access — here's our API." If your target is docs, articles, RSS, or JSON endpoints that answer a plain GET, you don't have to pay for that.
The artifact first: what the agent actually receives
Here's the real round-trip (terminal output, reformatted for readability — the raw print() repr is denser). I started the server, connected an MCP client over stdio, asked it to list tools, then called web_fetch on example.com:
=== TOOLS THE AGENT SEES ===
- web_fetch: Fetch a public web page and return clean readable text (no raw HTML).
=== call_tool web_fetch('https://example.com') ===
isError: False
Example Domain Example Domain This domain is for use in documentation examples
without needing permission. Avoid use in operations. Learn more
That's it. The agent asked for a URL and got back readable prose — not a wall of <div> soup. No tags, no <style> block, no nav chrome.
Stack I ran this on: mcp 1.27.2 (pip show mcp, installed 2026-06-11), httpx 0.28.1, Python 3.13.5. The MCP SDK API moves between versions, so I'll flag the parts that matter as we go.
Why free / self-hosted, and why I'm writing this
Here's the context. Pierluigi Vinciguerra runs The Web Scraping Club — one of the most-read voices in this niche. On 2026-06-07 he published a walkthrough titled, roughly, how to give Claude real-time web access with the Decodo MCP. Good post. But Decodo is a paid service. The same week, an HN front-pager pitched a "Bot Browser" MCP server that "saves 90% of tokens." The demand is obvious. The default answer everyone reaches for is a vendor.
For a big chunk of cases, that's overkill.
"Self-hosted" means the server runs as a local process you started. The traffic goes out from your IP, the rate limits are your own, and no third party logs which URLs your agent reads. For internal docs, public APIs, blog posts, changelogs, RSS — a plain GET is all you need, and a vendor in the middle is a cost and a dependency you didn't have to take on.
I'll be straight about the boundary, because this is where honesty matters: this server does not beat anti-bot systems. No headless browser, no fingerprint rotation, no JavaScript execution. Hit a Cloudflare-challenged or JS-rendered page and it returns nothing useful. That's a different tool for a different day. What this does cover is the long, boring, very common tail of sites that just answer.
The server, line by line
pip install mcp httpx and you're ready. The whole thing is one file.
The shape: create a FastMCP instance, decorate a function with @mcp.tool(), and the SDK turns the function's signature and docstring into a tool schema the agent can discover. Run it over stdio with mcp.run().
# server.py — runnable local: pip install mcp httpx → python server.py
import ipaddress, re, socket
from urllib.parse import urlparse
import httpx
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("web-fetch")
MAX_CHARS = 8000 # guardrail: don't blow up the agent's context window
@mcp.tool()
def web_fetch(url: str) -> str:
"""Fetch a public web page and return clean readable text (no raw HTML).
Use this when you need the current contents of a URL."""
_is_public_http_url(url)
headers = {"User-Agent":
"agent-web-fetch/0.1 (+https://blog.spinov.online; contact you@example.com)"}
with httpx.Client(headers=headers, timeout=15.0, follow_redirects=True) as client:
resp = client.get(url)
resp.raise_for_status() # 4xx/5xx -> raise, agent sees a real error
text = _html_to_text(resp.text)
if len(text) > MAX_CHARS:
text = text[:MAX_CHARS] + f"\n\n[truncated at {MAX_CHARS} chars]"
return text
if __name__ == "__main__":
mcp.run() # stdio transport by default
The two helpers (_is_public_http_url, _html_to_text) and imports round it out to exactly 60 lines. The full file is at the end. The docstring on web_fetch is not a comment — it becomes the tool description the model reads when deciding whether to call it, which is why the list_tools output above echoes that first line back. Write it for the agent, not for you.
A version note, because this bites people: I ran this on mcp 1.27.2. On that version from mcp.server.fastmcp import FastMCP, the @mcp.tool() decorator, and mcp.run() all exist and behave as shown. The low-level Server API and some helper signatures have shifted across releases — if you're on something older or newer, check pip show mcp and lean on the declarative FastMCP path. It's the most stable surface between versions, and it's what keeps this under sixty lines.
Talking to it from a client
To prove the agent can actually reach the tool, I connected over stdio with the SDK's own client:
# client.py — runnable local
import asyncio, sys
from mcp.client.session import ClientSession
from mcp.client.stdio import StdioServerParameters, stdio_client
async def main():
params = StdioServerParameters(command=sys.executable, args=["server.py"])
async with stdio_client(params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
print(await session.list_tools()) # tool is discoverable
print(await session.call_tool("web_fetch",
{"url": "https://example.com"}))
asyncio.run(main())
One snag worth saving you: I first wrote command="python", and on a clean venv that raised FileNotFoundError: 'python' — the binary wasn't on PATH, only python3 was. sys.executable points at the interpreter already running, so it just works. Small thing, ten minutes lost. Prefer it over a bare "python".
Don't want to write a client? npx @modelcontextprotocol/inspector python server.py gives you a UI to poke the tool, or you register server.py as a local MCP server in Claude Desktop / Claude Code and call it from there.
The four defaults that aren't decoration
This is the part I actually care about, and the reason I bothered writing instead of just linking the quickstart.
The naive version of this tool is three lines: httpx.get(url).text, return it, done. It demos fine. Then you point a real agent at the open web and it falls over in ways the quickstart never warned you about. These four defaults come straight off our scraping fleet — across roughly 2,190 production runs on 32 published actors (our Trustpilot scraper alone has logged 962 runs), and every one of these earned its place by causing pain when it was missing.
1. A polite, contactable User-Agent. The default httpx/python-requests UA is a fast way to get silently throttled or blocked. A UA that says who you are and how to reach you is the cheapest goodwill there is — and on a few of our actors it was the single line that flipped a site from 403 to 200. Feature → so what: your agent stops getting ghosted by servers that block unknown bots.
2. timeout + raise_for_status(). An agent that hangs on a slow server is worse than one that errors — it freezes the whole tool call with no signal. A 15-second timeout plus raising on 4xx/5xx means a bad URL surfaces as a real error the agent can react to, instead of an empty string it confidently treats as "the page said nothing." Silent garbage is the expensive failure mode; I've watched a scraper return empty arrays for days because nobody raised on a 429.
3. A size cap. MAX_CHARS = 8000 is a guardrail against your agent's context window, not the network. Some pages are enormous. Without a cap, one fetch of a bloated page can eat half your context and your budget with it. Truncating with a visible marker is honest and bounded. Tune the number to your model; the principle doesn't change.
4. Clean text instead of raw HTML. The tool strips tags and returns prose. On example.com — a trivial page — that took the payload from 559 raw HTML characters down to 142 of clean text, about 3.9× smaller (one page; real sites skew far higher because of nav, scripts, and inline styling). Why it matters for your token bill is its own rabbit hole, and I already measured it on real pages in Raw HTML Is a Token Tax — I Measured It. Short version: agents pay for every HTML character they're handed and read almost none of it. Hand them text.
The guardrail people skip: SSRF
One more, and it's the one I'd flag in code review. A web_fetch tool an LLM controls is a request your model can aim anywhere — including http://169.254.169.254/, the cloud metadata endpoint that leaks credentials, or http://localhost:6379 to poke your Redis. That's Server-Side Request Forgery, and an agent can be talked into it by a malicious page telling it to "fetch this URL."
So before any request, the server resolves the host and refuses private, loopback, link-local, and reserved addresses. Real output, same run:
=== call_tool web_fetch('http://169.254.169.254/') ===
isError: True
Error executing tool web_fetch: refusing to fetch a private/internal address
That's the metadata IP getting turned away. The check is ~10 lines and it's not optional if the URL can come from a model. It is not a complete SSRF defense — DNS rebinding and redirect-to-internal are still live concerns, and follow_redirects=True means you'd want to re-check the final hop in anything serious. But refusing the obvious internal targets is the floor, and most toy fetch tools don't even have that.
And the de-tagger is honest about being a toy. The regex _html_to_text is fine for a demo; it is not a real content extractor. For production, swap it for trafilatura or readability-lxml, which actually find the article body and drop boilerplate. I left the regex in so the file stays one dependency and sixty lines — but I'm telling you it's the first thing to replace.
What you've got, and the honest edges
Sixty lines, pip install mcp httpx, and your agent has a web_fetch tool that returns readable text, identifies itself politely, won't hang, won't flood your context, and refuses a direct metadata-IP URL. Free. On your machine. No vendor.
Where it stops, plainly: no JavaScript, no anti-bot evasion, no proxies, a toy extractor by default, and an SSRF guard that's a floor, not a fortress. For the docs/API/article tail, that's plenty. For Cloudflare-walled or JS-heavy targets, you're in headless-browser territory — a separate post.
The source for server.py (exactly 60 lines, the file I ran) is below, verified against mcp 1.27.2.
# server.py — a minimal MCP server that gives an AI agent ONE tool: web_fetch.
# runnable local: pip install mcp httpx → python server.py
# The defaults here (UA, timeout, redirects, raise_for_status, size cap) are the
# same ones I run across our scraping fleet — not decoration, they stop real pain.
import ipaddress
import re
import socket
from urllib.parse import urlparse
import httpx
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("web-fetch")
MAX_CHARS = 8000 # guardrail: don't blow up the agent's context window
def _is_public_http_url(url: str) -> None:
"""Reject non-http(s) and private/loopback targets (a small SSRF guard)."""
p = urlparse(url)
if p.scheme not in ("http", "https"):
raise ValueError("only http/https URLs are allowed")
host = p.hostname or ""
try:
ip = ipaddress.ip_address(socket.gethostbyname(host))
except (socket.gaierror, ValueError):
raise ValueError(f"cannot resolve host: {host}")
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
raise ValueError("refusing to fetch a private/internal address")
def _html_to_text(html: str) -> str:
"""Toy de-tag: good enough for a demo, NOT a real extractor.
For production use trafilatura or readability-lxml instead."""
html = re.sub(r"<(script|style|noscript)[^>]*>.*?</\1>", " ", html, flags=re.S | re.I)
text = re.sub(r"<[^>]+>", " ", html)
return re.sub(r"\s+", " ", text).strip()
@mcp.tool()
def web_fetch(url: str) -> str:
"""Fetch a public web page and return clean readable text (no raw HTML).
Use this when you need the current contents of a URL."""
_is_public_http_url(url)
headers = {
# Identify yourself. A polite, contactable UA is the cheapest way to
# not get silently blocked — and it's the right thing to do.
"User-Agent": "agent-web-fetch/0.1 (+https://blog.spinov.online; contact you@example.com)"
}
with httpx.Client(headers=headers, timeout=15.0, follow_redirects=True) as client:
resp = client.get(url)
resp.raise_for_status() # 4xx/5xx -> raise, so the agent sees a real error
text = _html_to_text(resp.text)
if len(text) > MAX_CHARS:
text = text[:MAX_CHARS] + f"\n\n[truncated at {MAX_CHARS} chars]"
return text
if __name__ == "__main__":
mcp.run() # stdio transport by default
I keep coming back to one design question and don't have a clean answer: when you hand an agent a tool, how much should the tool enforce versus how much you trust the model to behave? I put the SSRF check and size cap in the tool because I don't trust prompt-level rules to hold under adversarial input. But that bloats every tool with guardrail code. Where do you draw that line — in the tool, in a sandbox around it, or in the agent's policy?
What's the first tool you'd hand your agent — and what guardrail would you refuse to ship without? 👇
Follow for the next teardown from our production runs. I read every comment.
Written with AI assistance; all code was run and every output above is real terminal output, not generated. Tested on mcp 1.27.2 / Python 3.13.5 on 2026-06-11. Source: the official MCP docs at modelcontextprotocol.io.