A fast Rust-built web-to-Markdown fetcher for AI agents.
npx chidori-fetch https://example.com
chidori fetches a URL, extracts the main readable content, removes page noise, and prints Markdown to stdout by default. Logs and errors go to stderr so AI agents and shell scripts can safely pipe stdout into prompts, files, or other tools.
Quick demo
Web pages are built for browsers. Agents usually need the readable content, not the page chrome around it.
npx chidori-fetch https://taishikato.com
 I'm Taishi, a fullstack developer and solofounder based in Japan and Canada. Full-stack dev shipping AI infrastructure & tools to 18K+ users (Supabase hackathon winner). Two of them were ranked as the [#1 Product of the day on Product Hunt](https://www.producthunt.com/products/supavec#supavec). ========== [Hire me](/cdn-cgi/l/email-protection#493d28203a212067227970797a092e24282025672a2624) [My Work](/)[Stack](/stack)[Blog](/blog) [](https://www.supavec.com) ### Supavec ### OSS RAG as a Service - spin up vector search + chat API in \<5 min. 630▲ Product Hunt, 620⭐ GitHub. [supavec.com →](https://www.supavec.com) ## AI Tooling & Infrastructure ### [Supavec](https://www.supavec.com) ### \#RAG#Supabase#OpenSource Open-source RAG platform using pgvector. ### [MCP Server](https://www.supavec.com/blog/supavec-mcp-server) ### \#MCP#RAG#OpenSource MCP server streaming Supabase-secured RAG data to LLMs. ### [CiteAnalytics](https://x.com/supabase/status/1909552555333398630) ### \#OpenSource#Hackathon Supabase hackathon winning AI citation analytics app. ## Work Experience <img alt="Supabase" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/supabase.92465ed1.webp" width="44"> Supabase Frontend Developer (contract)Remote Here are [my PRs](https://github.com/supabase/supabase/issues?q=author%3Ataishikato) Jul 2025 - Sep 2025 <img alt="Whisperit" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/whisperit.302305fe.webp" width="44"> Whisperit Full Stack Developer (freelance)Switzerland (Remote) Aug 2024 - Jan 2025 <img alt="Semios" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/semios.1e5ff783.webp" width="44"> Semios Full Stack JS Developer (full-time)Vancouver 2020 - 2023 <img alt="Yahoo! Japan" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/yahoo.2fd5bb07.webp" width="44"> Yahoo! Japan Software Developer (full-time)Tokyo 2015 - 2019 <img alt="btrax" class="rounded-full w-11 grayscale" height="44" loading="lazy" src="https://taishikato.com/_next/static/media/btrax.204b56b0.webp" width="44"> btrax Software Developer (intern)San Francisco Apr 2013 - Jun 2013 ## Hackathon Wins [Supabase Hackathon for Launch Week 14 Winner Apr 2025](https://x.com/supabase/status/1909552555333398630) [Supabase Hackathon for Launch Week 6 Runner Up - Best Storage Project Jan 2023](https://supabase.com/blog/launch-week-6-hackathon-winners#runner-up-2) [Supabase Hackathon for Launch Week 5 Winner - Most Visually Pleasing Aug 2022](https://supabase.com/blog/launch-week-5-hackathon-winners#winner-1) ## OSS Contributions [Pull Request #79 · supabase/embeddings-generator](https://github.com/supabase/embeddings-generator/pull/79) [Pull Request #137 · langchain-ai/langchain-community](https://github.com/langchain-ai/langchain-community/pull/137) [Pull Request #30 · ueberdosis/tiptap-docs](https://github.com/ueberdosis/tiptap-docs/pull/30) [Pull Request #60822 · vercel/next.js](https://github.com/vercel/next.js/pull/60822) [Pull Request #19009 · supabase/supabase](https://github.com/supabase/supabase/pull/19009) [Pull Request #340 · modal-labs/modal-examples](https://github.com/modal-labs/modal-examples/pull/340) [Pull Request #1799 · nuxt/docs](https://github.com/nuxt/docs/pull/1799) [Pull Request #1797 · nuxt/docs](https://github.com/nuxt/docs/pull/1797) [Pull Request #24589 · nodejs/node](https://github.com/nodejs/node/pull/24589)
That output can go straight into an agent prompt, a RAG ingestion job, a note, or another shell command:
chidori https://example.com/article | llm "summarize this for a code review" chidori https://example.com/article --json | jq -r '.title, .markdown'
Why agents use it
- Pipeable by default: Markdown goes to stdout. Logs and errors go to stderr, so scripts can trust the output stream.
- Readable content first:
chidoristrips navigation, footers, forms, hidden content, related links, script noise, and common social/page chrome. - Agent-friendly Markdown: code blocks, links, images, simple tables, footnotes, math, and callouts are preserved in text form where possible.
- Useful metadata:
--jsonreturns the final URL, canonical URL, title, description, domain, language, meta tags, schema.org data, word count, and the extracted Markdown. - Fast local CLI: it is a Rust binary with no hosted service in the loop.
JavaScript-heavy pages can opt into your own renderer with
--render=auto.
How it is different
| Tool | What you usually get |
|---|---|
curl |
Raw HTML, scripts, navigation, and layout markup. |
| Browser automation | Accurate rendering, but heavier and harder to pipe through shell workflows. |
| Readability libraries | Good extraction primitives, usually embedded inside another app or runtime. |
chidori |
One command that fetches, extracts, cleans, converts to Markdown, and keeps stdout safe for agents. |
Installation
npm install -g chidori-fetch
Usage
chidori --help chidori https://example.com chidori https://example.com --json chidori https://example.com --output article.md chidori https://example.com --max-chars 20000 chidori https://example.com --lang ja chidori https://example.com --no-images chidori https://example.com --debug CHIDORI_RENDER_COMMAND=/path/to/render-page chidori https://example.com --render=auto
--json prints metadata plus markdown as JSON. Metadata includes url,
finalUrl, canonicalUrl, domain, title/description fields, metaTags,
schemaOrgData, and wordCount. With --json --debug, stdout also includes a
debug object; human-readable debug lines still go to stderr.
--render=auto is an optional fallback for JavaScript-heavy pages. When static
extraction fails or only finds a short app shell, chidori runs
CHIDORI_RENDER_COMMAND <url> if configured and expects rendered HTML on stdout.
If the renderer is unavailable and no custom user agent was supplied, chidori
still tries its bot user-agent fallback. Renderer output uses the same timeout
and 5 MiB fetch limit.
External renderer example
One simple renderer setup is a small Playwright script:
npm install -D playwright npx playwright install chromium
Create render-page.mjs:
#!/usr/bin/env node import { chromium } from "playwright"; const url = process.argv[2]; if (!url) { console.error("Usage: render-page.mjs <url>"); process.exit(2); } const browser = await chromium.launch({ headless: true }); try { const page = await browser.newPage(); await page.goto(url, { waitUntil: "networkidle", timeout: 10_000, }); process.stdout.write(await page.content()); } finally { await browser.close(); }
Then make it executable and pass it to chidori:
chmod +x render-page.mjs CHIDORI_RENDER_COMMAND=/absolute/path/to/render-page.mjs chidori https://example.com --render=auto
The renderer should write only HTML to stdout. Write logs and errors to stderr so
chidori can safely read stdout as the rendered document.
Design goals
AI agents need web pages as clean, pipeable Markdown. chidori is built for fast
CLI startup, deterministic shell behavior, and extraction that is easy to debug
when a page does something strange.