agentfetch-mcp
Web intelligence for AI agents — an MCP server that fetches URLs with token estimation, smart caching, and intelligent routing built in.
AgentFetch sits between your agent and the open web. Instead of integrating Jina, FireCrawl, pypdf, and your own caching layer separately, agents call one MCP tool and AgentFetch handles routing, caching, token budgeting, and clean Markdown extraction automatically.
This repository contains the open-source MCP server. For the hosted API + dashboard + billing, see www.agentfetch.dev.
What it does
| Tool | What it's for |
|---|---|
fetch_url |
Fetch a URL → clean Markdown + metadata + token count + cache info |
estimate_tokens |
Get a token count before fetching, so agents don't blow context windows on huge pages |
fetch_multiple |
Fetch up to 20 URLs concurrently |
search_and_fetch |
Web search + fetch top N results in one round-trip |
Under the hood, AgentFetch routes URLs to the cheapest effective fetcher:
- Trafilatura (free, local) for ~70% of standard web pages
- Jina Reader for the rest of HTML
- FireCrawl for JS-heavy pages (Twitter/X, LinkedIn, Notion, etc.)
- pypdf for PDFs (zero external cost)
