The Data Collector
Web scraping APIs for Bluesky, Substack, and Hacker News with x402 micropayment support. Built with FastAPI.
Live: https://frog03-20494.wykr.es
Features
- Search Bluesky posts (AT Protocol), Substack newsletters, and Hacker News stories
- Returns structured JSON with engagement metrics
- x402 micropayments ($0.05 USDC on Base per call) — no account needed
- API key authentication for regular use
- A2A Agent Card and MCP discovery endpoints
- OpenAPI spec with x402 payment metadata
API Endpoints
| Method | Endpoint | Description | Price |
|---|---|---|---|
| POST | /api/bluesky/search |
Search Bluesky posts by keyword | $0.05 |
| POST | /api/substack/search |
Scrape Substack newsletter articles | $0.05 |
| POST | /api/hn/search |
Search Hacker News stories | $0.05 |
Quick Start
code hiddenDocker
code hiddenEnvironment Variables
| Variable | Required | Description |
|---|---|---|
APIFY_TOKEN |
Yes | Apify API token for running scrapers |
API_KEY |
No | API key for authenticated access (X-API-Key header) |
BASE_URL |
No | Public URL of the server (default: https://frog03-20494.wykr.es) |
PAY_TO |
No | Wallet address for x402 payments |
PRICE_ATOMIC |
No | Price per call in USDC atomic units (default: 50000 = $0.05) |
