API reference

Microcrawl API

Same data as the web UI, over an HTTP API you can call from any language. Bearer-token auth, JSON responses, CSV exports. All endpoints served from https://api.microcrawl.com.

Auth

Create an API key from your account page. The plaintext key is shown once; store it somewhere safe (we only keep the hash). Include it on every request as a bearer token:

curl https://api.microcrawl.com/api/query \
  -H "Authorization: Bearer hc_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain":"stripe.com"}'

Keys authenticate as the user that created them. Active access (lifetime purchase or monthly API subscription) is required.

Endpoints

POST/api/query

Return the top linking domains for a target, ranked by host breadth.

Request body:

{ "domain": "stripe.com" }

Response:

{
  "domain": "stripe.com",
  "releaseID": "cc-main-2026-jan-feb-mar",
  "totalLinking": 702919,
  "results": [
    { "linkingDomain": "github.com",    "numHosts": 41823 },
    { "linkingDomain": "wordpress.com", "numHosts": 28410 }
    // ... up to 10,000 rows
  ]
}
POST/api/intersect

Find domains that link to every competitor but not to you.

Request body:

{
  "yourDomain": "my.site",
  "competitors": ["ahrefs.com", "semrush.com", "moz.com"]
}

Response:

{
  "yourDomain": "my.site",
  "competitors": ["ahrefs.com","semrush.com","moz.com"],
  "releaseID": "cc-main-2026-jan-feb-mar",
  "totalMatching": 3412,
  "missing": [],
  "results": [
    { "linkingDomain": "mozcommunity.com", "numHosts": 412 }
    // ...
  ]
}
POST/api/diff

Quarter-over-quarter change: linkers gained vs. lost since the previous release.

Request body:

{ "domain": "stripe.com" }

Response:

{
  "domain": "stripe.com",
  "currentRelease": "cc-main-2026-jan-feb-mar",
  "previousRelease": "cc-main-2025-oct-nov-dec",
  "currentTotal": 702919,
  "previousTotal": 706206,
  "gainedTotal": 138112,
  "lostTotal": 141399,
  "gained": [ /* up to 10,000 */ ],
  "lost":   [ /* up to 10,000 */ ]
}
POST/api/batch/csv

Bulk lookup: up to 50 domains in one call. Returns a single CSV with columns queried_domain, total_linking, linking_domain, num_hosts. Each domain costs one query against your rate limit.

Request body:

{ "domains": ["stripe.com", "github.com", "vercel.com"] }
GET/api/query/csv?domain=stripe.com

Full result set as a CSV download. Same auth. No row cap.

GET/api/intersect/csv?you=my.site&c=ahrefs.com&c=semrush.com

CSV export of an intersect. Repeat ?c= for each competitor.

GET/api/diff/gained.csv?domain=stripe.com

Gained linkers as CSV. Pair with /api/diff/lost.csv for the other half.

GET/api/release

No auth. Returns the current release ID + build time. Useful for drift detection.

Rate limits

Limits are per-account (shared across all your keys), not per-key. The defaults:

  • 60 queries per minute
  • 10,000 queries per day
  • Plus a per-IP ceiling of 120 queries/min across all users

Over the limit returns 429 Too Many Requests. Queries returning from cache don't count; we don't currently cache.

Errors

Errors are JSON with a consistent shape:

{ "error": { "message": "invalid domain" } }
  • 401 — missing or invalid bearer token
  • 403 — valid key but no active access
  • 400 — bad input (invalid domain, missing field)
  • 429 — rate limit
  • 503 — transient (Firebase key refresh failed, etc.)

Sample: Python

import os, requests

KEY = os.environ["MICROCRAWL_KEY"]
r = requests.post(
    "https://api.microcrawl.com/api/query",
    headers={"Authorization": f"Bearer {KEY}"},
    json={"domain": "stripe.com"},
    timeout=30,
)
r.raise_for_status()
data = r.json()
print(data["totalLinking"], "linking domains")
for row in data["results"][:20]:
    print(row["linkingDomain"], row["numHosts"])

MCP server (Claude, Cursor, etc.)

Use Microcrawl from inside an LLM chat via the Model Context Protocol server: microcrawl-mcp. Install once in Claude Desktop or any MCP-compatible client, then ask things like “how visible is stripe.com in LLM training data?” — the assistant picks the right tool, hits the API, and reasons over the structured result.

Five tools ship with the server:

  • microcrawl_query — top linkers for a target
  • microcrawl_ai_visibility — % of linkers on the LLM-source allowlist, with tier breakdown
  • microcrawl_intersect — competitor link-gap
  • microcrawl_diff — quarter-over-quarter change
  • microcrawl_release — historical-vault catalog

Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "microcrawl": {
      "command": "npx",
      "args": ["-y", "microcrawl-mcp"],
      "env": {
        "MICROCRAWL_API_KEY": "hc_live_YOUR_KEY"
      }
    }
  }
}

Same API key as the HTTP endpoints. Same rate limits. Source on GitHub under mcp/.

Stability

Endpoints are stable. We version breaking changes via a new path (e.g. /api/v2/…) so your existing scripts keep working forever. Field additions are non-breaking and land without notice.

Questions? Email [email protected].