# Common Crawl on Onekey

> 2 endpoints for Common Crawl (2 stable, 0 beta), 1 to 5 credits each. Category: Web tools. One key, one JSON shape.

Base URL: https://api.onekey.dev
Auth: header `x-api-key: <key>`. Call shape: `GET /v1/{platform}/{endpoint}?{params}`.
Every response is `{success, data, metadata, error}`; check `success` first. Cost is in `metadata.credits_charged`. Free when served from cache (0.2 credit for a cached response over 100 KB). Failed and not-found calls are refunded.
Status: stable = verified against the live source; beta = not yet verified, fields may be missing.

Page: https://onekey.dev/platforms/commoncrawl
Reference (Markdown): https://onekey.dev/docs/api/commoncrawl.md
All docs: https://onekey.dev/llms.txt

## Canonical fields returned

Unified-schema fields (same names on every platform), with how many of the 2 mapped Common Crawl endpoints return each:

- `entity.createdAt` (ISO 8601 date): 1
- `entity.id` (string): 1
- `entity.language` (string): 1
- `entity.name` (string): 1
- `entity.url` (URL): 1
- plus 8 platform-specific fields under `ext.`

## Endpoints

### commoncrawl.crawls

List Common Crawl crawl collections (id, name, date range, CDX API URL).
- Price: 1 credit per fresh call. Status: stable. Returns: a list.
- Request: `GET https://api.onekey.dev/v1/commoncrawl/crawls` with header `x-api-key: YOUR_API_KEY`
- Parameters: none
- Fields returned: `entity.id`, `entity.name`, `ext.from`, `ext.to`, `ext.cdxApi`
- Reference: https://onekey.dev/docs/api/commoncrawl/crawls.md

### commoncrawl.index

Common Crawl index captures for a URL or domain in one crawl.
- Price: 5 credits per fresh call. Status: stable. Returns: a list.
- Request: `GET https://api.onekey.dev/v1/commoncrawl/index?url=example.com` with header `x-api-key: YOUR_API_KEY`
- Parameters:
  - `url` (string, required): URL or domain (wildcards not allowed)
  - `crawl` (string, optional, default CC-MAIN-2026-39): Crawl id, e.g. CC-MAIN-2026-39 (see commoncrawl.crawls)
- Fields returned: `entity.url`, `entity.createdAt`, `entity.language`, `ext.httpStatus`, `ext.mimeType`, `ext.filename`, `ext.offset`, `ext.length`
- Reference: https://onekey.dev/docs/api/commoncrawl/index.md

## Sample request

```bash
curl "https://api.onekey.dev/v1/commoncrawl/crawls" \
  -H "x-api-key: YOUR_API_KEY"
```