# web.sitemap

**stable** **2 credits**

Parse an XML/gzip/text sitemap or sitemap index into URL entries.

`GET /v1/web/sitemap`

## What you get

A list of Web results in Onekey's unified shape, in `data` (an array). An empty array is a normal, successful answer and is charged like any other; a source that reports "nothing found" as an error returns a free `404 not_found` instead.

**Also known as:** `/on_page/page`, `/web/map` on APIs that nest their paths. See [Alternatives and equivalents](https://onekey.dev/docs/alternatives.md).

## What it costs

- **2 credits** for a fresh fetch, charged only when it succeeds.
- **Free when served from cache (0.2 credit for a cached response over 100 KB).**
- **Free** if the request is invalid (never charged) or the source fails or reports not found (refunded).

## Parameters

| Name | Type | Required | Default | What it is | Constraints |
| --- | --- | --- | --- | --- | --- |
| `url` | string | yes | (required) | Absolute http(s) URL of the page (ports 80/443 only; private/internal hosts are refused) | max 2048 chars; must match `https?://[A-Za-z0-9](?:[A-Za-z0-9.-]{0,251}[A-Za-z0-9])?(?::(?:80\|443))?(?:[/?][\x21-\x5b\x5d-\x7e]*)?`; trimmed |

## Request

```bash
curl "https://api.onekey.dev/v1/web/sitemap?url=https%3A%2F%2Fwww.bbc.com%2Fsitemap.xml" \
  -H "x-api-key: YOUR_API_KEY"
```

```python
import requests

res = requests.get(
    "https://api.onekey.dev/v1/web/sitemap",
    params={"url": "https://www.bbc.com/sitemap.xml"},
    headers={"x-api-key": "YOUR_API_KEY"},
)
body = res.json()

if body["success"]:
    print(body["data"])
else:
    print(body["error"]["code"], body["error"]["message"])
```

```javascript
const params = new URLSearchParams({ url: "https://www.bbc.com/sitemap.xml" });
const res = await fetch(`https://api.onekey.dev/v1/web/sitemap?${params}`, {
  headers: { "x-api-key": "YOUR_API_KEY" },
});
const body = await res.json();

if (body.success) console.log(body.data);
else console.error(body.error.code, body.error.message);
```

## Recorded response

A real Web response, recorded and returned in Onekey's shape. Fields the platform does not provide are `null`.

```json
{
  "success": true,
  "data": [
    {
      "entity": {
        "url": "https://www.bbc.com/scotland",
        "updatedAt": null
      },
      "ext": {
        "changeFrequency": "always",
        "priority": 0.5,
        "kind": "page"
      }
    },
    {
      "entity": {
        "url": "https://www.bbc.com/wales",
        "updatedAt": null
      },
      "ext": {
        "changeFrequency": "always",
        "priority": 0.5,
        "kind": "page"
      }
    }
  ],
  "metadata": {
    "platform": "web",
    "endpoint": "sitemap",
    "cached": false,
    "stale": false,
    "age_s": 0,
    "fetched_at": "2026-01-01T12:00:00+00:00",
    "credits_charged": 2,
    "request_id": "7f3a9c2e5b1d4c8fa2e6d0b9c4f18e37"
  },
  "error": null
}
```

## Field mapping

How this endpoint's source fields become Onekey's [unified schema](https://onekey.dev/docs/schema.md). Fields the source lacks come back as `null`.

| Source path | Onekey field | Transform |
| --- | --- | --- |
| `url` | `entity.url` | `safeUrl` |
| `lastModified` | `entity.updatedAt` | `toIsoDate` |
| `changeFrequency` | `ext.changeFrequency` | - |
| `priority` | `ext.priority` | `toNumber` |
| `kind` | `ext.kind` | - |

## Caching and freshness

- Fresh for about 10 minutes. Repeat calls in that window are served from cache and are free.
- Between 10 minutes and 1 hour the cached copy is still served instantly and marked `stale: true` while Onekey refreshes it in the background.
- If the source is down when a refresh is needed, the last copy can be served for up to 1 hour more (marked stale).
- A "not found" is remembered for 2 minutes and is free.
- The cache is shared across all customers, which is what keeps repeat calls cheap.

## What can go wrong

| Status | Code | What to do |
| --- | --- | --- |
| 401 | `unauthorized` | Check the `x-api-key` header. |
| 402 | `insufficient_credits` | [Add credits](https://onekey.dev/pricing); your balance must cover the price. |
| 404 | `not_found` | Unknown endpoint path. |
| 422 | `invalid_params` | Fix the parameters using the table above. Free. |
| 429 | `rate_limited` | Wait `Retry-After` seconds, then retry. |
| 502 | `upstream_error` | The source failed. Refunded; retry with backoff. |

See [Errors](https://onekey.dev/docs/errors.md) for the full list and retry guidance.

Status: stable (verified against the live source).