# web.unlocked_markdown

**beta** **8 credits** **needs a pack**

Markdown of a page that is hard to reach from ordinary servers.

`GET /v1/web/unlocked_markdown`

> [!NOTE] **Beta.** This endpoint has not yet been verified against the live source, so fields may be missing or differ until it becomes stable. Failed calls are refunded.

> [!NOTE] **Needs a pack.** Needs a purchased pack and a verified email; free credits do not reach this endpoint. See [pricing](https://onekey.dev/pricing).

## What you get

One Web record in Onekey's unified shape, in `data`. If the record does not exist you get a `404 not_found` and are not charged.

## What it costs

- **8 credits** for a fresh fetch, charged only when it succeeds.
- **Free when served from cache (0.2 credit for a cached response over 100 KB).**
- **Free** if the request is invalid (never charged) or the source fails or reports not found (refunded).

## Parameters

| Name | Type | Required | Default | What it is | Constraints |
| --- | --- | --- | --- | --- | --- |
| `url` | string | yes | (required) | Absolute http(s) URL of the page (ports 80/443 only; private/internal hosts are refused) | max 2048 chars; must match `https?://[A-Za-z0-9](?:[A-Za-z0-9.-]{0,251}[A-Za-z0-9])?(?::(?:80\|443))?(?:[/?][\x21-\x5b\x5d-\x7e]*)?`; trimmed |

## Request

```bash
curl "https://api.onekey.dev/v1/web/unlocked_markdown?url=https%3A%2F%2Fwww.python.org%2F" \
  -H "x-api-key: YOUR_API_KEY"
```

```python
import requests

res = requests.get(
    "https://api.onekey.dev/v1/web/unlocked_markdown",
    params={"url": "https://www.python.org/"},
    headers={"x-api-key": "YOUR_API_KEY"},
)
body = res.json()

if body["success"]:
    print(body["data"])
else:
    print(body["error"]["code"], body["error"]["message"])
```

```javascript
const params = new URLSearchParams({ url: "https://www.python.org/" });
const res = await fetch(`https://api.onekey.dev/v1/web/unlocked_markdown?${params}`, {
  headers: { "x-api-key": "YOUR_API_KEY" },
});
const body = await res.json();

if (body.success) console.log(body.data);
else console.error(body.error.code, body.error.message);
```

## Recorded response

A real Web response, recorded and returned in Onekey's shape. Fields the platform does not provide are `null`.

```json
{
  "success": true,
  "data": {
    "content": {
      "title": "Web scraping - Wikipedia",
      "text": "# Web scraping\n\nFrom Wikipedia, the free encyclopedia\n\nFor broader coverage of this topic, see [Data scraping](https://en.wikipedia.org/wiki/Data_scraping).\n\n\"Web scraper\" redirects here. For websites that scrape content, see [Scraper site](https://en.wikipedia.org/wiki/Scraper_site).\n\n|  | This article needs more citations. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: \"Web scraping\" – news · newspapers · books · scholar · JSTOR (April 2023) (Learn how and when to remove this message) |\n| --- | --- |\n\n**Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](https://en.wikipedia.org/wiki/Data_extraction) from [websites](https://en.wikipedia.org/wiki/Website).[1] Web scraping software may directly access the [World Wide Web](https://en.wikipedia.org/wiki/World_Wide_Web) using the [Hypertext Transfer Protocol](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol) or a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a [bot](https://en.wikipedia.org/wiki/Internet_bot) or [web crawler](https://en.wikipedia.org/wiki/Web_crawler). It is a form of copying in which specific data is gathered and copied from the web, typically into a central local [database](https://en.wikipedia.org/wiki/Database) or [spr",
      "url": "https://en.wikipedia.org/wiki/Web_scraping",
      "language": "en",
      "createdAt": "2005-09-17T18:57:30Z"
    },
    "author": {
      "displayName": "Contributors to Wikimedia projects"
    },
    "ext": {
      "wordCount": 6206
    }
  },
  "metadata": {
    "platform": "web",
    "endpoint": "unlocked_markdown",
    "cached": false,
    "stale": false,
    "age_s": 0,
    "fetched_at": "2026-01-01T12:00:00+00:00",
    "credits_charged": 8,
    "request_id": "7f3a9c2e5b1d4c8fa2e6d0b9c4f18e37"
  },
  "error": null
}
```

## Field mapping

How this endpoint's source fields become Onekey's [unified schema](https://onekey.dev/docs/schema.md). Fields the source lacks come back as `null`.

| Source path | Onekey field | Transform |
| --- | --- | --- |
| `title` | `content.title` | - |
| `markdown` | `content.text` | - |
| `finalUrl` | `content.url` | `safeUrl` |
| `language` | `content.language` | - |
| `publishedAt` | `content.createdAt` | `toIsoDate` |
| `byline` | `author.displayName` | - |
| `wordCount` | `ext.wordCount` | `toNumber` |

## Caching and freshness

- Fresh for about 10 minutes. Repeat calls in that window are served from cache and are free.
- Between 10 minutes and 1 hour the cached copy is still served instantly and marked `stale: true` while Onekey refreshes it in the background.
- If the source is down when a refresh is needed, the last copy can be served for up to 1 hour more (marked stale).
- A "not found" is remembered for 2 minutes and is free.
- The cache is shared across all customers, which is what keeps repeat calls cheap.

## What can go wrong

| Status | Code | What to do |
| --- | --- | --- |
| 401 | `unauthorized` | Check the `x-api-key` header. |
| 402 | `insufficient_credits` | [Add credits](https://onekey.dev/pricing); your balance must cover the price. |
| 404 | `not_found` | That record does not exist. Free; do not retry. |
| 422 | `invalid_params` | Fix the parameters using the table above. Free. |
| 429 | `rate_limited` | Wait `Retry-After` seconds, then retry. |
| 502 | `upstream_error` | The source failed. Refunded; retry with backoff. |

See [Errors](https://onekey.dev/docs/errors.md) for the full list and retry guidance.

Status: beta (not yet verified against the live source).