huggingface.papers_search

curl "https://api.onekey.dev/v1/huggingface/papers_search?q=attention" \
  -H "x-api-key: YOUR_API_KEY"

{
  "success": true,
  "data": [
    {
      "content": {
        "id": "1706.03762",
        "title": "Attention Is All You Need",
        "text": "The dominant sequence transduction models are based on complex recurrent or\nconvolutional neural networks in an encoder-decoder configuration. The best\nperforming models also connect the encoder and decoder through an attention\nmechanism. We propose a new simple network architecture, the Transformer, based\nsolely on attention mechanisms, dispensing with recurrence and convolutions\nentirely. Experiments on two machine translation tasks show these models to be\nsuperior in quality while being more parallelizable and requiring significantly\nless time to train. Our model achieves 28.4 BLEU on the WMT 2014\nEnglish-to-German translation task, improving over the existing best results,\nincluding ensembles by over 2 BLEU. On the WMT 2014 English-to-French\ntranslation task, our model establishes a new single-model state-of-the-art\nBLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction\nof the training costs of the best models from the literature. We show that the\nTransformer generalizes well to other tasks by applying it successfully to\nEnglish constituency parsing both with large and limited training data.",
        "createdAt": "2017-06-12T13:57:34Z"
      },
      "engagement": {
        "likes": 143,
        "comments": 14
      },
      "media": {
        "thumbnailUrl": "https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/1706.03762.png"
      }
    },
    {
      "content": {
        "id": "2108.12409",
        "title": "Train Short, Test Long: Attention with Linear Biases Enables Input\n  Length Extrapolation",
        "text": "Since the introduction of the transformer model by Vaswani et al. (2017), a\nfundamental question has yet to be answered: how does a model achieve\nextrapolation at inference time for sequences that are longer than it saw\nduring training? We first show that extrapolation can be enabled by simply\nchanging the position representation method, though we find that current\nmethods do not allow for efficient extrapolation. We therefore introduce a\nsimpler and more efficient position method, Attention with Linear Biases\n(ALiBi). ALiBi does not add positional embeddings to word embeddings; instead,\nit biases query-key attention scores with a penalty that is proportional to\ntheir distance. We show that this method trains a 1.3 billion parameter model\non input sequences of length 1024 that extrapolates to input sequences of\nlength 2048, achieving the same perplexity as a sinusoidal position embedding\nmodel trained on inputs of length 2048 but training 11% faster and using 11%\nless memory. ALiBi's inductive bias towards recency also leads it to outperform\nmultiple strong position methods on the WikiText-103 benchmark.",
        "createdAt": "2021-08-27T13:35:06Z"
      },
      "engagement": {
        "likes": 5,
        "comments": 0
      },
      "media": {
        "thumbnailUrl": "https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2108.12409.png"
      }
    }
  ],
  "metadata": {
    "platform": "huggingface",
    "endpoint": "papers_search",
    "cached": false,
    "stale": false,
    "age_s": 0,
    "fetched_at": "2026-01-01T12:00:00+00:00",
    "credits_charged": 5,
    "request_id": "7f3a9c2e5b1d4c8fa2e6d0b9c4f18e37"
  },
  "error": null
}