# Hugging Face Inference API returning short generated text with GPT-2 model

**URL:** <https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078>\
**Category:** Beginners\
**Created:** [July 17, 2023, 12:18pm UTC](https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078 "2023-07-17T12:18:52Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ai-ml-app](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@ai-ml-app](https://discuss.huggingface.co/u/ai-ml-app)\
**Post date:** [July 17, 2023, 12:18pm UTC](https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078/1 "2023-07-17T12:18:52Z")

</div>

I’m using the Hugging Face API with a GPT-2 model to generate text based on a prompt. However, I’m encountering an issue where the generated text is consistently too short, even though I’m specifying a maximum number of new tokens and using other parameters to try to generate longer text.

```auto
function GPT2()
{
const prompt = "Please give me a 100 words tell me about yourself";  

const API_TOKEN = "xxxxxxxxxxxx";
const MODEL_NAME = "gpt2";

const url = `https://api-inference.huggingface.co/models/${MODEL_NAME}`;
  const options = 
  {
    method: "post",
    headers: 
    {
      Authorization: `Bearer ${API_TOKEN}`,
      "Content-Type": "application/json",
    },
    payload: JSON.stringify
    ({
      inputs: prompt,
      options: 
      {
        contentType: "application/json",
        max_tokens: 200,
        max_length: 100, 
        num_return_sequences: 3,
        use_cache: false,
        return_full_text: true
      }
    })
  };
  const response = UrlFetchApp.fetch(url, options);
  const res = JSON.parse(response.getContentText());
  Logger.log(response);

  return res[0].generated_text.trim();
}

```

For example, when I input the prompt “Please give me a 100 words tell me about yourself.”, the generated text returned is only a few words long, such as “I’m a writer, and I”. I was expecting the generated text to be much longer.

Can anyone offer any suggestions for how I can generate longer text using the Hugging Face API with a GPT-2 model? Is there a problem with my code or input parameters that is causing the generated text to be too short?

I have also checked the API response to ensure that it is valid and contains the expected input and output values.

Thank you for your help.

---

<div class="post-metadata">

**Author:** ![VatsaDev](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vatsadev/32/17184_2.png) [@VatsaDev](https://discuss.huggingface.co/u/VatsaDev)\
**Post date:** [July 17, 2023, 1:13pm UTC](https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078/2 "2023-07-17T13:13:02Z")

</div>

I’ve faced the same issue with this model,even though I was trying to prompt it with an incomplete sentence, it generates a couple words at a time, and starts repeating itself

---

<div class="post-metadata">

**Author:** ![VatsaDev](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vatsadev/32/17184_2.png) [@VatsaDev](https://discuss.huggingface.co/u/VatsaDev)\
**Post date:** [July 17, 2023, 2:21pm UTC](https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078/3 "2023-07-17T14:21:10Z")

</div>

Try using another model, though they currently all give error 422

---

<div class="post-metadata">

**Author:** ![ai-ml-app](https://avatars.discourse-cdn.com/v4/letter/a/a87d85/32.png) [@ai-ml-app](https://discuss.huggingface.co/u/ai-ml-app)\
**Post date:** [July 18, 2023, 4:04am UTC](https://discuss.huggingface.co/t/hugging-face-inference-api-returning-short-generated-text-with-gpt-2-model/47078/4 "2023-07-18T04:04:39Z")

</div>

I used the “bigscience/bloom” model on Hugging Face, but I encountered same issue where the generated text was consistently too short. However, when I used the API for OpenAI GPT-3, I didn’t encounter any issues. So I assume the problem lies with my coding or with Hugging Face.