# How to return more tokens when calling the inference end point?

**URL:** <https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314>\
**Category:** Inference Endpoints on the Hub\
**Created:** [July 27, 2023, 3:03am UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314 "2023-07-27T03:03:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![maplesake](https://avatars.discourse-cdn.com/v4/letter/m/c68b51/32.png) [@maplesake](https://discuss.huggingface.co/u/maplesake)\
**Post date:** [July 27, 2023, 3:03am UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314/1 "2023-07-27T03:03:48Z")

</div>

Hi,

I purchased the pro account and spun up a WizardLM/WizardLM-13B-V1.1 end point for text generation. The API only seems to return a max of 75 characters. I’ve tried adding max\_tokens and min\_tokens and return\_full\_text, it doesn’t seem to have an effect.

How can I increase the amount of tokens returned?

Here’s an example of what I’ve tried:

```auto
curl https://abcd.us-east-1.aws.endpoints.huggingface.cloud \
-X POST \
-d '{"inputs":"I want to build a large parking structure. Can you tell me the steps I would take?", "options": {"min_tokens": 5000, "max_tokens": 5000, "return_full_text": true}}' \
-H "Authorization: Bearer $HF_TOKEN" \
-H "Content-Type: application/json"

```

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [July 27, 2023, 6:08am UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314/2 "2023-07-27T06:08:20Z")

</div>

Hell @maplesake please check the [documentation](https://huggingface.co/docs/inference-endpoints/supported_tasks#additional-parameters) and the [blog post](https://huggingface.co/blog/inference-endpoints-llm).

---

<div class="post-metadata">

**Author:** ![roland71](https://avatars.discourse-cdn.com/v4/letter/r/c37758/32.png) [@roland71](https://discuss.huggingface.co/u/roland71)\
**Post date:** [March 29, 2024, 10:15am UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314/3 "2024-03-29T10:15:50Z")

</div>

@maplesake Hello. I have the same problem. Have you found a solution?  
I don’t understand why this is not clearly written in the documentation, there is nothing at all on this subject.

---

<div class="post-metadata">

**Author:** ![Roelf](https://avatars.discourse-cdn.com/v4/letter/r/53a042/32.png) [@Roelf](https://discuss.huggingface.co/u/Roelf)\
**Post date:** [April 21, 2024, 12:48pm UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314/4 "2024-04-21T12:48:45Z")

</div>

I agree documentation can be much better. max\_new\_tokens parameter is what you are looking for.

---

<div class="post-metadata">

**Author:** ![monsteraenjoyer](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@monsteraenjoyer](https://discuss.huggingface.co/u/monsteraenjoyer)\
**Post date:** [May 9, 2024, 7:31pm UTC](https://discuss.huggingface.co/t/how-to-return-more-tokens-when-calling-the-inference-end-point/48314/5 "2024-05-09T19:31:11Z")

</div>

For people finding this thread at a later time. This is an example of how a request (for Llama 3 8B) might look like:

```auto
{
    "inputs": "<|begin_of_text|><|start_header_id|>user<|end_header_id|>How are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>",
    "parameters": {
        "max_new_tokens": 1000
    }
}

```