# The current text generation call will exceed the model's predefined maximum length

**URL:** <https://discuss.huggingface.co/t/the-current-text-generation-call-will-exceed-the-models-predefined-maximum-length/70393>\
**Category:** 🤗Transformers\
**Created:** [January 24, 2024, 10:09am UTC](https://discuss.huggingface.co/t/the-current-text-generation-call-will-exceed-the-models-predefined-maximum-length/70393 "2024-01-24T10:09:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![madoss](https://avatars.discourse-cdn.com/v4/letter/m/bbe5ce/32.png) [@madoss](https://discuss.huggingface.co/u/madoss)\
**Post date:** [January 24, 2024, 10:09am UTC](https://discuss.huggingface.co/t/the-current-text-generation-call-will-exceed-the-models-predefined-maximum-length/70393/1 "2024-01-24T10:09:53Z")

</div>

I’m currently working on a generation task using transformers and have encountered an issue. I’m using the [NSQL-2B](https://huggingface.co/NumbersStation/nsql-2B) model, and the generation configuration is set as follows:

`GenerationConfig(num_beams=5, early_stopping=True, pad_token=self.__model.config.pad_token_id, max_new_tokens=2048, do_sample=False,temperature=0.1)`

I got the following warning when generating the text:

`This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (2048). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.`

Followed by this error:

`index 2048 is out of bounds for dimension 0 with size 2048)`

The model max length is : 2048

I’m looking for advice or solutions on how to handle this. Should I adjust the `max_new_tokens` parameter, or is there another way to manage this limitation without compromising the generation quality? Any insights or experiences with similar issues would be greatly appreciated.

Thank you!

---

<div class="post-metadata">

**Author:** ![hust](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hust/32/36916_2.png) [@hust](https://discuss.huggingface.co/u/hust)\
**Post date:** [April 16, 2025, 10:30pm UTC](https://discuss.huggingface.co/t/the-current-text-generation-call-will-exceed-the-models-predefined-maximum-length/70393/2 "2025-04-16T22:30:32Z")

</div>

I have the same issue, too. In fact, you define the max\_new\_token to be 2048 is the key point that why you have this issue. When you call output = model.generate(\*\*input,…) the output actually includes the input, which will be count into the overall length. For example, you input is “what is your name”, which contains 4 token and 2 special token (bos or eos), these 6 tokens plus your max\_new\_token 2048 would be 2054(2048 + 6). As the length of output of your model is limited, the exceeded tokens would result in this issue and throw some errors. I advice you lowering the max\_new\_token to 1024 and most important making sure that your input token would not exceed 1024 if your model’s max token is 2048. For me, I define two parameters:

1. model\_max\_length It restricts the input length to 1024 for example.

2. max\_new\_token (this parameter is in model.generate) It restrict the output length to 1024.

The overall length would not exceed the maximum token of your model. Like for opt1.3b: 2048 llama: 4096 …

I hope my experience can help you solve the issue.