# ValidationError: Max token limit(\>=1) reached for finetuned models

**URL:** <https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168>\
**Category:** Amazon SageMaker\
**Created:** [December 9, 2023, 1:53pm UTC](https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168 "2023-12-09T13:53:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![seinfeld](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@seinfeld](https://discuss.huggingface.co/u/seinfeld)\
**Post date:** [December 9, 2023, 1:53pm UTC](https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168/1 "2023-12-09T13:53:38Z")

</div>

I finetuned the Llama2-7b model on sagemaker, but I get the token limit reached error during inference.  
Here is the error that I get most of the times.

```auto

An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message "{"error":"Input validation error: `max_new_tokens` must be <= 1. Given: 500","error_type":"validation"}"

```

Does anyone have any idea why this is and how I can solve this?  
@philschmid , maybe you can help?

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [December 11, 2023, 9:45am UTC](https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168/2 "2023-12-11T09:45:30Z")

</div>

You can configure this via then environment variables.

```bash
MAX_INPUT_LENGTH': json.dumps(1024), # Max length of input text
  'MAX_TOTAL_TOKENS': json.dumps(2048), # Max length of the generation (including input text)

```

---

<div class="post-metadata">

**Author:** ![seinfeld](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@seinfeld](https://discuss.huggingface.co/u/seinfeld)\
**Post date:** [December 20, 2023, 7:25pm UTC](https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168/3 "2023-12-20T19:25:32Z")

</div>

Sorry for the late reply. This worked thanks.  
For a longer input with system prompts, can we increase the input length or the token length?

---

<div class="post-metadata">

**Author:** ![dipakrimal](https://avatars.discourse-cdn.com/v4/letter/d/bbe5ce/32.png) [@dipakrimal](https://discuss.huggingface.co/u/dipakrimal)\
**Post date:** [December 28, 2023, 5:03pm UTC](https://discuss.huggingface.co/t/validationerror-max-token-limit-1-reached-for-finetuned-models/65168/4 "2023-12-28T17:03:59Z")

</div>

Where should these be configured exactly? I ran into the same error while trying to invoke deployed sagemaker endpoint. I passed these along when deploying the finetuned estimator to the endpoint but still getting the same errors.