# Confused about max\_length and max\_new\_tokens **URL:** \ **Category:** 🤗Transformers\ **Created:** [February 1, 2023, 11:07am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892 "2023-02-01T11:07:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![hyperknot](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hyperknot/32/13058_2.png) [@hyperknot](https://discuss.huggingface.co/u/hyperknot)\ **Post date:** [February 1, 2023, 11:07am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/1 "2023-02-01T11:07:24Z")
I’m trying to run the example code from flan-t5-small: > **[google/flan-t5-small · Hugging Face](https://huggingface.co/google/flan-t5-small)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ```auto from transformers import T5Tokenizer, T5ForConditionalGeneration tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-base") model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-base") input_text = "translate English to German: How old are you?" input_ids = tokenizer(input_text, return_tensors="pt").input_ids outputs = model.generate(input_ids) print(tokenizer.decode(outputs[0])) ``` I’m getting the following warning: > UserWarning: Neither `max_length` nor `max_new_tokens` has been set, `max_length` will default to 20 (`generation_config.max_length`). Controlling `max_length` via the config is deprecated and `max_length` will be removed from the config in v5 of Transformers – we recommend using `max_new_tokens` to control the maximum length of the generation. How should I configure this? Is this something like on OpenAI playground where the default setting is 256 but the model actually supports 4000 tokens? ---
**Author:** ![arastu-mudgal](https://avatars.discourse-cdn.com/v4/letter/a/d78d45/32.png) [@arastu-mudgal](https://discuss.huggingface.co/u/arastu-mudgal)\ **Post date:** [February 2, 2023, 4:13am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/2 "2023-02-02T04:13:11Z")
hey, was this resolved? ---
**Author:** ![hyperknot](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hyperknot/32/13058_2.png) [@hyperknot](https://discuss.huggingface.co/u/hyperknot)\ **Post date:** [February 3, 2023, 1:55am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/3 "2023-02-03T01:55:23Z")
No, I’m waiting for a reply. ---
**Author:** ![gputask009](https://avatars.discourse-cdn.com/v4/letter/g/7c8e57/32.png) [@gputask009](https://discuss.huggingface.co/u/gputask009)\ **Post date:** [March 13, 2023, 11:32am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/4 "2023-03-13T11:32:47Z")
This `outputs = model.generate(input_ids,max_length= 60)` worked for me without giving any error. ---
**Author:** ![ShaYuer](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@ShaYuer](https://discuss.huggingface.co/u/ShaYuer)\ **Post date:** [September 14, 2023, 3:29pm UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/5 "2023-09-14T15:29:04Z")
just give a generate number: outputs = model.generate(input\_ids,max\_new\_tokens=4000) ---
**Author:** ![Navanit-AI](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/navanit-ai/32/30195_2.png) [@Navanit-AI](https://discuss.huggingface.co/u/Navanit-AI)\ **Post date:** [December 14, 2023, 7:44am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/6 "2023-12-14T07:44:17Z")
I know I’m late now. But for any future preferences. what the max\_length and max\_new\_tokens do. In max\_length we get the maximum length including the input and output tokens. But in max\_new\_tokens we get the maximum output excluding the output. Let me show you using the code ```auto import torch from transformers import AutoModelForCausalLM, AutoTokenizer,AutoModelForSeq2SeqLM torch.set_default_device("cuda") model = AutoModelForCausalLM.from_pretrained("microsoft/phi-2", torch_dtype="auto", trust_remote_code=True) tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-2", trust_remote_code=True) inputs = tokenizer('''def print_prime(n): """ Print all primes between 1 and n """''', return_tensors="pt", return_attention_mask=False) outputs = model.generate(**inputs, max_length=200) text = tokenizer.batch_decode(outputs)[0] outputs_2 = model.generate(**inputs,max_new_tokens=200) text_2 = tokenizer.batch_decode(outputs_2)[0] prompt_tokens = tokenizer.convert_ids_to_tokens(inputs['input_ids'][0]) output_tokens_1 = tokenizer.convert_ids_to_tokens(outputs[0]) output_tokens_2 = tokenizer.convert_ids_to_tokens(outputs_2[0]) num_prompt_tokens = len(prompt_tokens) num_output_tokens = len(output_tokens_1) num_output_tokens_2 = len(output_tokens_2) print("Number of tokens in prompt:", num_prompt_tokens) print("Number of tokens from max_length output:", num_output_tokens) print("Number of tokens from max_new_tokens output:", num_output_tokens_2) ``` The output that I got Number of tokens in prompt: 23 Number of tokens from max\_length output: 200 Number of tokens from max\_new\_tokens output: 223 ## Hope it helps ---
**Author:** ![vaibhavdabhade](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vaibhavdabhade/32/27971_2.png) [@vaibhavdabhade](https://discuss.huggingface.co/u/vaibhavdabhade)\ **Post date:** [June 8, 2024, 8:17am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/7 "2024-06-08T08:17:29Z")
Yes, It worked without this error now. Thanks. ---
**Author:** ![ngogiahan](https://avatars.discourse-cdn.com/v4/letter/n/82dd89/32.png) [@ngogiahan](https://discuss.huggingface.co/u/ngogiahan)\ **Post date:** [September 5, 2024, 7:58am UTC](https://discuss.huggingface.co/t/confused-about-max-length-and-max-new-tokens/30892/8 "2024-09-05T07:58:24Z")
this solve my problem, thank you!