# Doubt on Tokenization in Pegasus **URL:** \ **Category:** Beginners\ **Created:** [November 30, 2020, 2:35pm UTC](https://discuss.huggingface.co/t/doubt-on-tokenization-in-pegasus/2365 "2020-11-30T14:35:10Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![airnicco8](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@airnicco8](https://discuss.huggingface.co/u/airnicco8)\ **Post date:** [November 30, 2020, 2:35pm UTC](https://discuss.huggingface.co/t/doubt-on-tokenization-in-pegasus/2365/1 "2020-11-30T14:35:10Z")
Hi, i created a 16-2 pegasus student with make\_student.py then tried to use finetune.py on XSUM dataset. The script i run is: python finetune.py --max\_source\_length 500 --data\_dir xsum --freeze\_encoder --freeze\_embeds --learning\_rate=1e-4 --do\_train --do\_predict --val\_check\_interval 0.1 --n\_val 1000 --max\_target\_length=60 --val\_max\_target\_length=60 --test\_max\_target\_length=100 --model\_name\_or\_path dpx\_xsum\_16\_2 --train\_batch\_size=1 --eval\_batch\_size=1 --sortish\_sampler --num\_train\_epochs=6 --warmup\_steps 500 --output\_dir distilpeg\_xsum\_sft\_16\_2 --gpus 0 --gradient\_accumulation\_steps 256 --adafactor --dropout 0.1 --attention\_dropout 0.1 --overwrite\_output\_dir The question is, is it normal if i don’t specify --max\_source\_length 500 i obtain an error during embedding? If i leave it like that the fine-tuning is efficient? Thanks in advance! ---
**Author:** ![airnicco8](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@airnicco8](https://discuss.huggingface.co/u/airnicco8)\ **Post date:** [November 30, 2020, 4:03pm UTC](https://discuss.huggingface.co/t/doubt-on-tokenization-in-pegasus/2365/2 "2020-11-30T16:03:42Z")
I noticed on the script in the repo --max\_source\_lenght 512 is set and so i ran with such setting. But i notice that starting r2 score is 0.0 in metrics.json. Is this a problem?