# Finetuning model with smaller sequence size and Dmodel

**URL:** <https://discuss.huggingface.co/t/finetuning-model-with-smaller-sequence-size-and-dmodel/5526>\
**Category:** Models\
**Created:** [April 15, 2021, 3:20pm UTC](https://discuss.huggingface.co/t/finetuning-model-with-smaller-sequence-size-and-dmodel/5526 "2021-04-15T15:20:57Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ierezell](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ierezell/32/2517_2.png) [@ierezell](https://discuss.huggingface.co/u/ierezell)\
**Post date:** [April 15, 2021, 3:20pm UTC](https://discuss.huggingface.co/t/finetuning-model-with-smaller-sequence-size-and-dmodel/5526/1 "2021-04-15T15:20:57Z")

</div>

Hi all !

I wanted to know if there is any way to finetune a model but with smaller parameters (for exemple : sequence length from 512 to 64 and dModel from 768 to 256).

I could use the pytorch TransformerEncoder and train my own from scratch but I wanted to know if there were some fine-tunning possible (i.e not starting from scratch).

Thanks in addvance for any help,

Have a great day !