# Changing pooling method in pre-trained models

**URL:** <https://discuss.huggingface.co/t/changing-pooling-method-in-pre-trained-models/43790>\
**Category:** 🤗Transformers\
**Created:** [June 19, 2023, 1:04pm UTC](https://discuss.huggingface.co/t/changing-pooling-method-in-pre-trained-models/43790 "2023-06-19T13:04:42Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![yasinor](https://avatars.discourse-cdn.com/v4/letter/y/73ab20/32.png) [@yasinor](https://discuss.huggingface.co/u/yasinor)\
**Post date:** [June 19, 2023, 1:04pm UTC](https://discuss.huggingface.co/t/changing-pooling-method-in-pre-trained-models/43790/1 "2023-06-19T13:04:42Z")

</div>

I would like to obtain sentence embeddings for the texts in my dataset. To achieve this, I am utilizing the Sentence Transformer model called “bert-base-nli-mean-tokens.” During the training of this pre-trained model, the developers used mean pooling as the pooling method. However, I would like to use the model with max-pooling instead, without undergoing any additional training. I have written the following code, which runs without errors, but I am unsure if it is a valid approach. Could you assist me with this?

model\_name = ‘sentence-transformers/bert-base-nli-mean-tokens’  
word\_embedding\_model = models.Transformer(model\_name)  
pooling\_model = models.Pooling(word\_embedding\_model.get\_word\_embedding\_dimension(),  
pooling\_mode\_mean\_tokens=False,  
pooling\_mode\_cls\_token=True,  
pooling\_mode\_max\_tokens=False)  
model = SentenceTransformer(modules=[word\_embedding\_model, pooling\_model])  
sentence\_embeddings=model.encode(my\_dataset)