# Deploy distiluse-base-multilingual-cased-v2 on Sagemaker

**URL:** <https://discuss.huggingface.co/t/deploy-distiluse-base-multilingual-cased-v2-on-sagemaker/51557>\
**Category:** Amazon SageMaker\
**Created:** [August 21, 2023, 5:32pm UTC](https://discuss.huggingface.co/t/deploy-distiluse-base-multilingual-cased-v2-on-sagemaker/51557 "2023-08-21T17:32:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ylevine84](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ylevine84/32/18646_2.png) [@ylevine84](https://discuss.huggingface.co/u/ylevine84)\
**Post date:** [August 21, 2023, 5:32pm UTC](https://discuss.huggingface.co/t/deploy-distiluse-base-multilingual-cased-v2-on-sagemaker/51557/1 "2023-08-21T17:32:52Z")

</div>

I’m trying to figure out if I can use sentence\_transformers/distiluse-base-multilingual-cased-v2 as a real-time inference endpoint on AWS Sagemaker to retrieve embeddings and run a model.

Using [this](https://github.com/huggingface/notebooks/blob/main/sagemaker/17_custom_inference_script/sagemaker-notebook.ipynb) guide from @philschmid on AWS I see how it uses the following code with the Transformers library to load a model stored in an S3 bucket and create and return the embeddings. This code is provided on most model pages to load the model with Transformers, but not on the distiluse-base-multilingual-cased-v2 page.

I’m able to upload it as in the tutorial, but the vectors returned from Sagemaker are len(768) instead of 512 when I load the model with the sentence\_transformers library. I know that this model has a custom dense layer that [doesn’t allow it to be fine-tuned with Transformers](https://github.com/UKPLab/sentence-transformers/issues/513), and I’m assuming that that when I load the model with Transformers it is missing the custom dense layer at the end. I’m wondering if there are any suggestions that would allow me to load the complete model into Sagemaker and return embeddings of len(512)

Thanks!!

---

<div class="post-metadata">

**Author:** ![pserotini](https://avatars.discourse-cdn.com/v4/letter/p/e480ec/32.png) [@pserotini](https://discuss.huggingface.co/u/pserotini)\
**Post date:** [January 25, 2024, 6:50pm UTC](https://discuss.huggingface.co/t/deploy-distiluse-base-multilingual-cased-v2-on-sagemaker/51557/2 "2024-01-25T18:50:43Z")

</div>

Were you able to solve it? Same problem here…