# Productionizing HuggingFace Transformers?

**URL:** <https://discuss.huggingface.co/t/productionizing-huggingface-transformers/23017>\
**Category:** Beginners\
**Created:** [September 12, 2022, 11:27am UTC](https://discuss.huggingface.co/t/productionizing-huggingface-transformers/23017 "2022-09-12T11:27:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vanlifecoder](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/vanlifecoder/32/6022_2.png) [@vanlifecoder](https://discuss.huggingface.co/u/vanlifecoder)\
**Post date:** [September 12, 2022, 11:27am UTC](https://discuss.huggingface.co/t/productionizing-huggingface-transformers/23017/1 "2022-09-12T11:27:03Z")

</div>

what’s a common reference architecture for companies that use sentence transformers via huggingface in production?

i was thinking:

api gateway → queue → serverless (sentence transformer module)

is it best to co-locate the model file in my lambda VPN? looking for any and all best practices.

---

<div class="post-metadata">

**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](https://discuss.huggingface.co/u/NimaBoscarino)\
**Post date:** [September 12, 2022, 11:55am UTC](https://discuss.huggingface.co/t/productionizing-huggingface-transformers/23017/2 "2022-09-12T11:55:07Z")

</div>

Hi there! I think the things that it depends on most are:

1. Your company’s existing stack
2. Your use-case (expected load, real-time vs. batched, etc.)

Can you share a bit more about what those things look like in your situation? Without that (IMO) it’s a bit difficult to give any recommendations. I could point you towards our Inference API service, for example ([Inference API - Hugging Face](https://huggingface.co/inference-api)), which lets you offload that to our infrastructure. Or you could take an approach like the one outlined here: [How to Deploy NLP Models in Production - neptune.ai](https://neptune.ai/blog/deploy-nlp-models-in-production). Some companies might set up entire CI/CD situations if they need to constantly monitor, retrain, and redeploy their models ([Continuous Delivery for Machine Learning](https://martinfowler.com/articles/cd4ml.html)).

If you have more details about your use-case I can definitely try to provide more details!