# Containerizing transformers with Docker and FastAPI

**URL:** <https://discuss.huggingface.co/t/containerizing-transformers-with-docker-and-fastapi/880>\
**Category:** 🤗Transformers\
**Created:** [August 26, 2020, 5:31pm UTC](https://discuss.huggingface.co/t/containerizing-transformers-with-docker-and-fastapi/880 "2020-08-26T17:31:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lmwilkin](https://avatars.discourse-cdn.com/v4/letter/l/f4b2a3/32.png) [@lmwilkin](https://discuss.huggingface.co/u/lmwilkin)\
**Post date:** [August 26, 2020, 5:31pm UTC](https://discuss.huggingface.co/t/containerizing-transformers-with-docker-and-fastapi/880/1 "2020-08-26T17:31:36Z")

</div>

Hi everyone!

I’ve been working on putting GPU accelerated transformer inference into production using Docker. I thought it would be helpful for me to share how I did it (link to [gist](https://gist.github.com/lmwilkin/359ef8ada2eb1766d049719e9fc7053a)):

```
<script src="https://gist.github.com/lmwilkin/359ef8ada2eb1766d049719e9fc7053a.js"></script>

```

You’ll need to have installed [nvidia-docker](https://github.com/NVIDIA/nvidia-docker)

I used FastAPI to set up a basic rest interface. The Dockerfile would be a good place to add an environmental variable for something like model name too if you want to set that dynamically.

My plan is to try write a container that can swap out models at request which I’ll share if I can get working.

Hope this helps!

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [August 28, 2020, 3:33pm UTC](https://discuss.huggingface.co/t/containerizing-transformers-with-docker-and-fastapi/880/2 "2020-08-28T15:33:15Z")

</div>

Hey @lmwilkin, great! Thank you for sharing this.