# Image-to-image Stable Diffusion Inference Endpoint? **URL:** \ **Category:** Inference Endpoints on the Hub\ **Created:** [October 14, 2022, 11:34am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438 "2022-10-14T11:34:36Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![fredrivett](https://avatars.discourse-cdn.com/v4/letter/f/65b543/32.png) [@fredrivett](https://discuss.huggingface.co/u/fredrivett)\ **Post date:** [October 14, 2022, 11:34am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/1 "2022-10-14T11:34:36Z")
Hey there, I’m trying to setup an image-to-image, stable diffusion Inference Endpoint. I saw [CompVis/stable-diffusion-v-1-4-original](https://huggingface.co/CompVis/stable-diffusion-v-1-4-original) which seems setup for the Inference Endpoint, but noticed this states it is only text-to-image, not image-to-image, so I assume this wouldn’t support generating images from an existing image. Then I looked at using [lambdalabs/sd-image-variations-diffusers](https://huggingface.co/lambdalabs/sd-image-variations-diffusers), but upon getting ready to set this up, I noticed this message: > Deploying this model with a custom task may fail as a handler.py file was not found in the repository. Refer to the `documentation` for more details. ![Screenshot 2022-10-14 at 12.23.47](https://us1.discourse-cdn.com/hellohellohello/original/2X/5/538a48e6efaaf12c533fc69c21792bdd2f48db6a.png) When searching for a stable-diffusion model, `lambdalabs/sd-image-variations-diffusers` was the only result. Is `lambdalabs/sd-image-variations-diffusers` setup to work with the Inference Endpoint, or is there another model that is? Thanks folks! Fred. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 14, 2022, 12:16pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/2 "2022-10-14T12:16:53Z")
Hello Fred, Yes, lambdalabs/sd-image-variations-diffusers is supported by Inference Endpoints by using a [custom handler](https://huggingface.co/docs/inference-endpoints/guides/custom_handler). Hugging Face Inference Endpoints be default support all of the [🤗 Transformers and Sentence-Transformers tasks](https://huggingface.co/docs/inference-endpoints/supported_tasks). if you want to deploy a custom model or customize a task, e.g. for diffusion you can do this by creating a [Create custom Inference Handler](https://huggingface.co/docs/inference-endpoints/guides/custom_handler) with a `handler.py`. The warning you saw indicates that the model doesn’t have a supported task and doesn’t include a `handler.py`. We already have an example on how you would deploy stable-diffusion: [philschmid/stable-diffusion-v1-4-endpoints · Hugging Face](https://huggingface.co/philschmid/stable-diffusion-v1-4-endpoints). With a custom handler you can customize the request payload to accept images and text, see this[zero-shot-image-classification example](https://huggingface.co/philschmid/clip-zero-shot-image-classification). ---
**Author:** ![fredrivett](https://avatars.discourse-cdn.com/v4/letter/f/65b543/32.png) [@fredrivett](https://discuss.huggingface.co/u/fredrivett)\ **Post date:** [October 14, 2022, 1:09pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/3 "2022-10-14T13:09:57Z")
Thanks so much for the swift reply here @philschmid. This all seems super powerful, and your comment makes sense. It seems like there’s two paths ahead: 1. Use `lambdalabs/sd-image-variations-diffusers` and look to implement my own custom handler (`handler.py`) 2. Use `CompVis/stable-diffusion-v-1-4-original` and try to customise it to accept an image payload I’ve used python a _little_ before, but I’m a front-end dev by trade, so a little out of my depth here, I’ll look into this and try and see how far I can get with it, I’ll probably prioritise trying #1. If you’ve any other tips or resources that are worth looking at, let me know! Cheers. ---
**Author:** ![dano1234](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dano1234/32/12175_2.png) [@dano1234](https://discuss.huggingface.co/u/dano1234)\ **Post date:** [October 21, 2022, 1:06pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/4 "2022-10-21T13:06:44Z")
Fred, Did you get this to work. My install fails repeatedly with this in the tail of the logs. tbghq 2022-10-21T12:58:51.065Z 2022-10-21 12:58:51,065 | INFO | Found custom pipeline at /repository/handler.py tbghq 2022-10-21T12:58:51.098Z File “/opt/conda/lib/python3.9/site-packages/starlette/routing.py”, line 566, in **aenter** tbghq 2022-10-21T12:58:51.098Z 2022-10-21 12:58:51,098 | ERROR | Traceback (most recent call last): tbghq 2022-10-21T12:58:51.098Z File “”, line 850, in exec\_module tbghq 2022-10-21T12:58:51.098Z File “/app/./huggingface\_inference\_toolkit/handler.py”, line 44, in get\_inference\_handler\_either\_custom\_or\_default\_handler tbghq 2022-10-21T12:58:51.098Z File “/opt/conda/lib/python3.9/site-packages/starlette/routing.py”, line 671, in lifespan tbghq 2022-10-21T12:58:51.098Z 2022-10-21 12:58:51,098 | ERROR | Application startup failed. Exiting. tbghq 2022-10-21T12:58:51.098Z inference\_handler = get\_inference\_handler\_either\_custom\_or\_default\_handler(HF\_MODEL\_DIR, task=HF\_TASK) tbghq 2022-10-21T12:58:51.098Z async with self.lifespan\_context(app): tbghq 2022-10-21T12:58:51.098Z ValueError: need to run on GPU tbghq 2022-10-21T12:58:51.098Z File “/opt/conda/lib/python3.9/site-packages/starlette/routing.py”, line 648, in startup tbghq 2022-10-21T12:58:51.098Z File “/repository/handler.py”, line 13, in tbghq 2022-10-21T12:58:51.098Z File “/app/./huggingface\_inference\_toolkit/utils.py”, line 202, in check\_and\_register\_custom\_pipeline\_from\_directory tbghq 2022-10-21T12:58:51.098Z spec.loader.exec\_module(handler) tbghq 2022-10-21T12:58:51.098Z await self.\_router.startup() tbghq 2022-10-21T12:58:51.098Z await handler() tbghq 2022-10-21T12:58:51.098Z File “/app/./webservice\_starlette.py”, line 56, in some\_startup\_task tbghq 2022-10-21T12:58:51.098Z raise ValueError(“need to run on GPU”) tbghq 2022-10-21T12:58:51.098Z tbghq 2022-10-21T12:58:51.098Z File “”, line 228, in \_call\_with\_frames\_removed tbghq 2022-10-21T12:58:51.098Z custom\_pipeline = check\_and\_register\_custom\_pipeline\_from\_directory(model\_dir) ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 21, 2022, 1:30pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/5 "2022-10-21T13:30:15Z")
Hello @dano1234, It seems that you are not using a GPU instance. `raise ValueError(“need to run on GPU”)` ---
**Author:** ![dano1234](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dano1234/32/12175_2.png) [@dano1234](https://discuss.huggingface.co/u/dano1234)\ **Post date:** [October 21, 2022, 2:27pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/6 "2022-10-21T14:27:29Z")
Phil, Thanks for quick reply. I get that error before I have a chance to specify anything about a GPU. I only filled in the “philschmid/stable-diffusion-v1-4-endpoints” AWS and region. Then it goes to next screen and eventually fails before I can set anything else. I am probably missing something. Dan ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 21, 2022, 2:41pm UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/7 "2022-10-21T14:41:44Z")
@dano1234 to select the instance type you need to click on “advanced configuration” and there you can select the instance type: ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/1/1252b634f344c058ab0abe16e97ccdabcc788d50.png) You can learn more about in the documentation: [Advanced Setup (Instance Types, Auto Scaling, Versioning)](https://huggingface.co/docs/inference-endpoints/guides/advanced) ---
**Author:** ![dano1234](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dano1234/32/12175_2.png) [@dano1234](https://discuss.huggingface.co/u/dano1234)\ **Post date:** [October 26, 2022, 9:45am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/8 "2022-10-26T09:45:09Z")
Thanks ! BTW when it says ~$450/month does it charge you that in a lump or is it pay as you go. IE can I try this out without being on the hook for monthly charge? ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 27, 2022, 6:41am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/9 "2022-10-27T06:41:02Z")
Hey @dano1234, At the end of each month the user or organization account will be charged for the compute resources used while Endpoints are up and running, that is, in the “running” state. So you will only pay the minutes of the uptime of you endpoint. You can learn more here: [Hugging Face Inference Endpoint Pricing](https://huggingface.co/docs/inference-endpoints/pricing) ---
**Author:** ![dano1234](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dano1234/32/12175_2.png) [@dano1234](https://discuss.huggingface.co/u/dano1234)\ **Post date:** [November 4, 2022, 11:02am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/10 "2022-11-04T11:02:24Z")
Thanks worked great. ---
**Author:** ![dskill](https://avatars.discourse-cdn.com/v4/letter/d/bc79bd/32.png) [@dskill](https://discuss.huggingface.co/u/dskill)\ **Post date:** [November 28, 2022, 4:17am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/11 "2022-11-28T04:17:18Z")
This is a total noob question, but I guess I have to ask it somewhere 🙂 The repo `philschmid/stable-diffusion-v1-4-endpoints` doesn’t seem to show up as an available model in the Model Repository dropdown of the `Create a new Endpoint` page. From this discussion, it looks like it used to show up since folks here seem to be using it as a starting point to set up a working stable diffusion based inference endpoint? ---
**Author:** ![dskill](https://avatars.discourse-cdn.com/v4/letter/d/bc79bd/32.png) [@dskill](https://discuss.huggingface.co/u/dskill)\ **Post date:** [November 29, 2022, 12:34am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/12 "2022-11-29T00:34:09Z")
Today the `philschmid/stable-diffusion-v1-4-endpoints` repo is showing up in the inference endpoint UI, along with a bunch of other diffusion models. Maybe there was a bug that was fixed recently. In any case, I’m unblocked now. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [November 29, 2022, 4:34am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/13 "2022-11-29T04:34:05Z")
Hey @dskill, We launched some new exciting features yesterday: [Philipp Schmid on LinkedIn: Stable Diffusion with Hugging Face Inference Endpoints](https://www.linkedin.com/feed/update/urn:li:activity:7003047028209426432/) Which added support for all `text-to-image` diffusion models based on StableDiffusionPipeline. ---
**Author:** ![dskill](https://avatars.discourse-cdn.com/v4/letter/d/bc79bd/32.png) [@dskill](https://discuss.huggingface.co/u/dskill)\ **Post date:** [November 29, 2022, 6:19am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/14 "2022-11-29T06:19:24Z")
Lucky timing for me! 🙂 I did see your excellent blog post earlier today and was able to get an inference endpoint up and running with text-to-image. It was extremely helpful. While I have you, one follow up question that’s been a bit confusing for me to untangle: How similar is the inference API to the endpoint API? Are they in fact identical? I also noticed that the inference API is turned off for `https://huggingface.co/stabilityai/stable-diffusion-2`. Is there a way for me to develop using model with the inference API somehow, or is using endpoint API the only way for now? Again, many thanks. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [November 29, 2022, 6:54am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/15 "2022-11-29T06:54:51Z")
The Inference API is free and designed for testing and experimenting with models on [huggingface.co](http://huggingface.co). It runs on shared infrastructure. This means that you are sharing the resources with other users, which could lead to high latency and low throughput there is no guarantee that the model has a GPU. The Inference API is not providing SLAs or other production-required features, like logs and monitoring. Inference Endpoints on the other side support all of this. ---
**Author:** ![dskill](https://avatars.discourse-cdn.com/v4/letter/d/bc79bd/32.png) [@dskill](https://discuss.huggingface.co/u/dskill)\ **Post date:** [November 29, 2022, 7:09am UTC](https://discuss.huggingface.co/t/image-to-image-stable-diffusion-inference-endpoint/24438/16 "2022-11-29T07:09:44Z")
OK got it - so in some cases the inference API may not be available, but that’s not something that I as an end user will have any control over. In general though, it sounds like the intended workflow is develop on the inference API where it’s available, and then graduate to endpoint API when you require the production solution. Thanks again for the responses!