# Using the prompt to switch model

**URL:** <https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039>\
**Category:** 🔒 Gradio\
**Created:** [January 19, 2023, 9:02pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039 "2023-01-19T21:02:01Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 19, 2023, 9:02pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/1 "2023-01-19T21:02:01Z")

</div>

Hello,

I am new to gradio and tried to implement a way to switch models by identifying keyword inside the prompt

see

> **[app.py · AlStable/stable-diffusion-2-1 at main](https://huggingface.co/spaces/AlStable/stable-diffusion-2-1/blob/main/app.py)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

the problem seems to be that whatever is selected  
it is ignored by gradio

Can someone explain, what is wrong in my code  
and if this is feasible or not?

Thanks

---

<div class="post-metadata">

**Author:** ![abidlabs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abidlabs/32/21878_2.png) [@abidlabs](https://discuss.huggingface.co/u/abidlabs)\
**Post date:** [January 21, 2023, 2:33am UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/3 "2023-01-21T02:33:44Z")

</div>

Hi @blogclif, you can’t use `gr.Interface.load()` with a `fn`, you can only load a single model at a time. However, it is possible to do what you have in mind by noting the fact that an a model loaded with `gr.Interface.load()` can be used as a function. So you could do something like this:

```plaintext
models = [
 gr.Interface.load("model1name"),
 gr.Interface.load("model2name")
 ...
]

def prediction(model_choice, input):
  model = models[model_choice]
  output = model(input)
  return output = model

gr.Interface(fn=prediciton, inputs=[gr.Dropdown(["model1name", "model2name"], type="index"), gr.Textbox()], outputs=gr.Image()].launch()

```

This is just a code skeleton but let me know if it makes sense!

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 21, 2023, 4:55pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/4 "2023-01-21T16:55:57Z")

</div>

Thanks a lot for looking into it

I will try, I like the fact that the prompt was including the model in a text format ; that seemed really transparent, but having to select the model via a menu list is surely better, and more reliable

thank a lot

---

<div class="post-metadata">

**Author:** ![radames](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/radames/32/28246_2.png) [@radames](https://discuss.huggingface.co/u/radames)\
**Post date:** [January 23, 2023, 7:14pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/5 "2023-01-23T19:14:25Z")

</div>

hi, this is a cool example using this approach [app.py · Omnibus/maximum\_diffusion at a451648ba3f1b4e498072f0e4a462d4c265fe713](https://huggingface.co/spaces/Omnibus/maximum_diffusion/blob/a451648ba3f1b4e498072f0e4a462d4c265fe713/app.py) from this project [Maximum Diffusion - a Hugging Face Space by Omnibus](https://huggingface.co/spaces/Omnibus/maximum_diffusion) currently down but I’ve ping @Omnibus to look at the error.

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 24, 2023, 3:24pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/7 "2023-01-24T15:24:36Z")

</div>

Thanks a lot for the help, I am writing some more code  
yesterday and today  
let’s see what will work best.

I will most likely go in the direction of a dropdown menu ; as I understand now that extracting the model from the prompt is not accepted by gradio, because gradio needs two variables, and won’t accept the second variable after it has been called.

I guess, that was the misunderstanding I had at the very beginning

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 24, 2023, 5:29pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/9 "2023-01-24T17:29:54Z")

</div>

@Omnibus

I did some more testing and found another user who had the interface I was looking for

 ![interface-al-stable](https://us1.discourse-cdn.com/hellohellohello/original/2X/9/9edfb4ad0330acd75fb55fc468941e15c9d437a1.png)

there is a new problem now

This error is popping up

> OSError: We couldn’t connect to ‘[https://huggingface.co](https://huggingface.co)’ to load this model, couldn’t find it in the cached files and it looks like models/ItsJayQz/Marvel\_WhatIf\_Diffusion is not the path to a directory containing a model\_index.json file.

for all the models

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 24, 2023, 5:48pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/11 "2023-01-24T17:48:34Z")

</div>

will try, here was the link to the other space that had the interface I was trying to have in the first place.

> **[Text To Image AllModels - a Hugging Face Space by Joeythemonster](https://huggingface.co/spaces/Joeythemonster/Text-To-image-AllModels)**
>
> Discover amazing ML apps made by the community

I left a comment in his discussion thread

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 24, 2023, 5:59pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/13 "2023-01-24T17:59:49Z")

</div>

Feels like your solution above is working

removing the “models/” and using get\_default\_dtype

now in the log I see

> Downloading: 71%|███████ | 2.47G/3.46G [00:35\<00:18, 54.4MB/s]

it still shows an ‘Error’ in the end

 ![interface-al-stable](https://us1.discourse-cdn.com/hellohellohello/original/2X/e/e4fe6ca278e310db4fed985e2b91212dba7fdb78.png)

and the logs are still scrolling … I can’t really get an error this time

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 24, 2023, 6:19pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/16 "2023-01-24T18:19:19Z")

</div>

This is what I have:

Cannot initialize model with low cpu memory usage because `accelerate` was not found in the environment. Defaulting to `low_cpu_mem_usage=False`. It is strongly recommended to install `accelerate` for faster and less memory-intense model loading. You can do so with:

```auto
pip install accelerate

```

.  
Traceback (most recent call last):  
File “/home/user/.local/lib/python3.8/site-packages/gradio/routes.py”, line 321, in run\_predict  
output = await app.blocks.process\_api(  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 1015, in process\_api  
result = await self.call\_function(fn\_index, inputs, iterator, request)  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 856, in call\_function  
prediction = await anyio.to\_thread.run\_sync(  
File “/home/user/.local/lib/python3.8/site-packages/anyio/to\_thread.py”, line 31, in run\_sync  
return await get\_asynclib().run\_sync\_in\_worker\_thread(  
File “/home/user/.local/lib/python3.8/site-packages/anyio/\_backends/\_asyncio.py”, line 937, in run\_sync\_in\_worker\_thread  
return await future  
File “/home/user/.local/lib/python3.8/site-packages/anyio/\_backends/\_asyncio.py”, line 867, in run  
result = context.run(func, \*args)  
File “app.py”, line 18, in TextToImage  
pipe = StableDiffusionPipeline.from\_pretrained(model\_id, torch\_dtype=torch.get\_default\_dtype)  
File “/home/user/.local/lib/python3.8/site-packages/diffusers/pipeline\_utils.py”, line 708, in from\_pretrained  
loaded\_sub\_model = load\_method(os.path.join(cached\_folder, name), \*\*loading\_kwargs)  
File “/home/user/.local/lib/python3.8/site-packages/transformers/modeling\_utils.py”, line 2325, in from\_pretrained  
dtype\_orig = cls.\_set\_default\_torch\_dtype(torch\_dtype)  
File “/home/user/.local/lib/python3.8/site-packages/transformers/modeling\_utils.py”, line 1102, in \_set\_default\_torch\_dtype  
if not dtype.is\_floating\_point:  
AttributeError: ‘builtin\_function\_or\_method’ object has no attribute ‘is\_floating\_point’  
Traceback (most recent call last):  
File “/home/user/.local/lib/python3.8/site-packages/gradio/routes.py”, line 321, in run\_predict  
output = await app.blocks.process\_api(  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 1013, in process\_api  
inputs = self.preprocess\_data(fn\_index, inputs, state)  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 923, in preprocess\_data  
processed\_input.append(block.preprocess(inputs[i]))  
IndexError: list index out of range  
Traceback (most recent call last):  
File “/home/user/.local/lib/python3.8/site-packages/diffusers/configuration\_utils.py”, line 326, in load\_config  
config\_file = hf\_hub\_download(  
File “/home/user/.local/lib/python3.8/site-packages/huggingface\_hub/utils/\_validators.py”, line 114, in \_inner\_fn  
validate\_repo\_id(arg\_value)  
File “/home/user/.local/lib/python3.8/site-packages/huggingface\_hub/utils/\_validators.py”, line 172, in validate\_repo\_id  
raise HFValidationError(  
huggingface\_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or ‘-’, '_', ‘.’, ‘–’ and ‘…’ are forbidden, ‘-’ and ‘.’ cannot start or end the name, max length is 96: ‘’.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
File “/home/user/.local/lib/python3.8/site-packages/gradio/routes.py”, line 321, in run\_predict  
output = await app.blocks.process\_api(  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 1015, in process\_api  
result = await self.call\_function(fn\_index, inputs, iterator, request)  
File “/home/user/.local/lib/python3.8/site-packages/gradio/blocks.py”, line 856, in call\_function  
prediction = await anyio.to\_thread.run\_sync(  
File “/home/user/.local/lib/python3.8/site-packages/anyio/to\_thread.py”, line 31, in run\_sync  
return await get\_asynclib().run\_sync\_in\_worker\_thread(  
File “/home/user/.local/lib/python3.8/site-packages/anyio/\_backends/\_asyncio.py”, line 937, in run\_sync\_in\_worker\_thread  
return await future  
File “/home/user/.local/lib/python3.8/site-packages/anyio/\_backends/\_asyncio.py”, line 867, in run  
result = context.run(func, \*args)  
File “app.py”, line 18, in TextToImage  
pipe = StableDiffusionPipeline.from\_pretrained(model\_id, torch\_dtype=torch.get\_default\_dtype)  
File “/home/user/.local/lib/python3.8/site-packages/diffusers/pipeline\_utils.py”, line 459, in from\_pretrained  
config\_dict = cls.load\_config(  
File “/home/user/.local/lib/python3.8/site-packages/diffusers/configuration\_utils.py”, line 363, in load\_config  
raise EnvironmentError(  
OSError: We couldn’t connect to ‘[https://huggingface.co](https://huggingface.co)’ to load this model, couldn’t find it in the cached files and it looks like is not the path to a directory containing a model\_index.json file.  
Checkout your internet connection or see how to run the library in offline mode at ‘[Installation](https://huggingface.co/docs/diffusers/installation#offline-mode)’.

* * *

To get this error I left the model as ‘blank’

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 8:33am UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/18 "2023-01-25T08:33:57Z")

</div>

@Omnibus  
Oh well, this time the generating part never ends  
there are no error it just won’t finish

I give up

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 9:54am UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/20 "2023-01-25T09:54:59Z")

</div>

Thanks again, for your assistance,

I let it run for 3 hours and just closed the tab (after taking this screenshot)

 ![workin-1](https://us1.discourse-cdn.com/hellohellohello/original/2X/0/06c3d9997b355b3bf8f49ee4de073c1ccced5d72.png)

the logs are showing a lot of downloads and sometime I see this

> Fetching 16 files: 100%|██████████| 16/16 [02:48\<00:00, 10.53s/it]  
> /home/user/.local/lib/python3.8/site-packages/transformers/models/clip/feature\_extraction\_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.  
> warnings.warn(

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 10:00am UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/21 "2023-01-25T10:00:56Z")

</div>

> [@Omnibus](#):
>
> Gradio.Interface.load(“models/Repo/Name”)

if this would load the models once and for all?  
that could really speed up the process

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 10:12am UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/24 "2023-01-25T10:12:55Z")

</div>

Thanks a lot, I will use this new code above  
and the solution provided by @abidlabs

Hopefully, this will work faster

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 2:24pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/27 "2023-01-25T14:24:36Z")

</div>

looks promising, I am not having the version working yet  
I made the space private in the meantime to get faster result

Will make the space public once I’m done

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 3:08pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/29 "2023-01-25T15:08:18Z")

</div>

I am finishing up, but it works now

Thanks a lot for your help, I am glad this is resolved  
and I learned a lot in the last 2 days thanks to you

Here is the final version:  
[https://alstable-marvel.hf.space](https://alstable-marvel.hf.space)

As a sidenote - I had to get rid of one the style

> Fetching model from: [DGSpitzer/Guan-Yu-Diffusion · Hugging Face](https://huggingface.co/DGSpitzer/Guan-Yu-Diffusion)  
> as it was giving an error  
> ValueError: Unsupported pipeline type: None

This thread can now be closed

---

<div class="post-metadata">

**Author:** ![blogclif](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/blogclif/32/13841_2.png) [@blogclif](https://discuss.huggingface.co/u/blogclif)\
**Post date:** [January 25, 2023, 3:24pm UTC](https://discuss.huggingface.co/t/using-the-prompt-to-switch-model/30039/30 "2023-01-25T15:24:43Z")

</div>

here is a fresh example

 ![blackwidow1](https://us1.discourse-cdn.com/hellohellohello/original/2X/0/0849f6f6a461d9f0997c156b1a9a750808c33c0f.jpeg)