# Error raised by inference API: Cannot override task for LLM models

**URL:** <https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169>\
**Category:** Models\
**Created:** [May 10, 2024, 7:42pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169 "2024-05-10T19:42:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adilmar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adilmar/32/52970_2.png) [@Adilmar](https://discuss.huggingface.co/u/Adilmar)\
**Post date:** [May 10, 2024, 7:42pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/1 "2024-05-10T19:42:19Z")

</div>

I have the same problem with different models as well (including mistralai/Mistral-7B-Instruct-v0.2 and meta-llama/Meta-Llama-3-8B)

# HuggingFaceHub fails with “Error raised by inference API: Cannot override task for LLM models”

This error started now, do you have any idea what it could be?

---

<div class="post-metadata">

**Author:** ![Adilmar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adilmar/32/52970_2.png) [@Adilmar](https://discuss.huggingface.co/u/Adilmar)\
**Post date:** [May 10, 2024, 7:44pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/2 "2024-05-10T19:44:18Z")

</div>

@abhishek can you help us?

---

<div class="post-metadata">

**Author:** ![abhishek](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abhishek/32/32360_2.png) [@abhishek](https://discuss.huggingface.co/u/abhishek)\
**Post date:** [May 10, 2024, 7:55pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/3 "2024-05-10T19:55:19Z")

</div>

where is this error? are there any logs or screenshots?

---

<div class="post-metadata">

**Author:** ![Adilmar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adilmar/32/52970_2.png) [@Adilmar](https://discuss.huggingface.co/u/Adilmar)\
**Post date:** [May 10, 2024, 8:57pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/4 "2024-05-10T20:57:52Z")

</div>

Log @abhishek

```auto
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
    exec(code, module. __dict__ )
File "/home/user/app/app.py", line 130, in <module>
    main()
File "/home/user/app/app.py", line 106, in main
    handle_userinput(user_question)
File "/home/user/app/app.py", line 71, in handle_userinput
    response = st.session_state.conversation({"question": user_question})
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 310, in __call__
    raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/site-packages/langchain/chains/conversational_retrieval/base.py", line 159, in _call
    answer = self.combine_docs_chain.run(
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 510, in run
    return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 310, in __call__
    raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 122, in _call
    output, extra_return_dict = self.combine_docs(
File "/usr/local/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 171, in combine_docs
    return self.llm_chain.predict(callbacks=callbacks, **inputs), {}
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 298, in predict
    return self(kwargs, callbacks=callbacks)[self.output_key]
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 310, in __call__
    raise e
File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 108, in _call
    response = self.generate([inputs], run_manager=run_manager)
File "/usr/local/lib/python3.10/site-packages/langchain/chains/llm.py", line 120, in generate
    return self.llm.generate_prompt(
File "/usr/local/lib/python3.10/site-packages/langchain/llms/base.py", line 507, in generate_prompt
    return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)
File "/usr/local/lib/python3.10/site-packages/langchain/llms/base.py", line 656, in generate
    output = self._generate_helper(
File "/usr/local/lib/python3.10/site-packages/langchain/llms/base.py", line 544, in _generate_helper
    raise e
File "/usr/local/lib/python3.10/site-packages/langchain/llms/base.py", line 531, in _generate_helper
    self._generate(
File "/usr/local/lib/python3.10/site-packages/langchain/llms/base.py", line 1053, in _generate
    self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)
File "/usr/local/lib/python3.10/site-packages/langchain/llms/huggingface_hub.py", line 112, in _call
    raise ValueError(f"Error raised by inference API: {response['error']}"

```

```auto

Function code

def get_conversation_chain(vectorstore:FAISS) -> ConversationalRetrievalChain:
    llm = HuggingFaceHub(
        repo_id="mistralai/Mixtral-8x7B-Instruct-v0.1",
        #repo_id="TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF"
        model_kwargs={"temperature": 0.5, "max_length": 1048},
    )

```

---

<div class="post-metadata">

**Author:** ![Adilmar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adilmar/32/52970_2.png) [@Adilmar](https://discuss.huggingface.co/u/Adilmar)\
**Post date:** [May 10, 2024, 8:59pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/5 "2024-05-10T20:59:54Z")

</div>

In this link, several people have the same problem, @abhishek

> <https://github.com/jupyterlab/jupyter-ai/issues/772>
>
> \<!-- Welcome! Thank you for contributing. These HTML comments will not render in… the issue.
> 
> Before creating a new issue:
> \* Search for relevant issues
> \* Follow the issue reporting guidelines:
> https://jupyterlab.readthedocs.io/en/latest/getting\_started/issue.html
> \--\>
> 
> \## Description
> 
> Hugging Face Hub Integration does not work. I can access HF Inference API from curl, but jupyter-ai chat fails with the \`ValueError: Error raised by inference API: Cannot override task for LLM models\` (see Context section for the full traceback). I've tried different open models: mistralai/Mistral-7B-Instruct-v0.2, bigcode/starcoder2-3b. None of them works.
> 
> I guess, the reason is that you \[set\](https://github.com/jupyterlab/jupyter-ai/blob/759c60c0796c8a70b8278d17cc1e6e4fcd8c5fec/packages/jupyter-ai-magics/jupyter\_ai\_magics/providers.py#L622) \`task\` argument explicitly, while langchain \[does not set it\](https://github.com/langchain-ai/langchain/blame/6e1e0c7d5c4f0983d8fa7bdf93a65e538adca2a3/libs/community/langchain\_community/llms/huggingface\_hub.py#L70). Moreover, Langchain uses InferenceClient instead of deprecated InferenceApi.
> 
> Anyway, the best solution, in my opinion, is to replace langchain's HuggingFaceHub with HuggingFaceEndpoint, because the first is \[deprecated\](https://github.com/langchain-ai/langchain/blame/6e1e0c7d5c4f0983d8fa7bdf93a65e538adca2a3/libs/community/langchain\_community/llms/huggingface\_hub.py#L26) in langchain.
> 
> \## Reproduce
> 
> 
> 
> 1. Go to Jupyter AI Chat Settings
> 2. Choose Hugging Face Hub as Language Model
> 3. Set Local Model ID to bigcode/starcoder2-3b
> 4. Set valid API key and click Save
> 5. Go back to chat and write "Test"
> 6. See error in the chat
> 
> \<!--Describe how you diagnosed the issue. See the guidelines at
> https://jupyterlab.readthedocs.io/en/latest/getting\_started/issue.html --\>
> 
> \## Expected behavior
> 
> Any answer from the model served on HF Inference API
> 
> \## Context
> 
> 
> 
> \<details\>\<summary\>Traceback From Chat\</summary\>
> \<pre\>
> Traceback (most recent call last):
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_ai/chat\_handlers/base.py", line 125, in on\_message
> await self.process\_message(message)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_ai/chat\_handlers/default.py", line 61, in process\_message
> response = await self.llm\_chain.apredict(input=message.body, stop=\["\\nHuman:"\])
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 333, in apredict
> return (await self.acall(kwargs, callbacks=callbacks))\[self.output\_key\]
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/\_api/deprecation.py", line 157, in awarning\_emitting\_wrapper
> return await wrapped(\*args, \*\*kwargs)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall
> return await self.ainvoke(
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke
> raise e
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
> await self.\_acall(inputs, run\_manager=run\_manager)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 298, in \_acall
> response = await self.agenerate(\[inputs\], run\_manager=run\_manager)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 165, in agenerate
> return await self.llm.agenerate\_prompt(
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/language\_models/llms.py", line 643, in agenerate\_prompt
> return await self.agenerate(
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/language\_models/llms.py", line 1018, in agenerate
> output = await self.\_agenerate\_helper(
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/language\_models/llms.py", line 882, in \_agenerate\_helper
> raise e
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/language\_models/llms.py", line 866, in \_agenerate\_helper
> await self.\_agenerate(
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain\_core/language\_models/llms.py", line 1338, in \_agenerate
> else await self.\_acall(prompt, stop=stop, \*\*kwargs)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_ai\_magics/providers.py", line 684, in \_acall
> return await self.\_call\_in\_executor(\*args, \*\*kwargs)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_ai\_magics/providers.py", line 322, in \_call\_in\_executor
> return await loop.run\_in\_executor(executor, \_call\_with\_args)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/concurrent/futures/thread.py", line 58, in run
> result = self.fn(\*self.args, \*\*self.kwargs)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_ai\_magics/providers.py", line 647, in \_call
> raise ValueError(f"Error raised by inference API: {response\['error'\]}")
> ValueError: Error raised by inference API: Cannot override task for LLM models
> \</pre\>
> \</details\>
> 
> \- Operating System and version: Ubuntu 20.04.6 LTS
> \- Browser and version: Firefox 120.0.1
> \- JupyterLab version: 4.1.8
> 
> 
> 
> \<details\>\<summary\>Troubleshoot Output\</summary\>
> \<pre\>
> $PATH:
> /home/user/.pyenv/versions/3.10.5/bin
> /home/user/.pyenv/libexec
> /home/user/.pyenv/plugins/python-build/bin
> /home/user/.pyenv/plugins/pyenv-virtualenv/bin
> /home/user/.pyenv/plugins/pyenv-update/bin
> /home/user/.pyenv/plugins/pyenv-jupyter-kernel/bin
> /home/user/.pyenv/plugins/pyenv-installer/bin
> /home/user/.pyenv/plugins/pyenv-doctor/bin
> /home/user/.local/bin
> /home/user/.pyenv/plugins/pyenv-virtualenv/shims
> /home/user/.pyenv/shims
> /home/user/.pyenv/bin
> /usr/local/sbin
> /usr/local/bin
> /usr/sbin
> /usr/bin
> /sbin
> /bin
> /usr/games
> /usr/local/games
> /snap/bin
> 
> sys.path:
> /home/user/.pyenv/versions/3.10.5/bin
> /home/user/.pyenv/versions/3.10.5/lib/python310.zip
> /home/user/.pyenv/versions/3.10.5/lib/python3.10
> /home/user/.pyenv/versions/3.10.5/lib/python3.10/lib-dynload
> /home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages
> 
> sys.executable:
> /home/user/.pyenv/versions/3.10.5/bin/python3.10
> 
> sys.version:
> 3.10.5 (main, Jun 16 2022, 15:48:16) \[GCC 9.4.0\]
> 
> platform.platform():
> Linux-5.4.0-177-generic-x86\_64-with-glibc2.31
> 
> which -a jupyter:
> /home/user/.pyenv/versions/3.10.5/bin/jupyter
> /home/user/.pyenv/shims/jupyter
> 
> pip list:
> Package Version
> ------------------------------------- ------------------
> absl-py 1.1.0
> adal 1.2.7
> aiofiles 22.1.0
> aiohttp 3.8.4
> aiosignal 1.3.1
> aiosqlite 0.19.0
> altair 5.0.1
> annotated-types 0.6.0
> anyio 3.6.1
> appdirs 1.4.4
> APScheduler 3.6.3
> argcomplete 3.3.0
> argon2-cffi 21.3.0
> argon2-cffi-bindings 21.2.0
> arrow 1.2.3
> astroid 2.11.6
> asttokens 2.0.5
> astunparse 1.6.3
> async-lru 2.0.4
> async-timeout 4.0.2
> attrs 23.1.0
> autopep8 1.6.0
> azure 4.0.0
> azure-applicationinsights 0.1.1
> azure-batch 4.1.3
> azure-common 1.1.28
> azure-core 1.24.1
> azure-cosmosdb-nspkg 2.0.2
> azure-cosmosdb-table 1.0.6
> azure-datalake-store 0.0.52
> azure-eventgrid 1.3.0
> azure-graphrbac 0.40.0
> azure-keyvault 1.1.0
> azure-loganalytics 0.1.1
> azure-mgmt 4.0.0
> azure-mgmt-advisor 1.0.1
> azure-mgmt-applicationinsights 0.1.1
> azure-mgmt-authorization 0.50.0
> azure-mgmt-batch 5.0.1
> azure-mgmt-batchai 2.0.0
> azure-mgmt-billing 0.2.0
> azure-mgmt-cdn 3.1.0
> azure-mgmt-cognitiveservices 3.0.0
> azure-mgmt-commerce 1.0.1
> azure-mgmt-compute 4.6.2
> azure-mgmt-consumption 2.0.0
> azure-mgmt-containerinstance 1.5.0
> azure-mgmt-containerregistry 2.8.0
> azure-mgmt-containerservice 4.4.0
> azure-mgmt-cosmosdb 0.4.1
> azure-mgmt-datafactory 0.6.0
> azure-mgmt-datalake-analytics 0.6.0
> azure-mgmt-datalake-nspkg 3.0.1
> azure-mgmt-datalake-store 0.5.0
> azure-mgmt-datamigration 1.0.0
> azure-mgmt-devspaces 0.1.0
> azure-mgmt-devtestlabs 2.2.0
> azure-mgmt-dns 2.1.0
> azure-mgmt-eventgrid 1.0.0
> azure-mgmt-eventhub 2.6.0
> azure-mgmt-hanaonazure 0.1.1
> azure-mgmt-iotcentral 0.1.0
> azure-mgmt-iothub 0.5.0
> azure-mgmt-iothubprovisioningservices 0.2.0
> azure-mgmt-keyvault 1.1.0
> azure-mgmt-loganalytics 0.2.0
> azure-mgmt-logic 3.0.0
> azure-mgmt-machinelearningcompute 0.4.1
> azure-mgmt-managementgroups 0.1.0
> azure-mgmt-managementpartner 0.1.1
> azure-mgmt-maps 0.1.0
> azure-mgmt-marketplaceordering 0.1.0
> azure-mgmt-media 1.0.1
> azure-mgmt-monitor 0.5.2
> azure-mgmt-msi 0.2.0
> azure-mgmt-network 2.7.0
> azure-mgmt-notificationhubs 2.1.0
> azure-mgmt-nspkg 3.0.2
> azure-mgmt-policyinsights 0.1.0
> azure-mgmt-powerbiembedded 2.0.0
> azure-mgmt-rdbms 1.9.0
> azure-mgmt-recoveryservices 0.3.0
> azure-mgmt-recoveryservicesbackup 0.3.0
> azure-mgmt-redis 5.0.0
> azure-mgmt-relay 0.1.0
> azure-mgmt-reservations 0.2.1
> azure-mgmt-resource 2.2.0
> azure-mgmt-scheduler 2.0.0
> azure-mgmt-search 2.1.0
> azure-mgmt-servicebus 0.5.3
> azure-mgmt-servicefabric 0.2.0
> azure-mgmt-signalr 0.1.1
> azure-mgmt-sql 0.9.1
> azure-mgmt-storage 2.0.0
> azure-mgmt-subscription 0.2.0
> azure-mgmt-trafficmanager 0.50.0
> azure-mgmt-web 0.35.0
> azure-nspkg 3.0.2
> azure-servicebus 0.21.1
> azure-servicefabric 6.3.0.0
> azure-servicemanagement-legacy 0.20.7
> azure-storage-blob 1.5.0
> azure-storage-common 1.4.2
> azure-storage-file 1.4.0
> azure-storage-queue 1.4.0
> Babel 2.10.3
> backcall 0.2.0
> backoff 2.2.1
> beautifulsoup4 4.11.1
> bleach 1.5.0
> blis 0.7.7
> boto 2.49.0
> boto3 1.24.8
> boto3-stubs 1.24.8
> botocore 1.27.8
> botocore-stubs 1.27.8
> bz2file 0.98
> cachetools 4.2.2
> catalogue 2.0.7
> certifi 2022.5.18.1
> cffi 1.15.0
> chardet 4.0.0
> charset-normalizer 2.0.12
> click 8.1.3
> clickhouse-connect 0.5.25
> cloudpickle 3.0.0
> cloudstorage 0.11.0
> coloredlogs 15.0.1
> commonmark 0.9.1
> configparser 5.2.0
> cryptography 37.0.2
> cycler 0.11.0
> cymem 2.0.6
> Cython 0.29.30
> dask 2024.3.1
> dataclasses-json 0.5.7
> DAWG-Python 0.7.2
> debugpy 1.6.0
> decorator 5.1.1
> deepmerge 1.1.1
> defusedxml 0.7.1
> Deprecated 1.2.13
> deprecation 2.1.0
> dill 0.3.5.1
> distlib 0.3.4
> distributed 2024.3.1
> docker-pycreds 0.4.0
> docopt 0.6.2
> dogpile.cache 1.1.6
> duckdb 0.8.0
> elasticsearch 7.17.4
> elasticsearch-dsl 7.4.0
> en-core-web-sm 3.3.0
> entrypoints 0.4
> et-xmlfile 1.1.0
> executing 0.8.3
> faiss-cpu 1.8.0
> fastjsonschema 2.15.3
> ffmpy 0.3.1
> filelock 3.7.1
> flake8 4.0.1
> flatbuffers 1.12
> fonttools 4.33.3
> fqdn 1.5.1
> frozenlist 1.3.3
> fsspec 2023.5.0
> ftfy 6.1.1
> fuzzywuzzy 0.18.0
> gast 0.4.0
> gensim 4.2.0
> gitdb 4.0.10
> GitPython 3.1.37
> google-api-core 2.8.1
> google-auth 2.7.0
> google-auth-oauthlib 0.4.6
> google-cloud-core 2.3.1
> google-cloud-storage 2.4.0
> google-crc32c 1.3.0
> google-pasta 0.2.0
> google-resumable-media 2.3.3
> googleapis-common-protos 1.56.2
> gpt4all 2.6.0
> gradio 3.40.1
> gradio\_client 0.4.0
> greenlet 2.0.2
> grpcio 1.46.3
> h11 0.14.0
> h5py 3.7.0
> hnswlib 0.7.0
> html5lib 0.9999999
> httpcore 1.0.2
> httptools 0.5.0
> httpx 0.26.0
> huggingface-hub 0.14.1
> humanfriendly 10.0
> idna 3.3
> importlib\_metadata 7.1.0
> importlib-resources 6.0.1
> inflection 0.5.1
> ipykernel 6.15.0
> ipympl 0.9.2
> ipython 8.4.0
> ipython-genutils 0.2.0
> ipywidgets 8.0.2
> iso8601 1.0.2
> isodate 0.6.1
> isoduration 20.11.0
> isort 5.10.1
> itsdangerous 2.1.2
> jedi 0.17.2
> jellyfish 0.9.0
> Jinja2 3.1.2
> jmespath 1.0.0
> joblib 1.1.0
> json5 0.9.8
> jsonpatch 1.33
> jsonpath-ng 1.6.1
> jsonpointer 2.3
> jsonschema 4.19.0
> jsonschema-specifications 2023.7.1
> jupyter\_ai 2.14.1
> jupyter\_ai\_magics 2.14.1
> jupyter\_client 8.3.1
> jupyter\_core 5.3.1
> jupyter-events 0.7.0
> jupyter-lsp 2.2.0
> jupyter\_server 2.7.3
> jupyter\_server\_fileid 0.9.0
> jupyter\_server\_terminals 0.4.4
> jupyter\_server\_ydoc 0.8.0
> jupyter-ydoc 0.2.5
> jupyterlab 4.1.8
> jupyterlab-lsp 5.0.0
> jupyterlab\_pygments 0.3.0
> jupyterlab\_server 2.27.1
> jupyterlab-widgets 3.0.3
> keras 2.9.0
> Keras-Preprocessing 1.1.2
> keystoneauth1 4.6.0
> kiwisolver 1.4.3
> langchain 0.1.17
> langchain-community 0.0.36
> langchain-core 0.1.50
> langchain-text-splitters 0.0.1
> langcodes 3.3.0
> langdetect 1.0.9
> langsmith 0.1.31
> lazy-object-proxy 1.7.1
> Levenshtein 0.21.1
> libclang 14.0.1
> lightgbm 3.3.2
> linkify-it-py 2.0.2
> llama-cpp-python 0.1.56
> locket 1.0.0
> lxml 4.9.0
> lz4 4.3.2
> Markdown 3.3.7
> markdown-it-py 2.2.0
> MarkupSafe 2.1.1
> marshmallow 3.19.0
> marshmallow-enum 1.5.1
> matplotlib 3.5.2
> matplotlib-inline 0.1.3
> mccabe 0.6.1
> mdit-py-plugins 0.3.3
> mdurl 0.1.2
> minio 6.0.0
> mistune 0.8.4
> monotonic 1.6
> mpmath 1.3.0
> msg-parser 1.2.0
> msgpack 1.0.4
> msrest 0.7.1
> msrestazure 0.6.4
> multidict 6.0.4
> munch 2.5.0
> murmurhash 1.0.7
> mypy-boto3-s3 1.24.0
> mypy-extensions 1.0.0
> nbclassic 0.3.7
> nbclient 0.6.4
> nbconvert 6.5.0
> nbformat 5.4.0
> nest-asyncio 1.5.5
> netifaces 0.11.0
> nltk 3.7
> notebook 6.4.12
> notebook\_shim 0.2.3
> numexpr 2.8.4
> numpy 1.22.4
> oauthlib 3.2.0
> olefile 0.46
> onnxruntime 1.15.0
> openapi-schema-pydantic 1.2.4
> opencorpora-tools 0.6
> openpyxl 3.1.2
> openstacksdk 0.17.2
> opt-einsum 3.3.0
> orjson 3.9.15
> os-service-types 1.7.0
> overrides 7.3.1
> packaging 23.2
> pandas 1.4.2
> pandocfilters 1.5.0
> parso 0.7.1
> partd 1.4.1
> pathtools 0.1.2
> pathy 0.6.1
> pbr 5.9.0
> pdfminer.six 20221105
> pexpect 4.8.0
> pickleshare 0.7.5
> pika 1.2.1
> Pillow 9.1.1
> pip 24.0
> pipenv 2022.7.4
> pipx 1.5.0
> platformdirs 2.5.2
> pluggy 1.0.0
> ply 3.11
> posthog 3.0.1
> preshed 3.0.6
> prometheus-client 0.14.1
> prompt-toolkit 3.0.29
> protobuf 3.19.4
> psutil 5.9.1
> ptyprocess 0.7.0
> pure-eval 0.2.2
> pyasn1 0.4.8
> pyasn1-modules 0.2.8
> pycodestyle 2.8.0
> pycparser 2.21
> pydantic 2.7.1
> pydantic\_core 2.18.2
> pydocstyle 6.1.1
> pydub 0.25.1
> pyemd 0.5.1
> pyenchant 3.2.2
> pyflakes 2.4.0
> Pygments 2.12.0
> PyJWT 2.4.0
> pylint 2.14.2
> pymorphy2 0.9.1
> pymorphy2-dicts-ru 2.4.417127.4579844
> pymystem3 0.2.0
> pypandoc 1.11
> pyparsing 3.0.9
> pypdf 3.16.4
> pyrsistent 0.18.1
> python-dateutil 2.8.2
> python-docx 0.8.11
> python-dotenv 1.0.0
> python-json-logger 2.0.7
> python-jsonrpc-server 0.4.0
> python-Levenshtein 0.21.1
> python-lsp-jsonrpc 1.0.0
> python-lsp-server 1.4.1
> python-magic 0.4.27
> python-multipart 0.0.6
> python-pptx 0.6.21
> python-telegram-bot 13.12
> pytz 2022.1
> pytz-deprecation-shim 0.1.0.post0
> PyYAML 6.0
> pyzmq 25.1.1
> rackspaceauth 0.2.0
> rackspacesdk 0.7.5
> rapidfuzz 3.2.0
> redis 4.3.3
> referencing 0.30.2
> regex 2022.6.2
> requests 2.31.0
> requests-oauthlib 1.3.1
> requestsexceptions 1.4.0
> rfc3339-validator 0.1.4
> rfc3986 1.5.0
> rfc3986-validator 0.1.1
> rich 13.0.1
> rope 1.1.1
> rpds-py 0.10.2
> rsa 4.8
> s3transfer 0.6.0
> scikit-learn 1.1.1
> scipy 1.8.1
> seaborn 0.11.2
> semantic-version 2.10.0
> Send2Trash 1.8.2
> sentry-sdk 1.31.0
> setproctitle 1.3.2
> setuptools 62.4.0
> six 1.16.0
> smart-open 5.2.1
> smmap 5.0.1
> sniffio 1.2.0
> snowballstemmer 2.2.0
> sortedcontainers 2.4.0
> soupsieve 2.3.2.post1
> spacy-legacy 3.0.9
> spacy-loggers 1.0.2
> SQLAlchemy 2.0.15
> srsly 2.4.3
> stack-data 0.3.0
> starlette 0.27.0
> stevedore 3.5.0
> sympy 1.12
> tblib 3.0.0
> tenacity 8.2.2
> tensorboard 2.9.1
> tensorboard-data-server 0.6.1
> tensorboard-plugin-wit 1.8.1
> tensorflow 2.9.1
> tensorflow-estimator 2.9.0
> tensorflow-gpu 2.9.1
> tensorflow-io-gcs-filesystem 0.26.0
> termcolor 1.1.0
> terminado 0.15.0
> threadpoolctl 3.1.0
> tiktoken 0.6.0
> tinycss2 1.1.1
> tokenizers 0.13.3
> toml 0.10.2
> tomli 2.0.1
> tomlkit 0.11.0
> toolz 0.12.0
> tornado 6.3.3
> tqdm 4.64.0
> traitlets 5.9.0
> transformers 4.29.2
> transliterate 1.10.2
> typer 0.4.2
> typing\_extensions 4.6.2
> typing-inspect 0.9.0
> tzdata 2022.1
> tzlocal 4.2
> uc-micro-py 1.0.2
> ujson 5.3.0
> unstructured 0.6.11
> uri-template 1.3.0
> urllib3 1.26.9
> userpath 1.9.2
> uvicorn 0.22.0
> uvloop 0.17.0
> virtualenv 20.15.1
> virtualenv-clone 0.5.7
> wandb 0.15.11
> wasabi 0.9.1
> watchfiles 0.19.0
> wcwidth 0.2.5
> webcolors 1.13
> webencodings 0.5.1
> websocket-client 1.3.2
> websockets 11.0.3
> Werkzeug 2.1.2
> wheel 0.37.1
> widgetsnbextension 4.0.3
> wordfreq 3.0.1
> wrapt 1.14.1
> xattr 0.9.9
> xlrd 2.0.1
> XlsxWriter 3.1.2
> xlwt 1.3.0
> y-py 0.6.0
> yapf 0.32.0
> yarl 1.9.2
> ypy-websocket 0.8.4
> zict 3.0.0
> zipp 3.8.0
> zstandard 0.21.0
> \</pre\>
> \</details\>
> 
> \<details\>\<summary\>Command Line Output\</summary\>
> \<pre\>
> \[W 2024-05-03 21:37:35.229 ServerApp\] ServerApp.password config is deprecated in 2.0. Use PasswordIdentityProvider.hashed\_password.
> /home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/\_api/module\_import.py:87: LangChainDeprecationWarning: Importing GuardrailsOutputParser from langchain.output\_parsers is deprecated. Please replace the import with the following:
> from langchain\_community.output\_parsers.rail\_parser import GuardrailsOutputParser
> warnings.warn(
> /home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/document\_loaders/\_\_init\_\_.py:36: LangChainDeprecationWarning: Importing document loaders from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:
> 
> \`from langchain\_community.document\_loaders import PyPDFLoader\`.
> 
> To install langchain-community run \`pip install -U langchain-community\`.
> warnings.warn(
> \[I 2024-05-03 21:37:36.796 ServerApp\] jupyter\_ai | extension was successfully linked.
> \[I 2024-05-03 21:37:36.796 ServerApp\] jupyter\_lsp | extension was successfully linked.
> \[I 2024-05-03 21:37:36.799 ServerApp\] jupyter\_server\_fileid | extension was successfully linked.
> \[I 2024-05-03 21:37:36.803 ServerApp\] jupyter\_server\_terminals | extension was successfully linked.
> \[I 2024-05-03 21:37:36.806 ServerApp\] jupyter\_server\_ydoc | extension was successfully linked.
> \[I 2024-05-03 21:37:36.811 ServerApp\] jupyterlab | extension was successfully linked.
> \[W 2024-05-03 21:37:36.812 ServerApp\] nbclassic | error linking extension: module 'notebook' has no attribute 'base'
> Traceback (most recent call last):
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_server/extension/manager.py", line 342, in link\_extension
> extension.link\_all\_points(self.serverapp)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_server/extension/manager.py", line 224, in link\_all\_points
> self.link\_point(point\_name, serverapp)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_server/extension/manager.py", line 214, in link\_point
> point.link(serverapp)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter\_server/extension/manager.py", line 136, in link
> linker(serverapp)
> File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/nbclassic/notebookapp.py", line 197, in \_link\_jupyter\_server\_extension
> notebook.base.handlers.IPythonHandler.get\_template = get\_template
> AttributeError: module 'notebook' has no attribute 'base'
> \[I 2024-05-03 21:37:36.815 ServerApp\] notebook\_shim | extension was successfully linked.
> \[I 2024-05-03 21:37:37.135 ServerApp\] notebook\_shim | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.136 AiExtension\] Configured provider allowlist: None
> \[I 2024-05-03 21:37:37.141 AiExtension\] Configured provider blocklist: None
> \[I 2024-05-03 21:37:37.141 AiExtension\] Configured model allowlist: None
> \[I 2024-05-03 21:37:37.141 AiExtension\] Configured model blocklist: None
> \[I 2024-05-03 21:37:37.141 AiExtension\] Configured model parameters: {}
> \[I 2024-05-03 21:37:37.168 AiExtension\] Registered model provider \`ai21\`.
> \[I 2024-05-03 21:37:37.169 AiExtension\] Registered model provider \`bedrock\`.
> \[I 2024-05-03 21:37:37.169 AiExtension\] Registered model provider \`bedrock-chat\`.
> \[W 2024-05-03 21:37:37.169 AiExtension\] Unable to load model provider \`anthropic\`. Please install the \`langchain\_anthropic\` package.
> \[W 2024-05-03 21:37:37.169 AiExtension\] Unable to load model provider \`anthropic-chat\`. Please install the \`langchain\_anthropic\` package.
> \[W 2024-05-03 21:37:37.170 AiExtension\] Unable to load model provider \`azure-chat-openai\`. Please install the \`langchain\_openai\` package.
> \[I 2024-05-03 21:37:37.170 AiExtension\] Registered model provider \`cohere\`.
> \[W 2024-05-03 21:37:37.170 AiExtension\] Unable to load model provider \`gemini\`. Please install the \`langchain\_google\_genai\` package.
> \[I 2024-05-03 21:37:37.170 AiExtension\] Registered model provider \`gpt4all\`.
> \[I 2024-05-03 21:37:37.170 AiExtension\] Registered model provider \`huggingface\_hub\`.
> \[W 2024-05-03 21:37:37.170 AiExtension\] Unable to load model provider \`nvidia-chat\`. Please install the \`langchain\_nvidia\_ai\_endpoints\` package.
> \[W 2024-05-03 21:37:37.171 AiExtension\] Unable to load model provider \`openai\`. Please install the \`langchain\_openai\` package.
> \[W 2024-05-03 21:37:37.171 AiExtension\] Unable to load model provider \`openai-chat\`. Please install the \`langchain\_openai\` package.
> \[I 2024-05-03 21:37:37.171 AiExtension\] Registered model provider \`qianfan\`.
> \[I 2024-05-03 21:37:37.171 AiExtension\] Registered model provider \`sagemaker-endpoint\`.
> \[I 2024-05-03 21:37:37.171 AiExtension\] Registered model provider \`togetherai\`.
> \[I 2024-05-03 21:37:37.199 AiExtension\] Registered embeddings model provider \`bedrock\`.
> \[I 2024-05-03 21:37:37.199 AiExtension\] Registered embeddings model provider \`cohere\`.
> \[I 2024-05-03 21:37:37.199 AiExtension\] Registered embeddings model provider \`gpt4all\`.
> \[I 2024-05-03 21:37:37.199 AiExtension\] Registered embeddings model provider \`huggingface\_hub\`.
> \[E 2024-05-03 21:37:37.200 AiExtension\] Unable to load embeddings model provider class from entry point \`openai\`: No module named 'langchain\_openai'.
> \[I 2024-05-03 21:37:37.200 AiExtension\] Registered embeddings model provider \`qianfan\`.
> \[I 2024-05-03 21:37:37.206 AiExtension\] Registered providers.
> \[I 2024-05-03 21:37:37.206 AiExtension\] Registered jupyter\_ai server extension
> \[I 2024-05-03 21:37:37.237 AiExtension\] Initialized Jupyter AI server extension in 102 ms.
> \[I 2024-05-03 21:37:37.238 ServerApp\] jupyter\_ai | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.240 ServerApp\] jupyter\_lsp | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] Configured File ID manager: ArbitraryFileIdManager
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] ArbitraryFileIdManager : Configured root dir: /home/user
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] ArbitraryFileIdManager : Configured database path: /home/user/.local/share/jupyter/file\_id\_manager.db
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] ArbitraryFileIdManager : Successfully connected to database file.
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] ArbitraryFileIdManager : Creating File ID tables and indices with journal\_mode = DELETE
> \[I 2024-05-03 21:37:37.241 FileIdExtension\] Attached event listeners.
> \[I 2024-05-03 21:37:37.242 ServerApp\] jupyter\_server\_fileid | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.242 ServerApp\] jupyter\_server\_terminals | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.243 ServerApp\] jupyter\_server\_ydoc | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.244 LabApp\] JupyterLab extension loaded from /home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyterlab
> \[I 2024-05-03 21:37:37.244 LabApp\] JupyterLab application directory is /home/user/.pyenv/versions/3.10.5/share/jupyter/lab
> \[I 2024-05-03 21:37:37.245 LabApp\] Extension Manager is 'pypi'.
> \[I 2024-05-03 21:37:37.255 ServerApp\] jupyterlab | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.258 ServerApp\] nbclassic | extension was successfully loaded.
> \[I 2024-05-03 21:37:37.258 ServerApp\] Serving notebooks from local directory: /home/user
> \[I 2024-05-03 21:37:37.258 ServerApp\] Jupyter Server 2.7.3 is running at:
> \[I 2024-05-03 21:37:37.258 ServerApp\] http://localhost:8888/lab
> \[I 2024-05-03 21:37:37.258 ServerApp\] http://127.0.0.1:8888/lab
> \[I 2024-05-03 21:37:37.258 ServerApp\] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
> \[I 2024-05-03 21:37:37.703 ServerApp\] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
> \[I 2024-05-03 21:37:41.017 ServerApp\] Client connected. ID: someid
> \[I 2024-05-03 21:38:11.943 AiExtension\] Switching chat language model from None to huggingface\_hub:bigcode/starcoder2-3b.
> \[E 2024-05-03 21:38:13.027 AiExtension\] Error raised by inference API: Cannot override task for LLM models
> \[I 2024-05-03 21:38:13.029 ServerApp\] Default chat handler resolved in 1087 ms.
> \[C 2024-05-03 21:38:27.574 ServerApp\] received signal 15, stopping
> \[I 2024-05-03 21:38:27.575 ServerApp\] Shutting down 8 extensions
> \[I 2024-05-03 21:38:27.575 AiExtension\] Closing Dask client.
> 
> 
> ESC\[1m\> Entering new ConversationChain chain...ESC\[0m
> Prompt after formatting:
> ESC\[32;1mESC\[1;3mYou are Jupyternaut, a conversational assistant living in JupyterLab to help users.
> You are not a language model, but rather an application built on a foundation model from Hugging Face Hub called bigcode/starcoder2-3b.
> You are talkative and you provide lots of specific details from the foundation model's context.
> You may use Markdown to format your response.
> Code blocks must be formatted in Markdown.
> Math should be rendered with inline TeX markup, surrounded by $.
> If you do not know the answer to a question, answer truthfully by responding that you do not know.
> The following is a friendly conversation between you and a human.
> 
> Current conversation:
> 
> Human: Test
> AI:ESC\[0m
> 
> \</pre\>
> \</details\>

---

<div class="post-metadata">

**Author:** ![abhishek](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abhishek/32/32360_2.png) [@abhishek](https://discuss.huggingface.co/u/abhishek)\
**Post date:** [May 10, 2024, 9:17pm UTC](https://discuss.huggingface.co/t/error-raised-by-inference-api-cannot-override-task-for-llm-models/86169/6 "2024-05-10T21:17:11Z")

</div>

this doesnt concern autotrain. unfortunately, i cannot help. since you are using langchain, i suggest you to raise it with their developers.