# Having problem running qwen2:7b locally. Getting 'AgentGenerationError: Error in generating model output'

**URL:** <https://discuss.huggingface.co/t/having-problem-running-qwen2-7b-locally-getting-agentgenerationerror-error-in-generating-model-output/159497>\
**Category:** Beginners\
**Created:** [June 16, 2025, 10:37am UTC](https://discuss.huggingface.co/t/having-problem-running-qwen2-7b-locally-getting-agentgenerationerror-error-in-generating-model-output/159497 "2025-06-16T10:37:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrmursalin](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mrmursalin/32/49456_2.png) [@mrmursalin](https://discuss.huggingface.co/u/mrmursalin)\
**Post date:** [June 16, 2025, 10:37am UTC](https://discuss.huggingface.co/t/having-problem-running-qwen2-7b-locally-getting-agentgenerationerror-error-in-generating-model-output/159497/1 "2025-06-16T10:37:13Z")

</div>

Im trying to run the code from the chapter [Building Agents That Use Code - Hugging Face Agents Course](https://huggingface.co/learn/agents-course/unit2/smolagents/code_agents) . After exceeding the inference limit , I have installed ollama on my machine and pulled the quen2:7b model and checked that ollama is running on the default url successfully.

**Here’s my code :**

```auto
!pip install smolagents -U
!pip install duckduckgo-search
!pip install 'smolagents[litellm]'

```

```auto
from smolagents import CodeAgent, DuckDuckGoSearchTool, LiteLLMModel
from smolagents import LiteLLMModel

model = LiteLLMModel(
        model_id="ollama_chat/qwen2:7b",  
        api_base="http://127.0.0.1:11434", 
        num_ctx=8192,
    )
agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=LiteLLMModel())
agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")

```

**Error logs :**

```auto
Error in generating model output:
litellm.AuthenticationError: Missing Anthropic API Key - A call is being made to anthropic but no key is set either
in the environment variables or via params. Please set ANTHROPIC_API_KEY in your environment vars

[Step 1: Duration 0.02 seconds]

---------------------------------------------------------------------------

AuthenticationError Traceback (most recent call last)

/usr/local/lib/python3.11/dist-packages/smolagents/agents.py in _step_stream(self, memory_step)
   1610 else:
-> 1611 chat_message: ChatMessage = self.model.generate(
   1612 input_messages,

12 frames

AuthenticationError: litellm.AuthenticationError: Missing Anthropic API Key - A call is being made to anthropic but no key is set either in the environment variables or via params. Please set ANTHROPIC_API_KEY in your environment vars

```

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [June 16, 2025, 2:03pm UTC](https://discuss.huggingface.co/t/having-problem-running-qwen2-7b-locally-getting-agentgenerationerror-error-in-generating-model-output/159497/2 "2025-06-16T14:03:02Z")

</div>

Hmm… Maybe you need `ollama_chat/` prefix.

> [@How to run agents from \`smolagents\` locally?](https://discuss.huggingface.co/t/how-to-run-agents-from-smolagents-locally/152874/3):
>
> I have been running everything locally by using either the TransfomerModel or MLXModel classes instead of the HfApiModel class. from smolagents import CodeAgent, MLXModel model = MLXModel( "mlx-community/Qwen2.5-Coder-32B-Instruct-4bit", { "temperature": 0.7, "top\_k": 20, "top\_p": 0.8, "min\_p": 0.05, "num\_ctx": 32768, }, ) agent = CodeAgent(tools=[], model=model, add\_base\_tools=True) agent.run( "Could you give me the 40th number in the…

---

<div class="post-metadata">

**Author:** ![ChicK00o](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/chick00o/32/51067_2.png) [@ChicK00o](https://discuss.huggingface.co/u/ChicK00o)\
**Post date:** [July 14, 2025, 10:31am UTC](https://discuss.huggingface.co/t/having-problem-running-qwen2-7b-locally-getting-agentgenerationerror-error-in-generating-model-output/159497/3 "2025-07-14T10:31:36Z")

</div>

`agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=LiteLLMModel())`  
replace with  
`agent = CodeAgent(tools=[DuckDuckGoSearchTool()], model=model)`  
check the last part of the code agent creation.  
when you dont pass info into it, then it tries to default to Claude