# Lama 3.23b performs great when I download and use using ollama but when I manually download the model or if I use the gguf model by unsloth, it gives me irrelevant response. Please help me out **URL:** \ **Category:** Beginners\ **Created:** [October 30, 2024, 7:05pm UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768 "2024-10-30T19:05:43Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![anikeths](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/anikeths/32/33694_2.png) [@anikeths](https://discuss.huggingface.co/u/anikeths)\ **Post date:** [October 30, 2024, 7:05pm UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/1 "2024-10-30T19:05:43Z")
Hi, Im very new to this but quite interested about LLM’s, I’m working on a project that requires me to fine tune an LLM but the issue is the gguf models of Llama 3.23b that I download and try to run give me weird outputs, like the one below. ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/d/f/dfaa39e4b37e290babe03c2395a31bafc88ad632.jpeg) But when I use the one from ollama itself(command: ollama run llama3.2)… it works fine ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 30, 2024, 11:35pm UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/2 "2024-10-30T23:35:48Z")
It seems that Ollama internally loads the **Instruct** model instead of **Base**. > [@How to Configure LLaMA-3:8B on HuggingFace to Generate Responses Similar to Ollama?](https://discuss.huggingface.co/t/how-to-configure-llama-3-8b-on-huggingface-to-generate-responses-similar-to-ollama/109960/1): > > Hi HuggingFace Community, I have been experimenting with the LLaMA-3:8B model using the following code: import transformers import torch model\_id = "meta-llama/Meta-Llama-3-8B" pipeline = transformers.pipeline( "text-generation", model=model\_id, model\_kwargs={"torch\_dtype": torch.bfloat16}, device\_map="auto" ) pipeline("Hey how are you doing today?") This code generates the following response: [{'generated\_text': 'Hey how are you doing today? My name is David and I’m a photographer bas… ---
**Author:** ![anikeths](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/anikeths/32/33694_2.png) [@anikeths](https://discuss.huggingface.co/u/anikeths)\ **Post date:** [October 31, 2024, 3:55am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/3 "2024-10-31T03:55:28Z")
Thank you for replying. Could you help me find the instruct model that works as a chatbot please. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 31, 2024, 4:14am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/4 "2024-10-31T04:14:18Z")
Okay. ## Official one (gated. requires authorization) > **[meta-llama/Llama-3.2-3B-Instruct · Hugging Face](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ## Unofficial ones (ungated) > **[unsloth/Llama-3.2-3B-Instruct · Hugging Face](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. > **[chuanli11/Llama-3.2-3B-Instruct-uncensored · Hugging Face](https://huggingface.co/chuanli11/Llama-3.2-3B-Instruct-uncensored)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ## GGUF files for Llamacpp > **[bartowski/Llama-3.2-3B-Instruct-GGUF · Hugging Face](https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. ---
**Author:** ![anikeths](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/anikeths/32/33694_2.png) [@anikeths](https://discuss.huggingface.co/u/anikeths)\ **Post date:** [October 31, 2024, 6:45am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/5 "2024-10-31T06:45:49Z")
Thank you! I used unsloth and bartowski but no luck 😂. It gives me the same kind of weird output. ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 31, 2024, 6:53am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/6 "2024-10-31T06:53:26Z")
How about this? > **[Llama 3.2 3B Instruct - a Hugging Face Space by huggingface-projects](https://huggingface.co/spaces/huggingface-projects/llama-3.2-3B-Instruct)** > > Chatbot > **[Llama 3.2 3B Uncensored Chat - a Hugging Face Space by chuanli11](https://huggingface.co/spaces/chuanli11/Chat-Llama-3.2-3B-Instruct-uncensored)** > > Discover amazing ML apps made by the community ---
**Author:** ![anikeths](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/anikeths/32/33694_2.png) [@anikeths](https://discuss.huggingface.co/u/anikeths)\ **Post date:** [October 31, 2024, 7:51am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/7 "2024-10-31T07:51:29Z")
Thank you so much @John6666 for taking your time and helping me out. I realized where I went wrong, I had to include a template in my model file before running it in ollama. Your first comment where you mentioned a link to a similar problem helped me out, I found this link in the comments… > and it said I had to include this template in my modelfile TEMPLATE “”“{{ if .System }}\<|im\_start|\>system {{ .System }}\<|im\_end|\> {{ end }}{{ if .Prompt }}\<|im\_start|\>user {{ .Prompt }}\<|im\_end|\> {{ end }}\<|im\_start|\>assistant “”” This solved the problem and now my models work like charm. Big thanks to you for helping me out! ---
**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\ **Post date:** [October 31, 2024, 8:11am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/8 "2024-10-31T08:11:44Z")
So it was a template issue. Good to have it resolved! ---
**Author:** ![anikeths](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/anikeths/32/33694_2.png) [@anikeths](https://discuss.huggingface.co/u/anikeths)\ **Post date:** [October 31, 2024, 8:31am UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/9 "2024-10-31T08:31:41Z")
Yuppp, Thank you ---
**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\ **Post date:** [October 31, 2024, 8:31pm UTC](https://discuss.huggingface.co/t/lama-3-23b-performs-great-when-i-download-and-use-using-ollama-but-when-i-manually-download-the-model-or-if-i-use-the-gguf-model-by-unsloth-it-gives-me-irrelevant-response-please-help-me-out/114768/10 "2024-10-31T20:31:52Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.