# Using alpaca with local embedding

**URL:** <https://discuss.huggingface.co/t/using-alpaca-with-local-embedding/39413>\
**Category:** Intermediate\
**Created:** [May 10, 2023, 3:47pm UTC](https://discuss.huggingface.co/t/using-alpaca-with-local-embedding/39413 "2023-05-10T15:47:43Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rmcf1902](https://avatars.discourse-cdn.com/v4/letter/r/71c47a/32.png) [@rmcf1902](https://discuss.huggingface.co/u/rmcf1902)\
**Post date:** [May 10, 2023, 3:47pm UTC](https://discuss.huggingface.co/t/using-alpaca-with-local-embedding/39413/1 "2023-05-10T15:47:43Z")

</div>

Hi there!

I am using huggingface model chavinlo/alpaca-native.  
However, when i use local embeddings, my output is always only 1 word long. Can anyone explain this?

```auto
model_nm = 'chavinlo/alpaca-native' 
save_path = '/content/drive/MyDrive/alpaca_native_pretrained_model_pytorch'

model = LlamaForCausalLM.from_pretrained(save_path, return_dict=True, load_in_8bit=True, device_map='auto')

tokenizer = AutoTokenizer.from_pretrained(save_path)

pipe = pipeline(
    "text-generation",
    model=model, 
    tokenizer=tokenizer, 
    max_length=248,
    temperature=0.4,
    top_p=0.95,
    repetition_penalty=1.2
)

local_llm = HuggingFacePipeline(pipeline=pipe)

qa = RetrievalQA.from_chain_type(
    llm=local_llm,
    chain_type="stuff", # "map_reduce",
    retriever=retriever,
    return_source_documents=True,
)

query = "xyz"
llm_response = qa(query)

```

Can anyone help me with that or suggest me alternative ways to embed pdf’s with an LLM, everything locally on colab?

Thanks!  
Yves

---

<div class="post-metadata">

**Author:** ![sarmadq](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@sarmadq](https://discuss.huggingface.co/u/sarmadq)\
**Post date:** [July 19, 2023, 12:58am UTC](https://discuss.huggingface.co/t/using-alpaca-with-local-embedding/39413/2 "2023-07-19T00:58:08Z")

</div>

The max\_length you’ve specified is 248. That might not be enough to include the context from the RetrievalQA embeddings, plus your question, and so the response returned is small because the prompt is exceeding the context window. Try increasing max\_length to start with.