# RAG on HF Inference for Pros - using Llama 2 + Llama 2 embeddings model

**URL:** <https://discuss.huggingface.co/t/rag-on-hf-inference-for-pros-using-llama-2-llama-2-embeddings-model/60236>\
**Category:** Models\
**Created:** [October 28, 2023, 9:13am UTC](https://discuss.huggingface.co/t/rag-on-hf-inference-for-pros-using-llama-2-llama-2-embeddings-model/60236 "2023-10-28T09:13:26Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![deekshith-rj](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/deekshith-rj/32/20720_2.png) [@deekshith-rj](https://discuss.huggingface.co/u/deekshith-rj)\
**Post date:** [October 28, 2023, 9:13am UTC](https://discuss.huggingface.co/t/rag-on-hf-inference-for-pros-using-llama-2-llama-2-embeddings-model/60236/1 "2023-10-28T09:13:26Z")

</div>

Hi,

I’m looking to use Hugging Face Inference for Pros along with one of the Llama 2 models + one of the Llama 2 embeddings model for one of my prototypes for Retrieval-Augmented Generation (RAG).

- Llama 2 embeddings model - [shalomma/llama-7b-embeddings · Hugging Face](https://huggingface.co/shalomma/llama-7b-embeddings)
- Llama 2 model - [Riiid/sheep-duck-llama-2-70b-v1.1 · Hugging Face](https://huggingface.co/Riiid/sheep-duck-llama-2-70b-v1.1)

My concerns about this approach include

- Are the models above compatible with each other?
- Can Inference for Pros handle them, considering that the main model has 70B params?
- Is there a better embeddings model to use with Llama 2 LLMs? Or is using Llama.cpp locally for embeddings a better approach?

Please advise.

Thanks & Regards,  
Deekshith