# Using LoRA Adapters

**URL:** <https://discuss.huggingface.co/t/using-lora-adapters/70392>\
**Category:** Beginners\
**Created:** [January 24, 2024, 10:08am UTC](https://discuss.huggingface.co/t/using-lora-adapters/70392 "2024-01-24T10:08:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![fame2312](https://avatars.discourse-cdn.com/v4/letter/f/b2d939/32.png) [@fame2312](https://discuss.huggingface.co/u/fame2312)\
**Post date:** [January 24, 2024, 10:08am UTC](https://discuss.huggingface.co/t/using-lora-adapters/70392/1 "2024-01-24T10:08:17Z")

</div>

Hello,  
for fine tuning LLama 2-7B with the LoraConfig I quantized it in int8.  
After that I saved the adapters.

Now for the inference, I have to load the base model and load the adapters in it, like:

model\_id = “meta-llama/Llama-2-7b-chat-hf”  
model = AutoModelForCausalLM.from\_pretrained(  
model\_id,  
device\_map=“auto”,  
cache\_dir=“/mnt/video/llm\_models/”,  
torch\_dtype=torch.float16)

model = PeftModel.from\_pretrained(model, path\_to\_the\_adpaters)

Should I use the basic model LLama 2 for the inference, as above, or the quantized one as below, with which I created the adapters with fine tuning

model\_id = “meta-llama/Llama-2-7b-chat-hf”  
model = AutoModelForCausalLM.from\_pretrained(  
model\_id,  
device\_map=“auto”,  
cache\_dir=“/mnt/video/llm\_models/”,  
load\_in\_8bit=True)

model = PeftModel.from\_pretrained(model, path\_to\_the\_adpaters)