# How to use llm (access fail)

**URL:** <https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076>\
**Category:** Beginners\
**Created:** [August 20, 2024, 3:04am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076 "2024-08-20T03:04:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![alice86](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@alice86](https://discuss.huggingface.co/u/alice86)\
**Post date:** [August 20, 2024, 3:04am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076/1 "2024-08-20T03:04:00Z")

</div>

[meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)

I follow the tutorial and it show the error.

‘’’  
Cannot access gated repo for url [https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/resolve/main/config.json](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/resolve/main/config.json).  
Access to model meta-llama/Meta-Llama-3.1-8B-Instruct is restricted. You must be authenticated to access it.  
‘’’

Q2.  
I download the Meta-Llama-3.1-8B-Instruct by git clone  
How to load it with the code?  
I dont see any solution?

```auto
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

```

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/52583_2.png) [@mahmutc](https://discuss.huggingface.co/u/mahmutc)\
**Post date:** [August 20, 2024, 8:18am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076/2 "2024-08-20T08:18:23Z")

</div>

hi @alice86  
You need to agree the license agreement to access this model. You can find it in the model card.

FAQ

> **[meta-llama/Meta-Llama-3.1-8B-Instruct · Access request FAQ](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/discussions/47)**
>
> You can get the Meta Llama models directly from Meta or through Hugging Face or Kaggle. However you get the models, you will first need to accept the license agreements for the models you want.

You will need this one:

> **[meta-llama/Meta-Llama-3-8B-Instruct · Denied permission to DL](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/discussions/51#666d991c357aeb334f1feca5)**
>
> HF indicates "Gated model You have been granted access to this model"

---

<div class="post-metadata">

**Author:** ![alice86](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@alice86](https://discuss.huggingface.co/u/alice86)\
**Post date:** [August 21, 2024, 12:15am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076/3 "2024-08-21T00:15:44Z")

</div>

Q1  
I already accept the license.  
And the official don’t show any method to download it.

It just direct use

```auto
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

```

Q2  
It is different question.  
If I download the model and how to load model without token? (already download model)  
Because I already download the model.

I expect the result is  
python test.py =\> it will automatically download model with token

python test.py -m ./model/llama3 =\> It will use the model that I already download without any token.

In fact, I dont see any code for python test.py -m ./model/llama3

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/52583_2.png) [@mahmutc](https://discuss.huggingface.co/u/mahmutc)\
**Post date:** [August 21, 2024, 11:10am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076/4 "2024-08-21T11:10:54Z")

</div>

hi @alice86  
About your first question:  
Did you create an access token? Did you add a permission for relevant repository?

You can check from [Hugging Face – The AI community building the future.](https://huggingface.co/settings/tokens).  
(Edit permissions → Repositories permissions)

You need to run something like this:

```auto
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
    token="hf_xxxxxxx",
)

```

> **[User access tokens](https://huggingface.co/docs/hub/en/security-tokens)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/52583_2.png) [@mahmutc](https://discuss.huggingface.co/u/mahmutc)\
**Post date:** [August 21, 2024, 11:31am UTC](https://discuss.huggingface.co/t/how-to-use-llm-access-fail/103076/5 "2024-08-21T11:31:20Z")

</div>

About your second question:  
You need to call the pipeline with `model='./path_for_local_model'`:

```auto
pipeline = transformers.pipeline(
    "text-generation",
    model='./path_for_local_model',
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

```