# Inference Endpoints 401 Error

**URL:** <https://discuss.huggingface.co/t/inference-endpoints-401-error/96833>\
**Category:** Intermediate\
**Created:** [July 11, 2024, 2:34pm UTC](https://discuss.huggingface.co/t/inference-endpoints-401-error/96833 "2024-07-11T14:34:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheRealPapaBear](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TheRealPapaBear](https://discuss.huggingface.co/u/TheRealPapaBear)\
**Post date:** [July 11, 2024, 2:34pm UTC](https://discuss.huggingface.co/t/inference-endpoints-401-error/96833/1 "2024-07-11T14:34:12Z")

</div>

I have a llama3-8B fine tuned pipeline in Colab built with transformers. I’ve tested it in Colab and working fine. The only problem I have is that I can’t seem to be able to get passed this error on how the model is gated.

I’ve created an env variable HF\_TOKEN\_API in my runtime, passing that everywhere in the code.

I’ve added it in the Inference Endpoint settings too as an env variable along with its value and getting the same thing.

```auto
OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/meta-llama/Meta-Llama-3-8B.
401 Client Error. (Request ID: Root=1-668feaf5-37525ba76173671b77b38d68;fad4290c-56e7-4679-a073-69c13b9ec2e8)

Cannot access gated repo for url https://huggingface.co/meta-llama/Meta-Llama-3-8B/resolve/main/config.json.
Access to model meta-llama/Meta-Llama-3-8B is restricted. You must be authenticated to access it.

Application startup failed. Exiting.

```

---

<div class="post-metadata">

**Author:** ![TheRealPapaBear](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TheRealPapaBear](https://discuss.huggingface.co/u/TheRealPapaBear)\
**Post date:** [July 15, 2024, 8:59am UTC](https://discuss.huggingface.co/t/inference-endpoints-401-error/96833/2 "2024-07-15T08:59:26Z")

</div>

I figured it out.

So HuggingFace has some preset environment variables that are there for us to use.

Link: [Environment variables (huggingface.co)](https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables)

The idea is that when you build your pipeline you need to login either via the hugging-face cli or import the login from huggingface\_hub like so:

```auto
from huggingface_hub import login

```

In order to get pass this error is to create and pass in your huggingface token to the preset hugging face environment variable called HF\_TOKEN like so:

```auto
# Declare env variable in the pipeline and use it where needed

os.environ["HF_TOKEN"] = "You hugging face token" - Also this needs to be either write or fine grained.

```

You will pass this everywhere you need to use the hugging face token like so:

```auto
# When you login

login(token=os.environ["HF_TOKEN"], add_to_git_credential=True)

# When you define training arguments

training_args = TrainingArguments(
    ... other trainig arguments,
    hub_token=os.environ["HF_TOKEN"],
)

#When you push the model and tokenizer to the HF repo

model.push_to_hub(repo_name, use_auth_token=os.environ["HF_TOKEN"])
tokenizer.push_to_hub(repo_name, use_auth_token=os.environ["HF_TOKEN"])

```

The last step is to declare the HF\_TOKEN as an env variable in the Inference Endpoints settings and pass in the value and then deploy the endpoint.

This should short out your issues.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [July 17, 2024, 1:16pm UTC](https://discuss.huggingface.co/t/inference-endpoints-401-error/96833/3 "2024-07-17T13:16:31Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.