# Is there llama3 api for hugging face to use?

**URL:** <https://discuss.huggingface.co/t/is-there-llama3-api-for-hugging-face-to-use/105411>\
**Category:** Beginners\
**Created:** [September 5, 2024, 8:53am UTC](https://discuss.huggingface.co/t/is-there-llama3-api-for-hugging-face-to-use/105411 "2024-09-05T08:53:03Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [September 8, 2024, 1:15pm UTC](https://discuss.huggingface.co/t/is-there-llama3-api-for-hugging-face-to-use/105411/6 "2024-09-08T13:15:41Z")

</div>

Hi,

HF provides the [serverless Inference API](https://discuss.huggingface.co/t/is-there-llama3-api-for-hugging-face-to-use/105411/6) to do just that. It comes with OpenAI-compatible APIs.

Usage is as follows (add your HF token):

```python
# instead of `from openai import OpenAI`
from huggingface_hub import InferenceClient

# instead of `client = OpenAI(...)`
client = InferenceClient(
    "meta-llama/Meta-Llama-3.1-8B-Instruct",
    token=<your-hf-token>,
)

output = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Count to 10"},
    ],
    stream=True,
    max_tokens=1024,
)

for chunk in output:
    print(chunk.choices[0].delta.content)

```

By getting a [PRO subscription](https://huggingface.co/subscribe/pro), you get higher rate limits.

---

_[View the full topic](https://discuss.huggingface.co/t/is-there-llama3-api-for-hugging-face-to-use/105411)._