# How do I send system prompts using inference api serverless, llama3 8b instruct model

**URL:** <https://discuss.huggingface.co/t/how-do-i-send-system-prompts-using-inference-api-serverless-llama3-8b-instruct-model/83639>\
**Category:** Beginners\
**Created:** [April 25, 2024, 3:51am UTC](https://discuss.huggingface.co/t/how-do-i-send-system-prompts-using-inference-api-serverless-llama3-8b-instruct-model/83639 "2024-04-25T03:51:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![betaalpha](https://avatars.discourse-cdn.com/v4/letter/b/47e85d/32.png) [@betaalpha](https://discuss.huggingface.co/u/betaalpha)\
**Post date:** [April 25, 2024, 3:51am UTC](https://discuss.huggingface.co/t/how-do-i-send-system-prompts-using-inference-api-serverless-llama3-8b-instruct-model/83639/1 "2024-04-25T03:51:37Z")

</div>

I’m using lama 3 8 b model, using the inference api serverless, like this:

```auto
import requests

API_URL = "https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct"
headers = {"Authorization": "Bearer mytoken"}

def query(payload):
	response = requests.post(API_URL, headers=headers, json=payload)
	return response.json()
	
output = query({
	"inputs": "Can you please let us know more details about your ",
})
print(output)

```

everything works great, but how do I use a system prompt, & do I need to send it with every query or only once, also is it possible to have it remember conversations or do I’ve to send old ones with every query, finally, how do I make it to complete or to chat, like on the huggingface chat ui

---

<div class="post-metadata">

**Author:** ![marcoguerrap](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@marcoguerrap](https://discuss.huggingface.co/u/marcoguerrap)\
**Post date:** [April 28, 2024, 3:38pm UTC](https://discuss.huggingface.co/t/how-do-i-send-system-prompts-using-inference-api-serverless-llama3-8b-instruct-model/83639/2 "2024-04-28T15:38:08Z")

</div>

Hey I found this post that helped me:

> [@Formatting Inference API call for LLama 2](https://discuss.huggingface.co/t/formatting-inference-api-call-for-llama-2/54901/3):
>
> I solved it by inputting a single string using the official Llama 2 format (see [Llama 2 is here - get it on Hugging Face](https://huggingface.co/blog/llama2#how-to-prompt-llama-2)). I don’t know why the default Sagemaker Llama endpoint doesn’t work that way. But this works for me: import json import requests API\_URL = "https://api-inference.huggingface.co/models/meta-llama/Llama-2-7b-chat-hf" headers = {"Authorization": f"Bearer hf\_XXXXXXXXXXXXXXXXXX", "Content-Type": "application/json",} def query(payload): json\_body = { "inputs":…

---

<div class="post-metadata">

**Author:** ![marcoguerrap](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@marcoguerrap](https://discuss.huggingface.co/u/marcoguerrap)\
**Post date:** [April 29, 2024, 12:27pm UTC](https://discuss.huggingface.co/t/how-do-i-send-system-prompts-using-inference-api-serverless-llama3-8b-instruct-model/83639/3 "2024-04-29T12:27:16Z")

</div>

[Meta Llama 3 | Model Cards and Prompt formats](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3) for the appropriate tags of Llama 3