# How can I get the logits from an endpoint call?

**URL:** <https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047>\
**Category:** Inference Endpoints on the Hub\
**Created:** [August 26, 2024, 1:19pm UTC](https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047 "2024-08-26T13:19:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mfixman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mfixman/32/29758_2.png) [@mfixman](https://discuss.huggingface.co/u/mfixman)\
**Post date:** [August 26, 2024, 1:19pm UTC](https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047/1 "2024-08-26T13:19:56Z")

</div>

I’m attempting to do a query similar to this one using the Huggingface inference endpoints.

```auto
api_url = 'https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3.1-70B-Instruct'
headers = {'Authorization': f'Bearer {token}'}
response = requests.post(api_url, headers = headers, json = {'inputs': 'What is the capital of France? The capital of France is : ')

```

I’m not just looking for the answer, but also for the logits of the generated search: I want to be able to calculate the probability of getting a certain answer.

I can do this with AutoModelForCausalInference, but most big models don’t fit my GPUs (and a HF Pro subscription is cheaper than another A100).

Is there any way to use the API this way?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [August 26, 2024, 1:33pm UTC](https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047/2 "2024-08-26T13:33:37Z")

</div>

Hi,

According to [Detailed parameters](https://huggingface.co/docs/api-inference/en/detailed_parameters#text-generation-task), the serverless inference API does not support returning logits.

If you want that, you could define a [custom handler](https://huggingface.co/docs/inference-endpoints/guides/custom_handler) on Inference Endpoints which also returns logits besides text.

---

<div class="post-metadata">

**Author:** ![mfixman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mfixman/32/29758_2.png) [@mfixman](https://discuss.huggingface.co/u/mfixman)\
**Post date:** [August 26, 2024, 1:56pm UTC](https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047/3 "2024-08-26T13:56:52Z")

</div>

> [@nielsr](#):
>
> According to [Detailed parameters](https://huggingface.co/docs/api-inference/en/detailed_parameters#text-generation-task), the serverless inference API does not support returning logits.

Just checking: can I use the API calls in inference endpoints with logits as part of an experiment I’m doing in my local computer, or do I have to use gradio, Spaces, or some other library like that?

---

<div class="post-metadata">

**Author:** ![kerrmetric](https://avatars.discourse-cdn.com/v4/letter/k/f14d63/32.png) [@kerrmetric](https://discuss.huggingface.co/u/kerrmetric)\
**Post date:** [August 30, 2024, 6:03am UTC](https://discuss.huggingface.co/t/how-can-i-get-the-logits-from-an-endpoint-call/104047/4 "2024-08-30T06:03:03Z")

</div>

+1 I’d be interested in an easy/standard way to do this as well.