# Token positions when using the Inference API

**URL:** <https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188>\
**Category:** Beginners\
**Created:** [November 24, 2020, 4:21pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188 "2020-11-24T16:21:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fdb](https://avatars.discourse-cdn.com/v4/letter/f/c89c15/32.png) [@fdb](https://discuss.huggingface.co/u/fdb)\
**Post date:** [November 24, 2020, 4:21pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188/1 "2020-11-24T16:21:44Z")

</div>

Hi,

I am looking to run some NER models via the Inference API, but I am running into some issues. My problem is that the Inference API does not seem to return token positions. Consider this request:

```
curl -X POST https://api-inference.huggingface.co/models/dslim/bert-base-NER \
-H "Authorization: Bearer <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d "Hello Sarah Jessia Parker who lives in New York."

```

It returns:

```
[
  {
    "entity_group": "PER",
    "score": 0.9959956109523773,
    "word": "Sarah Jessia Parker"
  },
  {
    "entity_group": "LOC",
    "score": 0.9994343519210815,
    "word": "New York"
  }
]

```

So it finds the right tokens (and, nicely, returns the identified tokens grouped correctly.) However, there is no indication of where the tokens start in the input text.

Confusingly, the model page ([https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessia+Parker+who+lives+in+New+York.)](https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessia+Parker+who+lives+in+New+York.)) highlights the right tokens, which would suggest you _can_ get token positions. (unless it’s doing something terribly hacky like looking for the first occurence of a particular token).

Is there an option I’m missing?

---

<div class="post-metadata">

**Author:** ![fdb](https://avatars.discourse-cdn.com/v4/letter/f/c89c15/32.png) [@fdb](https://discuss.huggingface.co/u/fdb)\
**Post date:** [November 24, 2020, 4:26pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188/2 "2020-11-24T16:26:17Z")

</div>

Indeed, the page seems to just be highlighting the first occurence of a token. Note how it (probably) picks up the wrong “Jessica” in this example: [https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York](https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York).

Is there a way to extract the token position from the Inference API?

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [November 25, 2020, 4:34pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188/3 "2020-11-25T16:34:29Z")

</div>

Hi @fdb,

Thanks for the note. You are perfectly right in your assumptions.  
In earlier versions of tokenizers/transformers the offsets of tokens were not necessarily available (because the `encode` operation was destructive). Because offsets were not available, the API did not send this information.

However thanks to `tokenizers` @anthony and in the near future 4.0 version of transformers, the `encode` operation won’t be destructive anymore, so `offsets` _will_ be available.

There is a PR for transformers for this pipeline [https://github.com/huggingface/transformers/pull/8781](https://github.com/huggingface/transformers/pull/8781)  
And the API inference is already running the fix:

```auto
curl -X POST -d 'Hello Sarah Jessica Parker who Jessica lives in New York' https://api-inference.huggingface.co/models/dslim/bert-base-NER

```

```auto
[
{"entity_group":"PER","score":0.9960219860076904,"word":"Sarah Jessica Parker","start":6,"end":26},
{"entity_group":"PER","score":0.9771094918251038,"word":"Jessica","start":31,"end":38},
{"entity_group":"LOC","score":0.9994266927242279,"word":"New York","start":48,"end":56}
]

```

[https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York](https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York). should soon follow.

---

<div class="post-metadata">

**Author:** ![fdb](https://avatars.discourse-cdn.com/v4/letter/f/c89c15/32.png) [@fdb](https://discuss.huggingface.co/u/fdb)\
**Post date:** [November 25, 2020, 5:50pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188/4 "2020-11-25T17:50:07Z")

</div>

Hi @Narsil,

Indeed, I now see start/end positions in my responses from the inference API. Thanks for adding those fields!

---

<div class="post-metadata">

**Author:** ![Narsil](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/narsil/32/40029_2.png) [@Narsil](https://discuss.huggingface.co/u/Narsil)\
**Post date:** [November 25, 2020, 5:53pm UTC](https://discuss.huggingface.co/t/token-positions-when-using-the-inference-api/2188/5 "2020-11-25T17:53:04Z")

</div>

The hard work was not done by me 🙂