# Access tokenizer from within predict\_fn

**URL:** <https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697>\
**Category:** Amazon SageMaker\
**Created:** [January 14, 2022, 3:08pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697 "2022-01-14T15:08:38Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![rogeriobromfman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rogeriobromfman/32/3727_2.png) [@rogeriobromfman](https://discuss.huggingface.co/u/rogeriobromfman)\
**Post date:** [January 14, 2022, 3:08pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/1 "2022-01-14T15:08:38Z")

</div>

I’m trying to overwrite `predict_fn` for a named-entity recognition task. Mostly because I provide very long sequences. In this case, I need to use the tokenizer to break the long sequence down, and then merge the predictions of all the sub-sequences.

I call the tokenizer as:

```auto
sentences = tokenizer(sentence, max_length=max_length, stride=stride, truncation=True,
                      return_overflowing_tokens=True)

```

Since I have a stride, I need to properly care for overlapping tokens and their predictions.

I can access the model, as it is received as a parameter of the `predict_fn`, but how do I access the tokenizer?

Thanks!

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [January 14, 2022, 3:55pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/2 "2022-01-14T15:55:09Z")

</div>

Hi Rogerio

`model_fn` actually takes the model directory as a parameter, not the model itself (see here: [Deploy models to Amazon SageMaker](https://huggingface.co/docs/sagemaker/inference#user-defined-code-and-modules)). This means that you can load the tokenizer within model\_fn like so:

`tokenizer = AutoTokenizer.from_pretrained(model_dir)`

Cheers  
Heiko

---

<div class="post-metadata">

**Author:** ![rogeriobromfman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rogeriobromfman/32/3727_2.png) [@rogeriobromfman](https://discuss.huggingface.co/u/rogeriobromfman)\
**Post date:** [January 14, 2022, 3:59pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/3 "2022-01-14T15:59:00Z")

</div>

Hi Heiko,

I apologise! I meant to say `predict_fn` and not `model_fn`. I’m correcting this in the original post.

Thanks!

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [January 14, 2022, 4:03pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/4 "2022-01-14T16:03:24Z")

</div>

Ah, I see. No worries - you can just save the tokenizer in a variable and you’ll be able to access it throughout your inference.py code. Check this one as an example: [https://github.com/marshmellow77/hf-summarization/blob/main/inference\_code/inference.py](https://github.com/marshmellow77/hf-summarization/blob/main/inference_code/inference.py)

Hope that helps 🙂

---

<div class="post-metadata">

**Author:** ![rogeriobromfman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rogeriobromfman/32/3727_2.png) [@rogeriobromfman](https://discuss.huggingface.co/u/rogeriobromfman)\
**Post date:** [January 14, 2022, 4:10pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/5 "2022-01-14T16:10:37Z")

</div>

This link is not working for me. Maybe it’s not a public repository?

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [January 14, 2022, 4:12pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/6 "2022-01-14T16:12:50Z")

</div>

Apologies - this is the correct link: [text-summarisation-project/inference.py at main · marshmellow77/text-summarisation-project · GitHub](https://github.com/marshmellow77/text-summarisation-project/blob/main/inference_code/inference.py)

---

<div class="post-metadata">

**Author:** ![rogeriobromfman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rogeriobromfman/32/3727_2.png) [@rogeriobromfman](https://discuss.huggingface.co/u/rogeriobromfman)\
**Post date:** [January 14, 2022, 4:14pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/7 "2022-01-14T16:14:59Z")

</div>

I see what you did there! Great idea to pass the tokenizer and model in a dictionary! I’ll implement and update here once I get it to work. Thank you so much once again!!

---

<div class="post-metadata">

**Author:** ![rogeriobromfman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rogeriobromfman/32/3727_2.png) [@rogeriobromfman](https://discuss.huggingface.co/u/rogeriobromfman)\
**Post date:** [January 14, 2022, 7:53pm UTC](https://discuss.huggingface.co/t/access-tokenizer-from-within-predict-fn/13697/8 "2022-01-14T19:53:36Z")

</div>

Worked like a charm! I can now run very long sequences through the token classifier. Thank you Heiko!