# Stopping criteria for Llama2 70b

**URL:** <https://discuss.huggingface.co/t/stopping-criteria-for-llama2-70b/48050>\
**Category:** 🤗Transformers\
**Created:** [July 25, 2023, 9:50am UTC](https://discuss.huggingface.co/t/stopping-criteria-for-llama2-70b/48050 "2023-07-25T09:50:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![anishAlapattu](https://avatars.discourse-cdn.com/v4/letter/a/b3f665/32.png) [@anishAlapattu](https://discuss.huggingface.co/u/anishAlapattu)\
**Post date:** [July 25, 2023, 9:50am UTC](https://discuss.huggingface.co/t/stopping-criteria-for-llama2-70b/48050/1 "2023-07-25T09:50:49Z")

</div>

I have used the following code for defining the stopping criteria for Llama2

```auto
from transformers import StoppingCriteria, StoppingCriteriaList

# define custom stopping criteria object
class StopOnTokens(StoppingCriteria):
    def __call__ (self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
        for stop_ids in stop_token_ids:
            if torch.eq(input_ids[0][-len(stop_ids):], stop_ids).all():
                return True
        return False

stopping_criteria = StoppingCriteriaList([StopOnTokens()])

```

This is my stop\_list = ['\nHuman: ', ‘\n```\n’, '\nAI: ', ‘\nQuestion’]  
However to my question “Who is the CEO of Meta?”, llama2 doesn’t stop on any of these stop tokens.  
Also attaching the code for conversion of tokens to longtensor

```auto
stop_token_ids = [tokenizer(x, return_tensors='pt')['input_ids'].squeeze() for x in stop_list]
stop_token_ids
stop_token_ids = [torch.LongTensor(x).to(device) for x in stop_token_ids]
stop_token_ids

```

Any help is appreciated! Thanks!

---

<div class="post-metadata">

**Author:** ![nghiemhnlp](https://avatars.discourse-cdn.com/v4/letter/n/59ef9b/32.png) [@nghiemhnlp](https://discuss.huggingface.co/u/nghiemhnlp)\
**Post date:** [October 19, 2023, 5:07pm UTC](https://discuss.huggingface.co/t/stopping-criteria-for-llama2-70b/48050/2 "2023-10-19T17:07:30Z")

</div>

I think you have to add “add\_special\_token=False” to restrict the tokenization to the phrases only.

---

<div class="post-metadata">

**Author:** ![crouchcd](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/crouchcd/32/22659_2.png) [@crouchcd](https://discuss.huggingface.co/u/crouchcd)\
**Post date:** [January 3, 2024, 11:11pm UTC](https://discuss.huggingface.co/t/stopping-criteria-for-llama2-70b/48050/3 "2024-01-03T23:11:11Z")

</div>

Adding a link to [add\_special\_tokens docs](https://huggingface.co/docs/transformers/main_classes/tokenizer#transformers.PreTrainedTokenizer. __call__.add_special_tokens) for reference. This fixed my problem, thanks!