# TypeError when loading tokenizer with from\_pretrained method for bart-large-mnli model **URL:** \ **Category:** 🤗Tokenizers\ **Created:** [January 25, 2021, 8:06pm UTC](https://discuss.huggingface.co/t/typeerror-when-loading-tokenizer-with-from-pretrained-method-for-bart-large-mnli-model/3378 "2021-01-25T20:06:50Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![StarDriver](https://avatars.discourse-cdn.com/v4/letter/s/41988e/32.png) [@StarDriver](https://discuss.huggingface.co/u/StarDriver)\ **Post date:** [January 25, 2021, 8:06pm UTC](https://discuss.huggingface.co/t/typeerror-when-loading-tokenizer-with-from-pretrained-method-for-bart-large-mnli-model/3378/1 "2021-01-25T20:06:50Z")
Hello everyone. Here is my problem, (I wish someone can help me, I try so hard in vain to resolve it T.T) : I use transformers 4.2.1 lib, and I am in a context where I only can use it in offline mode (no internet). I want to use the bart-large-mnli model so I upload it on a specific server and I download the model with the following link : > **[facebook/bart-large-mnli at main](https://huggingface.co/facebook/bart-large-mnli/tree/main)** > > We’re on a journey to advance and democratize artificial intelligence through open source and open science. Then I try to use from\_pretrained method like this : tokenizer = BartTokenizerFast.from\_pretrained(‘/appli/pretrainedModel/bart-large-mnli’) or like this : tokenizer = AutoTokenizer.from\_pretrained(‘/appli/pretrainedModel/bart-large-mnli’) But every time I do this I get the following error (more detailed log at the end of my post, I truncate the last line with […] to replace all the content of merges.txt) : “TypeError: Can’t convert [(‘Ä’, ‘t’), (‘Ä’, ‘a’), (‘h’, ‘e’), […] ] (list) to Union[Merges, Filename]” I definitively don’t know what’s going wrong with merges.txt but it seems like there is a problem… The content of /appli/pretrainedModel/bart-large-mnli is : config.json merges.txt pytorch\_model.bin rust\_model.ot tokenizer\_config.json vocab.json Does someone have any idea where is the problem ? Thanks in advance. More detailed error log : ``` --------------------------------------------------------------------------- ``` TypeError Traceback (most recent call last) in ----\> 1 tokenizer = AutoTokenizer.from\_pretrained(‘/appli/pretrainedModel/bart-large-mnli’) 2 pipeline(‘zero-shot-classification’, model=model, tokenizer=tokenizer) /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/models/auto/tokenization\_auto.py in from\_pretrained(cls, pretrained\_model\_name\_or\_path, \*inputs, \*\*kwargs) 383 tokenizer\_class\_py, tokenizer\_class\_fast = TOKENIZER\_MAPPING[type(config)] 384 if tokenizer\_class\_fast and (use\_fast or tokenizer\_class\_py is None): → 385 return tokenizer\_class\_fast.from\_pretrained(pretrained\_model\_name\_or\_path, \*inputs, \*\*kwargs) 386 else: 387 if tokenizer\_class\_py is not None: /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/tokenization\_utils\_base.py in from\_pretrained(cls, pretrained\_model\_name\_or\_path, \*init\_inputs, \*\*kwargs) 1767 1768 return cls.\_from\_pretrained( → 1769 resolved\_vocab\_files, pretrained\_model\_name\_or\_path, init\_configuration, \*init\_inputs, \*\*kwargs 1770 ) 1771 /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/tokenization\_utils\_base.py in \_from\_pretrained(cls, resolved\_vocab\_files, pretrained\_model\_name\_or\_path, init\_configuration, \*init\_inputs, \*\*kwargs) 1839 # Instantiate tokenizer. 1840 try: → 1841 tokenizer = cls(\*init\_inputs, \*\*init\_kwargs) 1842 except OSError: 1843 raise OSError( /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/models/roberta/tokenization\_roberta\_fast.py in **init** (self, vocab\_file, merges\_file, tokenizer\_file, errors, bos\_token, eos\_token, sep\_token, cls\_token, unk\_token, pad\_token, mask\_token, add\_prefix\_space, \*\*kwargs) 171 mask\_token=mask\_token, 172 add\_prefix\_space=add\_prefix\_space, → 173 \*\*kwargs, 174 ) 175 /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/models/gpt2/tokenization\_gpt2\_fast.py in **init** (self, vocab\_file, merges\_file, tokenizer\_file, unk\_token, bos\_token, eos\_token, add\_prefix\_space, \*\*kwargs) 139 eos\_token=eos\_token, 140 add\_prefix\_space=add\_prefix\_space, → 141 \*\*kwargs, 142 ) 143 /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/tokenization\_utils\_fast.py in **init** (self, \*args, \*\*kwargs) 87 elif slow\_tokenizer is not None: 88 # We need to convert a slow tokenizer to build the backend —\> 89 fast\_tokenizer = convert\_slow\_tokenizer(slow\_tokenizer) 90 elif self.slow\_tokenizer\_class is not None: 91 # We need to create and convert a slow tokenizer to build the backend /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/convert\_slow\_tokenizer.py in convert\_slow\_tokenizer(transformer\_tokenizer) 657 converter\_class = SLOW\_TO\_FAST\_CONVERTERS[tokenizer\_class\_name] 658 → 659 return converter\_class(transformer\_tokenizer).converted() /appli/.conda/envs/bf\_verbatim/lib/python3.7/site-packages/transformers/convert\_slow\_tokenizer.py in converted(self) 281 continuing\_subword\_prefix=“”, 282 end\_of\_word\_suffix=“”, → 283 fuse\_unk=False, 284 ) 285 ) TypeError: Can’t convert [(‘Ä’, ‘t’), (‘Ä’, ‘a’), (‘h’, ‘e’), […] ] (list) to Union[Merges, Filename] ---
**Author:** ![OptimusPrime](https://avatars.discourse-cdn.com/v4/letter/o/f14d63/32.png) [@OptimusPrime](https://discuss.huggingface.co/u/OptimusPrime)\ **Post date:** [July 8, 2021, 1:01pm UTC](https://discuss.huggingface.co/t/typeerror-when-loading-tokenizer-with-from-pretrained-method-for-bart-large-mnli-model/3378/2 "2021-07-08T13:01:18Z")
Hi, Were you able to resolve the issue? Thanks in advance