# Strange error when using the Longformer (HuggingFace developers, please reply) **URL:** \ **Category:** 🤗Transformers\ **Created:** [October 11, 2020, 1:52am UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468 "2020-10-11T01:52:49Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![h56cho](https://avatars.discourse-cdn.com/v4/letter/h/c2a13f/32.png) [@h56cho](https://discuss.huggingface.co/u/h56cho)\ **Post date:** [October 11, 2020, 1:52am UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/1 "2020-10-11T01:52:49Z")
Hello, I am trying to use `LongformerForMultipleChoice` model, and the code I am using is the following: ```auto # import the pre-trained HuggingFace Longformer tokenizer. longformer_tokenizer = LongformerTokenizer.from_pretrained('allenai/longformer-base-4096') # get the pre-trained HuggingFace Longformer best_model_longformer = LongformerForMultipleChoice.from_pretrained('allenai/longformer-base-4096', output_hidden_states = True) # my multiple choice question has 4 options. question_list = [main_question, main_question, main_question, main_question] option_list = [option1, option2, option3, option4] mc_labels = torch.tensor([my_answer]) encoded_dict = longformer_tokenizer(question_list, option_list, return_tensors = 'pt', add_prefix_space = True, padding = True) input_hidden_state = best_model_longformer( **{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:].detach() ``` and I am getting the error below: ```auto /home/ec2-user/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py:71: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(*, bool as_tuple) (Triggered internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.) sep_token_indices = (input_ids == sep_token_id).nonzero() Traceback (most recent call last): File "SEED_125_V20_15_LONGFORMER.py", line 427, in main_function('/home/ec2-user/G1G2.txt','/home/ec2-user/G1G2_answer_num.txt', num_iter) File "SEED_125_V20_15_LONGFORMER.py", line 389, in main_function best_model_longformer) File "SEED_125_V20_15_LONGFORMER.py", line 198, in fill_MC_loss_accuracy_tensor input_hidden_state = best_model_longformer(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:].detach() File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 1808, in forward loss = loss_fct(reshaped_logits, labels) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 948, in forward ignore_index=self.ignore_index, reduction=self.reduction) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2422, in cross_entropy return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction) File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2218, in nll_loss ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index) IndexError: Target 1 is out of bounds. ``` How can I fix this error? Thank you. ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [October 11, 2020, 4:04pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/2 "2020-10-11T16:04:22Z")
It probably means yours class number dont match with the one labels in the trainer(); ---
**Author:** ![h56cho](https://avatars.discourse-cdn.com/v4/letter/h/c2a13f/32.png) [@h56cho](https://discuss.huggingface.co/u/h56cho)\ **Post date:** [October 11, 2020, 4:05pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/3 "2020-10-11T16:05:05Z")
hello, I am not sure what you mean by this. could you please elaborate more? Thank you, ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [October 11, 2020, 4:06pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/4 "2020-10-11T16:06:58Z")
are you fine-tuning it? I am also new to transformers but last two days I have been struggling with this error, the issue was the my labels were [0,2] whereas the model was taking it as [0-1], so that was the error, once I resolved the label classes to [0,1] it worked. ---
**Author:** ![h56cho](https://avatars.discourse-cdn.com/v4/letter/h/c2a13f/32.png) [@h56cho](https://discuss.huggingface.co/u/h56cho)\ **Post date:** [October 11, 2020, 4:16pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/5 "2020-10-11T16:16:10Z")
my answers are coded in the right way, I don’ think that the Longformer is correctly getting that my multiple-choice questions have 4 options… ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [October 11, 2020, 4:27pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/6 "2020-10-11T16:27:25Z")
in my case I am struggling with cuda out of memory with longformer, I am using Google Colab pro ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [October 11, 2020, 4:33pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/7 "2020-10-11T16:33:52Z")
did you try this > ---
**Author:** ![h56cho](https://avatars.discourse-cdn.com/v4/letter/h/c2a13f/32.png) [@h56cho](https://discuss.huggingface.co/u/h56cho)\ **Post date:** [October 11, 2020, 4:37pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/8 "2020-10-11T16:37:50Z")
I don’t think this has anything to do with my problem…but thanks for sharing ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [October 12, 2020, 4:55pm UTC](https://discuss.huggingface.co/t/strange-error-when-using-the-longformer-huggingface-developers-please-reply/1468/9 "2020-10-12T16:55:19Z")
issue should be with `mc_labels`, labels should be between `[0..num_choices-1]`