# TFLongformer Shape Error

**URL:** <https://discuss.huggingface.co/t/tflongformer-shape-error/12651>\
**Category:** 🤗Transformers\
**Created:** [December 9, 2021, 3:03am UTC](https://discuss.huggingface.co/t/tflongformer-shape-error/12651 "2021-12-09T03:03:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sidharrth2002](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@sidharrth2002](https://discuss.huggingface.co/u/sidharrth2002)\
**Post date:** [December 9, 2021, 3:03am UTC](https://discuss.huggingface.co/t/tflongformer-shape-error/12651/1 "2021-12-09T03:03:45Z")

</div>

Hi, when trying to finetune the TFLongformer using the TFTrainer, I got this error

```auto
InvalidArgumentError: 2 root error(s) found.
  (0) INVALID_ARGUMENT: Incompatible shapes: [2,1024,12,514] vs. [2,1024,12,513]
	 [[node while/gradients/while/tf_longformer_for_sequence_classification/longformer/encoder/layer_._0/attention/self/SelectV2_4_grad/BroadcastGradientArgs_1
 (defined at /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:633)
]]
	 [[while/LoopCond/_568/_14]]
  (1) INVALID_ARGUMENT: Incompatible shapes: [2,1024,12,514] vs. [2,1024,12,513]
	 [[node while/gradients/while/tf_longformer_for_sequence_classification/longformer/encoder/layer_._0/attention/self/SelectV2_4_grad/BroadcastGradientArgs_1
 (defined at /usr/local/lib/python3.7/dist-packages/transformers/trainer_tf.py:633)

```

This is my train configuration:

```auto
training_args = TFTrainingArguments(
    output_dir='./results',          
    num_train_epochs=3,             
    per_device_train_batch_size=2,
    gradient_accumulation_steps=32,  
    per_device_eval_batch_size=16,   
    logging_steps=1,
)

with training_args.strategy.scope():
  model = TFLongformerForSequenceClassification.from_pretrained('allenai/longformer-base-4096', num_labels=5, return_dict=True, problem_type = "single_label_classification")

trainer = TFTrainer(model=model, args=training_args, train_dataset=train_dataset, eval_dataset=test_dataset)

```

Someone else had the [same error](https://github.com/huggingface/transformers/issues/10894) when using the Tensorflow version of the Longformer.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [December 9, 2021, 1:20pm UTC](https://discuss.huggingface.co/t/tflongformer-shape-error/12651/2 "2021-12-09T13:20:07Z")

</div>

I believe the `TFTrainer` is deprecated, one can now easily train using Keras’ .fit() method.

You can check out the [official example notebook](https://github.com/huggingface/notebooks/blob/master/examples/text_classification-tf.ipynb) or [script](https://github.com/huggingface/transformers/tree/master/examples/tensorflow/text-classification) for reference.

cc @Rocketknight1

---

<div class="post-metadata">

**Author:** ![sidharrth2002](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@sidharrth2002](https://discuss.huggingface.co/u/sidharrth2002)\
**Post date:** [December 31, 2021, 6:26pm UTC](https://discuss.huggingface.co/t/tflongformer-shape-error/12651/3 "2021-12-31T18:26:03Z")

</div>

Sorry for the late reply! Switching to the torch trainer fixed the issue!