# EncoderDecoderModel with Longformer and Bert

**URL:** <https://discuss.huggingface.co/t/encoderdecodermodel-with-longformer-and-bert/3601>\
**Category:** 🤗Transformers\
**Created:** [February 11, 2021, 2:33pm UTC](https://discuss.huggingface.co/t/encoderdecodermodel-with-longformer-and-bert/3601 "2021-02-11T14:33:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![cahya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cahya/32/48725_2.png) [@cahya](https://discuss.huggingface.co/u/cahya)\
**Post date:** [February 11, 2021, 2:33pm UTC](https://discuss.huggingface.co/t/encoderdecodermodel-with-longformer-and-bert/3601/1 "2021-02-11T14:33:06Z")

</div>

Hi,  
Is it possible to create EncoderDecoderModel using Longformer and Bert? I tried Longformer and Roberta and it works (the training runs), but if I use Longformer and Bert, I get this error message `RuntimeError: CUDA error: device-side assert triggered` when I train it.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [February 11, 2021, 7:48pm UTC](https://discuss.huggingface.co/t/encoderdecodermodel-with-longformer-and-bert/3601/2 "2021-02-11T19:48:19Z")

</div>

Normally Longformer and BERT should work in an encoder-decoder setting.

If you have a CUDA error, it’s advised to test the code on CPU and see if you’re getting an error that is more interpretable.