# BART with custom encoder and decoder

**URL:** <https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643>\
**Category:** Models\
**Created:** [June 2, 2022, 7:29am UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643 "2022-06-02T07:29:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Boglinger](https://avatars.discourse-cdn.com/v4/letter/b/9fc29f/32.png) [@Boglinger](https://discuss.huggingface.co/u/Boglinger)\
**Post date:** [June 2, 2022, 7:29am UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/1 "2022-06-02T07:29:43Z")

</div>

Hi, I am currently working on german abstractive summarisation. My goal is to have a custom style abstractive summarisation model that learned a certain style of summarisation.

I was therefore thinking if it is possible to train a German or multilingual GPT-2 model for language modeling and inserting it into a BART model as the decoder and a fine tuned BERT model as encoder.

BART always states to use a “GPT-like decoder” and “BERT-like encoder”, which made me wonder if the models are exchangeable. That would allow me use a GPT-2 decoder trained on the specific task and in the desired language and a German BERT model as encoder.

Does anyone have any experience on doing so? Also if only one part is exchangeable (decoder or encoder) this would also be very interesting.  
I am happy for any leads.

Cheers  
Lukas

---

<div class="post-metadata">

**Author:** ![Boglinger](https://avatars.discourse-cdn.com/v4/letter/b/9fc29f/32.png) [@Boglinger](https://discuss.huggingface.co/u/Boglinger)\
**Post date:** [June 9, 2022, 9:46am UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/2 "2022-06-09T09:46:08Z")

</div>

By simply calling model.decoder = gpt\_model I would add the decoder to the model but not exchanging it, is that correct?

---

<div class="post-metadata">

**Author:** ![AITony70100](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@AITony70100](https://discuss.huggingface.co/u/AITony70100)\
**Post date:** [May 24, 2023, 3:12pm UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/3 "2023-05-24T15:12:11Z")

</div>

It’s BARD, not Bart. Bart is a cartoon character from The Simpsons as well as Bert, a puppet character from Sesame Street.

---

<div class="post-metadata">

**Author:** ![Boglinger](https://avatars.discourse-cdn.com/v4/letter/b/9fc29f/32.png) [@Boglinger](https://discuss.huggingface.co/u/Boglinger)\
**Post date:** [May 24, 2023, 3:34pm UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/4 "2023-05-24T15:34:35Z")

</div>

> **[BART](https://huggingface.co/docs/transformers/model_doc/bart)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![Boglinger](https://avatars.discourse-cdn.com/v4/letter/b/9fc29f/32.png) [@Boglinger](https://discuss.huggingface.co/u/Boglinger)\
**Post date:** [May 24, 2023, 3:35pm UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/5 "2023-05-24T15:35:31Z")

</div>

It is BART not Bart… 😉

---

<div class="post-metadata">

**Author:** ![AITony70100](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@AITony70100](https://discuss.huggingface.co/u/AITony70100)\
**Post date:** [May 25, 2023, 8:56pm UTC](https://discuss.huggingface.co/t/bart-with-custom-encoder-and-decoder/18643/6 "2023-05-25T20:56:41Z")

</div>

Well, it also means “Bay Area Rapid Transit”. But for me it’s funny a model that has a popular cartoon character’s name. Ay Caramba!.