# Use EncoderDecoder models for text summarization **URL:** \ **Category:** 🤗 Course Projects\ **Created:** [November 9, 2021, 7:29pm UTC](https://discuss.huggingface.co/t/use-encoderdecoder-models-for-text-summarization/11525 "2021-11-09T19:29:20Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [November 9, 2021, 7:29pm UTC](https://discuss.huggingface.co/t/use-encoderdecoder-models-for-text-summarization/11525/1 "2021-11-09T19:29:21Z")
> 👋 Please read the topic category [description](https://discuss.huggingface.co/t/about-the-course-projects-category/11510) to understand what this is all about ### Description Most of the available [Transformer models](https://huggingface.co/models?pipeline_tag=summarization&sort=downloads) for text summarization are only available for English documents. At the same time, there are now many pretrained BERT-like models in non-English languages. The goal of this project is to explore whether the `[EncoderDecoder` architecture]([https://huggingface.co/transformers/model\_doc/encoderdecoder.html?highlight=encoderdecoder#encoder-decoder-models](https://huggingface.co/transformers/model_doc/encoderdecoder.html?highlight=encoderdecoder#encoder-decoder-models)) in 🤗 Transformers can be used to create summarization models using just the pretrained weights of encoder-based models. Your task is to pick a pretrained encoder in a non-English language and train it to summarise texts in that language. ### Model(s) See [here](https://huggingface.co/models?search=bert2bert) for example models that people have fine-tuned using this architecture. You task is to create your very own model with this technique! ### Datasets Search for `summarization` datasets on the [Hub](https://huggingface.co/datasets?task_ids=task_ids:summarization&sort=downloads) to get an appropriate corpus for this task ### Challenges Text summarization is a tricky NLP task, so the performance obtained with these models may not match what is observed for their English couterparts (where much more data is available) ### Desired project outcomes - Create a Streamlit or Gradio app on [🤗 Spaces](https://huggingface.co/spaces) that can summarize a document in your chosen language - Don’t forget to push all your models and datasets to the Hub so others can build on them! ### Additional resources - [Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://arxiv.org/abs/1907.12461) [PAPER] - [Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models](https://huggingface.co/blog/warm-starting-encoder-decoder) [BLOG POST] - Examples of these models on the Hub by @mrm8488: [https://twitter.com/mrm8488/status/1458475725565141001?s=20](https://twitter.com/mrm8488/status/1458475725565141001?s=20) ### Discord channel To chat and organise with other people interested in this project, head over to our [Discord](http://xwo7x.mjt.lu/lnk/CAAAAkdkZwAAAAAAAAAAALG-MTEAAYCr4H8AAAAAAAyFyQBhjpnimM4rcE5DT5ibCUtRT96tWwAMh8Y/15/U6BYNMNDksyyGG4jgtzPKQ/aHR0cHM6Ly9odWdnaW5nZmFjZS5jby9qb2luL2Rpc2NvcmQ) and: Follow the instructions on the `#join-course` channel. Then join one of the following channels: - `#encoder-decoder-es` channel (Spanish) Just make sure you comment here to indicate that you’ll be contributing to this project 🙂 ---
**Author:** ![edumunozsala](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/edumunozsala/32/26889_2.png) [@edumunozsala](https://discuss.huggingface.co/u/edumunozsala)\ **Post date:** [November 16, 2021, 11:59am UTC](https://discuss.huggingface.co/t/use-encoderdecoder-models-for-text-summarization/11525/2 "2021-11-16T11:59:00Z")
Interesting project, I am interested in a Spanish summarizer using Encoder-Decoder model. Anyone else interested in this approach? ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [November 16, 2021, 12:19pm UTC](https://discuss.huggingface.co/t/use-encoderdecoder-models-for-text-summarization/11525/3 "2021-11-16T12:19:50Z")
Hey @edumunozsala, cool to hear that you’re interesting in tackling this project! I’ve created a Discord channel (see topic description) in case you and others want to use it 🙂 ---
**Author:** ![Parmida](https://avatars.discourse-cdn.com/v4/letter/p/f0a364/32.png) [@Parmida](https://discuss.huggingface.co/u/Parmida)\ **Post date:** [December 28, 2023, 12:00pm UTC](https://discuss.huggingface.co/t/use-encoderdecoder-models-for-text-summarization/11525/4 "2023-12-28T12:00:31Z")
I am interested to train it for Persian (Farsi) language. how can I contribute?