# Bart input confusion

**URL:** <https://discuss.huggingface.co/t/bart-input-confusion/1103>\
**Category:** Beginners\
**Created:** [September 13, 2020, 6:34pm UTC](https://discuss.huggingface.co/t/bart-input-confusion/1103 "2020-09-13T18:34:30Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hildweig](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hildweig/32/190_2.png) [@Hildweig](https://discuss.huggingface.co/u/Hildweig)\
**Post date:** [September 13, 2020, 6:34pm UTC](https://discuss.huggingface.co/t/bart-input-confusion/1103/1 "2020-09-13T18:34:30Z")

</div>

In Bart for the summarization task, the input length is 1024 (1024 token), what does this input represents: (for example i have a document with s1, s2, s500) does this mean that we feed a sequence as sentence per sentence or the whole document must be as input as 1024 token ( all sentences must fit with truncation ) if this true doesn’t it cause information loss ?  
And if it’s sequence per sequence, let’s say 20 sentences at a time of 500, will the output at the top encoder change each time?  
To be honest I’m having a difficult time imagining how the encoder is processing the document.

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\
**Post date:** [September 14, 2020, 6:46am UTC](https://discuss.huggingface.co/t/bart-input-confusion/1103/2 "2020-09-14T06:46:12Z")

</div>

Hi @Hildweig

For BART max input length is 1024 tokens. You can think of a token as as a word for simplicity (words can be split in multiple tokens as well). It’s not sentences.

Here document means a seq with max 1024 tokens. Processing longer sequences than that is still a topic of ongoing research.

And I would recommend to read the original Transformers paper (Attention is all you need ) to get an idea about how a sequence is processed by the encoder. Or [the illustrated transformers](http://jalammar.github.io/illustrated-transformer/)

---

<div class="post-metadata">

**Author:** ![Hildweig](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hildweig/32/190_2.png) [@Hildweig](https://discuss.huggingface.co/u/Hildweig)\
**Post date:** [September 14, 2020, 9:36am UTC](https://discuss.huggingface.co/t/bart-input-confusion/1103/3 "2020-09-14T09:36:44Z")

</div>

So does it get truncated? If yes is there a special method they use for truncation?