# Bart Large CNN summarization **URL:** \ **Category:** Beginners\ **Created:** [January 25, 2021, 10:34pm UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383 "2021-01-25T22:34:09Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Billy](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Billy](https://discuss.huggingface.co/u/Billy)\ **Post date:** [January 25, 2021, 10:34pm UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/1 "2021-01-25T22:34:09Z")
On [facebook/bart-large-cnn · Hugging Face](https://huggingface.co/facebook/bart-large-cnn), an article can be pasted into the summarization tool. I am attempting to replicate this with the same model. By viewing the “use in transformers” button, the following code is able to be seen: ```auto from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") model = AutoModel.from_pretrained("facebook/bart-large-cnn") ``` Looking at the transformers/model\_doc/bart documentation, the summarization example at the bottom of the page uses the bart-large model, and not the cnn. However, attempting to combine the two I come up with: ```auto from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") model = AutoModel.from_pretrained("facebook/bart-large-cnn") ARTICLE_TO_SUMMARIZE = "My friends are cool but they eat too many carbs." inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='tf') # Generate Summary summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=5, early_stopping=True) print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]) ``` The following warning and trace is returned: ```auto Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`. --------------------------------------------------------------------------- ValueError Traceback (most recent call last) in 7 inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='tf') 8 # Generate Summary ----> 9 summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=5, early_stopping=True) 10 print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]) ~/.local/lib/python3.6/site-packages/torch/autograd/grad_mode.py in decorate_no_grad(*args, **kwargs) 47 def decorate_no_grad(*args, **kwargs): 48 with self: ---> 49 return func(*args, **kwargs) 50 return decorate_no_grad 51 ~/.local/lib/python3.6/site-packages/transformers/generation_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, **model_kwargs) 821 # init `attention_mask` depending on `pad_token_id` 822 model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation( --> 823 input_ids, pad_token_id, eos_token_id 824 ) 825 ~/.local/lib/python3.6/site-packages/transformers/generation_utils.py in _prepare_attention_mask_for_generation(self, input_ids, pad_token_id, eos_token_id) 360 self, input_ids: torch.Tensor, pad_token_id: int, eos_token_id: int 361 ) -> torch.LongTensor: --> 362 is_pad_token_in_inputs_ids = (pad_token_id is not None) and (pad_token_id in input_ids) 363 is_pad_token_not_equal_to_eos_token_id = (eos_token_id is None) or ( 364 (eos_token_id is not None) and (pad_token_id != eos_token_id) ~/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in __bool__ (self) 990 991 def __bool__ (self): --> 992 return bool(self._numpy()) 993 994 __nonzero__ = __bool__ ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ``` I have tried switching different hyperparameters, specifically the max\_length with truncation=True and more. End goal is to reproduce locally the output from [this article summary](https://huggingface.co/facebook/bart-large-cnn?text=The+Kansas+City+Chiefs+host+the+Denver+Broncos+for+a+Sunday+Night+Football+showdown.Will+the+reigning+Super+Bowl+champions+cover+this+13-point+spread+with+ease%3FOur+NFL+betting+analyst+Stuckey+previews+the+matchup+below.Broncos+vs.+Chiefs+Odds%0A%0A%0A%0ABroncos+Odds%0A%2B13+%5BBET+NOW%5D%0A%0A%0AChiefs+Odds%0A-13+%5BBET+NOW%5D%0A%0A%0AOver%2FUnder%0A51+%5BBET+NOW%5D%0A%0A%0AKickoff%0A8%3A20+p.m.+ET%0A%0A%0ATV%0ANBC%0A%0A%0AOdds+as+of+early+Sunday+and+via+PointsBet%2C+where+you+can+bet+%2420+to+win+%24125+if+Patrick+Mahomes+throws+for+at+least+one+yard.%0A%0A%0A%0AAt+least+the+Broncos+will+have+a+quarterback+on+Sunday+Night+Football+with+all+signs+pointing+to+Drew+Lock+returning+under+center+after+practice+squad+receiver+Kendall+Hinton+was+forced+into+action+last+week+against+New+Orleans.%0AHinton%E2%80%99s+outing+went+about+as+well+as+you+would+have+expected%2C+as+the+Broncos+became+the+first+team+to+finish+with+more+interceptions+than+completions+since+the+Ryan+Leaf-quarterbacked+Chargers+of+1998.%0AMeanwhile%2C+the+defending+Super+Bowl+champions+sit+at+10-1+with+another+division+title+all+but+wrapped+up.+They%E2%80%99ll+clinch+the+AFC+West+with+a+win+on+Sunday+night%2C+but+obviously+have+their+sights+set+on+bigger+goals+%E2%80%94+starting+with+the+lone+bye+in+the+AFC.+That+would+require+them+to+catch+the+Steelers%2C+who+currently+own+a+one-game+lead.%0AThe+Broncos+have+lost+10+straight+to+the+Chiefs%2C+including+three+straight+in+blowout+fashion.%0ALet%E2%80%99s+take+a+closer+look+at+the+matchup.%0A%0A%0A%0A%0A%0AThe+must-have+app+for+NFL+bettors%0A%0A%0A%0A%0A%0A%0ACustom+scoreboard+for+your+NFL+bets%0A%0A%0A%0AFree+picks+from+experts%0A%0A%0A%0ALive+win+probabilities+for+every+game%0A%0A%0A%0ADownload+Now%0A%0A%0ADenver+Broncos%0ALet%E2%80%99s+just+throw+the+obvious+out+there%3A+Denver%E2%80%99s+offense+is+miserable.%0AThe+Broncos+rank+32nd+in+Football+Outsiders%E2%80%99+DVOA+in+overall+and+passing+offense.+Outside+of+a+few+meaningless+late-game+situations%2C+Lock+hasn%E2%80%99t+been+good+by+any+measure.+And+while+the+running+game+has+been+a+bit+better%2C+it%E2%80%99s+worth+mentioning+Philip+Lindsay+is+questionable%2C+as+is+wide+receiver+Jerry+Jeudy.%0AGarett+Bolles+has+been+a+bright+spot+on+the+offensive+line%2C+grading+out+as+one+of+the+NFL%E2%80%99s+best+tackles+to+date%2C+but+rookie+center+Lloyd+Cushenberry+has+really+struggled.+That+could+be+problematic+against+Chris+Jones%2C+who+could+wreak+havoc+in+the+backfield+with+interior+pressure.%0AOn+the+other+side+of+the+ball%2C+Denver%E2%80%99s+defense+has+been+tremendous+despite+losing+Von+Miller+before+the+season+and+dealing+with+a+slew+of+other+injuries.+The+progress+in+Year+2+of+Vic+Fangio%E2%80%99s+scheme+is+evident+%E2%80%94+by+most+metrics%2C+Denver+has+been+a+top-10+overall+defense+this+season.%0AThe+defense+is+a+bit+stronger+against+the+pass+than+the+run%2C+but+the+Broncos+won%E2%80%99t+have+the+services+of+cornerback+Bryce+Callahan+on+Sunday%2C+who+has+graded+out+as+one+of+the+best+in+the+NFL+this+season.+That+hurts.%0A%0A%0ASNF+Promos%3A+Bet+%2420%2C+Win+%24125+if+Patrick+Mahomes+Throws+for+a+Yard%2C+Way+More%21Read+now+%0A%0AThe+Broncos+still+have+one+of+the+league%E2%80%99s+top+safety+duos%2C+but+A.J.+Bouye+is+no+longer+an+elite+corner+on+the+other+side+of+30.+And+per+Pro+Football+Focus%2C+Essang+Bassey+and+Michael+Ojemudia+have+graded+outside+the+top+90+out+of+125+qualified+cornerbacks.+Callahan+will+be+sorely+missed+against+the+Chiefs.%0ADenver%E2%80%99s+best+interior+defensive+lineman+Shelby+Harris+should+be+back%2C+but+Fangio+has+stated+Harris+won%E2%80%99t+get+his+normal+reps.%0AKansas+City+Chiefs%0AI+don%E2%80%99t+need+to+spend+much+time+on+the+Chiefs.%0AThey%E2%80%99re+the+best+team+in+the+NFL.+They+are+also+basically+the+same+team+as+they+were+last+season.%0ATheir+offense+is+elite%2C+starting+with+the+league%E2%80%99s+best+quarterback+by+a+wide+margin.+The+offensive+line%2C+skill+positions+and+great+play-calling+also+don%E2%80%99t+hurt.%0AThe+defense+still+has+weaknesses+%E2%80%94+the+unit+remains+very+vulnerable+against+the+run+%2832nd+in+DVOA%29+with+a+weak+group+of+linebackers+and+interior+defensive+linemen+outside+of+Jones.+However%2C+that+won%E2%80%99t+lose+you+games+if+teams+don%E2%80%99t+jump+out+to+a+lead+and+get+stops+against+the+Chiefs+offense.%0AI%E2%80%99m+a+bit+concerned+about+the+lack+of+edge+rush.+It%E2%80%99s+basically+become+nonexistent.+Frank+Clark+has+been+awful+all+season+and+hasn%E2%80%99t+generated+a+single+pressure+over+the+past+two+games.+But+while+this+defense+needs+work%2C+with+Patrick+Mahomes+and+that+offense%2C+it+ultimately+might+not+matter.%0A%0ABroncos-Chiefs+Pick%0ANot+every+game+has+betting+value%2C+especially+in+the+NFL.+This+is+one+of+them+%E2%80%94+I+make+this+spread+Chiefs+-13+with+a+total+of+50+points%2C+so+it%E2%80%99s+a+pass+for+me+from+a+pregame+perspective.%0AIt%E2%80%99s+also+difficult+to+even+picture+how+this+will+play+out%3A+Can+the+Broncos+get+a+few+scoring+drives+early+on+by+running+down+the+Chiefs%E2%80%99+throat%2C+which+would+also+keep+Mahomes+on+the+sidelines%3F+If+so%2C+Denver+can+probably+stay+within+this+number+and+Kansas+City+may+become+an+intriguing+live+target.%0AIf+Denver+can%E2%80%99t+start+fast%2C+though%2C+Kansas+City+should+build+a+lead.+If+so%2C+do+the+Chiefs+go+ultra+conservative+in+the+second+half%3F+I+would+think+so.+No+need+to+show+anything+more+here+%E2%80%94+that+would+open+up+a+backdoor+shot+for+Drew+Lock+late.%0ASo%2C+this+is+a+pure+%E2%80%9Cstay+away%E2%80%9D+for+me+from+a+side+and+pregame+perspective.+Follow+along+on+the+Action+app+to+see+if+I+come+in+on+anything+live+based+on+how+this+game+plays+out%2C+as+I+do+think+there+might+be+some+opportunities+to+jump+in.%0A%0A%0A%0A%0A%0AThe+ultimate+NFL+betting+cheat+code%0A%0A%0A%0A%0A%0A%0ABest+bets+for+every+game%0A%0A%0A%0AOur+NFL+model%E2%80%99s+biggest+weekly+edges%0A%0A%0A%0AProfitable+data-driven+system+picks%0A%0A%0A%0AStart+Free+Trial%0A%0A%0ASince+I+do+want+to+provide+some+actionable+advice+heading+into+the+game%2C+I+dug+into+potential+props+and+came+up+with+two+targets%3A%0A%0ANoah+Fant%0ASammy+Watkins%0A%0AFant+could+have+a+big+day+against+a+Kansas+City+defense+that+can+be+exploited+by+good+tight+ends.%0AAs+for+Watkins%2C+Fangio%E2%80%99s+primary+focus+is+preventing+the+big+play%2C+keeping+everything+in+front+of+the+defense+and+making+teams+work+their+way+down+the+field.+I+assume+Fangio%E2%80%99s+game+play+will+be+focused+on+taking+away+Tyreek+Hill+deep+with+a+safety+always+over+the+top+of+wherever+Hill+goes.%0AFangio+will+also+likely+bracket+Travis+Kelce.+Now%2C+a+lot+of+teams+probably+try+to+do+that%2C+but+as+I+mentioned+before%2C+Denver+has+the+luxury+of+having+two+of+the+league%E2%80%99s+best+safeties.+This+should+be+a+high-volume+day+for+Watkins%2C+who+has+historically+had+nice+outings+against+Denver+%E2%80%94+4.5+catches%2C+70+yards%2C+0.5+touchdown+on+average+%E2%80%94+and+he+should+be+open+all+night+underneath.%0AI+would+look+at+touchdown+props+for+both+Fant+and+Watkins%2C+in+addition+to+the+overs+on+their+reception+and+yardage+props.%0A%0A%0A%0AShop+real-time+Broncos-Chiefs+odds%0A%0A%0A%0A). Essentially my question boils down to what follows after the bart large cnn model instantiation in order to obtain the desired article summary from the link above? ```auto from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") model = AutoModel.from_pretrained("facebook/bart-large-cnn") inputs = tokenizer(params?) model.generate(params?) ``` transformers-cli env ```auto - `transformers` version: 4.2.2 - Platform: Linux-5.4.0-62-generic-x86_64-with-Ubuntu-18.04-bionic - Python version: 3.6.9 - PyTorch version (GPU?): 1.4.0 (True) - Tensorflow version (GPU?): 2.4.1 (False) - Using GPU in script?: tried both - Using distributed or parallel set-up in script?: no ``` I have also verified that ```auto python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))" ``` has worked for me in my environment. ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [January 26, 2021, 6:42am UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/2 "2021-01-26T06:42:23Z")
looks like you are returning tf tensors, from tokenizer, but the model is in torch. `inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='tf')` changing it to `pt` should fix the issue. ---
**Author:** ![Billy](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Billy](https://discuss.huggingface.co/u/Billy)\ **Post date:** [January 26, 2021, 2:36pm UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/3 "2021-01-26T14:36:18Z")
Changing just `return_tensors='pt'` returns a different error: ```auto from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn") model = AutoModel.from_pretrained("facebook/bart-large-cnn") ARTICLE_TO_SUMMARIZE = "My friends are cool but they eat too many carbs." inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors='pt') # Generate Summary summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=5, early_stopping=True) print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]) ``` Error: ```auto Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`. Traceback (most recent call last): File "para.py", line 56, in summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=5, early_stopping=True) File "/home/billy/.local/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 49, in decorate_no_grad return func(*args, **kwargs) File "/home/billy/.local/lib/python3.6/site-packages/transformers/generation_utils.py", line 958, in generate **model_kwargs, File "/home/billy/.local/lib/python3.6/site-packages/transformers/generation_utils.py", line 1596, in beam_search next_token_logits = outputs.logits[:, -1, :] AttributeError: 'Seq2SeqModelOutput' object has no attribute 'logits' ``` ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [January 29, 2021, 9:50am UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/4 "2021-01-29T09:50:00Z")
to load Seq2Seq (encoder-decoder) models, `AutoModelForSeq2SeqLM` should be used, in your snippet you are using `AutoModel` ---
**Author:** ![Billy](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Billy](https://discuss.huggingface.co/u/Billy)\ **Post date:** [January 29, 2021, 2:27pm UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/5 "2021-01-29T14:27:46Z")
Cheers, thank you. I was unable to find any relevant documentation for that specifying that model import, should it be updated and is more info located somewhere where I can read more about it as well? ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [February 1, 2021, 7:20am UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/6 "2021-02-01T07:20:00Z")
Auto models classes are documented here [https://huggingface.co/transformers/model\_doc/auto.html#](https://huggingface.co/transformers/model_doc/auto.html#) ---
**Author:** ![raveenb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/raveenb/32/987_2.png) [@raveenb](https://discuss.huggingface.co/u/raveenb)\ **Post date:** [February 5, 2021, 10:04am UTC](https://discuss.huggingface.co/t/bart-large-cnn-summarization/3383/7 "2021-02-05T10:04:31Z")
can you not use the BartModel and BartTokenizer directly and use the truncation=True to get over this? if you don’t intend to truncate then you may be interested in this discussion here [Summarization on long documents](https://discuss.huggingface.co/t/summarization-on-long-documents/920) Thanks