# Obtain output embeddings from summarization

**URL:** <https://discuss.huggingface.co/t/obtain-output-embeddings-from-summarization/5558>\
**Category:** 🤗Transformers\
**Created:** [April 16, 2021, 2:57pm UTC](https://discuss.huggingface.co/t/obtain-output-embeddings-from-summarization/5558 "2021-04-16T14:57:16Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![vincentw](https://avatars.discourse-cdn.com/v4/letter/v/9e8a1a/32.png) [@vincentw](https://discuss.huggingface.co/u/vincentw)\
**Post date:** [April 16, 2021, 2:57pm UTC](https://discuss.huggingface.co/t/obtain-output-embeddings-from-summarization/5558/1 "2021-04-16T14:57:17Z")

</div>

I am trying to obtain output embeddings by the transformer, along with some generated text summaries. So far, I have tried the following.

```auto
config = {'num_return_sequences': 3, 'num_beams': 4, ...}

translated = model.generate(tokenized_input, **config, output_scores=True, return_dict_in_generate=True, output_hidden_states=True)
d_model = model.config.d_model
# Embeddings of the final output layer, for every generation step
output_embeddings_per_step = [step[0] for step in translated['decoder_hidden_states']]
# zip(*output_embeddings_per_step) : creates (pos1_embedding, pos2_embedding, ...) for each sample in sampled sentences
current_output_embeddings = [torch.cat(sample, dim=0).reshape(-1).split(d_model) for sample in zip(*output_embeddings_per_step)]

# len(current_output_embeddings)=4, whereas len(translated['sequences'])=3

```

However, as I have num\_return\_sequences \< num\_beams. I got embeddings for more sequences than intended (incl. those not picked as well). Is there a way to identify which of the sequences as picked by num\_return\_sequences?

Thank you.