# Error while optimizing seq2seq model using optimum

**URL:** <https://discuss.huggingface.co/t/error-while-optimizing-seq2seq-model-using-optimum/101180>\
**Category:** 🤗Optimum\
**Created:** [August 6, 2024, 6:32pm UTC](https://discuss.huggingface.co/t/error-while-optimizing-seq2seq-model-using-optimum/101180 "2024-08-06T18:32:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![LearnToGrow](https://avatars.discourse-cdn.com/v4/letter/l/5fc32e/32.png) [@LearnToGrow](https://discuss.huggingface.co/u/LearnToGrow)\
**Post date:** [August 6, 2024, 6:32pm UTC](https://discuss.huggingface.co/t/error-while-optimizing-seq2seq-model-using-optimum/101180/1 "2024-08-06T18:32:05Z")

</div>

I could not reproduce this [example](https://huggingface.co/docs/optimum/v1.21.2/onnxruntime/usage_guides/optimization)  
transformers==4.42.4  
torch== 2.4.0+cpu  
onnx==1.16.2  
onnxruntime==1.18.1  
optimum==1.21.2

```auto
from transformers import AutoTokenizer
from optimum.onnxruntime import OptimizationConfig, ORTOptimizer, ORTModelForSeq2SeqLM
model_id = "sshleifer/distilbart-cnn-12-6"
save_dir = "distilbart_optimized"

# Load a PyTorch model and export it to the ONNX format
model = ORTModelForSeq2SeqLM.from_pretrained(model_id, export=True)

# Create the optimizer
optimizer = ORTOptimizer.from_pretrained(model)

# Define the optimization strategy by creating the appropriate configuration
optimization_config = OptimizationConfig(
    optimization_level=2,
    enable_transformers_specific_optimizations=True,
    optimize_for_gpu=False,
)

# Optimize the model
optimizer.optimize(save_dir=save_dir, optimization_config=optimization_config)
tokenizer = AutoTokenizer.from_pretrained(model_id)
optimized_model = ORTModelForSeq2SeqLM.from_pretrained(save_dir)
tokens = tokenizer("This is a sample input", return_tensors="pt")
outputs = optimized_model.generate(**tokens)

```

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/1/5/158d9b6c98eddb6b8f28b89513f350ba9051c310.png)

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/7/7/776c1f830aa201e0a0b981349273cc032a75204d.png)

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](https://discuss.huggingface.co/u/regisss)\
**Post date:** [September 16, 2024, 6:36pm UTC](https://discuss.huggingface.co/t/error-while-optimizing-seq2seq-model-using-optimum/101180/2 "2024-09-16T18:36:40Z")

</div>

@LearnToGrow Can you try the solution suggested [here](https://github.com/huggingface/optimum/issues/1983#issuecomment-2353639785)?