# Optimize AND quantize with Optimum **URL:** \ **Category:** 🤗Optimum\ **Created:** [September 27, 2022, 11:51am UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675 "2022-09-27T11:51:11Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![jorgealro](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@jorgealro](https://discuss.huggingface.co/u/jorgealro)\ **Post date:** [September 27, 2022, 11:51am UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/1 "2022-09-27T11:51:12Z")
Hi everyone, I would like to optimize AND quantize a NER model I have fine-tuned. I see that with the Optimum library it is easy to do both things separately, but I haven’t still managed to optimize and quantize the same model. Back with ONNXRuntime it was possible and easy to do. How can this be done with Optimum? Thanks!! ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 27, 2022, 12:10pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/2 "2022-09-27T12:10:09Z")
You can check out this blog post: [Optimizing Transformers with Hugging Face Optimum](https://www.philschmid.de/optimizing-transformers-with-optimum) or the documentation: [🤗 Optimum](https://huggingface.co/docs/optimum/index) ---
**Author:** ![jorgealro](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@jorgealro](https://discuss.huggingface.co/u/jorgealro)\ **Post date:** [September 27, 2022, 2:20pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/3 "2022-09-27T14:20:09Z")
Hi Phil, Thank you for your answer. The first blog post seems to solve my question. Anyways, I think it uses an old version of Optimum that has some attributes that actual ones don’t. When exporting the optimized/quantized model, in the blog post they use the `export` attribute. But, when I do the same (using version 1.4.0), it gives me an error saying `'ORTOptimizer' object has no attribute 'export'`. I have searched in the documentation but I haven’t find anything similar to the article for more recent versions. Thanks again!! ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 27, 2022, 6:17pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/4 "2022-09-27T18:17:05Z")
Take a look at the documentation there are the API changes included for version 1.4.0, e.g. for quantization here: [Quantization](https://huggingface.co/docs/optimum/onnxruntime/quantization) ---
**Author:** ![jorgealro](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@jorgealro](https://discuss.huggingface.co/u/jorgealro)\ **Post date:** [September 28, 2022, 3:00pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/5 "2022-09-28T15:00:30Z")
The problem I see with the latest version (1.4.0) compared to older versions is that it doesn’t have an `export` attribute implemented nor anything similar that could optimize a quantized model or quantize an optimized model. As far as I understand, in version 1.4.0, it only allows you to go from `model.onnx` to `model-optimized.onnx` or from `model.onnx` to `model-quantized.onnx`, but then you can’t quantize `model-optimized.onnx` or optimize `model-quantized.onnx` to get a `model-optimized-quantized.onnx`, while in other versions, with `export` you could (such as in the blog post you linked). Will be possible to do this in future versions? I think having a model quantized and optimized was such a good solution as it reduced a lot of space and achieved almost similar results as the original model. Thanks!! ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 28, 2022, 3:13pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/6 "2022-09-28T15:13:45Z")
@jorgealro, that is already possible and supported you can provide the `file_name` when either loading a `ORTModel` or creating a `Optimizer/Quantizer`. This is explained and document in our documentation. ---
**Author:** ![jorgealro](https://avatars.discourse-cdn.com/v4/letter/j/e68b1a/32.png) [@jorgealro](https://discuss.huggingface.co/u/jorgealro)\ **Post date:** [September 29, 2022, 3:50pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/7 "2022-09-29T15:50:33Z")
Yes, that works for me 🙂 Thank you very much!! ---
**Author:** ![f-dig](https://avatars.discourse-cdn.com/v4/letter/f/858c86/32.png) [@f-dig](https://discuss.huggingface.co/u/f-dig)\ **Post date:** [March 27, 2023, 10:27am UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/8 "2023-03-27T10:27:20Z")
Hi @jorgealro, could you please explain how you solved the `'ORTOptimizer' object has no attribute 'export'` issue? I’m facing the same problem trying to optimize a CrossEncoder with optimum 1.7.3. Many thanks in advance! ---
**Author:** ![f-dig](https://avatars.discourse-cdn.com/v4/letter/f/858c86/32.png) [@f-dig](https://discuss.huggingface.co/u/f-dig)\ **Post date:** [March 27, 2023, 11:10am UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/9 "2023-03-27T11:10:43Z")
I found the solution, it might be useful to someone until the [documentation](https://huggingface.co/docs/optimum/v1.3.0/en/onnxruntime/optimization) gets updated (currently available is for v1.3.0). As of optimum==1.7.3, you should use the `optimize` method, instead of the `export` one: ```auto optimizer = ORTOptimizer.from_pretrained('model_name_or_path') optimizer.optimize( save_dir='output_folder' ) ``` ---
**Author:** ![fxmarty](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fxmarty/32/23782_2.png) [@fxmarty](https://discuss.huggingface.co/u/fxmarty)\ **Post date:** [March 27, 2023, 9:30pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/10 "2023-03-27T21:30:36Z")
Hi @f-dig, the documention seem to have moved: [Optimization](https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/optimization) Sorry for the dead link! ---
**Author:** ![Gidz](https://avatars.discourse-cdn.com/v4/letter/g/8e8cbc/32.png) [@Gidz](https://discuss.huggingface.co/u/Gidz)\ **Post date:** [August 27, 2023, 1:04pm UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/11 "2023-08-27T13:04:43Z")
Had the same problem… and a bunch of other issues from the tutorial: [Accelerated Inference with Optimum and Transformers Pipelines](https://huggingface.co/blog/optimum-inference)… Easy way to solve… delete all the argument, run, and use the logged comments (CLI messages) to find out the correct arguments. ---
**Author:** ![ddahlmeier](https://avatars.discourse-cdn.com/v4/letter/d/f0a364/32.png) [@ddahlmeier](https://discuss.huggingface.co/u/ddahlmeier)\ **Post date:** [February 10, 2024, 9:12am UTC](https://discuss.huggingface.co/t/optimize-and-quantize-with-optimum/23675/12 "2024-02-10T09:12:28Z")
I managed to make the example from the blog [Accelerated Inference with Optimum and Transformers Pipelines](https://huggingface.co/blog/optimum-inference) work with the advise from this thread, but since this week the code breaks when quantizing the model with this error ```auto RuntimeError: Unable to find data type for weight_name='/roberta/encoder/layer.0/attention/output/dense/MatMul_output_0' ``` I have created a new post to explain the steps how to reproduce the error [Optimum library optimization and quantization fails - #2 by ddahlmeier](https://discuss.huggingface.co/t/optimum-library-optimization-and-quantization-fails/72629/2)