# Deepspeed ZeRO2, PEFT, bitsnbytes training

**URL:** <https://discuss.huggingface.co/t/deepspeed-zero2-peft-bitsnbytes-training/90331>\
**Category:** DeepSpeed\
**Created:** [June 4, 2024, 6:56pm UTC](https://discuss.huggingface.co/t/deepspeed-zero2-peft-bitsnbytes-training/90331 "2024-06-04T18:56:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![cdoern](https://avatars.discourse-cdn.com/v4/letter/c/f08c70/32.png) [@cdoern](https://discuss.huggingface.co/u/cdoern)\
**Post date:** [June 4, 2024, 6:56pm UTC](https://discuss.huggingface.co/t/deepspeed-zero2-peft-bitsnbytes-training/90331/1 "2024-06-04T18:56:46Z")

</div>

How can one train with a bitsnbytes quantized model, a PEFT config, and DeepSpeed? The docs don’t seem to cover this.

My training finishes successfully but then I have no pytorch\_model.bin. Because I am using PEFT, is there no `.bin`? Really just looking for a way to get the trained model, merge whatever adapters I need and produce a some final directory 🙏