# Does loading in 4bit override an 8bit model?

**URL:** <https://discuss.huggingface.co/t/does-loading-in-4bit-override-an-8bit-model/59265>\
**Category:** 🤗Transformers\
**Created:** [October 20, 2023, 10:39am UTC](https://discuss.huggingface.co/t/does-loading-in-4bit-override-an-8bit-model/59265 "2023-10-20T10:39:41Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![cassianlewis](https://avatars.discourse-cdn.com/v4/letter/c/e36b37/32.png) [@cassianlewis](https://discuss.huggingface.co/u/cassianlewis)\
**Post date:** [October 20, 2023, 10:39am UTC](https://discuss.huggingface.co/t/does-loading-in-4bit-override-an-8bit-model/59265/1 "2023-10-20T10:39:41Z")

</div>

Having saved a model in 8bit:

```auto
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    load_in_8bit = True,
    device_map = 'auto')

model.save_pretrained('model_weights_test')

```

I then loaded it in 4bit (with a different script):

```auto
double_quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)
    
model = AutoModelForCausalLM.from_pretrained(
    'model_weights_test',
    quantization_config=double_quant_config,
    device_map = 'auto')

```

However I am then running into OOM issues I was not seeing by just initially loading it in 4bit:

```auto
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=double_quant_config,
    device_map = 'auto')

```

My question is therefore: does the 4bit config fail to override the 8bit saved model? Am I essentially just training an 8bit model, hence the OOM issues?