# "normal\_kernel\_cpu" not implemented for 'Char' when trying to import 8-bit model

**URL:** <https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349>\
**Category:** 🤗Transformers\
**Created:** [February 8, 2024, 2:47am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349 "2024-02-08T02:47:26Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![cccx3](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cccx3/32/1546_2.png) [@cccx3](https://discuss.huggingface.co/u/cccx3)\
**Post date:** [February 8, 2024, 2:47am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/1 "2024-02-08T02:47:26Z")

</div>

Didn’t see this issue anywhere else while googling, so posting here; tried loading an 8-bit SFTT model as part of a TRL-based reinforcement learning pipeline. Returned the error as stated in the title:

`RuntimeError: "normal_kernel_cpu" not implemented for 'Char'`

Model config is as follows, with the loading line throwing the error:

```auto
config = PPOConfig(
    model_name="drive/MyDrive/fg_proj/lr_pipeline/sftt_models/feb7_2024_8bit",
    learning_rate=1.41e-5,
    log_with="tensorboard",#"wandb",
    project_kwargs={"logging_dir": "run_logs_107_3"}
)

model = AutoModelForCausalLMWithValueHead.from_pretrained(config.model_name)

```

My hunch is this related to loading in 8-bit, since the full-bit and 4-bit models I tried did not throw this error. Any ideas as to what I’d be missing for this to occur would be helpful.

---

<div class="post-metadata">

**Author:** ![solomonpm](https://avatars.discourse-cdn.com/v4/letter/s/8e7dd6/32.png) [@solomonpm](https://discuss.huggingface.co/u/solomonpm)\
**Post date:** [February 29, 2024, 9:14am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/2 "2024-02-29T09:14:49Z")

</div>

Received similar error. when i try to load the 8 bit quantized model from local system.

I tried it for owl-vit model.

---

<div class="post-metadata">

**Author:** ![daltonh](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@daltonh](https://discuss.huggingface.co/u/daltonh)\
**Post date:** [April 30, 2024, 1:42pm UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/3 "2024-04-30T13:42:17Z")

</div>

I receive the same error, also when loading in with 8-bit but not when loading in with 4-bit…

---

<div class="post-metadata">

**Author:** ![mkgs210](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@mkgs210](https://discuss.huggingface.co/u/mkgs210)\
**Post date:** [May 11, 2024, 11:30am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/4 "2024-05-11T11:30:06Z")

</div>

I have this problem too

---

<div class="post-metadata">

**Author:** ![danielmisrael](https://avatars.discourse-cdn.com/v4/letter/d/ecae2f/32.png) [@danielmisrael](https://discuss.huggingface.co/u/danielmisrael)\
**Post date:** [July 2, 2024, 12:45am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/5 "2024-07-02T00:45:20Z")

</div>

Same problem. Someone please help

---

<div class="post-metadata">

**Author:** ![gazm2k5](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/gazm2k5/32/30561_2.png) [@gazm2k5](https://discuss.huggingface.co/u/gazm2k5)\
**Post date:** [August 20, 2024, 11:35am UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/6 "2024-08-20T11:35:54Z")

</div>

I am also having this problem trying to load longformer with a bitsandbytes config.

---

<div class="post-metadata">

**Author:** ![ZCharon](https://avatars.discourse-cdn.com/v4/letter/z/ac8455/32.png) [@ZCharon](https://discuss.huggingface.co/u/ZCharon)\
**Post date:** [February 23, 2025, 12:46pm UTC](https://discuss.huggingface.co/t/normal-kernel-cpu-not-implemented-for-char-when-trying-to-import-8-bit-model/72349/7 "2025-02-23T12:46:48Z")

</div>

```auto
    def _init_weights(self, module):
        if module is self.rank_head:
            if hasattr(module, "weight") and module.weight is not None:
                module.weight.data = module.weight.data.to(torch.float32)
            return
        super()._init_weights(module)

```

overwrite the \_init\_weights method will be successful