# PEFT tuning error

**URL:** <https://discuss.huggingface.co/t/peft-tuning-error/49060>\
**Category:** Beginners\
**Created:** [August 2, 2023, 3:52am UTC](https://discuss.huggingface.co/t/peft-tuning-error/49060 "2023-08-02T03:52:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![woyaofawenzhang](https://avatars.discourse-cdn.com/v4/letter/w/b5e925/32.png) [@woyaofawenzhang](https://discuss.huggingface.co/u/woyaofawenzhang)\
**Post date:** [August 2, 2023, 3:52am UTC](https://discuss.huggingface.co/t/peft-tuning-error/49060/1 "2023-08-02T03:52:53Z")

</div>

Hi, I was finetuning a DNABERT model using LORA.  
However, there seems to have a data loading error after applying LORA.  
My data looks like

> Dataset({  
> features: [‘binding\_energy’, ‘label’, ‘input\_ids’, ‘token\_type\_ids’, ‘attention\_mask’, ‘labels’],  
> num\_rows: 9261  
> })

and my code is

```auto
    config = BertConfig.from_pretrained(
        data_training_args.model_path,
        num_labels=1,
    )
    tokenizer = DNATokenizer.from_pretrained(
        data_training_args.model_path,
        do_lower_case=False,
    )
    model = DNABertForSequenceClassification.from_pretrained(
        data_training_args.model_path,
        from_tf=bool(".ckpt" in data_training_args.model_path),
        config=config,
    )
    
    # Define LoRA Config
    lora_config = LoraConfig(
                     r=LORA_R,
                     lora_alpha=LORA_ALPHA,
                     lora_dropout=LORA_DROPOUT,
                     bias="none",
                    inference_mode=False
    )
    model = get_peft_model(model, lora_config)
    model.print_trainable_parameters()
    
    def collate_fn(examples):
        rdict = {}
        print(examples)
        for k in examples[0].keys():
            if k not in [
                "binding_energy",
                "input_ids",
                "token_type_ids",
                "attention_mask",
                "labels",
            ]:
                continue
            rdict[k] = torch.stack([torch.tensor(example[k]) for example in examples])
        rdict["binding_energy"] = torch.unsqueeze(rdict["binding_energy"], 1)
        return rdict

    train_dataset = load_from_disk(data_training_args.train_data_dir)
    dev_dataset = load_from_disk(data_training_args.dev_data_dir)

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=dev_dataset,
        tokenizer=tokenizer,
        data_collator=collate_fn,
        compute_metrics=compute_metrics
    )

```

Then there is a key error:

> rdict[“binding\_energy”] = torch.unsqueeze(rdict[“binding\_energy”], 1)  
> KeyError: ‘binding\_energy’  
> When I print **examples** , only **label** is there, but my loaded data has more than that. And without LORA won’t have such issue.  
> Wondering why would this happen? Thanks!