# Custom BCEWithLogitsLoss for Sequence Classification using Auto Model

**URL:** <https://discuss.huggingface.co/t/custom-bcewithlogitsloss-for-sequence-classification-using-auto-model/156809>\
**Category:** Beginners\
**Created:** [May 26, 2025, 9:31am UTC](https://discuss.huggingface.co/t/custom-bcewithlogitsloss-for-sequence-classification-using-auto-model/156809 "2025-05-26T09:31:29Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sapphicart](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@sapphicart](https://discuss.huggingface.co/u/sapphicart)\
**Post date:** [May 26, 2025, 9:31am UTC](https://discuss.huggingface.co/t/custom-bcewithlogitsloss-for-sequence-classification-using-auto-model/156809/1 "2025-05-26T09:31:29Z")

</div>

I am training a BERT based model with `AutoModelForSequenceClassification`. I want to use class weights for the loss function so I followed this [discussion](https://discuss.huggingface.co/t/class-weights-for-bertforsequenceclassification/1674/8) and implemented a `BCEWithLogitsLoss`.

Problem is, HuggingFace advices using `num_labels=2` for Binary Classification with the Auto Class. When I try to implement `BCEWithLogitsLoss` I run into the error that

```auto
Value Error: Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 2]))

```

I understand why this is happening, the size of the `b_labels` is the ground truth while the `logits` have the shape `[32, 2]` but I can’t figure out how to fix it. Here’s the code:

```auto
criterion = nn.BCEWithLogitsLoss(weight=class_weights, reduction='mean')
logits = outputs.logits
loss = criterion(logits, b_labels)

```

Here, class\_weights is `tensor([1.0712, 0.9377])` Should I just use `CrossEntropyLoss` instead? I have dataset imbalance that’s why I want to switch to custom loss and see what happens. Please guide me towards a solution.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [May 26, 2025, 11:10am UTC](https://discuss.huggingface.co/t/custom-bcewithlogitsloss-for-sequence-classification-using-auto-model/156809/2 "2025-05-26T11:10:35Z")

</div>

Hmm…

> **[HELP with multilabel classification and BCEWithLogitsLoss](https://discuss.pytorch.org/t/help-with-multilabel-classification-and-bcewithlogitsloss/208094)**
>
> Im trying to classificate multilabels for sentiment analysis. first i separate the data and pass the batch on BERT layer, that encodes and generate embedding. So i create a NN that uses this data from BERT to classify and output a 16 layer data...

> [@Mismatched target and input size for BCE using "multi\_label\_classification"](https://discuss.huggingface.co/t/mismatched-target-and-input-size-for-bce-using-multi-label-classification/8706):
>
> I am trying to build a multi-label, multi-class classification model. Any input text can have zero or more labels, up to 11 possible classes. I have been trying to use the problem\_type="multi\_label\_classification" and everything looks OK, but I get ValueError: Target size (torch.Size([16, 11])) must be the same as input size (torch.Size([16, 2])) when it tries to calculate the binary\_cross\_entropy\_with\_logits I presume my data is in the wrong shape somehow, but I can’t see where exactly. Any s…

> **[ValueError: Target size (torch.Size(\[32\])) must be the same as input size...](https://discuss.pytorch.org/t/valueerror-target-size-torch-size-32-must-be-the-same-as-input-size-torch-size-32-25/182495)**
>
> Hi, Trying to fine-tune a Convnext Tiny model. This is an initialization of the model. from transformers import AutoModelForImageClassification,ConvNextFeatureExtractor,get\_linear\_schedule\_with\_warmup model =...

> BCEWithLogitsLoss is for Binary 0 or 1 use torch.nn.CrossEntropyLoss