# Model doesn't accept int32 **URL:** \ **Category:** Beginners\ **Created:** [April 29, 2024, 5:03pm UTC](https://discuss.huggingface.co/t/model-doesnt-accept-int32/84388 "2024-04-29T17:03:40Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![Esuradi](https://avatars.discourse-cdn.com/v4/letter/e/d07c76/32.png) [@Esuradi](https://discuss.huggingface.co/u/Esuradi)\ **Post date:** [April 29, 2024, 5:03pm UTC](https://discuss.huggingface.co/t/model-doesnt-accept-int32/84388/1 "2024-04-29T17:03:40Z")
I noticed that the pytorch tokenizers output tensors with int64 precision, and the model (I’m experimenting with different transformer models) only accepts int64 tensors, otherwise it throws an error: RuntimeError Traceback (most recent call last) in \() ----\> 1 model(input\_ids = input\_ids, attention\_mask=attention\_mask, labels=labels) 6 frames /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in \_wrapped\_call\_impl(self, \*args, \*\*kwargs) 1516 return self.\_compiled\_call\_impl(\*args, \*\*kwargs) # type: ignore[misc] 1517 else: → 1518 return self.\_call\_impl(\*args, \*\*kwargs) 1519 1520 def \_call\_impl(self, \*args, \*\*kwargs): /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in \_call\_impl(self, \*args, \*\*kwargs) 1525 or \_global\_backward\_pre\_hooks or \_global\_backward\_hooks 1526 or \_global\_forward\_hooks or \_global\_forward\_pre\_hooks): → 1527 return forward\_call(\*args, \*\*kwargs) 1528 1529 try: /usr/local/lib/python3.10/dist-packages/transformers/models/t5/modeling\_t5.py in forward(self, input\_ids, attention\_mask, decoder\_input\_ids, decoder\_attention\_mask, head\_mask, decoder\_head\_mask, cross\_attn\_head\_mask, encoder\_outputs, past\_key\_values, inputs\_embeds, decoder\_inputs\_embeds, labels, use\_cache, output\_attentions, output\_hidden\_states, return\_dict) 1776 # move labels to correct device to enable PP 1777 labels = labels.to(lm\_logits.device) → 1778 loss = loss\_fct(lm\_logits.view(-1, lm\_logits.size(-1)), labels.view(-1)) 1779 # TODO(thom): Add z\_loss [mesh/mesh\_tensorflow/layers.py at fa19d69eafc9a482aff0b59ddd96b025c0cb207d · tensorflow/mesh · GitHub](https://github.com/tensorflow/mesh/blob/fa19d69eafc9a482aff0b59ddd96b025c0cb207d/mesh_tensorflow/layers.py#L666) 1780 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in \_wrapped\_call\_impl(self, \*args, \*\*kwargs) 1516 return self.\_compiled\_call\_impl(\*args, \*\*kwargs) # type: ignore[misc] 1517 else: → 1518 return self.\_call\_impl(\*args, \*\*kwargs) 1519 1520 def \_call\_impl(self, \*args, \*\*kwargs): /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py in \_call\_impl(self, \*args, \*\*kwargs) 1525 or \_global\_backward\_pre\_hooks or \_global\_backward\_hooks 1526 or \_global\_forward\_hooks or \_global\_forward\_pre\_hooks): → 1527 return forward\_call(\*args, \*\*kwargs) 1528 1529 try: /usr/local/lib/python3.10/dist-packages/torch/nn/modules/loss.py in forward(self, input, target) 1177 1178 def forward(self, input: Tensor, target: Tensor) → Tensor: → 1179 return F.cross\_entropy(input, target, weight=self.weight, 1180 ignore\_index=self.ignore\_index, reduction=self.reduction, 1181 label\_smoothing=self.label\_smoothing) /usr/local/lib/python3.10/dist-packages/torch/nn/functional.py in cross\_entropy(input, target, weight, size\_average, ignore\_index, reduce, reduction, label\_smoothing) 3051 if size\_average is not None or reduce is not None: 3052 reduction = \_Reduction.legacy\_get\_string(size\_average, reduce) → 3053 return torch.\_C.\_nn.cross\_entropy\_loss(input, target, weight, \_Reduction.get\_enum(reduction), ignore\_index, label\_smoothing) 3054 3055 RuntimeError: expected scalar type Long but found Int Is there a way that allows the model to accept tesnors with preciosn int 32 ? (I’m aware that I can change the input type to int64, but due to memory constraints I would like my input to be int32)