# RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA\_LAUNCH\_BLOCKING=1 Compile with \`TORCH\_USE\_CU

**URL:** <https://discuss.huggingface.co/t/runtimeerror-cuda-error-device-side-assert-triggered-cuda-kernel-errors-might-be-asynchronously-reported-at-some-other-api-call-so-the-stacktrace-below-might-be-incorrect-for-debugging-consider-passing-cuda-launch-blocking-1-compile-with-torch-use-cu/114664>\
**Category:** 🤗Transformers\
**Created:** [October 30, 2024, 8:05am UTC](https://discuss.huggingface.co/t/runtimeerror-cuda-error-device-side-assert-triggered-cuda-kernel-errors-might-be-asynchronously-reported-at-some-other-api-call-so-the-stacktrace-below-might-be-incorrect-for-debugging-consider-passing-cuda-launch-blocking-1-compile-with-torch-use-cu/114664 "2024-10-30T08:05:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![saulml](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saulml/32/33535_2.png) [@saulml](https://discuss.huggingface.co/u/saulml)\
**Post date:** [October 30, 2024, 8:05am UTC](https://discuss.huggingface.co/t/runtimeerror-cuda-error-device-side-assert-triggered-cuda-kernel-errors-might-be-asynchronously-reported-at-some-other-api-call-so-the-stacktrace-below-might-be-incorrect-for-debugging-consider-passing-cuda-launch-blocking-1-compile-with-torch-use-cu/114664/1 "2024-10-30T08:05:12Z")

</div>

Hi All,

I am getting below error and tried pterry much everything. But did not work out. There are 8 clases in the dataset. I am not sure how to resolve the issue.

The code is:

# Convert data to InputExample format

# Convert labels to integers

for example in train\_examples:  
example.label = int(example.label)  
#print(example.label)

# Wrap the training examples in a SentencesDataset for compatibility with DataLoader

train\_dataset = SentencesDataset(train\_examples, model)  
train\_dataloader = DataLoader(train\_dataset, shuffle=True, batch\_size=16)

# Define the loss function with SoftmaxLoss for binary classification

train\_loss = losses.SoftmaxLoss(  
model=model,  
sentence\_embedding\_dimension=model.get\_sentence\_embedding\_dimension(),  
num\_labels=8  
)

# Train the model using fit()

model.fit(  
train\_objectives=[(train\_dataloader, train\_loss)],  
epochs=4,  
warmup\_steps=100  
)

The error is:  
RuntimeError: CUDA error: device-side assert triggered  
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.  
For debugging consider passing CUDA\_LAUNCH\_BLOCKING=1  
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Thank you,

Seyhan

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 30, 2024, 9:45am UTC](https://discuss.huggingface.co/t/runtimeerror-cuda-error-device-side-assert-triggered-cuda-kernel-errors-might-be-asynchronously-reported-at-some-other-api-call-so-the-stacktrace-below-might-be-incorrect-for-debugging-consider-passing-cuda-launch-blocking-1-compile-with-torch-use-cu/114664/2 "2024-10-30T09:45:43Z")

</div>

That error says CUDA, but it’s actually an error that often appears in places that have nothing to do with CUDA. Sometimes it can be fixed by tweaking the upper and lower limit settings.

> <https://github.com/pytorch/pytorch/issues/120901>
>
> \### 🐛 Describe the bug
> 
> Codes:
> \`\`\`
> import torch
> 
> data1 = torch.tensor(\[\[1, 0…, 3\], \[0, 5, 0\]\])
> data2 = torch.tensor(\[\[0, 0\],
> \[0, 2\],
> \[1, 1\]\])
> 
> data1 = data1.cuda()
> data2 = data2.cuda()
> 
> print(data1\[data2\])
> \`\`\`
> 
> Outputs:
> \`\`\`
> ../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: \[0,0,0\], thread: \[9,0,0\] Assertion \`-sizes\[i\] \<= index && index \< sizes\[i\] && "index out of bounds"\` failed.
> ../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: \[0,0,0\], thread: \[10,0,0\] Assertion \`-sizes\[i\] \<= index && index \< sizes\[i\] && "index out of bounds"\` failed.
> ../aten/src/ATen/native/cuda/IndexKernel.cu:92: operator(): block: \[0,0,0\], thread: \[11,0,0\] Assertion \`-sizes\[i\] \<= index && index \< sizes\[i\] && "index out of bounds"\` failed.
> 
> RuntimeError: CUDA error: device-side assert triggered
> CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
> For debugging consider passing CUDA\_LAUNCH\_BLOCKING=1.
> Compile with \`TORCH\_USE\_CUDA\_DSA\` to enable device-side assertions.
> \`\`\`
> 
> If CUDA is not used, clear error can be obtained:
> \`\`\`
> IndexError: index 2 is out of bounds for dimension 0 with size 2
> \`\`\`
> 
> Similar to \[#42452\](https://github.com/pytorch/pytorch/issues/42452), any subsequent calls to CUDA (even with valid instructions), result in:
> \`\`\`
> RuntimeError: CUDA error: device-side assert triggered
> \`\`\`
> 
> \### Versions
> 
> Collecting environment information...
> PyTorch version: 2.2.0+cu121
> Is debug build: False
> CUDA used to build PyTorch: 12.1
> ROCM used to build PyTorch: N/A
> 
> OS: Ubuntu 20.04.3 LTS (x86\_64)
> GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
> Clang version: Could not collect
> CMake version: Could not collect
> Libc version: glibc-2.31
> 
> Python version: 3.8.18 (default, Sep 11 2023, 13:40:15) \[GCC 11.2.0\] (64-bit runtime)
> Python platform: Linux-5.15.0-97-generic-x86\_64-with-glibc2.17
> Is CUDA available: True
> CUDA runtime version: Could not collect
> CUDA\_MODULE\_LOADING set to: LAZY
> GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090 Ti
> Nvidia driver version: 535.154.05
> cuDNN version: Could not collect
> HIP runtime version: N/A
> MIOpen runtime version: N/A
> Is XNNPACK available: True
> 
> CPU:
> Architecture: x86\_64
> CPU op-mode(s): 32-bit, 64-bit
> Byte Order: Little Endian
> Address sizes: 46 bits physical, 48 bits virtual
> CPU(s): 28
> On-line CPU(s) list: 0-27
> Thread(s) per core: 2
> Core(s) per socket: 14
> Socket(s): 1
> NUMA node(s): 1
> Vendor ID: GenuineIntel
> CPU family: 6
> Model: 85
> Model name: Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz
> Stepping: 4
> CPU MHz: 3100.000
> CPU max MHz: 4400.0000
> CPU min MHz: 1200.0000
> BogoMIPS: 6199.99
> Virtualisation: VT-x
> L1d cache: 448 KiB
> L1i cache: 448 KiB
> L2 cache: 14 MiB
> L3 cache: 19.3 MiB
> NUMA node0 CPU(s): 0-27
> Vulnerability Gather data sampling: Mitigation; Microcode
> Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled
> Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
> Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable
> Vulnerability Meltdown: Mitigation; PTI
> Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
> Vulnerability Retbleed: Mitigation; IBRS
> Vulnerability Spec rstack overflow: Not affected
> Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
> Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and \_\_user pointer sanitization
> Vulnerability Spectre v2: Mitigation; IBRS, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
> Vulnerability Srbds: Not affected
> Vulnerability Tsx async abort: Mitigation; Clear CPU buffers; SMT vulnerable
> Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant\_tsc art arch\_perfmon pebs bts rep\_good nopl xtopology nonstop\_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds\_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4\_1 sse4\_2 x2apic movbe popcnt tsc\_deadline\_timer aes xsave avx f16c rdrand lahf\_lm abm 3dnowprefetch cpuid\_fault epb cat\_l3 cdp\_l3 invpcid\_single pti ssbd mba ibrs ibpb stibp tpr\_shadow vnmi flexpriority ept vpid ept\_ad fsgsbase tsc\_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt\_a avx512f avx512dq rdseed adx smap clflushopt clwb intel\_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm\_llc cqm\_occup\_llc cqm\_mbm\_total cqm\_mbm\_local dtherm ida arat pln pts hwp hwp\_act\_window hwp\_epp hwp\_pkg\_req md\_clear flush\_l1d arch\_capabilities
> 
> Versions of relevant libraries:
> \[pip3\] numpy==1.24.4
> \[pip3\] torch==2.2.0
> \[pip3\] triton==2.2.0
> \[conda\] numpy 1.24.4 pypi\_0 pypi
> \[conda\] torch 2.2.0 pypi\_0 pypi
> \[conda\] triton 2.2.0 pypi\_0 pypi
> 
> cc @ptrblck

---

<div class="post-metadata">

**Author:** ![saulml](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/saulml/32/33535_2.png) [@saulml](https://discuss.huggingface.co/u/saulml)\
**Post date:** [November 1, 2024, 4:15am UTC](https://discuss.huggingface.co/t/runtimeerror-cuda-error-device-side-assert-triggered-cuda-kernel-errors-might-be-asynchronously-reported-at-some-other-api-call-so-the-stacktrace-below-might-be-incorrect-for-debugging-consider-passing-cuda-launch-blocking-1-compile-with-torch-use-cu/114664/3 "2024-11-01T04:15:38Z")

</div>

Thanks Pal!