# Tokenizer setting for model = LlamaForCausalLM.from\_pretrained(model\_path, device\_map='auto')

**URL:** <https://discuss.huggingface.co/t/tokenizer-setting-for-model-llamaforcausallm-from-pretrained-model-path-device-map-auto/52060>\
**Category:** Models\
**Created:** [August 25, 2023, 1:35am UTC](https://discuss.huggingface.co/t/tokenizer-setting-for-model-llamaforcausallm-from-pretrained-model-path-device-map-auto/52060 "2023-08-25T01:35:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![CHOJW1004](https://avatars.discourse-cdn.com/v4/letter/c/bbce88/32.png) [@CHOJW1004](https://discuss.huggingface.co/u/CHOJW1004)\
**Post date:** [August 25, 2023, 1:35am UTC](https://discuss.huggingface.co/t/tokenizer-setting-for-model-llamaforcausallm-from-pretrained-model-path-device-map-auto/52060/1 "2023-08-25T01:35:35Z")

</div>

I’m using 2 a100 GPU so i set model device\_map=‘auto’ than I got this error.

```auto
tokenizer = AutoTokenizer.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token
model = LlamaForCausalLM.from_pretrained(model_path, device_map='auto')
model.resize_token_embeddings(len(tokenizer))

conversation_str = 'test....................'
inputs = tokenizer(conversation_str, return_tensors='pt').to('cuda')
generate_ids = model.generate(inputs.input_ids, max_length=4096)

```

```auto
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

```

I guess is data parallelizing issue but can’t solve.