# Setting specific device for Trainer **URL:** \ **Category:** Beginners\ **Created:** [August 20, 2020, 12:11pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784 "2020-08-20T12:11:55Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![ben9004](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@ben9004](https://discuss.huggingface.co/u/ben9004)\ **Post date:** [August 20, 2020, 12:11pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/1 "2020-08-20T12:11:56Z")
Hi I’m trying to fine-tune model with Trainer in transformers, Well, I want to use a specific number of GPU in my server. My server has two GPUs,(index 0, index 1) and I want to train my model with GPU index 1. I’ve read the Trainer and TrainingArguments documents, and I’ve tried the CUDA\_VISIBLE\_DEVICES thing already. but it didn’t worked for me. ``` import os os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" os.environ["CUDA_VISIBLE_DEVICES"] = "1" ``` (I did it within Jupyter, before I import all libraries) It gave me a runtime error when the trainer tries to initiate self.model = model.to(args.device) line. and the error says like RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable. ``` trainer = Trainer( model=model, args=training_args, data_collator=data_collator, train_dataset=train_dataset) ``` ![123](https://us1.discourse-cdn.com/hellohellohello/original/2X/7/70a8e2cc16268855cbebac02c87f206343a30b12.jpeg) I’ve also tried torch.cuda.set\_device(1), it also didn’t work. I don’t know how to set it up. It seems like I don’t have any options in argument of class Please help me to handle this problem. Thank you. ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [August 20, 2020, 12:37pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/2 "2020-08-20T12:37:18Z")
If `torch.cuda_set_device(1)` doesn’t work, the problem is in your install. Does the command `nvidia-smi` show up two GPUs? ---
**Author:** ![ben9004](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@ben9004](https://discuss.huggingface.co/u/ben9004)\ **Post date:** [August 21, 2020, 2:08am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/3 "2020-08-21T02:08:36Z")
Yes. My code is here ``` torch.cuda.set_device(1) torch.cuda.current_device() 1 epochs = 3 training_args = TrainingArguments( do_predict=True, output_dir=f'./results', overwrite_output_dir=True, do_train=True, num_train_epochs=epochs, per_device_train_batch_size=190, logging_steps=10, learning_rate=5e-05, warmup_steps=500, save_total_limit = 100, logging_dir='./logs', save_steps=50) training_args.device ``` out is ``` device(type='cuda', index=0) ``` and ``` trainer = Trainer( model=model, args=training_args, data_collator=data_collator, train_dataset=train_dataset) !nvidia-smi ``` ![2233](https://us1.discourse-cdn.com/hellohellohello/original/2X/7/7dfad86687c9209850c7f55ae04560d39d01903c.jpeg) I can see two GPUs and nvidia-smi shows that the process is running with GPU #0 still. It seems that torch.cuda.set\_device(1) doens’t work at all. ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [August 21, 2020, 1:10pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/4 "2020-08-21T13:10:35Z")
Mmmm, looking at the code, I can see that Trainer ignores the default device indeed. Will fix that soon. ---
**Author:** ![ben9004](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@ben9004](https://discuss.huggingface.co/u/ben9004)\ **Post date:** [August 22, 2020, 3:03am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/5 "2020-08-22T03:03:14Z")
I got a discussion with my mate, He suggested to reinstall GPU driver or restart the server. I’ll try it and tell what happens to you. Thanks a lot. ---
**Author:** ![bhadresh-savani](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bhadresh-savani/32/30594_2.png) [@bhadresh-savani](https://discuss.huggingface.co/u/bhadresh-savani)\ **Post date:** [May 6, 2021, 5:38am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/6 "2021-05-06T05:38:44Z")
Is it implemented?, I could not figure out how to set device one for training i tried above methods. ---
**Author:** ![FBH](https://avatars.discourse-cdn.com/v4/letter/f/db5fbb/32.png) [@FBH](https://discuss.huggingface.co/u/FBH)\ **Post date:** [October 25, 2022, 5:49am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/7 "2022-10-25T05:49:47Z")
Hi I am experiencing the same issue when using setfit is there any solution? ---
**Author:** ![ithieund](https://avatars.discourse-cdn.com/v4/letter/i/f475e1/32.png) [@ithieund](https://discuss.huggingface.co/u/ithieund)\ **Post date:** [November 29, 2022, 8:12pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/8 "2022-11-29T20:12:41Z")
Hi @sgugger , Did you fix this bug? Today I faced with the same issue when I set visible GPU to 1, but GPU 0 is still used ```auto import os os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" os.environ["CUDA_VISIBLE_DEVICES"] = "1" ``` ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/c/cc67862a8072615c47400b891756872b03e69c63.jpeg) Note: I’m using the latest transformers module in python 3.7 ---
**Author:** ![debashis](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@debashis](https://discuss.huggingface.co/u/debashis)\ **Post date:** [March 1, 2023, 1:10pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/9 "2023-03-01T13:10:06Z")
> [@ithieund](#): > > GPU to 1, but GPU 0 is still used I am having similar issue. Did you manage to get it work? ---
**Author:** ![ndvb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ndvb/32/17239_2.png) [@ndvb](https://discuss.huggingface.co/u/ndvb)\ **Post date:** [April 10, 2023, 12:34pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/10 "2023-04-10T12:34:38Z")
Same issue. Why when loading a model using .from\_pretrained we can specify device\_map, but we can’t specify device\_map when defining a Trainer? ---
**Author:** ![tumbleweed](https://avatars.discourse-cdn.com/v4/letter/t/43a26b/32.png) [@tumbleweed](https://discuss.huggingface.co/u/tumbleweed)\ **Post date:** [May 18, 2023, 9:28pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/11 "2023-05-18T21:28:03Z")
This issue still exists. Thanks. ---
**Author:** ![Harsh786](https://avatars.discourse-cdn.com/v4/letter/h/858c86/32.png) [@Harsh786](https://discuss.huggingface.co/u/Harsh786)\ **Post date:** [May 25, 2023, 7:34am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/12 "2023-05-25T07:34:07Z")
Meanwhile, this can do a trick: ```auto trainer = Trainer( model = model, . . device=0 # id of the device to use (int) -> for GPU(s): 0, 1 or for CPU: -1 ) ``` ---
**Author:** ![yaanhaan](https://avatars.discourse-cdn.com/v4/letter/y/e9a140/32.png) [@yaanhaan](https://discuss.huggingface.co/u/yaanhaan)\ **Post date:** [June 14, 2023, 6:12pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/13 "2023-06-14T18:12:04Z")
sorry to bother you, but what do you mean here? Trainer doesn’t have a parameter `device` ---
**Author:** ![noobmldude](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/noobmldude/32/48407_2.png) [@noobmldude](https://discuss.huggingface.co/u/noobmldude)\ **Post date:** [June 15, 2023, 4:13pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/14 "2023-06-15T16:13:59Z")
Is there any solution to this? Still can’t get Trainer to use a particular device in transformers `v4.30.2` even after setting environment variable ```python import os os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" os.environ["CUDA_VISIBLE_DEVICES"]= "2" ``` It still takes GPU device 0 ```python [In]: trainer.args.device [Out] :device(type='cuda', index=0) ``` ---
**Author:** ![rodio](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@rodio](https://discuss.huggingface.co/u/rodio)\ **Post date:** [July 10, 2023, 4:19am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/15 "2023-07-10T04:19:57Z")
The same issue still exist, literally dealing with this right now ---
**Author:** ![grofte](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@grofte](https://discuss.huggingface.co/u/grofte)\ **Post date:** [July 21, 2023, 8:48am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/16 "2023-07-21T08:48:38Z")
Really? I just did ```python import torch torch.cuda.set_device(1) ``` and it works flawlessly. Doesn’t fix that the models are pretty big relative to my GPU memory and I have to use quite small batch size 😄 Using `torch==2.0.1` and SetFit library. ---
**Author:** ![LoveLord](https://avatars.discourse-cdn.com/v4/letter/l/919ad9/32.png) [@LoveLord](https://discuss.huggingface.co/u/LoveLord)\ **Post date:** [July 30, 2023, 1:47am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/17 "2023-07-30T01:47:32Z")
It still doesn’t work. Any updates? Thanks! ---
**Author:** ![grofte](https://avatars.discourse-cdn.com/v4/letter/g/57b2e6/32.png) [@grofte](https://discuss.huggingface.co/u/grofte)\ **Post date:** [August 31, 2023, 1:11pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/18 "2023-08-31T13:11:58Z")
`pyNVML` is kinda terrible but if you can’t access the GPU using `pyNVML` then the problem is with your Python / Jupyter, not `torch` or any of the libraries that build on it. `!nvidia-smi` is not enough to verify that since the command is just sent directly to bash. Here is the website. > **[nvidia-ml-py](https://pypi.org/project/nvidia-ml-py/)** > > Python Bindings for the NVIDIA Management Library EDIT: But it sounds like the GPU is busy. Are you sure you don’t have another Jupyter session running? Even if it is done training it won’t release the GPU until you shut down the kernel. ---
**Author:** ![josejames00](https://avatars.discourse-cdn.com/v4/letter/j/3e96dc/32.png) [@josejames00](https://discuss.huggingface.co/u/josejames00)\ **Post date:** [October 5, 2023, 9:07pm UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/19 "2023-10-05T21:07:44Z")
A solution that works for me: import os os.environ[“CUDA\_DEVICE\_ORDER”]=“PCI\_BUS\_ID” os.environ[“CUDA\_VISIBLE\_DEVICES”]=“0” But import this FIRST, before anything from the pytorch or transformer libraries. ---
**Author:** ![LeoTungAnh](https://avatars.discourse-cdn.com/v4/letter/l/3ab097/32.png) [@LeoTungAnh](https://discuss.huggingface.co/u/LeoTungAnh)\ **Post date:** [October 13, 2023, 7:19am UTC](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784/20 "2023-10-13T07:19:43Z")
Thanks @josejames00, your solution works to me. [Next page](https://discuss.huggingface.co/t/setting-specific-device-for-trainer/784.md?page=2)