# Transformers 4.6.0 on SageMaker? **URL:** \ **Category:** Amazon SageMaker\ **Created:** [May 17, 2021, 7:51pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217 "2021-05-17T19:51:36Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![nreamaroon](https://avatars.discourse-cdn.com/v4/letter/n/eada6e/32.png) [@nreamaroon](https://discuss.huggingface.co/u/nreamaroon)\ **Post date:** [May 17, 2021, 7:51pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/1 "2021-05-17T19:51:36Z")
Hi all, Is there a timeline for when `Transformers 4.6.0` will be supported in the HuggingFace SDK on SageMaker? I’ve recently been having issues with CUDA running out of memory while training a distilBert model: > RuntimeError: CUDA out of memory. Tried to allocate 6.87 GiB (GPU 0; 15.78 GiB total capacity; 7.35 GiB already allocated; 2.79 GiB free; 11.78 GiB reserved in total by PyTorch) It seems like this has been acknowledged and fixed in a [recent commit](https://github.com/huggingface/transformers/pull/11318) - also similarly described [here](https://discuss.huggingface.co/t/run-mlm-py-cuda-error-memory-after-resuming-a-training/5549). Looks like this has also been added to `Transformers 4.6.0` and I can confirm that using this latest version (without the HuggingFace SDK) fixes the OOM issues for me. When configuring the HuggingFace estimator, it seems like the latest supported version of `Transformers` is version `4.5.0` > ValueError: Unsupported huggingface version: 4.6.0. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.4, 4.5. Does anyone have an idea of when we can expect version `4.6.0` to be supported? Thanks! ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [May 18, 2021, 6:21am UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/2 "2021-05-18T06:21:48Z")
hey @nreamaroon, We already opened a PR for a DLC with `transformers 4.6.0`. I hope we can get as merged as soon as possible. > > > … 4.6.0 > > \*Issue #, if available:\* > > \## PR Checklist > \- \[x\] I've prepended PR …tag with frameworks/job this applies to : \[mxnet, tensorflow, pytorch\] | \[ei/neuron\] | \[build\] | \[test\] | \[benchmark\] | \[ec2, ecs, eks, sagemaker\] > \- \[x\] (If applicable) I've documented below the DLC image/dockerfile this relates to > \- \[x\] (If applicable) I've documented below the tests I've run on the DLC image > \- \[x\] (If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list. See \[https://www.apache.org/legal/resolved.html\](https://www.apache.org/legal/resolved.html). > \- \[x\] (If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them. > > \## Pytest Marker Checklist > \- \[\] (If applicable) I have added the marker \`@pytest.mark.model("\")\` to the new tests which I have added, to specify the Deep Learning model that is used in the test (use \`"N/A"\` if the test doesn't use a model) > \- \[\] (If applicable) I have added the marker \`@pytest.mark.integration("\")\` to the new tests which I have added, to specify the feature that will be tested > \- \[\] (If applicable) I have added the marker \`@pytest.mark.multinode(\)\` to the new tests which I have added, to specify the number of nodes used on a multi-node test > \- \[\] (If applicable) I have added the marker \`@pytest.mark.processor(\<"cpu"/"gpu"/"eia"/"neuron"\>)\` to the new tests which I have added, if a test is specifically applicable to only one processor type > > \#### EIA/NEURON Checklist > \* When creating a PR: > \- \[\] I've modified \`src/config/build\_config.py\` in my PR branch by setting \`ENABLE\_EI\_MODE = True\` or \`ENABLE\_NEURON\_MODE = True\` > \* When PR is reviewed and ready to be merged: > \- \[\] I've reverted the code change on the config file mentioned above > \#### Benchmark Checklist > \* When creating a PR: > \- \[\] I've modified \`src/config/test\_config.py\` in my PR branch by setting \`ENABLE\_BENCHMARK\_DEV\_MODE = True\` > \* When PR is reviewed and ready to be merged: > \- \[\] I've reverted the code change on the config file mentioned above > > \## Reviewer Checklist > \* For reviewer, before merging, please cross-check: > \- \[\] I've verified the code change on the config file mentioned above has already been reverted > > \*Description:\* > > This PRs updates the \`transformers\` version inside the \`huggingface\` PyTorch and TensorFlow DLC. > > \*Tests run:\* > > https://github.com/huggingface/transformers/tree/master/tests/sagemaker > > > | ID | Description | Platform | #GPUS | Collected & evaluated metrics | > |-------------------------------------|-------------------------------------------------------------------|-----------------------------|-------|------------------------------------------| > | pytorch-transfromers-test-single | test bert finetuning using BERT fromtransformerlib+PT | SageMaker createTrainingJob | 1 | train\_runtime, eval\_accuracy & eval\_loss | > | pytorch-transfromers-test-2-ddp | test bert finetuning using BERT from transformer lib+ PT DPP | SageMaker createTrainingJob | 16 | train\_runtime, eval\_accuracy & eval\_loss | > | pytorch-transfromers-test-2-smd | test bert finetuning using BERT from transformer lib+ PT SM DDP | SageMaker createTrainingJob | 16 | train\_runtime, eval\_accuracy & eval\_loss | > | pytorch-transfromers-test-1-smp | test roberta finetuning using BERT from transformer lib+ PT SM MP | SageMaker createTrainingJob | 8 | train\_runtime, eval\_accuracy & eval\_loss | > | tensorflow-transfromers-test-single | Test bert finetuning using BERT from transformer lib+TF | SageMaker createTrainingJob | 1 | train\_runtime, eval\_accuracy & eval\_loss | > | tensorflow-transfromers-test-2-smd | test bert finetuning using BERT from transformer lib+ TF SM DDP | SageMaker createTrainingJob | 16 | train\_runtime, eval\_accuracy & eval\_loss | > > \*DLC image/dockerfile:\* > > The Hugging Face DLCs for PyTorch and TensorFlow > > > \*Additional context:\* > !\[Bildschirmfoto 2021-04-06 um 14 20 34\](https://user-images.githubusercontent.com/32632186/114026575-afec6480-9876-11eb-892e-55144707367d.png) > > > > By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. ---
**Author:** ![benG](https://avatars.discourse-cdn.com/v4/letter/b/b3f665/32.png) [@benG](https://discuss.huggingface.co/u/benG)\ **Post date:** [June 11, 2021, 12:28am UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/3 "2021-06-11T00:28:33Z")
Hi @philschmid - Thank you for all your work on this. I have just tried to use `Transformers` version `4.6.0` in a Huggingface estimator in Sagemaker, but get the same error as reported by @nreamaroon > ValueError: Unsupported huggingface version: 4.6. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.4.2, 4.5.0, 4.4, 4.5. I saw that the PR that you [referenced](https://github.com/aws/deep-learning-containers/pull/1103) has now been merged so am wondering if there’s anything else I need to do in order to be able to use `Transformers` version `4.6.0`? Thanks again! ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [June 11, 2021, 7:07am UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/4 "2021-06-11T07:07:01Z")
Hi @benG, There were some issues with the python sdk. But it is now merged and should be released tomorrow. You can use the new sagemaker-sdk with `pip install git+https://github.com/aws/sagemaker-python-sdk.git` or you can wait until tomorrow and install `pip install sagemaker` and then define `transformers_version='4.6'` (`4.6.1`). ```python huggingface_estimator = HuggingFace(entry_point='train.py', source_dir='./scripts', instance_type='ml.p3.2xlarge', instance_count=1, role=role, transformers_version='4.6', pytorch_version='1.7', py_version='py36', hyperparameters = hyperparameters) ``` ---
**Author:** ![pierreguillou](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pierreguillou/32/19626_2.png) [@pierreguillou](https://discuss.huggingface.co/u/pierreguillou)\ **Post date:** [December 1, 2021, 7:51pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/5 "2021-12-01T19:51:23Z")
Hi @philschmid. In the [AWS SageMaker HF workshops notebooks](https://github.com/philschmid/huggingface-sagemaker-workshop-series), we need to install the following versions `!pip install "sagemaker>=2.48.0" "transformers==4.6.1" "datasets[s3]==1.6.2"` Are they still the right versions? How could we know in the future which compatible versions to install? Thanks. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [December 2, 2021, 7:10am UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/6 "2021-12-02T07:10:58Z")
Hey @pierreguillou, Yes, versions still are compatible, but you can upgrade them as you want. You can find an overview of all DLCs + versions at [Reference](https://huggingface.co/docs/sagemaker/reference#deep-learning-container), which should be updated soon. I created a PR to update them [add new container version by philschmid · Pull Request #514 · huggingface/huggingface\_hub · GitHub](https://github.com/huggingface/huggingface_hub/pull/514/files). The latest `Transformers` version is currently `4.12.3`. You can also check here for notebook example with newer versions [notebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/master/sagemaker/14_train_and_push_to_hub/sagemaker-notebook.ipynb) ---
**Author:** ![pierreguillou](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/pierreguillou/32/19626_2.png) [@pierreguillou](https://discuss.huggingface.co/u/pierreguillou)\ **Post date:** [December 2, 2021, 1:00pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/7 "2021-12-02T13:00:14Z")
Thanks @philschmid Just to be sure, here is my understanding. ### Notebook In the notebook [notebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/master/sagemaker/14_train_and_push_to_hub/sagemaker-notebook.ipynb) that you mentioned, the libraries versions are the following ones: - sagemaker\>=2.69.0 - transformers==4.12.3 - datasets==1.13 The installation is done by the following code: ```auto !pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade # using older dataset due to incompatibility of sagemaker notebook & aws-cli with > s3fs and fsspec to >= 2021.10 !pip install "datasets==1.13" --upgrade ``` I noticed that unlike the first notebook (used in your workshop), you no longer install `datasets[s3]` but directly `datasets`. **I’m just curious as to why and what are the differences between these 2 libraries?** Thanks. ### Reference & PR In the [Reference](https://huggingface.co/docs/sagemaker/reference#deep-learning-container) page, the transformers version goes up to but `4.6.1` (and compatible datasets `1.6.2`) but your recent [PR](https://github.com/huggingface/huggingface_hub/pull/514/files) shows that the current situation of Training DLC is `transformers 4.12.3, datasets 1.15.1, PyTorch 1.9.1, python 3.8` **Does it mean I could run the following code in a notebook instance of AWS SageMaker?** ```auto !pip install "sagemaker>=2.69.0" "transformers==4.12.3" --upgrade !pip install "datasets==1.15.1" --upgrade ``` If yes, I would need to update as well Pytorch and python as I do not see in the available kernel, this configuration. **How can I do this?** (or do you recommend to create a new conda kernel? through the jupyter terminal? could you give use the commands?) ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [December 3, 2021, 7:54am UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/8 "2021-12-03T07:54:55Z")
The [PR](https://github.com/huggingface/huggingface_hub/pull/514/files) is the overview of the version inside the DLCs. Meaning the container you use when running `huggingface_estimator.fit()`. In the notebook, you can install every version you want, but it makes most of the time sense to use the same version as you will in the `HuggingFace` estimator. Thanks for finding the pip installation it should be `datasets[s3]` to make sure we install the S3 utils to push our datasets to s3. ---
**Author:** ![cgpeltier-janes](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@cgpeltier-janes](https://discuss.huggingface.co/u/cgpeltier-janes)\ **Post date:** [September 9, 2022, 2:34pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/9 "2022-09-09T14:34:57Z")
I’m currently getting this error with the Sagemaker SDK trying to use `Transformers==4.19.4`. Hoping to use \>=4.19.0 to take advantage of some deBERTa models. 🙂 ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 9, 2022, 3:03pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/10 "2022-09-09T15:03:46Z")
What error are you getting? ---
**Author:** ![cgpeltier-janes](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@cgpeltier-janes](https://discuss.huggingface.co/u/cgpeltier-janes)\ **Post date:** [September 9, 2022, 3:06pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/11 "2022-09-09T15:06:41Z")
This - `ValueError: Unsupported huggingface version: 4.19.4. You may need to upgrade your SDK version (pip install -U sagemaker) for newer huggingface versions. Supported huggingface version(s): 4.6.1, 4.10.2, 4.11.0, 4.12.3, 4.17.0, 4.6, 4.10, 4.11, 4.12, 4.17.` In my pipeline’s `requirements.txt` I have it using `sagemaker==2.108.0`. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 9, 2022, 3:18pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/12 "2022-09-09T15:18:10Z")
There is currently no available DLC with this `transformers` version, but you can add a `requirements.txt` with the version you want to install. ---
**Author:** ![cgpeltier-janes](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@cgpeltier-janes](https://discuss.huggingface.co/u/cgpeltier-janes)\ **Post date:** [September 9, 2022, 3:22pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/13 "2022-09-09T15:22:06Z")
Ah, ok. So even if the DLC/Sagemaker SDK doesn’t have the version I need, I can just specify it myself as a requirement – thanks. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 9, 2022, 3:30pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/14 "2022-09-09T15:30:21Z")
Yes, thats correct but we are working on updating the version. ---
**Author:** ![cgpeltier-janes](https://avatars.discourse-cdn.com/v4/letter/c/e56c9b/32.png) [@cgpeltier-janes](https://discuss.huggingface.co/u/cgpeltier-janes)\ **Post date:** [September 9, 2022, 4:28pm UTC](https://discuss.huggingface.co/t/transformers-4-6-0-on-sagemaker/6217/15 "2022-09-09T16:28:40Z")
Good to hear, thank you! We’re getting errors in our terragrunt apply when we add any transformers version to our requirements.txt for some reason. Strangely the build works fine specifying any other packages besides `transformers`. We’ll keep digging on that unless you have any recommendations.