# Huggingface / Pytorch versions on Sagemaker

**URL:** <https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315>\
**Category:** Amazon SageMaker\
**Created:** [November 18, 2022, 3:01pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315 "2022-11-18T15:01:40Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [November 18, 2022, 3:01pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/1 "2022-11-18T15:01:40Z")

</div>

When trying to use Huggingface estimator on sagemaker, [Run training on Amazon SageMaker](https://huggingface.co/docs/sagemaker/train) e.g.

```auto
# create the Estimator
huggingface_estimator = HuggingFace(
        entry_point='train.py',
        source_dir='./scripts',
        instance_type='ml.p3.2xlarge',
        instance_count=1,
        role=role,
        transformers_version='4.17',
        pytorch_version='1.10',
        py_version='py38',
        hyperparameters = hyperparameters
)

```

When I tried to increase the version to ` transformers_version='4.24'`, it throws an error where the maximum version supported is 4.17.

**Is there a page that lists the version that Sagemaker supports?**

**What are the possible versions for the following arguments?**

- transformers\_version
- pytorch\_version
- py\_version

**Also, are CPU instances supported by the Huggingface estimator on sagemaker?**

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [November 18, 2022, 3:25pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/2 "2022-11-18T15:25:03Z")

</div>

Hi @alvations - this [thread](https://discuss.huggingface.co/t/how-is-transfomers-verion-and-pytorch-version-determined/25802/2) should hopefully answer all your questions 😃

---

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [November 18, 2022, 5:48pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/3 "2022-11-18T17:48:24Z")

</div>

Thank you for the prompt reply!

There’s a pointer in the thread to `requirements.txt` for deployment when loading the model, is there some documentation to using `requirements.txt` when trying to use the huggingface estimator for training?

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [November 18, 2022, 6:24pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/4 "2022-11-18T18:24:12Z")

</div>

Yup, that would be in this [post](https://discuss.huggingface.co/t/deploying-open-ais-whisper-on-sagemaker/24761/9) 😉

---

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [November 22, 2022, 1:17am UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/5 "2022-11-22T01:17:02Z")

</div>

@marshmellow77, I’ve tried following the post [GitHub - aws/sagemaker-huggingface-inference-toolkit](https://github.com/aws/sagemaker-huggingface-inference-toolkit#-user-defined-codemodules) and tried a few combinations.

Is it right that the only way for me to use transformers version `>=4.17` with a `Trainer` object, I’ll have to

- first load the model in python
- save to disk in a directory
- add the `requirements.txt`, in the directory
- load the model with the `requirements.txt`

In code, something like:

```auto
import torch

from datasets import load_dataset
from transformers import EncoderDecoderModel
from transformers import AutoTokenizer
from transformers import Seq2SeqTrainer, Seq2SeqTrainingArguments

multibert = EncoderDecoderModel.from_encoder_decoder_pretrained(
    "bert-base-multilingual-uncased", "bert-base-multilingual-uncased"
)

multibert.save_to_disk("my-model/")

with open("my-model/code/requirements.txt", "w") as fout:
    fout.write("transformers==4.24")

```

Then inside my `./scripts/train-v4-24.py`, load the model like:

```auto
# Instead of 
# multibert = EncoderDecoderModel.from_encoder_decoder_pretrained(
# "bert-base-multilingual-uncased", "bert-base-multilingual-uncased"
# )
# load the model like this

def train(): 
    multibert = EncoderDecoderModel.load_from_disk("my-model/")

    trainer = Trainer(
    model=model,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
    tokenizer=tokenizer,
    )

    trainer.train()

def __main__ ():
    train()
    

```

Then in the Sagemaker, I can do this without the version:

```auto
huggingface_estimator = HuggingFace(
        entry_point='train-v4-24.py',
        source_dir='./scripts',
        instance_type='ml.p3.2xlarge',
        instance_count=1,
        role=role,
        hyperparameters = hyperparameters
)

```

**Is the above the only way to use the Trainer object with other versions `>=4.17`?**

---

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [November 23, 2022, 2:10pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/6 "2022-11-23T14:10:07Z")

</div>

Also asked on [docker - How to use AWS Sagemaker with newer version of Huggingface Estimator? - Stack Overflow](https://stackoverflow.com/q/74548143/610569) for wider audience

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [November 29, 2022, 5:06pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/8 "2022-11-29T17:06:13Z")

</div>

I published a repo that shows how you can extend the existing HF DLCs in order to use `transformers` versions higher than 4.17.0.

The repo shows how to extend the Inference Container, but the same applies to the Training DLC. Hope that helps!

> **[GitHub - marshmellow77/sm-extend-container: Extend a pre-built container to...](https://github.com/marshmellow77/sm-extend-container)**
>
> Extend a pre-built container to run latest transformer workloads - GitHub - marshmellow77/sm-extend-container: Extend a pre-built container to run latest transformer workloads

---

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [December 16, 2022, 12:23pm UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/9 "2022-12-16T12:23:59Z")

</div>

Pardon the hiatus. After trying out different methods, I’ve managed to get different transformers versions to work using either:

# Custom Docker with `image_uri`

Reference: [Adapting your own training container - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/adapt-training-container.html)

**Supported version of Huggingface:** Any version that you use to create your Docker image.

```auto
from sagemaker.huggingface import HuggingFace
import sagemaker
import boto3

client = boto3.client('sts')
account = client.get_caller.identit()['Account']
sess = boto3.sessions.Session()
role = sagemaker.get_execution_role()

region = sess.region_name
image_name = "huggingface-custom"
tag = "latest"

ecr_uri = f"{account}.dkr.ecr.{region}.amazonaws.com/{image_name}:{tag}"

estimator = HuggingFace(
  entry_point="train.py",
  source_dir="./scripts",
  instance_type="ml.p3.8large", 
  instance_count=1
  role=role,
  image_uri=ecr_uri, # Custom image.
  py_version="py38" # Somehow this is needed.
)

estimator.fit()

```

# Using Sagmaker Training Compiler Containers

Reference:

- [Amazon SageMaker Training Compiler - Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/training-compiler.html)
- [amazon-sagemaker-examples/sagemaker-training-compiler/huggingface/pytorch\_multiple\_gpu\_single\_node/language-modeling/gpt-2.ipynb at main · aws/amazon-sagemaker-examples · GitHub](https://github.com/aws/amazon-sagemaker-examples/blob/main/sagemaker-training-compiler/huggingface/pytorch_multiple_gpu_single_node/language-modeling/gpt-2.ipynb)

Using images from [deep-learning-containers/available\_images.md at master · aws/deep-learning-containers · GitHub](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#sagemaker-training-compiler-containers)

**Supported version of Huggingface:** `4.21.1`

```auto
from sagemaker.pytorch import PyTorch, TrainingCompilerConfig

estimator = HuggingFace(
  entry_point="train.py",
  source_dir="./scripts",
  instance_type="ml.p3.8large", 
  instance_count=1
  role=role,
  pytorch_version='1.11.0',
  transformers_version="4.21",
  py_version='py3',                  
  compiler_config=TrainingCompilerConfig(), # Needed to use the Training Compiler
  distribution ={'pytorchxla' : { 'enabled': True }} # To enable optimized training.
)

```

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [December 17, 2022, 7:28am UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/10 "2022-12-17T07:28:10Z")

</div>

I also meant to share this blog post I wrote around the repo I mentioned earlier: [Unlock the Latest Transformer Models with Amazon SageMaker | by Heiko Hotz | Dec, 2022 | Towards Data Science](https://towardsdatascience.com/unlock-the-latest-transformer-models-with-amazon-sagemaker-7fe65130d993)

---

<div class="post-metadata">

**Author:** ![alvations](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alvations/32/97_2.png) [@alvations](https://discuss.huggingface.co/u/alvations)\
**Post date:** [December 20, 2022, 9:41am UTC](https://discuss.huggingface.co/t/huggingface-pytorch-versions-on-sagemaker/26315/11 "2022-12-20T09:41:32Z")

</div>

Thank you for the share. Great job on the blogpost!