# Need help deploying a HF model to AWS Sagemaker

**URL:** <https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794>\
**Category:** Amazon SageMaker\
**Created:** [September 26, 2024, 2:55pm UTC](https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794 "2024-09-26T14:55:17Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhashwarsengupta](https://avatars.discourse-cdn.com/v4/letter/b/74df32/32.png) [@bhashwarsengupta](https://discuss.huggingface.co/u/bhashwarsengupta)\
**Post date:** [September 26, 2024, 2:55pm UTC](https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794/1 "2024-09-26T14:55:17Z")

</div>

I’m trying to deploy a private HF model to AWS Sagemaker by executing the following code locally:

```auto
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel
from huggingface_hub import login
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()

# Get the access token
token = os.environ.get("HF_LOGIN", None)

login(token)

hub = {
    "HF_MODEL_ID": "Org/model-name",
    "HF_TASK": "text-classification",
    "HUGGING_FACE_HUB_TOKEN": token
}

iam = boto3.client('iam')
role = iam.get_role(RoleName="AmazonSageMaker-ExecutionRole")["Role"]["Arn"]

session = sagemaker.Session(boto3.Session(region_name="ap-southeast-2"))

huggingface_model = HuggingFaceModel(
    env=hub,
    role=role, # The IAM role with necessary permissions
    transformers_version="4.17", # Use the appropriate transformers version
    pytorch_version="1.10", # Use the appropriate PyTorch version
    py_version="py38", # Python version
    sagemaker_session=session
)

predictor = huggingface_model.deploy(
    initial_instance_count=1, # Number of instances (e.g., 1 for small load)
    instance_type="ml.m5.xlarge" # Choose the instance type based on the resources required
)

```

However, the `deploy()` method call just keeps executing and never finishes. I’m not sure why this is happening. Any advice would be greatly appreciated.

Thanks!

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/52583_2.png) [@mahmutc](https://discuss.huggingface.co/u/mahmutc)\
**Post date:** [September 27, 2024, 8:59am UTC](https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794/2 "2024-09-27T08:59:56Z")

</div>

hi @bhashwarsengupta  
I’m not familiar with AWS Sagemaker but let me share this. I hope checking `/aws/sagemaker/Endpoints/<your_endpoint_name>` gives more details.

> <https://github.com/aws/amazon-sagemaker-examples/issues/210#issuecomment-371549251>
>
> I am trying to deploy a BYOB (bring your own model) keras model. I pushed the im…age to ECR with the 'latest' tag. All local testing passed, and I am able to successfully train the model e.g.:
> 
> \`\`\`python
> 
> image = '{}.dkr.ecr.{}.amazonaws.com/my-model:latest'.format(account, region)
> 
> dl = sage.estimator.Estimator(image,
> role, 1, 'ml.c4.2xlarge',
> output\_path="s3://{}/output".format(sess.default\_bucket()),
> sagemaker\_session=sess)
> 
> \`\`\`
> 
> However attempting to deploy gives me the error:
> 
> \`\`\`
> Failed Reason: The primary container for production variant AllTraffic did not pass the ping health check.
> \`\`\`
> I am not quite sure where this stems from given local health check passed. Any insight would be great! Thanks.

---

<div class="post-metadata">

**Author:** ![bhashwarsengupta](https://avatars.discourse-cdn.com/v4/letter/b/74df32/32.png) [@bhashwarsengupta](https://discuss.huggingface.co/u/bhashwarsengupta)\
**Post date:** [September 27, 2024, 9:20am UTC](https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794/3 "2024-09-27T09:20:27Z")

</div>

Thanks @mahmutc for sharing. I actually had figured out the problem. The `deploy()` method call was trying to find a file named `.sagemaker-code-config` throughout my filesystem. All I did was to create the same file in the cwd and it was resolved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [September 27, 2024, 9:20pm UTC](https://discuss.huggingface.co/t/need-help-deploying-a-hf-model-to-aws-sagemaker/108794/4 "2024-09-27T21:20:54Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.