# Endpoint Deployment

**URL:** <https://discuss.huggingface.co/t/endpoint-deployment/10077>\
**Category:** Amazon SageMaker\
**Created:** [September 17, 2021, 11:35am UTC](https://discuss.huggingface.co/t/endpoint-deployment/10077 "2021-09-17T11:35:26Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![marlon89](https://avatars.discourse-cdn.com/v4/letter/m/6de8d8/32.png) [@marlon89](https://discuss.huggingface.co/u/marlon89)\
**Post date:** [September 17, 2021, 11:35am UTC](https://discuss.huggingface.co/t/endpoint-deployment/10077/1 "2021-09-17T11:35:26Z")

</div>

Hello everyone,

I deployed my BERT classification model for batch jobs on Sagemaker with

```auto
create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data=model_uri, # configuration for loading model from Hub
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version='py36', # python version used
)

# create Transformer to run our batch job
batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.g4dn.xlarge',
    output_path=output_s3_path, # we are using the same s3 path to save the output with the input
    strategy='SingleRecord')

# starts batch transform job and uses s3 data as input
batch_job.transform(
    data=s3_file_uri,
    content_type='application/json',    
    split_type='Line')

utput_file = f"{dataset_jsonl_file}.out"
output_path = s3_path_join(output_s3_path,output_file)

# download file
S3Downloader.download(output_path,'.')

batch_transform_result = []
with open(output_file) as f:
    for line in f:
        # converts jsonline array to normal array
        line = "[" + line.replace("[","").replace("]",",") + "]"
        batch_transform_result = literal_eval(line) 

```

Anyways, when I want to predict a new batch it feels like I always have to start my Notebook on Sagemaker Studio etc. Is there a way to create an API which I can feed with my data to predict from the outside? Is there any cool tutorial or anything? Thanks in advance

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [September 20, 2021, 7:13am UTC](https://discuss.huggingface.co/t/endpoint-deployment/10077/2 "2021-09-20T07:13:03Z")

</div>

Hey @marlon89,

I sadly don’t have yet a tutorial or a sample for it. I am looking into creating something like that in the next weeks, with `cdk` support.  
I created an architecture of how this API can look like.

 ![SageMaker-batch-cycle.drawio](https://us1.discourse-cdn.com/hellohellohello/original/2X/6/62ac9922023f50737fa3f531e28e476aef80ea46.png)  
You can basically leverage AWS Lambda with S3 Trigger to start an create you batch transform jobs after a file Is uploaded to a prefix on an s3 bucket.