# Create a batch transform job with custom trained biobert model

**URL:** <https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107>\
**Category:** Amazon SageMaker\
**Created:** [January 25, 2022, 6:28pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107 "2022-01-25T18:28:10Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 25, 2022, 6:28pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/1 "2022-01-25T18:28:10Z")

</div>

Hi Team,

We have trained a biobert model on custom data using pytorch framework outside of sagemaker. we want to bring this model to sagemkaer to run the batch transform job on it.

Is there any different way we can try or any suggestion you have.

Thanks,  
Akash

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [January 25, 2022, 7:25pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/2 "2022-01-25T19:25:09Z")

</div>

Hello @akash97715,

can the model be loaded with `.from_pretrained`? If so I see no problem for batch transform. It then just depends if your model is stored on S3 or [Models - Hugging Face](http://hf.co/models).

You can check-out this example: [notebooks/sagemaker-notebook.ipynb at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/master/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb)

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 26, 2022, 4:18pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/3 "2022-01-26T16:18:49Z")

</div>

yes we are loading the model from pretrained. Shall i load my custom trained model to S3 and i can use this?

Also the extension of the saved model is “model.pt”. do i need to covert this extension or directly i can use this for my batch transform jobs?

Thanks,  
Akash

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/8/833d9f5e8080c25e649ded0a8eb375fc9f1ff773.png)

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 26, 2022, 4:20pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/4 "2022-01-26T16:20:33Z")

</div>

Also PFA the code that we are using to saving the model in our local environment. We are planning to take this saved model and perform batch transform job on it.

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/e/e1a700b5a4fb739c018b04fb8759d1085b3297b9.png)

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [January 26, 2022, 5:08pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/5 "2022-01-26T17:08:37Z")

</div>

Hi Akash - in order to use your model on Sagemaker you will have to create a model package called `model.tar.gz` that includes all the required model files. You can find all the info here: [Deploy models to Amazon SageMaker](https://huggingface.co/docs/sagemaker/inference#create-a-model-artifact-for-deployment)

Hope that helps!

Cheers  
Heiko

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 26, 2022, 6:48pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/6 "2022-01-26T18:48:13Z")

</div>

Hi Heiko,

Thank you for your response. When you say creating tar.gz file i understand your point.

Aa i mentioned I’m saving model in .pt extension as we are using pytorch framework.

Can you help me how to create a tar.gz file as I have saved model in .pt extension(refer to my code snippet). Does anybody done this earlier or do we have a sample notebook which explains how to convert .pt extension to .tar.gz file?

Or do i need to follow other approch while saving the model, i mean I need to save my pytorch model using other approach.

Thanks,  
Akash

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 26, 2022, 6:52pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/7 "2022-01-26T18:52:30Z")

</div>

Also one thing to keep in mind, i have a usecase where I need to use batch transform not any endpoint approach because we have multiple models for multiple hospital products and large data volume and creating multiple endpoints is not a feasible solution

---

<div class="post-metadata">

**Author:** ![marshmellow77](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marshmellow77/32/3279_2.png) [@marshmellow77](https://discuss.huggingface.co/u/marshmellow77)\
**Post date:** [January 26, 2022, 11:44pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/8 "2022-01-26T23:44:51Z")

</div>

Hi Akash, there is no need for a different approach when you save your model. Once you have saved your model, create the `model.tar.gz` file with the following command: `tar zcvf model.tar.gz * `, just like described in the documentation I linked to earlier.

I looked for example notebooks and found this one: [midas\_depth\_estimation.ipynb · GitHub](https://gist.github.com/harusametime/cb597d84459f5f8e8ab68935549d3832)

The markdowns are in Japanese (I think) but the code should be helpful for you to get an idea how to go about it.

Hope that helps.

Cheers  
Heiko

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [January 28, 2022, 8:08am UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/9 "2022-01-28T08:08:11Z")

</div>

Hi,  
To answer your question we are not loading our custom trained biobert model from ".from\_pretrained. we are using torch.load() to load the model, as i mentioned i am saving this model in .pt extension. The process we are following below:

we are downloading the pretrained model called “microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract” from model hub, we are applying transfer learning to it with our custom hospital data and then we are saving the model. Now we want to use this saved model to generate the classification labels on sagemaker.

Can you please guide me the way of saving model so that it can be loaded from “.from\_pretrained”

Thanks,  
Akash

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [January 28, 2022, 10:42am UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/10 "2022-01-28T10:42:21Z")

</div>

@akash97715 the easiest way to use the HuggingFace DLCs for batch transform job with zero-code configuration is by saving your model using `transformers`.  
Meaning to would need to replace the `torch.save` in your training script with.

```python
model.save_pretrained("my_dir")
tokenizer.save_pretrained("my_dir")

```

After that, you can create a compatible `model.tar.gz`

1. Create a `tar` file:

```auto
cd {my_dir}
tar zcvf model.tar.gz *

```

1. Upload `model.tar.gz` to S3:

```auto
aws s3 cp model.tar.gz <s3://{my-s3-path}>

```

Now you can provide the S3 URI to the `model_data` argument when creating your batch transform job.

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [February 19, 2022, 4:09pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/11 "2022-02-19T16:09:17Z")

</div>

Thanks a lot for guiding us the right approach. I was able to implement it in my project 🙂

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [February 19, 2022, 4:16pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/12 "2022-02-19T16:16:02Z")

</div>

Thank you for your precious suggestions. I was able to implement the inference pipeline in my project.

Moving to next question:  
My task is to build retraining pipeling using sagemaker, that means i want to bring my model training code on sagemaker and try to train the model on the training data stored in S3 bucket. Also once the training is completed i want to save my transfer learning trained model back to s3 bucket and plan is to use that model for production prediction. We are using step function on AWS to build the orchestrator pipeline.

My question is how we can build a retraining pipeline using Pytorch estimator on sagemker. If you point me to some implementation github page link, it will be great

Thank you,  
Akash

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [February 21, 2022, 3:54pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/13 "2022-02-21T15:54:40Z")

</div>

Hello @akash97715,

last oct-nov we did a whole workshop series on [“Enterprise-Scale NLP with Hugging Face & Amazon SageMaker”](https://github.com/philschmid/huggingface-sagemaker-workshop-series). With part 1 training models using Amazon SageMaker, Part 2 Scaling Inference and Part 3 MLOPs using SageMaker pipelines.

All videos and resources are available on-demand at: [GitHub - philschmid/huggingface-sagemaker-workshop-series: Enterprise Scale NLP with Hugging Face & SageMaker Workshop series](https://github.com/philschmid/huggingface-sagemaker-workshop-series)

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [February 22, 2022, 12:33pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/14 "2022-02-22T12:33:27Z")

</div>

Thank you for providing the link.  
I wanna understand how to convert normal dataset to “datasets.arrow\_dataset.Dataset” format. I see you are loading imdb dataset from directly from load\_dataset function from dataset[s3] module.

I have my csv file with 3 columns 1)unique id 2)description column(ML input) 3) Labels.  
I want to convert this dataframe to the “datasets.arrow\_dataset.Dataset” format.

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/6/6839ef10bd4a77115ce3bbf3fb32b06946338217.png)  
 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/e/e78f1ff0cc969de7d1fefcbed76066115c417986.png)  
 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/9/9fc0ae0e17eb22b594d676d77269e3d4be866959.png)

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [February 22, 2022, 12:50pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/15 "2022-02-22T12:50:46Z")

</div>

@akash97715 we have excellent documentation for the `datasets` library, which also includes how to load `csv` datasets. Please take a look there. [Load — datasets 1.18.3 documentation](https://huggingface.co/docs/datasets/loading.html)

I am always happy to help when you run into issues, but reading documentation has never hurt anyone.

```python
>>> data_files = {"train": "train.csv", "test": "test.csv"}
>>> dataset = load_dataset("namespace/your_dataset_name", data_files=data_files)

```

---

<div class="post-metadata">

**Author:** ![akash97715](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/akash97715/32/3852_2.png) [@akash97715](https://discuss.huggingface.co/u/akash97715)\
**Post date:** [February 22, 2022, 6:52pm UTC](https://discuss.huggingface.co/t/create-a-batch-transform-job-with-custom-trained-biobert-model/14107/16 "2022-02-22T18:52:19Z")

</div>

Thank you for your response and really sorry for this silly question. I will definitely have look into the documentation from now. I looked when u provided the link. It was really awesome documentation and also this has clarified my further doubts as well, like how directly i can load from s3 bucket. 🙂

Thank you once again