# Facebook/bart-large-mnli inference when deployed on SageMaker

**URL:** <https://discuss.huggingface.co/t/facebook-bart-large-mnli-inference-when-deployed-on-sagemaker/17320>\
**Category:** Amazon SageMaker\
**Created:** [April 29, 2022, 5:12am UTC](https://discuss.huggingface.co/t/facebook-bart-large-mnli-inference-when-deployed-on-sagemaker/17320 "2022-04-29T05:12:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![striki-ai](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/striki-ai/32/4624_2.png) [@striki-ai](https://discuss.huggingface.co/u/striki-ai)\
**Post date:** [April 29, 2022, 5:12am UTC](https://discuss.huggingface.co/t/facebook-bart-large-mnli-inference-when-deployed-on-sagemaker/17320/1 "2022-04-29T05:12:08Z")

</div>

Hi,

Any idea/documentation on how to compose json payload for inferencing facebook/bart-large-mnli model, deployed on SageMaker, used the code as provided on the "Deploy → Amazon SageMaker → AWS:

from sagemaker.huggingface import HuggingFaceModel  
import sagemaker

role = sagemaker.get\_execution\_role()

# Hub Model configuration. [Models – Hugging Face](https://huggingface.co/models)

hub = {  
‘HF\_MODEL\_ID’:‘facebook/bart-large-mnli’,  
‘HF\_TASK’:‘zero-shot-classification’  
}

# create Hugging Face Model Class

huggingface\_model = HuggingFaceModel(  
transformers\_version=‘4.17.0’,  
pytorch\_version=‘1.10.2’,  
py\_version=‘py38’,  
env=hub,  
role=role,  
)

# deploy model to SageMaker Inference

predictor = huggingface\_model.deploy(  
initial\_instance\_count=1, # number of instances  
instance\_type=‘ml.m5.xlarge’ # ec2 instance type  
)

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [April 29, 2022, 6:35am UTC](https://discuss.huggingface.co/t/facebook-bart-large-mnli-inference-when-deployed-on-sagemaker/17320/2 "2022-04-29T06:35:21Z")

</div>

Hello @striki-ai,

This documentation link should help you: [Reference](https://huggingface.co/docs/sagemaker/reference#inference-toolkit-api)

example

```python
predictor.predict({
  "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!",
  "parameters": {
    "candidate_labels": ["refund", "legal", "faq"]
  }
})

```