# Modelerror when deploying openchat3.5

**URL:** <https://discuss.huggingface.co/t/modelerror-when-deploying-openchat3-5/79859>\
**Category:** Amazon SageMaker\
**Created:** [April 2, 2024, 5:54pm UTC](https://discuss.huggingface.co/t/modelerror-when-deploying-openchat3-5/79859 "2024-04-02T17:54:58Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Faiz4work](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/faiz4work/32/25389_2.png) [@Faiz4work](https://discuss.huggingface.co/u/Faiz4work)\
**Post date:** [April 2, 2024, 5:54pm UTC](https://discuss.huggingface.co/t/modelerror-when-deploying-openchat3-5/79859/1 "2024-04-02T17:54:58Z")

</div>

Hi all,  
I am deploying [openchat3.5](https://huggingface.co/openchat/openchat-3.5-0106) model on sagemaker. first I uploaded all the files into the S3 storage and manged access and deployed the model successfully. I am getting error when trying to ask questions with it. Anybody knows how to use “predict” or “invoke\_endpoint” with my given model?  
below is the code which describes the functionality and error:

hub2 = {  
‘HF\_TASK’: ‘text-generation’,  
}  
model\_path = “s3://penchatbotmodel/model.tar.gz”  
huggingface\_model2 = HuggingFaceModel(  
role=role,  
env=hub2,  
py\_version=‘py36’,  
transformers\_version=‘4.6.1’,  
pytorch\_version=‘1.7.1’,  
model\_data=model\_path,  
)  
predictor = huggingface\_model2.deploy(  
initial\_instance\_count=1,  
instance\_type=“ml.g5.2xlarge”,  
endpoint\_name=“ChatBotPoint2”,  
)  
prompt=“”“\<|prompter|\>How can i stay more active during winter? Give me 3 tips.\<|endoftext|\>\<|assistant|\>”“”

# hyperparameters for llm

payload = {  
“inputs”: prompt,  
“parameters”: {  
“do\_sample”: True,  
“top\_p”: 0.7,  
“temperature”: 0.7,  
“top\_k”: 50,  
“max\_new\_tokens”: 256,  
# “repetition\_penalty”: 1.03,  
# “stop”: [“\<|endoftext|\>”]  
}  
}  
predictor.predict(payload )

The Error:  
[ModelError :](ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{  
“code”: 400,  
“type”: “InternalServerException”,  
“message”: “\u0027mistral\u0027”  
})