# Model\_fn and predict\_fn called multiple times?

**URL:** <https://discuss.huggingface.co/t/model-fn-and-predict-fn-called-multiple-times/32117>\
**Category:** Amazon SageMaker\
**Created:** [February 20, 2023, 8:40pm UTC](https://discuss.huggingface.co/t/model-fn-and-predict-fn-called-multiple-times/32117 "2023-02-20T20:40:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abesadi](https://avatars.discourse-cdn.com/v4/letter/a/c5a1d2/32.png) [@Abesadi](https://discuss.huggingface.co/u/Abesadi)\
**Post date:** [February 20, 2023, 8:40pm UTC](https://discuss.huggingface.co/t/model-fn-and-predict-fn-called-multiple-times/32117/1 "2023-02-20T20:40:26Z")

</div>

I am using the prebuilt SageMaker Hugging face container  
In the entry\_point,I include a script which carries out the batch transformation job.

‘’’  
def model\_fn():  
…

def input\_fn():  
…

def predict\_fn():

‘’’

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/1/15f41daeef5f11d970b1e68777bdc0a72fcf681d.png)

---

<div class="post-metadata">

**Author:** ![wichofer](https://avatars.discourse-cdn.com/v4/letter/w/d26b3c/32.png) [@wichofer](https://discuss.huggingface.co/u/wichofer)\
**Post date:** [October 11, 2024, 5:48am UTC](https://discuss.huggingface.co/t/model-fn-and-predict-fn-called-multiple-times/32117/2 "2024-10-11T05:48:47Z")

</div>

Did you find a solution? I am having the same issue (model\_fn called multiple times)

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [October 11, 2024, 8:16am UTC](https://discuss.huggingface.co/t/model-fn-and-predict-fn-called-multiple-times/32117/3 "2024-10-11T08:16:17Z")

</div>

Perhaps unresolved issue since 2020…?

> <https://github.com/aws/amazon-sagemaker-examples/issues/1073>
>
> I am using the prebuilt SageMaker SKLearn container (https://github.com/aws/sage…maker-scikit-learn-container) version 0.20.0.
> In the entry\_point,I include a script which carries out the batch transformation job.
> \`\`\`py
> def model\_fn():
> ...    
> 
> def input\_fn():
> ...
> 
> def predict\_fn():
> '''
> A long running process to preprocess the data before calling model
> https://aws.amazon.com/blogs/machine-learning/preprocess-input-data-before-making-predictions-using-amazon-sagemaker-inference-pipelines-and-scikit-learn/
> '''
> time.sleep(60 \* 11) # sleep for 11 mins to simulate a long running process
> ....
> 
> def output\_fn():
> ....
> \`\`\`
> 
> I noticed that the model\_fn() was called multiple times in the cloudwatch log
> \`\`\`
> 21:11:43 model\_fn called /opt/ml/model 0.3710819465747405
> 21:11:43 model\_fn called /opt/ml/model 0.1368146211634631
> 21:11:44 model\_fn called /opt/ml/model 0.09153953459183728
> \`\`\`
> 
> The input\_fn() was also called multiple times
> \`\`\`
> 20:41:31 input\_data \<class 'str'\> application/json 0.3936440317990033 {
> 20:51:30 input\_data \<class 'str'\> application/json 0.4852180186010707 {
> 21:01:30 input\_data \<class 'str'\> application/json 0.9954036507047136 {
> 21:11:30 input\_data \<class 'str'\> application/json 0.0806271844985188 {
> \`\`\`
> Precisely, it's called every 10 minutes.
> 
> 
> I used ml.m4.xlarge, BatchStrategy = SingleRecord and SplitType of None. I also used the environmental variable SAGEMAKER\_MODEL\_SERVER\_TIMEOUT = '9999' to overcome the 60s timeout. I expected that the model\_fn or input\_fn would only be called once, but in this case, they were called multiple times. In the end, the container crashed with "Internal Server Error".
> 
> I saw a similar related issue before https://github.com/awslabs/amazon-sagemaker-examples/issues/341 where the model\_fn was called on each invocation. But in this case, there is no /invocations, the model\_fn, input\_fn, predict\_fn, and output\_fn were called multiple time. In the end, the container \*crashed\* with Internal Server Error.