# Async TEI Deployment Cannot Handle Requests greater than 2mb

**URL:** <https://discuss.huggingface.co/t/async-tei-deployment-cannot-handle-requests-greater-than-2mb/107529>\
**Category:** Amazon SageMaker\
**Created:** [September 20, 2024, 8:45pm UTC](https://discuss.huggingface.co/t/async-tei-deployment-cannot-handle-requests-greater-than-2mb/107529 "2024-09-20T20:45:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mavjan](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mavjan](https://discuss.huggingface.co/u/mavjan)\
**Post date:** [September 20, 2024, 8:45pm UTC](https://discuss.huggingface.co/t/async-tei-deployment-cannot-handle-requests-greater-than-2mb/107529/1 "2024-09-20T20:45:51Z")

</div>

We are sending in this config:

```auto
    'HF_MODEL_ID':'BAAI/bge-m3',
    "DTYPE": "float16",
    "MAX_BATCH_TOKENS": "163840000",
    "MAX_CLIENT_BATCH_SIZE": "320000",
    "PAYLOAD_LIMIT": "200000000",
    "MAX_BATCH_REQUESTS": "30",
    'MMS_MAX_REQUEST_SIZE': '2000000000',
    'MMS_MAX_RESPONSE_SIZE': '2000000000',
    'MMS_DEFAULT_RESPONSE_TIMEOUT': '900',

```

However, our endpoint fails on async inferences with files greater than 2mb despite setting the payload higher and using async inference.

```auto
Received client error (413) from primary with message "Failed to buffer the request body: length limit exceeded"

```

---

<div class="post-metadata">

**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\
**Post date:** [November 4, 2024, 7:20am UTC](https://discuss.huggingface.co/t/async-tei-deployment-cannot-handle-requests-greater-than-2mb/107529/2 "2024-11-04T07:20:45Z")

</div>

Quick update an i digged a bit. You can increase the size by setting the `PAYLOAD_LIMIT` env when creating a TEI endpoint, default is 2000000 =\> 2MB

---

<div class="post-metadata">

**Author:** ![mavjan](https://avatars.discourse-cdn.com/v4/letter/m/a698b9/32.png) [@mavjan](https://discuss.huggingface.co/u/mavjan)\
**Post date:** [November 4, 2024, 7:47am UTC](https://discuss.huggingface.co/t/async-tei-deployment-cannot-handle-requests-greater-than-2mb/107529/3 "2024-11-04T07:47:25Z")

</div>

> [@mavjan](#):
>
> `200000000`

Unfortunately, we did try this setting the PAYLOAD\_LIMIT to 200000000 =\> 200 mb but it did not seem to resolve the errors