# Hitting Deployed Endpoint \*Outside\* of Notebook **URL:** \ **Category:** Amazon SageMaker\ **Created:** [October 22, 2021, 9:35pm UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006 "2021-10-22T21:35:49Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![rosenjcb](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@rosenjcb](https://discuss.huggingface.co/u/rosenjcb)\ **Post date:** [October 22, 2021, 9:35pm UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006/1 "2021-10-22T21:35:49Z")
All the tutorials tend to end at: ```auto predictor.predict({"input": "YOUR_TEXT_GOES_HERE"}) ``` It’s great that the notebooks deliver you to inference, but I have no idea how to hit this endpoint outside of the context of a Jupyter Notebook. I basically have Amazon AWS Java sdk code that does this: ```java AmazonSageMakerRuntime runtime = AmazonSageMakerRuntimeClientBuilder.defaultClient(); String body = "{\"instances\": [{\"data\": { \"input\": \"Hello World\"}}]}"; ByteBuffer bodyBuffer = ByteBuffer.wrap(body.getBytes()); InvokeEndpointRequest request = new InvokeEndpointRequest() .withEndpointName("huggingface-pytorch-training-....") .withBody(bodyBuffer); InvokeEndpointResult invokeEndpointResult = runtime.invokeEndpoint(request); ``` Unfortunately, I get an error: ```auto { "code": 400, "type": "InternalServerException", "message": "Content type is not supported by this framework.\n\n Please implement input_fn to to deserialize the request data or an output_fn to\n serialize the response. For more information, see the SageMaker Python SDK README." } ``` Am I missing something? ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 25, 2021, 7:57am UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006/3 "2021-10-25T07:57:41Z")
Hey @rosenjcb, Thank you for opening this thread. Yes you can use the endpoint with the aws sdk for this you can use the [InvokeEndpoint](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html) method. [Java doc](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/sagemakerruntime/SageMakerRuntimeClient.html#invokeEndpoint--) It looks like you are already doing this and there are only a few missing parts i guess. The Endpoint expects a `JSON` as HTTP Body and as the error says you are missing the `Content-Type: application/json` for that. I have to say i have no JAVA experience at all but i found this on StackOverflow: ```java InvokeEndpointRequest invokeEndpointRequest = new InvokeEndpointRequest(); invokeEndpointRequest.setContentType("application/x-image"); ByteBuffer buf = ByteBuffer.wrap(image); invokeEndpointRequest.setBody(buf); invokeEndpointRequest.setEndpointName(endpointName); invokeEndpointRequest.setAccept("application/json"); AmazonSageMakerRuntime amazonSageMaker = AmazonSageMakerRuntimeClientBuilder.defaultClient(); InvokeEndpointResult invokeEndpointResult = amazonSageMaker.invokeEndpoint(invokeEndpointRequest) ``` maybe this helps you crafting your request. You can also find a example on using the aws sdk with python below ```python response = client.invoke_endpoint( EndpointName=ENDPOINT_NAME, ContentType="application/json", Accept="application/json", Body=JSON_STRING, ) ``` ---
**Author:** ![rosenjcb](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@rosenjcb](https://discuss.huggingface.co/u/rosenjcb)\ **Post date:** [October 28, 2021, 5:04pm UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006/4 "2021-10-28T17:04:09Z")
@philschmid I found your example and many others after an hour or so of digging. Once you get the model into SageMaker, the inferrence instructions are pretty easy to google (you don’t even need to mention HuggingFace anymore since it’s abstracted from the SageMaker platform. As you mentioned, I needed to set the `Content-Type` of the request to `application/json` and I also needed to correct my query string to this: `{"inputs": "Hello World"}`. That’s it, no need to plaster on the `instances` or `data` structures - if you’re only asking for one query you can just pass the request along as you normally would when using the HuggingFace inference API. Thanks for the help! The way this integrates into AWS without too much hassle is super important and will encourage NLP adoption across many teams - no doubt. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 29, 2021, 6:21am UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006/5 "2021-10-29T06:21:39Z")
Hello @rosenjcb, Great to hear that you could solve it! And yes the API Contract is similar to the Hugging Face API. You can find more information in the documentation [Reference](https://huggingface.co/docs/sagemaker/reference#inference-toolkit-api). ---
**Author:** ![sow3110](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@sow3110](https://discuss.huggingface.co/u/sow3110)\ **Post date:** [July 4, 2023, 1:09pm UTC](https://discuss.huggingface.co/t/hitting-deployed-endpoint-outside-of-notebook/11006/6 "2023-07-04T13:09:01Z")
Hello, Found this thread when searching for the same issue. I have deployed the sentence-transformers/miniLM model as a sagemaker endpoint. [sentence-transformers/all-MiniLM-L6-v2 · Hugging Face](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) The predictor.predict method works for me. However when using client.invoke\_endpoint from another notebook, I get an error when I pass json asking me to pass bytes. When I pass bytes I get a model error as well. Any idea?