# How can I create lambda function code after deploying bart transformer for summarization in sagemaker? **URL:** \ **Category:** Amazon SageMaker\ **Created:** [September 25, 2021, 7:58am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295 "2021-09-25T07:58:00Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [September 25, 2021, 7:58am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/1 "2021-09-25T07:58:00Z")
I deployed bart-large-cnn model in aws sagemaker for summarization task by following the code. > from sagemaker.huggingface import HuggingFaceModel > import sagemaker > > role = sagemaker.get\_execution\_role() > > # Hub Model configuration. [Models – Hugging Face](https://huggingface.co/models) > > hub = { > ‘HF\_MODEL\_ID’:‘facebook/bart-large-cnn’, # model\_id from [Models – Hugging Face](http://hf.co/models) > ‘HF\_TASK’:‘summarization’ # NLP task you want to use for predictions > } > > # create Hugging Face Model Class > > huggingface\_model = HuggingFaceModel( > env=hub, > role=role, # iam role with permissions to create an Endpoint > transformers\_version=“4.6”, # transformers version used > pytorch\_version=“1.7”, # pytorch version used > py\_version=“py36”, # python version of the DLC > ) # deploy model to SageMaker Inference predictor = huggingface\_model.deploy( initial\_instance\_count=1, instance\_type=“ml.m5.xlarge” ) Then tested with text to extract summaries and it performed well. What code i needed to add in lambda function, for creating a rest api to call the summariZation model endpoint using aws lambda and api gateway. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 27, 2021, 7:43am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/2 "2021-09-27T07:43:41Z")
Hello @vihaary, Create to hear it worked as expected. Invoking your endpoint from an AWS Lambda function is pretty easy you can either use the `boto3` `sagemaker-runtime` or install the `sagemaker` sdk into your lambda function. `boto3` snippet: ```python import boto3 import json client = boto3.client('sagemaker-runtime') response = client.invoke_endpoint( EndpointName=ENDPOINT_NAME, ContentType="application/json", Accept="application/json", Body=json.dumps(payload), ) print(response['Body'].read().decode()) ``` `sagemaker` snippet ```python from sagemaker.huggingface import HuggingFacePredictor predictor = HuggingFacePredictor(ENDPOINT_NAME) response = predictor.predict(payload) print(response) ``` ---
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [September 27, 2021, 7:54am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/3 "2021-09-27T07:54:04Z")
Hello @philschmid , thanks for your response. Will I get summary as response output for text paragraphs as input with this lambda code?. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 27, 2021, 12:45pm UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/4 "2021-09-27T12:45:41Z")
No that is just how you can invoke your endpoint in your AWS Lambda function. ---
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [September 28, 2021, 4:26am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/5 "2021-09-28T04:26:09Z")
@philschmid { “errorMessage”: “An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from model with message “{\n “code”: 400,\n “type”: “InternalServerException”,\n “message”: “\u0027numpy.ndarray\u0027 object has no attribute \u0027pop\u0027”\n}\n”. See [https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/huggingface-pytorch-inference-2021-09-27-12-40-12-385](https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/sagemaker/Endpoints/huggingface-pytorch-inference-2021-09-27-12-40-12-385) in account 215401044306 for more information.”, “errorType”: “ModelError”, “stackTrace”: [ " File “/var/task/lambda\_function.py”, line 20, in lambda\_handler\n Body=payload)\n", " File “/var/runtime/botocore/client.py”, line 386, in \_api\_call\n return self.\_make\_api\_call(operation\_name, kwargs)\n", " File “/var/runtime/botocore/client.py”, line 705, in \_make\_api\_call\n raise error\_class(parsed\_response, operation\_name)\n" ] } I am getting this error. Also can i get summary as output for text paragraphs as input after invoking my endpoint in aws lambda function? ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 28, 2021, 7:06am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/6 "2021-09-28T07:06:10Z")
> [@vihaary](#): > > “\u0027numpy.ndarray\u0027 object has no attribute \u0027pop\u0027”\n}\n” You are not sending a `json` input to the endpoint. > [@vihaary](#): > > I am getting this error. Also can i get summary as output for text paragraphs as input after invoking my endpoint in aws lambda function? Yes ---
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [September 28, 2021, 8:02am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/7 "2021-09-28T08:02:57Z")
@philschmid Thanks for the reply. I tried this one import os import io import boto3 import json import csv # grab environment variables ENDPOINT\_NAME = os.environ[‘ENDPOINT\_NAME’] runtime= boto3.client(‘runtime.sagemaker’) def lambda\_handler(event, context): print("Received event: " + json.dumps(event, indent=2)) ``` data = json.loads(json.dumps(event)) payload = data['data'] print(payload) response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME, ContentType='text/csv', Body=payload) print(response) result = json.loads(response['Body'].read().decode()) print(result) ``` But i am not getting the summary output. Can you provide a sample code for lambda function where i can give paragraph as input and i should get summary as output for my huggingface bart model endpoint. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [September 28, 2021, 11:25am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/8 "2021-09-28T11:25:12Z")
Your `ContentType` is wrong if you compare it to what I have shared. I am going to create a sample for this but not before next week. ---
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [September 30, 2021, 4:10am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/9 "2021-09-30T04:10:07Z")
Thanks @philschmid . Will be waiting for your response ---
**Author:** ![vihaary](https://avatars.discourse-cdn.com/v4/letter/v/f05b48/32.png) [@vihaary](https://discuss.huggingface.co/u/vihaary)\ **Post date:** [October 5, 2021, 5:01am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/10 "2021-10-05T05:01:16Z")
Hi @philschmid . Can you try to create a lambda function sample for the above bart model summarization task?. Thanks in advance. ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [October 5, 2021, 6:52am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/11 "2021-10-05T06:52:32Z")
Hey, I’ll post about it later on Social Media (Twitter, LinkedIn) but here is the example: [cdk-samples/aws-lambda-sagemaker-endpoint-huggingface at master · philschmid/cdk-samples · GitHub](https://github.com/philschmid/cdk-samples/tree/master/aws-lambda-sagemaker-endpoint-huggingface) It is an example using the CDK so if you want to deploy a BART model for summarization you can follow the readme. clone the repository ```bash git clone https://github.com/philschmid/cdk-samples.git cd aws-lambda-sagemaker-endpoint-huggingface ``` Install the cdk required dependencies. Make your you have the [cdk](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html#getting_started_install) installed. ```bash pip3 install -r requirements.txt ``` [Bootstrap](https://docs.aws.amazon.com/cdk/latest/guide/bootstrapping.html) your application in the cloud. ```bash cdk bootstrap ``` Deploy your Hugging Face Transformer model to Amazon SageMaker ```bash cdk deploy \ -c model="facebook/bart-large-cnn" \ -c task="summarization" ``` clean up ```bash cdk destroy \ -c model="facebook/bart-large-cnn" \ -c task="summarization" ``` If you only want to know/see the lambda code you can look here [cdk-samples/handler.py at master · philschmid/cdk-samples · GitHub](https://github.com/philschmid/cdk-samples/blob/master/aws-lambda-sagemaker-endpoint-huggingface/lambda_src/handler.py) ---
**Author:** ![Suva](https://avatars.discourse-cdn.com/v4/letter/s/c67d28/32.png) [@Suva](https://discuss.huggingface.co/u/Suva)\ **Post date:** [January 21, 2022, 6:25am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/12 "2022-01-21T06:25:17Z")
Hi @vihaary, Do you able to solve that problem ? If yes then please share the code with me . I need that code as I am working on a project regarding that things… Thanks ---
**Author:** ![philschmid](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/philschmid/32/38022_2.png) [@philschmid](https://discuss.huggingface.co/u/philschmid)\ **Post date:** [January 21, 2022, 7:36am UTC](https://discuss.huggingface.co/t/how-can-i-create-lambda-function-code-after-deploying-bart-transformer-for-summarization-in-sagemaker/10295/13 "2022-01-21T07:36:07Z")
Yes, as mentioned above I created an example on how to do do this using AWS CDK. > [@philschmid](#): > > If you only want to know/see the lambda code you can look here [cdk-samples/handler.py at master · philschmid/cdk-samples · GitHub](https://github.com/philschmid/cdk-samples/blob/master/aws-lambda-sagemaker-endpoint-huggingface/lambda_src/handler.py)