# Cannot import "Conversation" from transformers\_utils.py

**URL:** <https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556>\
**Category:** Amazon SageMaker\
**Created:** [June 11, 2024, 1:58pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556 "2024-06-11T13:58:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vocabua](https://avatars.discourse-cdn.com/v4/letter/v/dc4da7/32.png) [@Vocabua](https://discuss.huggingface.co/u/Vocabua)\
**Post date:** [June 11, 2024, 1:58pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/1 "2024-06-11T13:58:41Z")

</div>

I am deploying a Huggingface model with Sagemaker. This is the code that I use to deploy:

```auto
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel

sess = sagemaker.Session()
role = sagemaker.get_execution_role()
client = boto3.client('sagemaker')

def new_endpoint(name, model_path, instance):
  # create HuggingFaceModel
  model = HuggingFaceModel(
    model_data=model_path+"output/model.tar.gz",
    role=role,
    transformers_version='4.28',
    pytorch_version='2.0',
    py_version='py310',
  )
  
  # Deploy model to an endpoint
  deployed = model.deploy(
    initial_instance_count=1,
    instance_type=instance,
    endpoint_name=name,
  )

```

This has previously worked without issue, but in the last few days, the model no longer successfully deploys, even though I have not modified the code. Looking in the CloudWatch logs, I see that this specific line causes the error:

```auto
2024-06-11T13:30:54,406 [INFO] W-9000-model-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - from sagemaker_huggingface_inference_toolkit.transformers_utils import (

```

…

```auto
2024-06-11T13:30:54,406 [INFO] W-9000-model-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - from transformers.pipelines import Conversation, Pipeline

```

I have looked at the source code of the transformer\_utils.py file, and the line that causes this error can be seen:

```auto
from huggingface_hub import HfApi, login, snapshot_download
from transformers import AutoTokenizer, pipeline
from transformers.file_utils import is_tf_available, is_torch_available
from transformers.pipelines import Conversation, Pipeline

```

I am unable to resolve this issue. Testing it locally, this line does not work for me either. What surprises me the most is that this error has occurred spontaneously, without the sagemaker-huggingface-inference-toolkit or transformers libraries having been updated.  
Any help would be appreciated, thanks.

---

<div class="post-metadata">

**Author:** ![Vocabua](https://avatars.discourse-cdn.com/v4/letter/v/dc4da7/32.png) [@Vocabua](https://discuss.huggingface.co/u/Vocabua)\
**Post date:** [June 11, 2024, 2:19pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/2 "2024-06-11T14:19:19Z")

</div>

I should add that while experimenting, I have found this line to work:

```auto
from transformers.pipelines.conversational import Conversation

```

However, the line that is in the source code does not work:

```auto
from transformers.pipelines import Conversation, Pipeline

```

---

<div class="post-metadata">

**Author:** ![wizardh](https://avatars.discourse-cdn.com/v4/letter/w/82dd89/32.png) [@wizardh](https://discuss.huggingface.co/u/wizardh)\
**Post date:** [June 28, 2024, 10:59am UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/3 "2024-06-28T10:59:35Z")

</div>

It seems a change in the packages in the **transformers** project was introduced some weeks ago [Remove ConversationalPipeline and Conversation object by Rocketknight1 · Pull Request #31165 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/pull/31165)

We observed the same issue in SageMaker with our hosted model. The issue should be solved by pinning the **transformers** dependency to version **4.41.2** which still has the object in the expected package.

I already reported the issue, so the team can look at it: [SageMaker fails because Conversation object is not found · Issue #129 · aws/sagemaker-huggingface-inference-toolkit · GitHub](https://github.com/aws/sagemaker-huggingface-inference-toolkit/issues/129)

---

<div class="post-metadata">

**Author:** ![Nitro1234](https://avatars.discourse-cdn.com/v4/letter/n/f05b48/32.png) [@Nitro1234](https://discuss.huggingface.co/u/Nitro1234)\
**Post date:** [July 8, 2024, 8:24am UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/4 "2024-07-08T08:24:11Z")

</div>

Thank you!

---

<div class="post-metadata">

**Author:** ![philhd](https://avatars.discourse-cdn.com/v4/letter/p/ea5d25/32.png) [@philhd](https://discuss.huggingface.co/u/philhd)\
**Post date:** [September 30, 2024, 12:20pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/5 "2024-09-30T12:20:11Z")

</div>

Hi @wizardh , may I ask how did you pin the version.  
Requirements.txt inside the model does not seem to work since hf version is depending on the container [deep-learning-containers/available\_images.md at master · aws/deep-learning-containers · GitHub](https://github.com/aws/deep-learning-containers/blob/master/available_images.md)  
None of the older versions is working for me . I event tried 4.26

```auto
        transformers_version="4.26", # transformers version used
        pytorch_version="1.13", # pytorch version used
        py_version='py39', # python version used

```

Help would be really appreciated

---

<div class="post-metadata">

**Author:** ![philhd](https://avatars.discourse-cdn.com/v4/letter/p/ea5d25/32.png) [@philhd](https://discuss.huggingface.co/u/philhd)\
**Post date:** [September 30, 2024, 5:25pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/6 "2024-09-30T17:25:56Z")

</div>

I was able to solve by having a `requirements.txt` inside the model folder beside inference.py . My assumption that the transformers lib will be the version from container is not right

---

<div class="post-metadata">

**Author:** ![nightbloomer](https://avatars.discourse-cdn.com/v4/letter/n/ecc23a/32.png) [@nightbloomer](https://discuss.huggingface.co/u/nightbloomer)\
**Post date:** [October 3, 2024, 6:23pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/7 "2024-10-03T18:23:57Z")

</div>

@philhd Can you mention what is present in your `requirements.txt`?

---

<div class="post-metadata">

**Author:** ![sdYethi](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@sdYethi](https://discuss.huggingface.co/u/sdYethi)\
**Post date:** [October 25, 2024, 5:45am UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/8 "2024-10-25T05:45:20Z")

</div>

Issue resolved by pin-ing down the transformers version. But the problem is while using Llama3.2 weights in the model.tar.tz any transformers version below 4.43 gives rope\_scaling error (Error: [meta-llama/Llama-3.1-8B-Instruct · ValueError: `rope_scaling` must be a dictionary with two fields](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct/discussions/15)).

Any walkarounds for that ??

---

<div class="post-metadata">

**Author:** ![ahkrion](https://avatars.discourse-cdn.com/v4/letter/a/ecccb3/32.png) [@ahkrion](https://discuss.huggingface.co/u/ahkrion)\
**Post date:** [January 9, 2025, 4:14pm UTC](https://discuss.huggingface.co/t/cannot-import-conversation-from-transformers-utils-py/91556/9 "2025-01-09T16:14:51Z")

</div>

Bump on this question, any workaround for this?