# Inference speed between pipelines and Heads

**URL:** <https://discuss.huggingface.co/t/inference-speed-between-pipelines-and-heads/35415>\
**Category:** 🤗Transformers\
**Created:** [April 3, 2023, 3:07pm UTC](https://discuss.huggingface.co/t/inference-speed-between-pipelines-and-heads/35415 "2023-04-03T15:07:40Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![brosand](https://avatars.discourse-cdn.com/v4/letter/b/3e96dc/32.png) [@brosand](https://discuss.huggingface.co/u/brosand)\
**Post date:** [April 3, 2023, 3:07pm UTC](https://discuss.huggingface.co/t/inference-speed-between-pipelines-and-heads/35415/1 "2023-04-03T15:07:40Z")

</div>

Is there any difference in latency between the general class `pipeline` and a more task-specific implementation? For example, between these code blocks, is there a speed difference? What about with batching, does that change things?

```auto
from transformers import pipeline

generator = pipeline(model="gpt2")
generator("I can't believe you did such a ", do_sample=False)

# These parameters will return suggestions, and only the newly created text making it easier for prompting suggestions.
outputs = generator("My tart needs some")

```

The other method would be:

```auto
from transformers import GPT2Tokenizer, GPT2Model
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2Model.from_pretrained('gpt2')
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

```

Moreover, is there any records on the fastest way to run HF models generally? Should they be exported somewhere else for optimization?