# Batching on Vanilla CPU for Inference

**URL:** <https://discuss.huggingface.co/t/batching-on-vanilla-cpu-for-inference/47048>\
**Category:** 🤗Transformers\
**Created:** [July 17, 2023, 7:11am UTC](https://discuss.huggingface.co/t/batching-on-vanilla-cpu-for-inference/47048 "2023-07-17T07:11:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shamik](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@Shamik](https://discuss.huggingface.co/u/Shamik)\
**Post date:** [July 17, 2023, 7:11am UTC](https://discuss.huggingface.co/t/batching-on-vanilla-cpu-for-inference/47048/1 "2023-07-17T07:11:48Z")

</div>

Hello,

Infrastructure: Vanilla CPU(any intel based CPU without any specialised ASIC/TPUs or anything)

How does transformers pipeline class handle batching for a system only with CPU, e.g.

```python
 pipeline(task='token-classification',
                     model=model,
                     tokenizer=tokenizer,
                     batch_size = 128,
                     device='cpu',
                     )

```

If the above code(changing device to ‘cuda’) was working on a GPU then it would do a batch inferencing unless the batch size is too much and it encounters a OOM allocation error.

However, how would the same code for a CPU? How does the pipeline class natively handle it when the CPU isn’t capable of handling batch computing?

Thank you