# Regarding max steps, streaming in language modeling

**URL:** <https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295>\
**Category:** 🤗Optimum\
**Created:** [April 11, 2024, 6:16pm UTC](https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295 "2024-04-11T18:16:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Palash123](https://avatars.discourse-cdn.com/v4/letter/p/90ced4/32.png) [@Palash123](https://discuss.huggingface.co/u/Palash123)\
**Post date:** [April 11, 2024, 6:16pm UTC](https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295/1 "2024-04-11T18:16:58Z")

</div>

Since --max\_steps is needed when streaming is switched on -  
If my training data has 10B tokens, seq\_len or block\_size of 1024, global batch size 128:  
then for say 5 epochs is my calculation for max\_steps, correct?

Calculation for max\_steps:

1. (1024 \* 128) tokens per step.
2. max\_steps = (10B / (1024 \* 128)) \* 5

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](https://discuss.huggingface.co/u/regisss)\
**Post date:** [April 12, 2024, 7:20am UTC](https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295/2 "2024-04-12T07:20:30Z")

</div>

Hi @Palash123, that looks right to me. Can you specify the exact example you’re referring to in order to make sure of this?

---

<div class="post-metadata">

**Author:** ![Palash123](https://avatars.discourse-cdn.com/v4/letter/p/90ced4/32.png) [@Palash123](https://discuss.huggingface.co/u/Palash123)\
**Post date:** [April 13, 2024, 10:27am UTC](https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295/3 "2024-04-13T10:27:35Z")

</div>

Hi @regisss, this is regarding an example from language-modeling.

> **[optimum-habana/examples/language-modeling at main · huggingface/optimum-habana](https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling#streaming)**
>
> Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU) - huggingface/optimum-habana

```auto
python run_clm.py \
    --model_name_or_path gpt2 \
    --dataset_name wikitext \
    --dataset_config_name wikitext-2-raw-v1 \
    --per_device_train_batch_size 4 \
    --per_device_eval_batch_size 4 \
    --do_train \
    --output_dir /tmp/test-clm \
    --gaudi_config_name Habana/gpt2 \
    --use_habana \
    --use_lazy_mode \
    --use_hpu_graphs_for_inference \
    --throughput_warmup_steps 3 \
    --streaming \
    --max_steps 1000 \
    --do_eval

```

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](https://discuss.huggingface.co/u/regisss)\
**Post date:** [April 13, 2024, 1:36pm UTC](https://discuss.huggingface.co/t/regarding-max-steps-streaming-in-language-modeling/81295/4 "2024-04-13T13:36:31Z")

</div>

Thanks, so yeah your calculation looks right 🙂