# Split document into sentences for sentence embedding

**URL:** <https://discuss.huggingface.co/t/split-document-into-sentences-for-sentence-embedding/3553>\
**Category:** Beginners\
**Created:** [February 9, 2021, 10:25am UTC](https://discuss.huggingface.co/t/split-document-into-sentences-for-sentence-embedding/3553 "2021-02-09T10:25:47Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![LostGoatOnHill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lostgoatonhill/32/1001_2.png) [@LostGoatOnHill](https://discuss.huggingface.co/u/LostGoatOnHill)\
**Post date:** [February 9, 2021, 10:25am UTC](https://discuss.huggingface.co/t/split-document-into-sentences-for-sentence-embedding/3553/1 "2021-02-09T10:25:47Z")

</div>

Hi,

Wondering if there is a Huggingface alternative to the Gensim split\_sentences method to take a document and split into sentences ready for model.encode()?

[https://www.kite.com/python/docs/gensim.summarization.textcleaner.split\_sentences](https://www.kite.com/python/docs/gensim.summarization.textcleaner.split_sentences)

A first timer says many thanks 🙂

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](https://discuss.huggingface.co/u/BramVanroy)\
**Post date:** [February 9, 2021, 12:10pm UTC](https://discuss.huggingface.co/t/split-document-into-sentences-for-sentence-embedding/3553/2 "2021-02-09T12:10:36Z")

</div>

So you want to split a text into sentences and then create a sentence embedding for each sentence? Just use a parser like stanza or spacy to tokenize/sentence segment your data. This is typically the first step in many NLP tasks.

---

<div class="post-metadata">

**Author:** ![LostGoatOnHill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lostgoatonhill/32/1001_2.png) [@LostGoatOnHill](https://discuss.huggingface.co/u/LostGoatOnHill)\
**Post date:** [February 9, 2021, 12:26pm UTC](https://discuss.huggingface.co/t/split-document-into-sentences-for-sentence-embedding/3553/3 "2021-02-09T12:26:29Z")

</div>

Indeed, just wondered if Huggingface had their own variant. Thanks