# Preparing datasets for NLP tasks

**URL:** <https://discuss.huggingface.co/t/preparing-datasets-for-nlp-tasks/8746>\
**Category:** 🤗Datasets\
**Created:** [July 28, 2021, 3:45am UTC](https://discuss.huggingface.co/t/preparing-datasets-for-nlp-tasks/8746 "2021-07-28T03:45:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![hgarg](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hgarg/32/1592_2.png) [@hgarg](https://discuss.huggingface.co/u/hgarg)\
**Post date:** [July 28, 2021, 3:45am UTC](https://discuss.huggingface.co/t/preparing-datasets-for-nlp-tasks/8746/1 "2021-07-28T03:45:35Z")

</div>

Can someone point me to resources on how to go about preparing custom datasets for fine-tuning NLP models?

Suppose I want to fine-tune a text completion model on a particular author’s works so that the outputs are in their style and I have access to the text in pdf format, how do I go from bunch of pdfs to a dataset that is acceptable for input to the model?

---

<div class="post-metadata">

**Author:** ![ahmedlone123](https://avatars.discourse-cdn.com/v4/letter/a/c4cdca/32.png) [@ahmedlone123](https://discuss.huggingface.co/u/ahmedlone123)\
**Post date:** [July 28, 2021, 6:31am UTC](https://discuss.huggingface.co/t/preparing-datasets-for-nlp-tasks/8746/2 "2021-07-28T06:31:12Z")

</div>

I think you might need to convert the pdfs to a readable format like json or csv . Then you can clean and process it through pandas or the datasets library provided by hugging face …