# How to construct Chinese dataset with gpt2 fine tune

**URL:** <https://discuss.huggingface.co/t/how-to-construct-chinese-dataset-with-gpt2-fine-tune/31673>\
**Category:** 🤗Transformers\
**Created:** [February 14, 2023, 5:52am UTC](https://discuss.huggingface.co/t/how-to-construct-chinese-dataset-with-gpt2-fine-tune/31673 "2023-02-14T05:52:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![LycheeX](https://avatars.discourse-cdn.com/v4/letter/l/f1d935/32.png) [@LycheeX](https://discuss.huggingface.co/u/LycheeX)\
**Post date:** [February 14, 2023, 5:52am UTC](https://discuss.huggingface.co/t/how-to-construct-chinese-dataset-with-gpt2-fine-tune/31673/1 "2023-02-14T05:52:22Z")

</div>

> <https://github.com/huggingface/transformers/issues/21618>
>
> \### System Info
> 
> \- \`transformers\` version: 4.26.0.dev0
> \- Platform: Linux-3.10.0…-1160.81.1.el7.x86\_64-x86\_64-with-glibc2.17
> \- Python version: 3.9.15
> \- Huggingface\_hub version: 0.11.1
> \- PyTorch version (GPU?): 1.13.1+cu116 (True)
> \- Tensorflow version (GPU?): not installed (NA)
> \- Flax version (CPU?/GPU?/TPU?): not installed (NA)
> \- Jax version: not installed
> \- JaxLib version: not installed
> \- Using GPU in script?: \<fill in\>
> \- Using distributed or parallel set-up in script?: \<fill in\>
> 
> 
> \### Who can help?
> 
> @vanpelt @pvl @arfon @xeb @kashif @sgugger 
> 
> \### Information
> 
> \- \[X\] The official example scripts
> \- \[\] My own modified scripts
> 
> \### Tasks
> 
> \- \[\] An officially supported task in the \`examples\` folder (such as GLUE/SQuAD, ...)
> \- \[X\] My own task or dataset (give details below)
> 
> \### Reproduction
> 
> want to know how to construct my own data,
> and does it support chinese words with gpt2,
> my dataset is Chinese sentence , the format is txt, and no label 
> could you pls help me ?
> 
> 
> \### Expected behavior
> 
> must have \[CLS\] in the beginning of the sentence ?
> and what's the meaning of the "==" as follow (from wiki.test.raw) :
> = Robert Boulter = 
>  
> Robert Boulter is an English film , television and theatre actor . He had a guest @-@ starring role on the television series The Bill in 2000 . This was followed by a starring role in the play Herons written by Simon Stephens , which was performed in 2001 at the Royal Court Theatre . He had a guest role in the television series Judge John Deed in 2002 . In 2004 Boulter landed a role as " Craig " in the episode " Teddy 's Story " of the television series The Long Firm ; he starred alongside actors Mark Strong and Derek Jacobi . He was cast in the 2005 theatre productions of the Philip Ridley play Mercury Fur , which was performed at the Drum Theatre in Plymouth and the Menier Chocolate Factory in London . He was directed by John Tiffany and starred alongside Ben Whishaw , Shane Zaza , Harry Kent , Fraser Ayres , Sophie Stanton and Dominic Hall . 
> In 2006 , Boulter starred alongside Whishaw in the play Citizenship written by Mark Ravenhill . He appeared on a 2006 episode of the television series , Doctors , followed by a role in the 2007 theatre production of How to Curse directed by Josie Rourke . How to Curse was performed at Bush Theatre in the London Borough of Hammersmith and Fulham . Boulter starred in two films in 2008 , Daylight Robbery by filmmaker Paris Leonti , and Donkey Punch directed by Olly Blackburn . In May 2008 , Boulter made a guest appearance on a two @-@ part episode arc of the television series Waking the Dead , followed by an appearance on the television series Survivors in November 2008 . He had a recurring role in ten episodes of the television series Casualty in 2010 , as " Kieron Fletcher " . Boulter starred in the 2011 film Mercenaries directed by Paris Leonti . 
>  
> = = Career = =

want to use my own dataset, run clm  
must have [cls] in the beginning ?