# GPT2 with TensorFlow?

**URL:** <https://discuss.huggingface.co/t/gpt2-with-tensorflow/2051>\
**Category:** 🤗Transformers\
**Created:** [November 13, 2020, 1:41pm UTC](https://discuss.huggingface.co/t/gpt2-with-tensorflow/2051 "2020-11-13T13:41:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![eddieone](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/eddieone/32/466_2.png) [@eddieone](https://discuss.huggingface.co/u/eddieone)\
**Post date:** [November 13, 2020, 1:41pm UTC](https://discuss.huggingface.co/t/gpt2-with-tensorflow/2051/1 "2020-11-13T13:41:24Z")

</div>

I tried doing this at the start without luck. I was wondering if anyone was successful at eval/retrain GPT2 with TensorFlow instead of pyTorch? I guess there are some ports directly in TF. However, my question is related to the Transformers library.

---

<div class="post-metadata">

**Author:** ![Jung](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jung/32/21989_2.png) [@Jung](https://discuss.huggingface.co/u/Jung)\
**Post date:** [November 14, 2020, 7:51am UTC](https://discuss.huggingface.co/t/gpt2-with-tensorflow/2051/2 "2020-11-14T07:51:14Z")

</div>

I am not aware of public notebooks regarding training Huggingface’s TFGPT2, but we have public notebooks of training TFT5 and TFXLM-R , which should give you some ideas

TF-T5

> **[Google Colaboratory](https://colab.research.google.com/github/snapthat/TF-T5-text-to-text/blob/master/snapthatT5/notebooks/TF-T5-Datasets%20Training.ipynb)**

TF-XLM-R

> **[Finetune XLM-Roberta on Jigsaw test data with MLM](https://www.kaggle.com/riblidezso/finetune-xlm-roberta-on-jigsaw-test-data-with-mlm)**
>
> Explore and run machine learning code with Kaggle Notebooks | Using data from multiple data sources