# \[Nov 15th Event\] Thomas Wolf: Transfer Learning and the birth of the Transformers library **URL:** \ **Category:** Course\ **Created:** [November 15, 2021, 2:16pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748 "2021-11-15T14:16:56Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 2:16pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/1 "2021-11-15T14:16:56Z")
Use this topic to ask your questions to Thomas Wolf during his talk: _Transfer Learning and the birth of the Transformers library_ You can watch it on [YouTube](https://www.youtube.com/watch?v=6NTHvcXAl90) or on [Twitch](https://www.twitch.tv/HuggingFace) at 8am PST [Slides](https://docs.google.com/presentation/d/1PrXbNuaPbMmdkKSKuKElVbSb1RSyEdNSsFZ4lWSVCbE/edit?usp=sharing) ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 3:30pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/2 "2021-11-15T15:30:52Z")
This is an example of test question. You can test the like button below. ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [November 15, 2021, 4:01pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/3 "2021-11-15T16:01:49Z")
Hello I have a question! ---
**Author:** ![NDugar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ndugar/32/40501_2.png) [@NDugar](https://discuss.huggingface.co/u/NDugar)\ **Post date:** [November 15, 2021, 4:05pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/4 "2021-11-15T16:05:30Z")
What are the best ways to do transfer learning on huge models? And what are the best ways to pick metrics for NLP models? ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [November 15, 2021, 4:05pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/5 "2021-11-15T16:05:54Z")
Is Huggingface gradually becoming commercial from open source, there are many parts of API that are for paid now, e.g., model training, please help me understand the portion of open source vs paid. ---
**Author:** ![excalibur](https://avatars.discourse-cdn.com/v4/letter/e/f07891/32.png) [@excalibur](https://discuss.huggingface.co/u/excalibur)\ **Post date:** [November 15, 2021, 4:06pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/6 "2021-11-15T16:06:19Z")
Is this presentation document going to be shared? Thanks! ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 4:06pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/7 "2021-11-15T16:06:40Z")
> Is this presentation document going to be shared? Thanks! Yes, the slides will be shared after the talk 🙂 ---
**Author:** ![shamikbose89](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shamikbose89/32/4370_2.png) [@shamikbose89](https://discuss.huggingface.co/u/shamikbose89)\ **Post date:** [November 15, 2021, 4:07pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/8 "2021-11-15T16:07:16Z")
Is there a systematic way to estimate how many annotated samples will be required to train a custom model (let’s say NER) by finetuning an existing model? ---
**Author:** ![wgpubs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/wgpubs/32/303_2.png) [@wgpubs](https://discuss.huggingface.co/u/wgpubs)\ **Post date:** [November 15, 2021, 4:15pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/9 "2021-11-15T16:15:53Z")
What does Thomas use to build/train models? notebooks? vscode? vim? other??? ---
**Author:** ![jairo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jairo/32/26898_2.png) [@jairo](https://discuss.huggingface.co/u/jairo)\ **Post date:** [November 15, 2021, 4:15pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/10 "2021-11-15T16:15:54Z")
How did the name HuggingFace and the use of 🤗 come about? ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 4:18pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/11 "2021-11-15T16:18:30Z")
> [@shamikbose89](#): > > Is there a systematic way to estimate how many annotated samples will be required to train a custom model (let’s say NER) by finetuning an existing model? This is answered at 17:15 on the main stream. ---
**Author:** ![MoritzLaurer](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moritzlaurer/32/18932_2.png) [@MoritzLaurer](https://discuss.huggingface.co/u/MoritzLaurer)\ **Post date:** [November 15, 2021, 4:18pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/12 "2021-11-15T16:18:39Z")
When explaining the reasons for the main revolution in 2018, you pointed out two separate points: (1) transfer learning and (2) transformers for increased efficiency. Does that mean that there is not inherent link between transfer learning and transformers? Transformers are only/mostly used because they are very efficient? ---
**Author:** ![kalpro](https://avatars.discourse-cdn.com/v4/letter/k/7feea3/32.png) [@kalpro](https://discuss.huggingface.co/u/kalpro)\ **Post date:** [November 15, 2021, 4:22pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/13 "2021-11-15T16:22:58Z")
Will the video/recording also be shared later? (It is becoming late here…) ---
**Author:** ![muellerzr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/muellerzr/32/21583_2.png) [@muellerzr](https://discuss.huggingface.co/u/muellerzr)\ **Post date:** [November 15, 2021, 4:23pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/14 "2021-11-15T16:23:31Z")
> [@kalpro](#): > > video/recording also be shared later The same YouTube link for streaming will have the video I believe 🙂 ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 4:23pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/15 "2021-11-15T16:23:53Z")
> Will the video/recording also be shared later? (It is becoming late here…) The live stream can be viewed at any time on YouTube, and we will also edit to share each talk in a separate YouTube video 🙂 ---
**Author:** ![Abirate](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abirate/32/3119_2.png) [@Abirate](https://discuss.huggingface.co/u/Abirate)\ **Post date:** [November 15, 2021, 4:25pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/16 "2021-11-15T16:25:22Z")
What about using Graph Neural networks (GNN) in NLP tasks? Graph structures could capture data with complex structures and relationships and GNN provide the opportunity to study and model complex data representation for NLP tasks… ---
**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](https://discuss.huggingface.co/u/shainaraza)\ **Post date:** [November 15, 2021, 4:26pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/17 "2021-11-15T16:26:47Z")
looking into reply for gnn ---
**Author:** ![ivanlau](https://avatars.discourse-cdn.com/v4/letter/i/b38774/32.png) [@ivanlau](https://discuss.huggingface.co/u/ivanlau)\ **Post date:** [November 15, 2021, 4:29pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/18 "2021-11-15T16:29:19Z")
Interesting! Looking into this reply ---
**Author:** ![sumedh](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sumedh/32/2097_2.png) [@sumedh](https://discuss.huggingface.co/u/sumedh)\ **Post date:** [November 15, 2021, 4:40pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/19 "2021-11-15T16:40:42Z")
can you tell us about the distributed training efforts taken by the hf community? ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [November 15, 2021, 4:41pm UTC](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748/20 "2021-11-15T16:41:41Z")
> What about using Graph Neural networks (GNN) in NLP tasks? Graph structures could capture data with complex structures and relationships and GNN provide the opportunity to study and model complex data representation for NLP tasks… This is answered at 41:00 of the main stream. [Next page](https://discuss.huggingface.co/t/nov-15th-event-thomas-wolf-transfer-learning-and-the-birth-of-the-transformers-library/11748.md?page=2)