# Few shot learning

**URL:** <https://discuss.huggingface.co/t/few-shot-learning/19444>\
**Category:** Beginners\
**Created:** [June 22, 2022, 5:54am UTC](https://discuss.huggingface.co/t/few-shot-learning/19444 "2022-06-22T05:54:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mujin](https://avatars.discourse-cdn.com/v4/letter/m/74df32/32.png) [@mujin](https://discuss.huggingface.co/u/mujin)\
**Post date:** [June 22, 2022, 5:54am UTC](https://discuss.huggingface.co/t/few-shot-learning/19444/1 "2022-06-22T05:54:31Z")

</div>

How can I do few shot learning using the transformers library and gpt-j-6B?  
I have found the contents of the following link helpful but it only describes few shot learning using the API.

[https://huggingface.co/blog/few-shot-learning-gpt-neo-and-inference-api](https://huggingface.co/blog/few-shot-learning-gpt-neo-and-inference-api)  
Thanks

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [June 22, 2022, 8:40am UTC](https://discuss.huggingface.co/t/few-shot-learning/19444/2 "2022-06-22T08:40:00Z")

</div>

Hi,

I do have a notebook on that here: [Transformers-Tutorials/Inference\_with\_GPT\_J\_6B.ipynb at master · NielsRogge/Transformers-Tutorials · GitHub](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/GPT-J-6B/Inference_with_GPT_J_6B.ipynb)

---

<div class="post-metadata">

**Author:** ![mujin](https://avatars.discourse-cdn.com/v4/letter/m/74df32/32.png) [@mujin](https://discuss.huggingface.co/u/mujin)\
**Post date:** [June 22, 2022, 12:47pm UTC](https://discuss.huggingface.co/t/few-shot-learning/19444/3 "2022-06-22T12:47:56Z")

</div>

Thanks for providing the code. When I run this code, it gives the following errors–\>

```auto
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.

```

` **RuntimeError** : "LayerNormKernelImpl" not implemented for 'Half'`  
Can you please help me with these?

---

<div class="post-metadata">

**Author:** ![kabirfarhad](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kabirfarhad](https://discuss.huggingface.co/u/kabirfarhad)\
**Post date:** [November 16, 2022, 5:08am UTC](https://discuss.huggingface.co/t/few-shot-learning/19444/4 "2022-11-16T05:08:30Z")

</div>

If you using CPU , instead of GPU please chnage revision=“float32” instead of revision=“float16”