# Project: Create a new zero-shot model with NLI data **URL:** \ **Category:** 🤗 Course Projects\ **Created:** [November 16, 2021, 8:45am UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858 "2021-11-16T08:45:39Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![MoritzLaurer](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moritzlaurer/32/18932_2.png) [@MoritzLaurer](https://discuss.huggingface.co/u/MoritzLaurer)\ **Post date:** [November 16, 2021, 8:45am UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/1 "2021-11-16T08:45:39Z")
### Description The zero-shot classification pipeline has becomes very popular on Hugging Face. It allows you to classify a text in any category without having to fine-tune a model for the specific classification task you are interested in. The zero-shot pipeline is based on models trained on Natural Language Inference (NLI). This project will train a new NLI model, which can then be used in the zero-shot classification pipeline. ### Model(s) Any base-model can be used. Since there are already several NLI models on the model hub, I suggest to train a new model based on Microsoft’s [DeBERTa-v3](https://arxiv.org/abs/2006.03654) model. [Version three](https://huggingface.co/microsoft/deberta-v3-base) was only published few weeks ago and can outperform larger models (see an example [here](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli)). We can probably create a new SOTA NLI model with the new DeBERTa version and enough NLI data. ### Datasets Established NLI datasets include: [MultiNLI](https://huggingface.co/datasets/multi_nli) [SNLI](https://huggingface.co/datasets/snli) [ANLI](https://huggingface.co/datasets/anli) Other interesting NLI datasets include: [FEVER-NLI](https://github.com/easonnie/combine-FEVER-NSMN/blob/master/other_resources/nli_fever.md) [DocNLI](https://github.com/salesforce/DocNLI) [LingNLI](https://github.com/Alicia-Parrish/ling_in_loop/tree/master) More datasets can be included! ### Challenges - NLI models can be trained as either 3-class classifiers (entailment/neutral/contradiction) or as 2-class classifiers (entailment/not\_entailment). Both setups have different advantages and disadvantages - There is a lot of NLI data (2 mio++ texts in the datasets linked above), which makes training computationally expensive. Optimising the training pipeline is a challenge. - Many different datasets can be translated into NLI-format. Including more datasets can be beneficial, but requires manual transformation of datasets. ### Desired project outcomes - Create a Streamlit or Gradio app on [🤗 Spaces](https://huggingface.co/spaces) that provides an interface for zero-shot classification with a new NLI model in the backend. ### Additional resources See the links to the datasets above. Also see Joe Davidson’s original [blog post](https://discuss.huggingface.co/t/new-pipeline-for-zero-shot-text-classification/681) on the zeroshot pipeline ### Discord channel To chat and organise with other people interested in this project, head over to our [Discord](http://xwo7x.mjt.lu/lnk/CAAAAkdkZwAAAAAAAAAAALG-MTEAAYCr4H8AAAAAAAyFyQBhjpnimM4rcE5DT5ibCUtRT96tWwAMh8Y/15/U6BYNMNDksyyGG4jgtzPKQ/aHR0cHM6Ly9odWdnaW5nZmFjZS5jby9qb2luL2Rpc2NvcmQ) and: - Follow the instructions on the `#join-course` channel - Join the `#zero-shot` channel Just make sure you comment here to indicate that you’ll be contributing to this project 🙂 ---
**Author:** ![HarrySaini](https://avatars.discourse-cdn.com/v4/letter/h/ee59a6/32.png) [@HarrySaini](https://discuss.huggingface.co/u/HarrySaini)\ **Post date:** [November 16, 2021, 12:51pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/2 "2021-11-16T12:51:00Z")
Hyee! I’d love to contribute in this one. I guess further discussion will take place in Discord? ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [November 16, 2021, 1:44pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/3 "2021-11-16T13:44:05Z")
Hey @HarrySaini, yep Discord is probably the best place to coordinate / discuss more efficiently 🙂 ---
**Author:** ![Rajaram1996](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rajaram1996/32/3025_2.png) [@Rajaram1996](https://discuss.huggingface.co/u/Rajaram1996)\ **Post date:** [November 16, 2021, 3:45pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/4 "2021-11-16T15:45:07Z")
Interested. Messaged in Discord as well. ---
**Author:** ![adorkin](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/adorkin/32/2704_2.png) [@adorkin](https://discuss.huggingface.co/u/adorkin)\ **Post date:** [November 18, 2021, 5:35pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/5 "2021-11-18T17:35:34Z")
I happen to have trained a bilingual MNLI model recently, so I decided to give this project a [shot](https://huggingface.co/spaces/AlekseyDorkin/ZeroShotClassificationEnRu) as well (_without taking a place on the team_). In theory this should work on English and Russian. Any feedback is very welcome. ---
**Author:** ![jordiclive](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jordiclive/32/28000_2.png) [@jordiclive](https://discuss.huggingface.co/u/jordiclive)\ **Post date:** [January 31, 2022, 6:09pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/6 "2022-01-31T18:09:28Z")
@MoritzLaurer was there any update on this? I would be very interested in a superior model to facebook/bart-large-mnli ---
**Author:** ![MoritzLaurer](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/moritzlaurer/32/18932_2.png) [@MoritzLaurer](https://discuss.huggingface.co/u/MoritzLaurer)\ **Post date:** [January 31, 2022, 6:23pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/7 "2022-01-31T18:23:42Z")
yeah, this one is better than bart-large-mnli: [MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli · Hugging Face](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli) (or this one: [MoritzLaurer/DeBERTa-v3-base-mnli-fever-docnli-ling-2c · Hugging Face](https://huggingface.co/MoritzLaurer/DeBERTa-v3-base-mnli-fever-docnli-ling-2c)) and they should also be faster ---
**Author:** ![jordiclive](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jordiclive/32/28000_2.png) [@jordiclive](https://discuss.huggingface.co/u/jordiclive)\ **Post date:** [February 5, 2022, 5:55pm UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/8 "2022-02-05T17:55:41Z")
Great! Thanks, I will be comparing them for zero-shot text classification. ---
**Author:** ![MaziyarPanahi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/maziyarpanahi/32/23330_2.png) [@MaziyarPanahi](https://discuss.huggingface.co/u/MaziyarPanahi)\ **Post date:** [April 3, 2023, 8:35am UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/9 "2023-04-03T08:35:54Z")
Hi @MoritzLaurer Is there a way to update the first post with some example notebooks on how to take any transformer-based models and one or more NLI-based datasets and fine-tune a new Zero-Shot text classifier? (some architectures are missing from the Models Hub like BERT, I would like to add those) ---
**Author:** ![MaziyarPanahi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/maziyarpanahi/32/23330_2.png) [@MaziyarPanahi](https://discuss.huggingface.co/u/MaziyarPanahi)\ **Post date:** [April 11, 2023, 7:47am UTC](https://discuss.huggingface.co/t/project-create-a-new-zero-shot-model-with-nli-data/11858/10 "2023-04-11T07:47:22Z")
I used the code in the `run_xnli` script and it worked well! 🙂 > **[transformers/examples/pytorch/text-classification at main ·...](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification)** > > main/examples/pytorch/text-classification