# Get text generation in particular format

**URL:** <https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313>\
**Category:** Research\
**Created:** [April 17, 2024, 11:48pm UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313 "2024-04-17T23:48:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivek9840](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@vivek9840](https://discuss.huggingface.co/u/vivek9840)\
**Post date:** [April 17, 2024, 11:48pm UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/1 "2024-04-17T23:48:03Z")

</div>

hi, i am currently inferring different LLMs for classification and their inherit understanding of requirement engineering.

this is a zero shot prompt designed for inferring the LLMs.

few LLMs are giving the output in described format. but few models, even with higher number of parameters are not giving answer. rather they just giving output like this:

‘generated\_text’ : \nSentence1\n\nSentence2\n\nSentence3.

expected output is: label1, label2, label3

test\_zero\_prompt = “”"

Functional requirements specify the functions or behaviors that a system must provide. They describe what the system should do.  
Non-functional requirements specify constraints, qualities, or characteristics that the system must possess, such as performance, usability, or security, etc. They describe how the system should behave or perform.  
Information (also known as non-requirements) refers to statements that provide context, background, or explanations but do not specify any requirements for the system.

classify the given sentences into functional requirement, non-functional requirement, or information. Give the answer in the classification labels only.

Sentence 1: this is a functional requirement.

Sentence 2: this is a non-functional requirement.

Sentence 3: this is an information.

Sentence 4: this is a functional requirement.

Answers:

functional requirement, non-functional requirement, information, functional requirement

Classify the following Sentences according to format above: “”"

can anyone suggest, how can i improve the prompt or change it.

or find mistake or error in prompt?

---

<div class="post-metadata">

**Author:** ![RaushanTurganbay](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@RaushanTurganbay](https://discuss.huggingface.co/u/RaushanTurganbay)\
**Post date:** [April 18, 2024, 8:58am UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/2 "2024-04-18T08:58:44Z")

</div>

You may be interested in libraries like [outlines](https://github.com/outlines-dev/outlines/tree/main) that enable enforcing a format/json/regex on generated text

---

<div class="post-metadata">

**Author:** ![vivek9840](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@vivek9840](https://discuss.huggingface.co/u/vivek9840)\
**Post date:** [April 18, 2024, 9:11am UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/3 "2024-04-18T09:11:47Z")

</div>

Is it compatible with hugging face libraries. Like accelerate or pipeline?

---

<div class="post-metadata">

**Author:** ![RaushanTurganbay](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@RaushanTurganbay](https://discuss.huggingface.co/u/RaushanTurganbay)\
**Post date:** [April 18, 2024, 9:30am UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/4 "2024-04-18T09:30:44Z")

</div>

If you want compatibility with `transformers`, look into [this](https://github.com/Saibo-creator/transformers-CFG/tree/main) which implemented constraints as a separate LogitsProcessor. I believe it should be compatible by simply passing `logits_processor=[grammar_processor]` into `model.generate()` 🤗

---

<div class="post-metadata">

**Author:** ![vivek9840](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@vivek9840](https://discuss.huggingface.co/u/vivek9840)\
**Post date:** [April 18, 2024, 9:45am UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/5 "2024-04-18T09:45:01Z")

</div>

Thanks, I think I can pass logits processor in pipeline or use it with accelerate.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [April 18, 2024, 9:45pm UTC](https://discuss.huggingface.co/t/get-text-generation-in-particular-format/82313/6 "2024-04-18T21:45:20Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.