# A minimalist DSL to enforce deterministic code generation with LLMs

**URL:** <https://discuss.huggingface.co/t/a-minimalist-dsl-to-enforce-deterministic-code-generation-with-llms/179149>\
**Category:** Show and Tell\
**Created:** [August 23, 2026, 3:36pm UTC](https://discuss.huggingface.co/t/a-minimalist-dsl-to-enforce-deterministic-code-generation-with-llms/179149 "2026-08-23T15:36:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lecyberbill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lecyberbill/32/39845_2.png) [@Lecyberbill](https://discuss.huggingface.co/u/Lecyberbill)\
**Post date:** [August 23, 2026, 3:36pm UTC](https://discuss.huggingface.co/t/a-minimalist-dsl-to-enforce-deterministic-code-generation-with-llms/179149/1 "2026-08-23T15:36:02Z")

</div>

Hi everyone,

When integrating LLMs into production pipelines, we often face stochastic issues: hallucinations, architectural over-engineering, and a lack of reproducibility. Instead of stacking layers of fragile prompt engineering, I approached this from a language design perspective: **constraining the expression space to enforce determinism.**

I designed **IPL (Intent Programming Language)** with a pragmatic goal: **making software generation reliable while drastically cutting development costs and cycles.**

To build this minimalist DSL, I leveraged an LLM to help identify and structure a universal, irreducible core vocabulary designed to serve as a deterministic bridge between human intent and executable code.

### Core Architecture

- **Minimal Grammar:** A strictly reduced syntax with core action primitives and standardized types. The structure is lightweight enough that **any basic or local LLM** can translate natural language into an `.ipl` file zero-shot.

- **Separation of Concerns:** Business logic is isolated in a human-readable, auditable intermediate artifact, completely decoupled from the target tech stack.

- **Multi-File & Heterogeneous Stacks:** Multi-file `.ipl` support allows orchestrating complex architectures mixing different backend and frontend layers.

- **Constrained Transpilation:** By locking the lexical scope, the LLM cannot hallucinate exotic architectures. It acts purely as a reliable transpilation engine.

### Quick Example

1. **Natural Language Intent:**

2. **Intermediary Intent Contract (`.ipl`):**

3. **Target Code Generation:**

The repository includes the specification, parser logic, and the Studio IDE interface:

🔗 **GitHub:** [GitHub - lecyberbill/IPL\_STUDIO: An AI-powered polyglot intent-based IDE with autonomous auto-healing agentic capabilities. · GitHub](https://github.com/lecyberbill/IPL_STUDIO)

I would love to hear your thoughts, feedback, or critiques on this constraint-first approach to LLM-driven development!

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [August 24, 2026, 11:27pm UTC](https://discuss.huggingface.co/t/a-minimalist-dsl-to-enforce-deterministic-code-generation-with-llms/179149/2 "2026-08-24T23:27:57Z")

</div>

For now, I tried a few lightweight checks in Colab;

* * *

I think the **constraint-first direction itself is quite reasonable**. In particular, I like the separation between a small human-readable intent representation and the deterministic/static/runtime checks around the LLM. There are related patterns in things such as [Microsoft’s DSL Copilot](https://github.com/microsoft/dsl-copilot?utm_source=chatgpt.com) and [TypeChat](https://github.com/microsoft/TypeChat?utm_source=chatgpt.com): the formal representation helps, but practical reliability still comes from combining it with validation, diagnostics, and repair rather than expecting the model alone to behave like a compiler.

After reading through [IPL Studio](https://github.com/lecyberbill/IPL_STUDIO?utm_source=chatgpt.com) and doing a couple of small controlled runs, my main thought is that it may help to separate **which freedoms IPL itself is supposed to eliminate** from the freedoms intentionally left to the architecture/code-generation stages.

Something roughly like this:

```plaintext
Natural-language requirement
        ↓
IPL / intent contract
        ↓
(optional normalized / typed authoritative IR)
        ↓
architecture / topology contract
        ↓
code lowering
        ↓
static gates
        ↓
runtime / behavioral verification
        ↓
repair

```

That separation would let several different meanings of “deterministic” coexist without forcing IPL to solve all of them:

| Goal | Probably the important contract |
| --- | --- |
| Same requirement → same observable behavior | IPL + behavioral verifier |
| Same requirement → roughly same architecture | explicit topology/architecture contract |
| Preserve exact names/types/formulas | semantic-lowering contract |
| Same input → identical source tree | deterministic lowering after an authoritative IR |
| Repeatability across inference runs | model/backend/version/seed/runtime controls too |

So I would not necessarily make IPL larger or stricter first. My default route would be to **keep the minimalist vocabulary, but make the boundaries measurable**.

The highest-information, relatively low-cost additions seem to be:

1. report **first-pass** , deterministic-repair, and LLM-repair results separately;
2. ensure the initial generator and behavioral oracle have the same requirement/fixture information;
3. measure **semantic preservation** separately from final runtime PASS;
4. measure architecture/topology variance separately from business behavior;
5. keep an equal-information direct natural-language baseline, mainly as a control rather than as a competitor to IPL.

The third item in particular seems useful for this project. A final PASS/FAIL can currently combine several very different things:

```plaintext
identifier / output-key preservation
type preservation
formula / arithmetic preservation
control-flow preservation
topology / entrypoint conformance
cross-file dependency closure
runtime behavior

```

If those become separate receipts, then when generation fails you can say _which layer still had freedom_, rather than only saying that the whole generated application failed.

> **A very small Colab sanity check**
>
> I used the two-pass harness with `Qwen/Qwen2.5-Coder-7B-Instruct` on a T4, and pinned the repository to commit `e91c547...` so that the experiment was at least reproducible. The model used here is the public [Qwen2.5-Coder-7B-Instruct checkpoint](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct?utm_source=chatgpt.com).
> 
> This is only a small sanity check, not a benchmark of IPL in general.
> 
> I first noticed that some benchmark oracles contained concrete fixture/output information not present in their initial IPL. So I made a second control where the IPL and natural-language versions carried the same task facts, removed repair entirely, and varied the generation seed.
> 
> The first-pass results were:
> 
> | Representation | Parking | Coffee |
> | --- | --- | --- |
> | equal-information IPL | 0/3 PASS | 1/3 PASS |
> | equal-information natural language | 2/3 PASS | 0/3 PASS (`WARN` in all three) |
> 
> With only three seeds per task, I do **not** think this supports “NL beats IPL” or “IPL beats NL.”
> 
> What was more useful was the failure _shape_.
> 
> On the IPL side I saw examples such as:
> 
> - exact output keys being normalized from `isVip` / `grandTotal` into `is_vip` / `grand_total`;
> - a rounding formula being lowered incorrectly;
> - `grandTotal` becoming a string in one generated program;
> - a final floating-point total being emitted as `5.779999999999999` instead of the requested rounded value;
> - a top-level `return` becoming invalid target-language code;
> - a generated module using `json.dumps` without importing `json`.
> 
> On the NL side, the failures were more often around project integration:
> 
> - a generated CLI topology without the expected `main.py`;
> - cross-file symbols/imports such as constants or helper classes being referenced but not imported;
> - entrypoint/dependency closure problems.
> 
> That makes me suspect the useful distinction is not simply:
> 
> ```plaintext
> IPL vs natural language
> 
> ```
> 
> but something closer to:
> 
> ```plaintext
> intent preservation
> +
> architecture preservation
> +
> target-language lowering
> +
> project integration
> +
> runtime verification
> 
> ```
> 
> There was also an interesting reproducibility distinction. With the exact prompt/model/runtime/seed held fixed, my repeated generation was highly stable. But changing the Pass-1 seed for the same IPL produced materially different project topologies — for example, different file decompositions and helper modules.
> 
> So I would distinguish:
> 
> > “IPL substantially reduces the allowable intent space”
> 
> from the stronger statement:
> 
> > “IPL uniquely determines the project architecture.”
> 
> The former can still be very useful even if the latter is intentionally not a goal.
> 
> One caveat: the checks above are specifically against `e91c547...`. When the Colab run started, the remote repository HEAD had already moved on to `f301e260...`, so I would treat any exact source-code observations here as snapshot observations rather than claims about whatever is currently on `main`.

> **One benchmark distinction that may be worth making explicit**
>
> At the snapshot I tested, several behavioral tests had information on the oracle side that was not available to the initial generator.
> 
> For example, the parking/coffee tests expected concrete demo records and exact outputs, while the original IPL mostly described the entities and business rules. `typed-order` had a similar case with concrete demo identity/output expectations.
> 
> The newer `seed` verb seems like a very natural way to resolve this **when those exact records are part of the intended contract**.
> 
> For example, conceptually:
> 
> ```plaintext
> seed Vehicle StandardCar {
> plate: "AB-123",
> ...
> }
> 
> seed Vehicle VipCar {
> plate: "VIP-7",
> ...
> }
> 
> ```
> 
> Then the requirement and oracle are talking about the same concrete world.
> 
> On the other hand, if `"AB-123"` itself is not semantically important, another legitimate route is to make the verifier test the underlying property rather than that particular fixture.
> 
> So I see two clean branches:
> 
> ```plaintext
> Exact fixture is part of the requirement
> → put it in IPL/input data
> 
> Exact fixture is only test scaffolding
> → verify the behavioral property instead
> 
> ```
> 
> There is a second accounting distinction around repair.
> 
> If a first attempt only sees IPL, but the repair attempt later receives the verifier’s expected runtime behavior, then the two successes mean different things:
> 
> ```plaintext
> first-pass success
> = generated from the original contract
> 
> post-verifier repair success
> = converged after receiving additional diagnostic/oracle information
> 
> ```
> 
> Both are useful measurements. I would just keep them as separate lines in the scorecard.
> 
> This is also consistent with the way other validated-generation systems are usually described. TypeChat, for example, validates the generated representation and then explicitly makes a second model call containing the validation diagnostics when repair is needed.

> **Why I think an architecture contract could complement IPL rather than compete with it**
>
> The 13-verb vocabulary appears to be mostly about **business/behavioral intent** :
> 
> ```plaintext
> add
> read
> compute
> if
> for
> send
> ...
> 
> ```
> 
> That does not necessarily determine questions such as:
> 
> ```plaintext
> one file vs five files
> which module owns a calculation
> entrypoint name
> framework choice
> dependency layout
> CLI vs service
> public API shape
> 
> ```
> 
> Those are a different dimension.
> 
> IPL Studio already partly recognizes this through form-factor/topology guidance and gates, so I wonder if making that separation first-class would actually preserve the minimalism of IPL:
> 
> ```plaintext
> Intent contract
> "what must happen"
> 
> Architecture contract
> "what implementation shape is permitted"
> 
> Lowering
> "produce target-language source that satisfies both"
> 
> ```
> 
> This is conceptually close to the broader specification-driven-development pattern where “what the system must do” and “technical implementation plan” are distinct artifacts. It does not require making IPL itself into a large architecture language.
> 
> It also gives a useful evaluation rule:
> 
> ```plaintext
> different topology + same behavior
> → possibly permitted architecture freedom
> 
> different behavior
> → contract underspecification or lowering error
> 
> same intended topology + different actual topology
> → architecture-contract failure
> 
> ```
> 
> That seems more informative than counting all three cases simply as nondeterminism.

> **I would also consider an IPL → code semantic-preservation receipt**
>
> One thing the small probe made very visible is that an LLM can understand an IPL statement approximately while still changing a detail that is contractual.
> 
> For example:
> 
> ```plaintext
> grandTotal
> 
> ```
> 
> becoming:
> 
> ```plaintext
> grand_total
> 
> ```
> 
> is perfectly idiomatic Python, but it is wrong if `grandTotal` is part of the externally visible schema.
> 
> Likewise, these are different classes of error:
> 
> ```plaintext
> round(x * 100) / 100
> 
> ```
> 
> versus an incorrectly lowered rounding expression;
> 
> or:
> 
> ```plaintext
> number
> 
> ```
> 
> becoming a formatted string before the output boundary.
> 
> So perhaps each generation could cheaply produce a small preservation receipt:
> 
> ```plaintext
> [] required identifiers/keys preserved
> [] required types preserved
> [] formulas preserved
> [] branch/control-flow relationships preserved
> [] required entrypoint exists
> [] imports/dependencies close
> [] runtime behavior passes
> 
> ```
> 
> The nice thing about this is that it helps IPL rather than requiring a redesign. It turns the intermediate representation into something that can be audited against its output.
> 
> There is a related idea in validated whole-project code translation work: instead of accepting a large translation because it compiles, translation can be decomposed and checked for localized semantic equivalence and type compatibility. The domains are different, but the principle — **validate the mapping itself, not only the final artifact** — seems very applicable here.

> **Where this seems to sit relative to some existing work**
>
> I do not mean these as “IPL already exists under another name”; the interesting part of IPL seems to be the particular combination of a tiny, readable intent vocabulary with polyglot project generation.
> 
> But there are useful neighboring patterns.
> 
> ### Microsoft DSL Copilot
> 
> [DSL Copilot](https://github.com/microsoft/dsl-copilot?utm_source=chatgpt.com) gives the model a DSL grammar/examples, generates DSL code, runs a compiler, and feeds compiler diagnostics back for retry.
> 
> That is a useful precedent for the general pattern:
> 
> ```plaintext
> constrain
> → generate
> → mechanically validate
> → repair from diagnostics
> 
> ```
> 
> rather than expecting the grammar alone to remove all hallucination. Its own documented flow explicitly includes compilation and retry.
> 
> ### TypeChat
> 
> [TypeChat](https://github.com/microsoft/TypeChat?utm_source=chatgpt.com) takes another nearby route: user intent is translated to a typed representation, validated, and — if validation fails — the diagnostics are sent back for a repair attempt.
> 
> I find its distinction useful here: a representation can be parsable and schema-valid while still requiring application-level semantic checks. The type/schema boundary limits what the LLM can hand to the application, but it does not magically prove every meaning of the data.
> 
> ### Text-to-DSL work
> 
> Recent Text2DSL work is useful evidence for the more modest proposition that giving an LLM an explicit formal target language — BNF grammar, API information, and a permitted vocabulary — can substantially improve DSL generation validity.
> 
> I would treat that as support for IPL’s **constraint/context idea** , not evidence that a particular 13-verb vocabulary guarantees deterministic arbitrary software generation.
> 
> ### Hard constrained decoding
> 
> There is also a stronger option than prompt-level grammar guidance: structured/constrained decoders can mask tokens that violate a grammar or schema during generation.
> 
> That could be interesting if a future IPL layer needs an **authoritative machine-valid artifact**.
> 
> But I would not assume that “harder is always better.” Work such as [CRANE](https://arxiv.org/abs/2502.09061?utm_source=chatgpt.com) studies exactly the tension between restrictive constrained generation and the reasoning freedom the model sometimes needs. Its result is a good reason to think in terms of _where_ to place the hard boundary, rather than constraining every stage equally.
> 
> A possible hybrid is therefore:
> 
> ```plaintext
> free(er) planning/reasoning
> ↓
> strict authoritative IR
> ↓
> validated lowering
> 
> ```
> 
> rather than requiring one representation to serve all three roles.

> **About the '13 verbs' question**
>
> I would probably **not expand the vocabulary just because edge cases can be imagined**.
> 
> In the existing curated IPL corpus I checked, I did not see ordinary semantic statements constantly falling through into arbitrary generic prose. The generic nodes I found were essentially imports. So I do not see evidence that the current vocabulary is already obviously too small.
> 
> At the same time, a corpus written with the grammar in mind cannot establish that the vocabulary is universal.
> 
> A measurement I would find more useful than arguing about the number 13 is something like **semantic escape rate** :
> 
> > When a new real-world requirement is encoded, how much of the meaning lands in canonical IPL structure, and how much has to survive as prose or be reconstructed from the model’s prior?
> 
> If that rate stays low across new domains, the minimalist claim becomes stronger without adding verbs.
> 
> If it rises sharply around some recurring concept — transactions/rollback, authorization/capabilities, retry/idempotency, temporal ordering, concurrency, invariants, etc. — that identifies a concrete reason to add or separate a construct.
> 
> So I would treat “13 verbs are sufficient” as a testable design hypothesis, not something that needs to be proven or abandoned up front.

> **A small terminology/evaluation note on 'deterministic'**
>
> I think the word is workable as long as its scope is stated.
> 
> For example:
> 
> ```plaintext
> IPL determinism
> ≠ architecture determinism
> ≠ source-text determinism
> ≠ behavioral equivalence
> ≠ inference-backend reproducibility
> 
> ```
> 
> Even below the IPL layer, inference systems have their own reproducibility conditions. For example, vLLM treats batch-invariant/reproducible execution as a separate runtime concern rather than something a prompt or seed universally guarantees.
> 
> So a statement such as:
> 
> > deterministic/reproducible under a fixed model, model revision, backend, configuration, and seed
> 
> is a different claim from:
> 
> > any implementation generated from this IPL must be semantically equivalent.
> 
> Keeping those separate would make positive results easier to interpret too.

If I had to pick only **one** next step, I would not start by adding more verbs or replacing the current pipeline.

I would make the existing scorecard more layer-aware:

```plaintext
1. IPL parse / semantic-contract checks
2. topology / entrypoint checks
3. identifier + type + formula preservation
4. project integration / dependency checks
5. first-pass runtime behavior
6. deterministic repair
7. LLM repair after diagnostics

```

Then the interesting claim becomes much easier to test:

> **Which classes of freedom does the minimalist IPL remove, and which classes remain in the LLM-controlled stages?**

To me, that is actually a stronger and more useful question than requiring IPL to make every generated source tree identical. The minimalist intent layer can remain minimalist, while the surrounding contracts progressively remove exactly the kinds of freedom that turn out to matter in practice.

---

<div class="post-metadata">

**Author:** ![Lecyberbill](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lecyberbill/32/39845_2.png) [@Lecyberbill](https://discuss.huggingface.co/u/Lecyberbill)\
**Post date:** [August 25, 2026, 9:31am UTC](https://discuss.huggingface.co/t/a-minimalist-dsl-to-enforce-deterministic-code-generation-with-llms/179149/3 "2026-08-25T09:31:36Z")

</div>

Thank you so much for this extremely detailed, insightful, and constructive feedback. I really appreciate that you took the time to test things in Colab and lay out such a clear and rigorous breakdown.

You hit the nail right on the head regarding what has been the main weak spot of the project so far: **the ability to objectively measure, evaluate, and quantify the approach.**

While I had the intuition and empirical evidence from my own use cases that constraining generation with a minimalist DSL drastically cuts down stochastic noise, I was missing this exact methodological framework to decouple and measure where degrees of freedom remain versus where determinism actually operates.

Your insights are going to be immensely helpful in several concrete ways:

1. **A layer-aware scorecard:** Moving away from a simple binary `PASS/FAIL` to track semantic preservation, topology conformance, and repair dynamics (first-pass vs. deterministic repair vs. LLM repair) is exactly the right direction.

2. **Contract decoupling:** Clarifying what belongs to the intent contract (IPL) versus the architecture/topology contract strongly reinforces my decision to **keep the minimalist vocabulary without adding more verbs** , and instead better frame the lowering and verification stages.

3. **The baseline protocol:** Measuring variance against an equal-information natural-language baseline to properly isolate and showcase the DSL’s actual contribution.

Your analysis provides a crystal-clear roadmap for the next benchmarking iterations and the formalization of IPL. Thanks again for such a high-value contribution!