Awesome-LLM-Constrained-Decoding icon indicating copy to clipboard operation
Awesome-LLM-Constrained-Decoding copied to clipboard

Published 4 months ago • verified publisher icon Saibo-creator
→

Metadata

A curated list of papers related to constrained decoding of LLM, along with their relevant code and resources.

Awesome-LLM-Constrained-Decoding

Towards reliable, controllable and more efficient generation with Large Language Models (LLMs)

A curated list of papers related to constrained decoding of LLM, along with their relevant code and resources.

Table of Contents

  • Awesome-LLM-Constrained-Decoding
    • Table of Contents
    • Libraries
    • Papers
    • Benchmark & Datasets & Evaluation
    • Survey
    • Blog Posts
    • Related Awesome Lists
    • Disclaimer
    • Contributing

Libraries

Library Feature Stars
guidance-ai/guidance CFG, Regex, JSON Schema, Token Forcing, compatible with Transformers, LLAMA-CPP Stars
outlines-dev/outlines CFG, Unicode support, Hugging Face ecosystem, VLLM support Stars
eth-sri/lmql Regex support, various constraints, more powerful control flow Stars
jxnl/instructor Try-Reject-Repeat approach to ensure constraints are met Stars
microsoft/aici A general framework of LLM controller with native support for CFG, Regex, JSON Schema Stars
noamgat/lm-format-enforcer Regex, JSON Schema, Beam Search etc. Stars
mlc-ai/xgrammar CFG, careful system optimizations Stars
epfl-dlab/transformers-CFG CFG (EBNF Interface), Compatible with Transformers, Easy to extend for research Stars
uiuc-focal-lab/syncode CFG generation that supports builtin grammars like JSON, Python, Go, and more Stars
Dan-wanna-M/formatron Regex, JSON Schema, CFG, etc Stars
genlm/genlm-control Arbitrary programmable syntactic and semantic constraints, Constrained decoding as posterior inference, Sequential Monte Carlo Stars
structuredllm/itergen CFG generation and backtracking to handle semantic constraints Stars
eth-sri/type-constrained-code-generation TypeScript, including type-safety. Stars
snowkylin/circuit-transformer Generating logical circuits that are strictly equivalent to given Boolean functions. Stars
epfl-dlab/jsonschemabench A benchmarking framework for evaluating constrained decoding engines on JSON Schema. Supports Guidance, Outlines, XGrammar, OpenAI and more. Stars
eth-sri/constrained-diffusion Diffusion LLMs, Multi-Region Infilling (CFG) Stars
grammarllm LLprefix, DPDA Stars

Disclaimer:

  • The libraries listed above are not exhaustive and are subject to change.
  • The features mentioned are 100% not exhaustive and I strongly recommend checking the respective repositories for more details.
  • The libraries are listed by the Github stars
  • If you are the author of a library and would like to add or update the information, please open an issue or submit a pull request.

Papers

Papers with are newly added papers (not necessarily newly published papers).

Constrained Decoding for Autoregressive Models

Date Paper Publication
2025-07 Grammar-Constrained Decoding Makes Large Language Models Better Logical Parsers ACL
2025-07 GRAMMAR-LLM: Grammar-Constrained Natural Language Generation ACL Findings
2025-07 Pre3: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation ACL
2025-07 Syntactic Control of Language Models by Posterior Inference ACL Findings
2025-06 Earley-Driven Dynamic Pruning for Efficient Structured Decoding ICML
2025-05 Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling Preprint
2025-05 Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo ICLR
2025-04 Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Models AISTATS
2025-02 CRANE: Reasoning with constrained LLM generation ICML
2025-05 Type-Constrained Code Generation with Language Models PLDI
2025-02 Lost in Space: Optimizing Tokens for Grammar-Constrained Decoding Preprint
2025-02 Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema Adherence Preprint
2025-02 Flexible and Efficient Grammar-Constrained Decoding Preprint
2025-01 Circuit Transformer: A Transformer That Preserves Logical Equivalence ICLR
2025-01 Generating Structured Outputs from Language Models: Benchmark and Studies Preprint
2024-11 XGRAMMAR: FLEXIBLE AND EFFICIENT STRUCTURED GENERATION ENGINE FOR LARGE LANGUAGE MODELS Preprint
2024-10 IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking ICLR
2024-08 Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models Preprint
2024-08 FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking Preprint
2024-07 Automata-based constraints for language model decoding CoLM
2024-06 Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Access ACL
2024-05 Grammar-Aligned Decoding Preprint
2024-03 SynCode: LLM Generation with Grammar Augmentation Preprint
2024-03 Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation ICML
2024-02 Constrained Decoding for Cross-lingual Label Projection ICLR
2024-02 Constrained Decoding for Code Language Models via Efficient Left and Right Quotienting of Context-Sensitive Grammars Preprint
2024-02 Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents Preprint
2023-12 SGLang: Efficient Execution of Structured Language Model Programs Preprint
2023-12 Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context NeurIPS
2023-11 Prompt Sketching for Large Language Models Preprint
2023-11 Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs PADL
2023-10 Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decoding Preprint
2023-10 Amortizing intractable inference in large language models ICLR
2023-10 Terminology-Aware Translation with Constrained Decoding and Large Language Model Prompting EMNLP
2023-10 KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection EMNLP
2023-10 Lazy-k Decoding: Constrained Decoding for Information Extraction EMNLP
2023-07 Efficient Guided Generation for Large Language Models Preprint
2023-06 Grammar Prompting for Domain-Specific Language Generation with Large Language Models NeurIPS
2023-06 Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning EMNLP
2023-06 Prompting Is Programming: A Query Language for Large Language Models PLDI
2023-05 Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing EMNLP Findings
2023-04 Tractable Control for Autoregressive Language Generation ICML
2022-11 Validating Large Language Models with ReLM MLSys
2022-11 CodePAD: Sequence-based Code Generation with Pushdown Automaton ISSTA
2022-05 Controllable Text Generation with Neurally-Decomposed Oracle NeurIPS
2022-05 Gradient-Based Constrained Sampling from Language Models EMNLP
2022-02 COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics NeurIPS
2022-01 Synchromesh: Reliable code generation from pre-trained language models ICLR
2021-12 PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models EMNLP
2021-12 Constrained Language Models Yield Few-Shot Semantic Parsers EMNLP
2021-12 Controlled Text Generation as Continuous Optimization with Multiple Constraints NeurIPS
2021-06 NEUROLOGIC DECODING:(Un)supervised Neural Text Generation with Predicate Logic Constraints NAACL
2019-05 A General-Purpose Algorithm for Constrained Sequential Inference CoNLL
2019-05 Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting NAACL
2018-09 CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling AAAI
2018-05 Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation NAACL
2018-04 Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method AAAI
2017-12 Guided Open Vocabulary Image Captioning with Constrained Beam Search EMNLP
2017-06 Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search ACL

Constrained Decoding for Diffusion Models

Date Paper Publication
2025-08 Constrained Decoding of Diffusion LLMs with Context-Free Grammars Preprint
2025-05 DINGO: Constrained Inference for Diffusion LLMs Preprint
2025-03 Constrained Language Generation with Discrete Diffusion Models Preprint

Benchmark & Datasets & Evaluation

Date Paper Publication
2025-08 JSON-mode Eval Cleaned/Extended, SMILES Eval and HumanEval MRI C++ HF Hub
2025-01 JsonSchemaBench: Generating Structured Outputs from Language Models: Benchmark and Studies Preprint
2024-05 COLLIE: Systematic Construction of Constrained Text Generation Tasks ICLR
2024-02 JSON-mode Eval dataset HF hub
2023-12 BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing NeurIPS Track on Datasets and Benchmarks
2023-10 Evaluating Large Language Models on Controlled Generation Tasks Preprint
2023-09 Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? Preprint
2021-10 NLV corpus CHI
2020-12 CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning EMNLP Findings
2018-09 Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task EMNLP

Survey

Date Paper Publication
2024-04 "We Need Structured Output": Towards User-centered Constraints on Large Language Model Output Preprint

Blog Posts

Many of the blogs are written by Outlines team, many thanks to them for their great work! ❤️

Related Awesome Lists

Disclaimer

This list is not exhaustive and will be updated regularly. If you have any suggestions or want to add a paper, please feel free to open an issue or submit a pull request. We hope to include all relevant papers in this list.

Contributing

Contributions are welcome! Feel free to submit a pull request or open an issue. Please make sure to read the Contributing Guidelines before contributing.

← Metadata

295
Stars
13
Forks
295
Watchers

Owner

verified publisher icon Saibo-creator

Metadata

A curated list of papers related to constrained decoding of LLM, along with their relevant code and resources.

Back