xgrammar0.1.18
xgrammar0.1.18
Published
Efficient, Flexible and Portable Structured Generation
pip install xgrammar
Package Downloads
Authors
Requires Python
<4,>=3.8
Dependencies
- pydantic
- sentencepiece
- tiktoken
- torch
>=1.10.0 - transformers
>=4.38.0 - triton
; platform_system == "Linux" and platform_machine == "x86_64" - mlx-lm
; platform_system == "Darwin" and platform_machine == "arm64" - ninja
- pytest
; extra == "test" - protobuf
; extra == "test" - huggingface-hub
[cli]; extra == "test" - transformers
<4.50.0; platform_system == "Darwin" and extra == "test"
XGrammar
Efficient, Flexible and Portable Structured Generation
News
- [2025/02] XGrammar has been officially integrated into Modular's MAX
- [2025/01] XGrammar has been officially integrated into TensorRT-LLM.
- [2024/12] XGrammar has been officially integrated into vLLM.
- [2024/12] We presented research talks on XGrammar at CMU Catalyst, Berkeley SkyLab, MIT HANLAB, THU IIIS, SJTU, Ant Group, SGLang Meetup, Qingke AI, Camel AI. The slides can be found here.
- [2024/11] XGrammar has been officially integrated into SGLang.
- [2024/11] XGrammar has been officially integrated into MLC-LLM.
- [2024/11] We officially released XGrammar v0.1.0!
Overview
XGrammar is an open-source library for efficient, flexible, and portable structured generation. It supports general context-free grammar to enable a broad range of structures while bringing careful system optimizations to enable fast executions. XGrammar features a minimal and portable C++ backend that can be easily integrated into multiple environments and frameworks, and is co-designed with the LLM inference engine and enables zero-overhead structured generation in LLM inference.
Get Started
Please visit our documentation to get started with XGrammar.