Rohan Paul

Open source meets type-safe AI, with TypeLLM. Chat models give you a reply and decision models give you a choice, but TypeLLM is built to run your whole workflow in one API call. Most production pipelines need something in between: several typed answers, some depending on others, a few that should exist only under certain conditions. TypeLLM takes that whole shape as one JSON Schema and executes it as a dependency graph over a single shared context. Typed LLM outputs, built in the open. You can get schema-guaranteed outputs from a stock Qwen checkpoint without touching its weights. The guarantee comes from how the requests are built and decoded. TypeLLM sits on SGLang and turns a JSON Schema into constrained decoding: one scored token for an enum or boolean, bounded digits for numbers, and a dependency graph that reuses cached prefixes. So a field can’t return a value outside its enum, because the decoder only considers the allowed candidates. https://x.com/TypeLLM/status/2110281195380736292/video/1