draft-models

Here are 3 public repositories matching this topic...

xMinCHsia / Marlin

Speculative decoding orchestrator for LLM serving: draft-model ensemble with rejection sampling, KV-cache-aware admission control and self-tuning draft lengths

golang performance orchestration developer-tools rejection-sampling serving kv-cache llm-inference speculative-decoding draft-models

Updated Sep 12, 2026
Go

pazara18 / DuckBoard

Star

Minimal server-rendered dashboard framework with zero dependencies - layouts, widgets, and escape-safe rendering in plain Node.js

golang performance orchestration developer-tools rejection-sampling serving kv-cache llm-inference speculative-decoding draft-models

Updated Sep 8, 2026
JavaScript

Crynge / SpecInferKit

Star

Production-grade speculative decoding toolkit for LLMs — multi-algorithm, FP8 quantization, online training, distributed serving, and comprehensive evaluation suite.

python nlp benchmarking machine-learning deep-learning eagle pytorch medusa gpu-acceleration distributed-training model-acceleration inference-optimization ai-inference llm-serving llm-inference speculative-decoding fp8-quantization draft-models

Updated Jul 1, 2026
Python

Add this topic to your repo

To associate your repository with the draft-models topic, visit your repo's landing page and select "manage topics."

Learn more