LLM Infrastructure, ranked by momentum

What’s trending in LLM Infrastructure: the projects accelerating fastest right now — ranked by how quickly their growth is speeding up, not by how many stars they already have.

63 tracked projects · updated daily · 1 Oct 2026

#ScoreProjectAcceleratingStars

195

unslothai/unslothUnsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

+13.676.7k 290

syv-ai/qwen38-27b-rtx3090Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks

+31.31.7k 390

ggml-org/llama.cppLLM inference in C/C++

+2.4129.4k 489

vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs

+2.492.6k 585~

youssofal/MTPLXThe fastest way to run Qwen 3.8 Flash Next and Qwen 3.8 27B on a Mac: 125 tok/s in OpenCode on an M5 Max. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

+10.02.4k 683

mudler/LocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

+1.449.3k 783

AtomicBot-ai/Atomic-ChatLocal AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

+14.11.6k 882

FareedKhan-dev/kimi-k3-in-cA 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

+3.18.5k 980

flashinfer-ai/flashinferFlashInfer: Kernel Library for LLM Serving

+0.96.5k 1078

hiyouga/LlamaFactoryUnified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

+4.675.0k 1172

skypilot-org/skypilotThe AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

+1.010.7k 1265

tingaicompass/AI-Compass“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向，无论你是初学者还是进阶开发者，都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势，并通过实践掌握从理论到落地的全过程。

+1.6962 1364

liguodongiot/llm-action本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

+1.725.1k 1464

linkedin/Liger-KernelEfficient Triton Kernels for LLM Training

+0.66.6k 1563

spiceai/spiceaiAdd a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

+0.93.1k 1661

gpustack/gpustackA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

-0.15.8k 1760

alibaba/rtp-llmRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

+0.61.4k 1858

vllm-project/vllm-ascendCommunity maintained hardware plugin for vLLM on Ascend

-0.12.9k 1958

lm-sys/FastChatAn open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

+1.339.6k 2057

thu-pacman/chituHigh-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

+0.63.0k 2155

bentoml/OpenLLMRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

+0.612.5k 2253

kubeflow/trainerDistributed AI Model Training and LLM Fine-Tuning on Kubernetes

-0.12.2k 2353

ome-projects/omeOpen Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

+0.3512 2453

axolotl-ai-cloud/axolotlGo ahead and axolotl questions

-0.712.5k 2552

bentoml/BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

+0.18.9k

View all 63 on the board →

What counts as LLM Infrastructure here

Inference servers, fine-tuning, and runtimes for large language models. This page lists only projects whose primary category is LLM Infrastructure; projects that merely connect to it are listed under Related projects below. Rankings cover the 1017 repositories Breakwave tracks, not all of GitHub.

Related projects (22)

These belong to other categories first, but show up when people explore LLM Infrastructure.

What counts as LLM Infrastructure here

Related projects (22)

Other categories