vllm · 46 topics

alibaba shrinks qwen3 32b to fit on a 24gb consumer gpu

edge0 audio8 asr infinite

vericache lossless kv cache

cohere labs north small translate beats deepl and google translate across 50 languages

deploying qwen3 8 2 4t a95b on amazon sagemaker hyperpod with vllm

qwen3 8 flash next fp8

qwen3 tts speed cost frontier

ornith 1 5 35b a3b gguf

z lab qwen3 8 27b dflash2

qwen3 8 2 4t a95b fp8

unsloth qwen3 8 27b nvfp4

qwen qwen3 8 27b

tencent open sources angelspec

nvidia rtx spark local ai agents

qwen3 6 27b dflash speculative decoding

unsloth qwen3 6 27b gguf

build real time voice applications with vllm omni on sagemaker ai

ukisai swift qwen38 27b gguf

aiperf llm benchmarking

vllm boosts kimi k3 throughput

dealignai glm 53 cybersecurity fp8

nvidia qwen3 flash next nvfp4

reduce llm latency with prefix aware routing on amazon sagemaker inference

coheres open source megakernel beats vllm

speculative decoding in vllm on amd gpus

k2 horizon local verify

vllm v0280 release

vllm v0 28 0 sparse attention speculative decoding

llms could control their host machines by exploiting inference engines

nvidia dynamo shadow engine recovery

why your local llm feels dumber than it is

vllm dspark deepseek v4

the inference engineering masterclass

airllm 70b inference single 4gb gpu

poolside laguna s 2 1 rate limits

mac laguna ollama llama cpp vllm

deploying kimi k3 on aws

vllm v0 26 0 inkling 1t

poolside laguna s 21 nvfp4

native speed vllm transformers backend

internscience agents a1

amd strix halo rdma cluster setup guide

unlimited ocr

tiny vllm cpp cuda inference engine

advanced quantization algorithm for llms

random attention rethinking kv cache eviction for efficient reasoning