中文版

FreeToken（GitHub：https://github.com/FlashML-org/FreeToken ，2026-07-20 创建，今晨抓取时已 15,403★）是边缘原生的 MoE（混合专家）推理引擎，目标是把数据中心级的前沿开源模型搬到个人与消费级硬件上本地跑——README 的说法是：在游戏 PC 上以交互速度跑 290B+ 参数的 MoE 模型。

它的关键技术是把 GPU、CPU、内存和互连当成一个统一的弹性推理平台：带宽自适应的 CPU-GPU 协同执行、全局 LRU 专家缓存、运行时动态重分显存（专家缓存与 KV 内存之间切换不用重启引擎）、语义感知缓存（agent 的工具调用、思考块等上下文编辑不用重复算）。支持的模型包括 DeepSeek-V4-Flash、Qwen3.6-35B-A3B、GLM-5.2 等，量化格式覆盖 MXFP4、NVFP4、FP8、BF16；接口兼容 Anthropic/OpenAI，可直接接 Codex、Claude Code、OpenCode 等 agent；硬件覆盖 RTX 30/40/50 系列。提供 Windows/Linux 桌面 App 与 CLI 两种用法，Apache-2.0 开源。

为什么值得了解

• 本地跑大模型长期卡在“小模型才行”的刻板印象里；这个项目把 MoE 架构的稀疏特性用到了极致，让“前沿级模型本地跑”从口号变成可下载的引擎
• 对已有游戏 PC 的人尤其值得一试：不用租云 GPU、不用按量付费，自己的显卡就能成为推理机；且 API 兼容意味着现有 agent 工具链不用改就能接
• 但要理性看待“290B+”：它依赖 MoE 只激活部分专家的特性与充足的内存/显存配置，上手前先按仓库的硬件要求核对自己的机器，别只看标题数字

English Version

FreeToken (GitHub: https://github.com/FlashML-org/FreeToken , created 2026-07-20, 15,403 stars at this morning’s fetch) is an edge-native Mixture-of-Experts (MoE) serving engine built to run frontier-scale open-weight models on personal and consumer hardware — the README’s claim is 290B+ parameter MoE models running locally on a gaming PC at interactive speeds.

Its key idea is treating GPUs, CPUs, host memory and interconnects as one unified, elastic inference platform: bandwidth-adaptive CPU–GPU co-execution, a global LRU expert cache, dynamic VRAM re-allocation between expert caches and KV memory without engine restarts, and semantic-aware caching so agentic context edits (tool calls, thinking blocks) skip redundant recomputation. Supported models include DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2, with MXFP4, NVFP4, FP8 and BF16 quantization; its Anthropic/OpenAI-compatible APIs plug straight into Codex, Claude Code, OpenCode and similar agents; hardware coverage spans NVIDIA RTX 30, 40 and 50 series. It ships as a Windows/Linux desktop app or a CLI, under Apache-2.0.

Why it’s worth knowing

• Local model serving has long been stuck at “only small models work”; this project exploits MoE sparsity to the full, turning “frontier models, run locally” from a slogan into a downloadable engine
• Especially worth trying if you already own a gaming PC: no rented cloud GPU, no metered bills — your own graphics card becomes the inference machine, and API compatibility means existing agent tooling needs no changes
• Keep the “290B+” claim in perspective: it relies on MoE activating only a fraction of experts plus ample memory/VRAM, so check your machine against the repo’s hardware requirements before diving in rather than trusting the headline number