# 腾讯混元发布了Hy3模型的GGUF量化版本，支持llama.cpp直接运行和MTP自推测解码，方便开发者部署

- 来源：Tencent Hunyuan
- 发布时间：2026-07-15 11:57
- AIWatch 分数：66
- AIWatch 标记：当日精选
- AIWatch 链接：https://aiwatch.icu/events/evt_01ky1erpbnr22jza11s9ydkgbe
- 原文链接：https://x.com/TencentHunyuan/status/2077241060523426067

## 精选理由

常规快讯，保留列表

## AI 摘要

腾讯混元发布了Hy3模型的GGUF量化版本，支持llama.cpp直接运行和MTP自推测解码，方便开发者部署。

## 正文

AngelSlim

 Dedicated to building a more intuitive, comprehensive, and efficient LLMs compression toolkit.

 📖 Documentation   |   🤗 Hugging Face   |   🤖 ModelScope   |   💬 WeChat

 Hy3 llama.cpp Quantization

Low-bit quantized Hy3 (hy_v3) GGUFs for llama.cpp — ready to run out of the box, with MTP self-speculative decoding. This repo ships the quantized models, plus the mixed-precision recipes to build your own from a calibration set.

Two parts below: Deploy (build & run) and Quantization.

 Quick Start

 Build

Clone the latest llama.cpp (must include the hy_v3 support merged in PR #25395 — i.e. any commit after 505b1ed; the current master is fine) and build it following the official build guide:

git clone https://github.com/ggml-org/llama.cpp llama.cpp-hyv3
# build per docs/build.md (CUDA / Metal / CPU as appropriate for your machine)

Binaries land in llama.cpp-hyv3/build/bin/.

Already downloaded an older GGUF? Earlier GGUFs here were built against a patched llama.cpp and won't load correctly on today's upstream. To build and run those, follow the previous README (the setup_hyv3_llama.sh + patches flow) instead.

 Run

# plain serve (no speculative decoding)
./llama.cpp-hyv3/build/bin/llama-server -m /path/to/Hy3.gguf -ctk q8_0 -ctv q8_0 -fa on -c 65536

# serve + MTP self-speculative decoding (needs an MTP gguf, i.e. converted WITHOUT --no-mtp)
./llama.cpp-hyv3/build/bin/llama-server -m /path/to/Hy3-mtp.gguf --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 1 -ctk q8_0 -ctv q8_0 -ctkd q8_0 -ctvd q8_0 -fa on -c 65536