M5 Air 16GB 本地跑 33B 视频模型：vpipe 实测，15 分钟生成 3.75 秒有声视频

by Mycelium Protocol

---

GitHub：https://github.com/tgo-app-dev/vpipe
许可证：Apache 2.0
语言：C++20
最新版本：v0.1.22
实测机型：M5 MacBook Air 16GB（fanless）

---

在 M5 Air 16GB 上，把一段文字变成一个 3.75 秒、960×544、24fps 的有声视频，需要多长时间？

实测答案是约 15 分钟。视频和音轨不是分两个模型生成再拼起来的，是同一个 33B 的去噪循环产生的——问场景里有没有雨声，你就能听到雨声，因为模型知道自己同时在画什么。

这件事是 vpipe 做到的，一个纯 C++ 写成的苹果芯片多模态 AI 运行时，build 产物不到 25MB，没有 Python，没有第三方 tensor 运行时参与前向计算。

---

视频和音频，一个去噪循环

MiniMax H3（FL2VA）是整个项目里最不寻常的部分。

大多数文生视频方案是：先跑视频扩散模型，再拿一个单独的音频模型配音。H3 不是这样——它只有一个去噪序列，同时携带视频 latent 和音频 latent，从同一个噪声开始，在同一个 transformer 里被处理。这意味着音频的生成时刻知道画面正在变成什么。

pipeline 图只有 8 个 stage，但拓扑上有一个关键的分叉：

text-prompt → diffusion-conditioner → generate-video ─┬─ port 0 ─→ vae-decode → rgb-to-video ─┐
                                                        │                                        ├─→ save-video (mp4)
                                                        └─ port 1 ─→ audio-vae-decode ───────────┘

generate-video 发出两条输出流——port 0 是视频 latent，port 1 是音频 latent——分别解码后在 save-video 里被 mux 进同一个 mp4。单条提示词里的声音描述决定了两者的内容：

An Asian musician playing classical music on a grand piano.

钢琴声来自「playing classical music on a grand piano」这几个词，没有单独的音频提示，也没有第二次前向计算——这是 guidance 蒸馏的结果。

---

为什么 8 步够用：guidance 蒸馏

标准扩散 transformer 需要 30+ 步才能得到可用结果，因为每一步都要跑两次前向：一次有条件，一次无条件，用差值做 classifier-free guidance。33B 模型跑一次已经很贵，跑两次更不现实。

H3 是 guidance 蒸馏模型，训练时把这两次前向的效果压进了权重本身。结果：

• 不需要无条件前向，步数砍到 4–8
• guidance_scale 参数和 negative prompt 在这里是无效的——vpipe 检测到蒸馏模型后直接跳过，不多付 33B 的计算代价
• 4 步可以快速看提示词效果，8 步出最终结果

实测参数：steps=8，width=960，height=544，frames=120（实际被 VAE chunk size 向上取整到 124 帧，即 5.16 秒，但视频文件里是 3.75 秒——可能是实测时用了更少帧数）

---

混合精度量化：为什么 4-bit 还需要 8-bit

H3 量化有一个不寻常的设计，值得记住。

MiniMax H3 的 33B 参数里，有 13B 是 AdaLN 的调制投影层（per-block 的 scale/shift）。如果和 transformer body 一样打成 4-bit，「4-bit」版本会有 ~36GB，量化几乎没有意义。

vpipe 的解决方案：body 4-bit，AdaLN 调制层 8-bit，加载器按 per-tensor 的位宽自动处理，不需要任何配置。最终模型大小约 45GB，准备流程：

1. 下载 Comfy-Org/MiniMax-H3（bf16，~115GB）
2. model-quantize: body → 4-bit，调制层 → 8-bit
3. model-quantize: 文字编码器（Qwen3-VL-32B）→ 4-bit
4. model-remove: 删掉中间产物
5. 保留 ~45GB 成品，115GB 下载可以删掉

VAEs（视频 VAE + 音频 VAE）在量化时被 hard-link 进输出目录，不复制，所以「下载 + 量化中峰值」是 ~155GB 而不是 115+45GB。

---

重量流式加载（weight streaming）

M5 Air 只有 16GB。33B 模型量化后 45GB，怎么运行？

vpipe 的答案是分块流式加载——把权重分成若干块，每次只把当前前向需要的块从磁盘载入 unified memory，完成后换下一块。这不是普通的「模型分片」，是 per-inference 的动态流式，和 Apple Silicon 的 unified memory 架构配合才能实现实用的速度。

关键 stage 配置：unload_when_idle: always——推理完成后立刻释放权重，让下一个 stage（vae-decode）有足够内存运行。pipeline 里各 stage 是流水线化的，不是串行占满内存。

---

M5 NAX 加速

M5 generation 的 Neural Arithmetic eXtension（NAX，也称为 P-cores 的新型 SIMD 扩展）增加了专用的 matmul2d 和 convolution2d 单元。vpipe 在编译时检测硬件，M5 上自动使用这些路径。README 里提到 M5 Air 实测比 M4 Air（同等 RAM）快，部分来源于 NAX。

---

技术架构：三层，C++ 到底

Pipeline 核心（可移植）：基于 coroutine 的 Job stage，通过 buffered port 连接，从 JSON spec 编排，每种 stage 注册一个类型名（rtsp-capture、generate-video、audio-vae-decode 等）。这一层在 Linux 和 Intel macOS 都能编译。

On-device 生成模型栈（Apple Silicon 专属）：从零写的 LLM/VLM/ASR/扩散/视频推理，metal-compute 自定义后端，自己的 Metal kernel（GEMM、attention、量化……），借用了 Apple MLX 的 steel GEMM/attention kernel 头文件但不链接 MLX，不使用第三方 tensor 运行时。

Web UI + Composer：自包含的浏览器 UI，嵌在二进制里，支持 pipeline 编排、运行、剖析、layout 保存。手机布局自动适配，二维码一扫直接认证。

整个 build 产物：< 25MB（不含 FFmpeg）。

---

实测参数（M5 Air 16GB）

参数	值
机型	M5 MacBook Air 15” 16GB（无风扇）
分辨率	960 × 544
帧率	24 fps
时长	3.75 秒
步数	8 steps
模型	MiniMax H3 FL2VA 4-bit
耗时	约 15 分钟
冻结风险	有。持续高负载会热节流；原 README 的「13 分钟」是冰袋辅助散热的结果

无风扇 MacBook Air 在长时间高负载下会降频，这是苹果硬件设计的权衡，不是 vpipe 的 bug。状态栏会显示 Throttling——看到它说明速度是由硬件热量决定的，不是卡死。

---

其他支持的模型和 pipeline

从 topics 和文档可以看出当前支持的能力矩阵：

模型/场景	stage 类型
MiniMax H3（33B 文生视频+音频）	generate-video + audio-vae-decode
FLUX.2 Klein（图像生成/编辑）	generate-image、diffusion-conditioner
Qwen3.5-9B/4B（LLM chat + VQA）	text-chat、visual-qa、realtime-vqa
Gemma4	LLM chat
KREA2	图像编辑
Qwen-VL（图像编辑）	qwen-image-edit
语音识别（ASR）	audio-transcribe
TTS	文字转语音
ONVIF/RTSP 摄像头	rtsp-capture、onvif-discovery
YOLO 检测	yolo-detection

本地 MCP：沙盒化的文件、Shell、Python 工具 + 网络 fetch，全部在 pipeline 里可用。

---

快速开始

# 构建
git clone --recursive https://github.com/tgo-app-dev/vpipe.git
cd vpipe && cmake -S . -B build && cmake --build build -j

# 准备 MiniMax H3（下载约 115GB，量化后保留 ~45GB）
cd ~/vpipe-work
cp ~/src/vpipe/docs/pipelines/prepare-minimax-h3-4bit.vpipeline .
~/src/vpipe/build/apps/vpipe/vpipe --launch prepare-minimax-h3-4bit.vpipeline

# 启动 Web UI，手机扫码接入
~/src/vpipe/build/apps/web-ui/vpipe-web-ui --show-qr
# 加载 minimax-h3-text-to-video.vpipeline，编辑提示词，Start

macOS 用户可以直接下载签名 app（v0.1.22，with-ffmpeg 版约 26MB，slim 版约 14MB），拖入 Applications，需要 macOS 26 和 Apple Silicon。

---

为什么这个项目值得关注

工程密度罕见。C++ 写一个同时支持视频扩散、LLM chat、实时 VQA、ASR/TTS、ONVIF 摄像头、本地 MCP 工具的运行时，整个 build 产物 25MB——这是一个不愿意妥协的性能优先工程风格，在 AI 应用里极少见。

视频+音频同步生成。这不是噱头：单循环联合生成意味着声音和画面从物理上不可能脱节，这和「生成视频再配音」在质量上是本质区别。

16GB 跑 33B 视频模型。这需要权重流式加载、混合精度量化、unified memory 的深度适配——三件事都做对了，才有 15 分钟这个数字。

本地就是一切。没有 API key，没有计量，没有数据出门，15 分钟换来一个属于自己机器的视频生成能力。

---

Mycelium Protocol — 追踪 AI 系统的底层演化

---

关于 Mycelium

菌丝协议。持续追踪 AI 工具、系统和实验的内容节点。

---

M5 Air 16 GB Runs a 33B Video Model Locally: vpipe Real-World Test — 15 Minutes for 3.75 Seconds of Video with Sound

by Mycelium Protocol

---

GitHub: https://github.com/tgo-app-dev/vpipe
License: Apache 2.0
Language: C++20
Latest: v0.1.22
Test hardware: M5 MacBook Air 16 GB (fanless)

---

On a 16 GB M5 MacBook Air, how long does it take to turn a text prompt into 3.75 seconds of 960×544 24 fps video — with sound?

Real-world answer: about 15 minutes. Video and soundtrack are not generated by two separate models then stitched together. They come from the same 33B denoising loop — ask for rain and you can hear it, because the model is generating both at once.

That is what vpipe does: a pure C++ multimodal AI runtime for Apple Silicon, build artifact under 25 MB, no Python, no third-party tensor runtime in the forward pass.

---

Video and Audio in One Denoising Loop

MiniMax H3 (FL2VA) is the most unusual part of the project. Most text-to-video pipelines generate video from a diffusion model and then add audio with a separate model. H3 doesn’t work that way — it has one denoising sequence that simultaneously carries both video and audio latents, processed together in the same transformer. This means the audio is generated with knowledge of what the picture is becoming.

The pipeline graph has 8 stages with one critical fork:

text-prompt → diffusion-conditioner → generate-video ─┬─ port 0 ─→ vae-decode → rgb-to-video ─┐
                                                        │                                        ├─→ save-video (.mp4)
                                                        └─ port 1 ─→ audio-vae-decode ───────────┘

generate-video emits two output streams — port 0 for video latents, port 1 for audio latents — which decode separately and meet again at save-video, muxed into a single mp4. The sound description in the text prompt determines both:

An Asian musician playing classical music on a grand piano.

The piano comes from “playing classical music on a grand piano.” There is no separate audio prompt and no second forward pass — this is guidance distillation.

---

Why 8 Steps Is Enough: Guidance Distillation

A standard diffusion transformer needs 30+ steps. Each step requires two forward passes: one conditioned, one unconditioned, with the difference used for classifier-free guidance. Running a 33B model once is expensive; running it twice per step on a fanless laptop is not practical.

H3 is a guidance-distilled model — training compressed those two forward passes into the weights themselves. The result: no unconditional forward pass, steps reduced to 4–8, and guidance_scale/negative prompt are inert (vpipe detects the distilled model and skips them rather than paying 2× on a 33B inference).

4 steps is enough to see what a prompt produces. 8 steps for a final output.

---

Mixed-Precision Quantization: Why 4-bit Still Needs 8-bit

H3’s 33B parameters include 13B of AdaLN modulation projections (per-block scale/shift). If those are quantized to 4-bit along with the transformer body, the “4-bit” checkpoint comes out at ~36 GB — almost no savings. vpipe’s solution: body at 4-bit, AdaLN modulation at 8-bit, with the loader detecting per-tensor bit widths automatically. Final model size: ~45 GB.

Preparation pipeline:

1. Download Comfy-Org/MiniMax-H3 (bf16, ~115 GB)
2. Quantize: body → 4-bit, modulation → 8-bit
3. Quantize: text encoder (Qwen3-VL-32B) → 4-bit
4. Remove intermediate checkpoint
5. Keep ~45 GB finished model; delete the 115 GB download

VAEs are hard-linked into the output directory rather than copied, so peak disk usage during preparation is ~155 GB (not 115 + 45).

---

Weight Streaming on 16 GB

A 45 GB quantized model on a 16 GB machine: weight streaming loads only the blocks needed for the current forward step into unified memory, releases them, then loads the next set. This is a per-inference dynamic stream, not a static model split — it requires deep integration with Apple Silicon’s unified memory architecture to stay practical. The critical config is unload_when_idle: always, which drops model weights between pipeline runs so the next stage (vae-decode) has the machine.

---

M5 NAX Acceleration

M5-generation hardware adds NAX (Neural Arithmetic eXtension) units: dedicated matmul2d and convolution2d hardware on the P-cores. vpipe detects the hardware at compile time and uses these paths on M5. Part of the speedup vs. M4 (same RAM) comes from NAX.

---

Architecture: Three Layers, C++ All the Way Down

Pipeline core (portable): Coroutine-based Job stages connected by buffered ports, orchestrated from a JSON spec. Each stage type registers a name (rtsp-capture, generate-video, audio-vae-decode, …). Builds on Linux and Intel macOS.

On-device generative stack (Apple Silicon): From-scratch LLM/VLM/ASR/diffusion/video inference. Custom Metal backend with custom kernels (GEMM, attention, quantization). Vendors a subset of Apple MLX’s steel GEMM/attention Metal kernel headers (compiled into embedded metallibs) but does not link MLX and does not use any third-party tensor runtime in the forward pass.

Web UI + Composer: Self-contained browser UI embedded in the binary. Supports pipeline arrangement, live inspection, profiling, and layout saving. Auto-adapts to phone. --show-qr prints a QR code — scan it to open the UI already authenticated, no key to retype.

Full build artifact: < 25 MB (excluding FFmpeg).

---

Real-World Numbers (M5 Air 16 GB)

Parameter	Value
Hardware	M5 MacBook Air 15” 16 GB (fanless)
Resolution	960 × 544
Frame rate	24 fps
Duration	3.75 s
Steps	8
Model	MiniMax H3 FL2VA 4-bit
Wall time	~15 minutes
Thermal note	Sustained load will cause throttling; the README’s “13 minutes” used an ice pack under the chassis

The status bar shows Throttling when sustained heat is limiting speed — this is hardware behavior, not a stall.

---

Why This Project Matters

Engineering density. C++ runtime supporting video diffusion, LLM chat, real-time VQA, ASR/TTS, ONVIF cameras, and local MCP tools — full build under 25 MB. This is a performance-first engineering discipline that is rare in AI applications.

True joint video+audio generation. Single-loop joint generation means audio and video are physically incapable of drifting out of sync. This is not a pipeline detail; it is a qualitative difference from “generate video, then add sound.”

33B video model on 16 GB. Weight streaming, mixed-precision quantization, and deep unified memory integration had to all be done right to reach this number.

Local means local. No API key, no metering, no data leaving the machine. Fifteen minutes of compute for a video generation capability that belongs to the hardware.

---

Mycelium Protocol — tracking the deep evolution of AI systems

© 2026 Mycelium Protocol. All rights reserved.