MOSS Transcribe Diarize 0.9B

MOSS Transcribe Diarize 0.9B jointly generates transcript text, timestamps, and anonymous speaker labels. speech-swift now provides a native 128K MLX runtime for long recordings alongside the existing short-form CoreML backend.

At a Glance

Use

CoreML INT8 remains the compatibility default. Select MLX for the upstream long-context behavior; INT5 is the MLX default.

# CoreML INT8: short-form default.
.build/release/speech transcribe recording.wav --engine moss

# MLX INT5: dynamic 128K context.
.build/release/speech transcribe meeting.wav --engine moss --backend mlx

# MLX INT8 quality reference, using the default FP16 KV cache.
.build/release/speech transcribe meeting.wav --engine moss --backend mlx --model int8

Model Links

• MLX INT5 bundle
• MLX INT8 bundle
• CoreML INT8 bundle
• CoreML FP16 bundle
• speech-swift model docs
• speech-swift inference docs
• MLX INT5 / INT8 benchmark
• Technical report

How the 128K path works

The encoder still processes Whisper-sized 30-second chunks. The runtime concatenates every resulting audio embedding into one prompt before decoding, so speaker labels can use the complete recording context. Transcript tokens are then generated autoregressively; “single pass” does not mean simultaneous token generation.

INT5 vs INT8

INT5 is the recommended MLX default: its two weight files total 987.0 MiB instead of 1,200.1 MiB, with a 0.28-point English WER cost and effectively equal DER on this five-file slice. The benchmark used identical FP16 audio weights, greedy decoding, and FP16 KV caches.

The ASR fixture is the first 80 English-US test clips from google/fleurs. The diarization files are from the CC-BY-4.0 VoxConverse test mirror; they are not the Community-1 development slice used by the separate CoreML benchmark.

Limits and memory

• Audio consumes 12.5 context tokens per second, so 90 minutes is about 67,500 audio tokens before instructions and transcript output.
• At that audio-token count, the analytical KV-cache estimate is 7.74 GB FP16 or 4.11 GB INT8, before weights and temporary tensors.
• An earlier short-fixture check preserved structured output with INT8 KV cache. Current checks omit speaker labels; use FP16 KV for structured output. INT4 cache dropped timestamps and repeated a word on the earlier quality gate, so it is not exposed.
• The upstream model claims 50+ languages and up to about 90 minutes. The matched speech-swift comparison currently covers English ASR and five English VoxConverse recordings.
• Speaker labels are anonymous within each file and are not persistent speaker identities.

MLX buffer cache

From weight loading through transcription, MLX limits its process-wide reusable-buffer cache to the smaller of 2 GiB and 25% of physical RAM. Lower caller limits, including 0 to disable caching, are preserved. MOSS and Qwen3-ASR share this budget; releasing the final model restores the caller's prior limit unless the caller changed it during use.

This limits unused reusable buffers, not total memory. Live KV tensors and audio data still grow with recording length; weights and other runtime allocations require additional memory.

Use the default FP16 KV cache for structured speaker output. Current INT8 KV checks omit speaker labels with both weight variants on the original and capped runtimes; the buffer-cache ceiling does not fix this separate quality issue.