Benchmarks

Compression numbers on representative real-world checkpoints.

Setup

• Corpus
  • openai/gpt-oss-20b — MXFP4-quantized MoE, 13.76 GB raw (FP4 nibbles + E8M0 scales + BF16 layers).
  • nvidia/Llama-3_3-Nemotron-Super-49B-v1_5-NVFP4 — NVFP4-quantized Llama derivative, 31.07 GB raw (FP4 nibbles + FP8-E4M3 scales).
• Configurations
  • huffman_only — PTWM with the preprocessing graph forced to passthrough; pure byte-Huffman per tensor. Isolates the contribution of the preprocessing graph.
  • microscale_full — PTWM with the production preprocessing graph and the MICROSCALE method enabled.
• Hardware — CPU-only host, 16 cores, 64 GB RAM.

Compression ratios

Ratio is compressed bytes ÷ raw bytes. Lower is better.

Combined-corpus ratio (44.83 GB raw): microscale_full lands at 0.8881.

Throughput

The encode and decode pipelines run across a rayon thread pool. Full-model throughput through the safetensors file integration on a 32-core CPU host:

Decode — the model-load hot path — runs at 0.6–1.6 GB/s: every tensor in a shard is decoded across the pool in a single call. Encode favours ratio over speed by trial-encoding the full codec menu per plane; the opt-in compress_safetensors_file(fast=True) mode forces a single codec for a 4–8× encode speedup at parity ratio on float models.

See pipeline validation for the full dtype, scale, memory, and edge-case matrix (including the fast-mode and thread-count dials that bound peak encode memory).

Key findings

1. Microscaling formats are already near-incompressible at the byte level. Trained FP4 / NVFP4 nibble-packed bytes are close to uniform; no amount of plane-aware coding extracts much further.
2. The remaining lossless headroom is in the scale plane. The MICROSCALE method's Order-1 ScaleAC codec compresses E8M0 scales to ≈ 0.16 of original. The whole-model effect is bounded by the scale-mass fraction (≈ 5%), which is why the on-disk gain over byte-Huffman is small in absolute terms.
3. The PTWM preprocessing graph adds value over pure byte-Huffman, but only modestly on microscaling formats. huffman_only (graph forced to passthrough) hits 0.8804 on gpt-oss-20b vs 0.8603 with the full graph — a 2.0 percentage-point delta, most of which is the scale codec; the rest is dtype-aware nibble routing.

On-disk vs. algorithmic size

The numbers above report on-disk .ptwm directory size, including the multi-tensor container header, tensor index, per-plane codec dispatch records, and payload hashes. Container overhead runs roughly 1.6 percentage points on gpt-oss-20b, the price of features the raw entropy coders lack (random access, hash verification, multi-tensor packaging).

For a pure entropy-coder comparison, subtract the container overhead: microscale_full on gpt-oss-20b drops from 0.8603 to roughly 0.844 algorithmically. The on-disk number is reported here because it matches what users measure with du.

Preprocessing ablations

Effect of preprocessing-graph variants on the byte-Huffman ratio across dtypes. Higher ratio is better (compressed bytes ÷ raw bytes is the inverse of huf_ratio).

The FP8-E4M3 row (e4m3_nibble_split) demonstrates the dtype-aware nibble routing: splitting the byte into two 4-bit planes drops entropy from 6.64 to ~3.3 bits/byte per plane, yielding a 2× compression ratio on a dtype that is otherwise incompressible byte-wise.

Plane entropy

Per-plane entropy on facebook/encodec_24khz (fp32). The reorder strategy moves the IEEE-754 exponent bits into a dedicated plane, exposing the trained-weight exponent distribution (≈ 3.25 bits/byte) that is otherwise scattered across byte boundaries.

Method comparison (synthetic)

Compression ratio and throughput on synthetic uniform-random data across the four high-level PTWM methods. Ratios above 1.0 reflect compression; values at or below 1.0 reflect the method passing through (or in the FP8 cases, the raw input being smaller than the container's per-tensor overhead at 500 kB).

Codec comparison (real models)

Per-codec ratio and throughput on real fp32 models, holding the preprocessing graph fixed at reorder+split for the Huffman and rANS rows and varying the ZSTD prefilter across the rest.

Random-access tradeoff

Per-tensor compression preserves random access but pays a small ratio penalty against monolithic compression. On distilbert-base-uncased the gap is negligible for ZSTD (1.1811 vs 1.1803) and reversed for Huffman and rANS, where the per-tensor coder benefits from dtype-aware preprocessing while the monolithic stream cannot recognise tensor boundaries.

Source data for distilbert/distilbert-base-uncased (267,687,144 raw bytes, 40 fp32 tensors).