Container format

The .ptwm file is a multi-tensor container. A single file holds many tensors with a shared header, a tensor index, and per-plane codec dispatch.

Layout

graph TD
    subgraph Container [".ptwm File Container"]
        M[Magic + Version<br/>'PTWM' + byte]
        P[Prelude<br/>Shared State & Codec Menu]
        I[Tensor Index<br/>Name → Offset Maps]

        subgraph T1 [Tensor 1: Plane Dispatch Records]
            R1_0[Plane 0: Codec, Shape, Hash]
            R1_1[Plane 1: Codec, Shape, Hash]
        end

        subgraph TN [Tensor N: Plane Dispatch Records]
            RN_0[Plane 0: Codec, Shape, Hash]
            RN_1[...]
        end

        subgraph Payloads [Concatenated Plane Payloads]
            PL1_0[Tensor 1 Plane 0 Bytes]
            PL1_1[Tensor 1 Plane 1 Bytes]
            PLN_0[...]
        end
    end

    M --> P
    P --> I
    I --> T1
    I --> TN
    T1 --> Payloads
    TN --> Payloads

See crates/ptwm-core/src/compressor.rs for the exact wire format fields and field widths.

Random access

The tensor index is small (a manifest of name → offset records). Opening a .ptwm and fetching one tensor touches only:

1. The header + index (a few KB).
2. The plane records for that tensor (≤ 200 bytes per plane).
3. The plane payloads for that tensor.

Use TensorIndex to do this from Python.

Hash verification

An xxHash annotates each plane payload. The decoder verifies the hash before returning bytes to the caller. A mismatch raises CompressionMethodNotSupportedError, which lets callers distinguish corruption from unsupported features.

Modes for safetensors input

The compress_safetensors_file entry point writes the compressed payload in one of two shapes; the mode argument selects which:

Both modes round-trip through PTWM. Mode b is what the safetensors integration emits by default.