Same loss with less validation compute

Cartesia Research dates the post October 28, 2026, and does not print a clock. The topic board lists the item at 17:53:06 UTC that day, which is 01:53 in Beijing on October 29, before this edition's window and after the backfill floor. The title is the first of a pair on moving from tokens to bytes, and the subject is scaling for hierarchical networks. The page says hierarchical networks were introduced last year, in a paper and a blog, as an end-to-end architecture that models multiple levels of abstraction. This post supplies a new compute comparison. It is not the first time the name appears.

The comparison is built near Figure 4. One rung of a BPE ladder is converted into a byte-level one-stage H-Net. The main backbone keeps the same architecture and runs on units of about 4.5 bytes in both models. The H-Net's outer networks replace the tokenization layers and are matched to them in parameters and FLOPs. A separate sentence says that on this data mix, each BPE token covers about 4.57 bytes. The 4.5 and the 4.57 belong to different sentences. One describes the unit. The other describes compression on this mix.

The in-domain figures are two sentences and do not collapse into one. After compute-to-loss power laws are fit, the page says the H-Net is 1.28 times more compute efficient than the BPE Transformer. The figure note says the H-Net reaches the BPE Transformer's loss with 22 percent less compute on held-out pretraining data, measured as validation bits per byte. Both the 1.28 times and the 22 percent are printed in that section. The page also says that at every rung, the H-Net run reached a lower loss than the matching BPE Transformer.

Downstream evaluation is a further widening, not the same ratio. The page names Paloma and OLMES, each a mixture of dozens of datasets and tasks. Soft accuracy is measured as the bits per byte the model assigns to the correct answer, and compute-to-bits scaling laws are refit for each suite. The gap grows to around 2 times. Around 2 times is what the page says about those suites. It is not a restatement of the in-domain 1.28 times.

The opening carries two larger claims, and the comparisons differ. Carefully controlled one-stage H-Nets are said to provide compute-efficiency gains of up to 3 to 5 times, purely through learned tokenization. A more flexible two-stage H-Net, once scaled up, is said to outperform a matched open-source model with 10 times less compute and data. That sentence does not name the open-source model. Up to 3 to 5 times, around 2 times, 1.28 times, 22 percent less, and 10 times less are not gathered into one number. A 128,000-word vocabulary for Llama-3 appears only as an example of how large a BPE vocabulary can be. It is not the model in the comparison.

The blog does not add a clock, a name for the model in the 10-times sentence, or an equation that turns 1.28 times into 3 to 5 times. The picture with this column is the cover diagram: bytes along the bottom, and two stages of blocks grouping them into longer spans. The efficiency ratios are in the body and in Figure 4, not in the colors of that cover.

Matching the outer networks to the tokenization layers in parameters and FLOPs is how the page says the rungs were balanced. It is not an external board. Lower loss at every rung is then read, on the page, as dynamic tokenization beating a static tokenizer. A later post is flagged for a wider set of models and for why the efficiency appears in language modeling. Biology and vision are named in the summary as ongoing work, and that sentence does not attach a number.