What one flipped bit costs on the ET-SoC-1

← AI Foundry lab dashboard › Worklog

AI Foundry lab · experiment E60 · Tue 20 Oct 2026 · aifoundry3, aifoundry2 and aifoundry1's card 0 · 609 of 630 runs

Horace He showed that a GPU multiplies predictable data faster because it flips fewer transistors. Tonight we turned that into a measurement on the ET-SoC-1: hold the clock, the voltage, the launch temperature and the arithmetic fixed, change only how many operand bits flip from one operation to the next, and read the slope. One flipped B-operand bit of an fp32 multiply-add costs 117 ± 1 fJ of board energy, everything it sets off included; the same bit carried into a unit that only XORs it costs 20.8 ± 0.1 fJ on the way in and 31.2 ± 0.2 fJ on the way out. And the same 4,096 multiply-adds, put in an order that flips more bits, cost +0.68 W ± 0.01 more.

Names on this page. A minion is one of the chip's 1,088 small RISC-V cores (these kernels use 1,024); each has an 8-lane vector unit. The minion rail is the supply that feeds them, metered on the card separately from the board's 12 V input. TensorFMA32 is the instruction that multiplies two 16×16 fp32 tiles into an accumulator, 512 eight-lane multiply-adds per operation; the Horace kernel is the lab's test that repeats it on one tile pair, named after the post. fxor.pi and fand.pi are the vector unit's bitwise XOR and AND on 8 lanes of 32 bits. Energies are board energies (the card's 12 V input) unless marked rail.

The question

Horace He's post (Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data!, 2024) explained the effect as power: random data flips more transistors, so a power-capped GPU clocks lower. He wrote that he could not count how many times each transistor flips. On this chip the multiply-add unit's RTL is open, and the Horace experiment (21 Sep) showed one matmul drawing 38.3, 46.7 and 63.4 W on zeros, ones and random data at one speed, with counted toggles predicting 14 patterns to half a watt. What was missing was a dose: turn one knob, the number of bits that flip, and watch the watts follow. Prior work measured GPU power against value distributions and bit similarity (ALUPower, Lucas and Juurlink, 2016; Gregersen, Patel and Choukse, 2024), and side-channel analysis has modelled power as Hamming distance since 2004; none of it priced a flip in joules on a chip whose flips can be counted.

The experiment

Pre-registered (tools/claims-v3/hl/README.md: the design, the analysis, the decision rules and the RTL predictions, committed before the first measured pass) and run tonight on three cards:

• The Hamming ladder. The Horace kernel unchanged. In every vector lane the B operand alternates, every cycle, between a word and the same word with its low h mantissa bits flipped; A is the same everywhere. h = 0, 1, 2, 4, 8, 12, 16, 23, with a quiet partner (A = 1.0) and a busy one (A = π).
• Same arithmetic, different order. Every A column holds the same 16 values, differing in 4 mantissa bits, in Gray order (1 bit flips a step), binary, shuffled, or the order that flips the most (3.5 bits a step, the most a cycle of 16 codes allows). Permuting A's rows permutes C's: the same 4,096 multiply-adds, only their order differs.
• Which bit. One B bit at a time flips every cycle: the sign, an exponent bit, mantissa bits.
• One bit, one gate. The vector integer unit's fxor.pi and fand.pi, with the eight source registers set so that a known number of bits flips on the read ports and in the result from one instruction to the next, and nothing else changes: the sources are never written, and every destination is rewritten with the value it already holds.

Protocol. Every run launched as the die's reading first fell to a fixed value (58 °C; 57 °C on aifoundry1's card 0) after a short heater burst, the 21 Sep method; 4 s runs; the board's power over two seconds once the clock had settled, corrected to the launch reading with each card's own leakage slope; every item compared with its zero rung inside the same pass; every error bar the scatter between passes. Each device process stayed under 10 s, and the cards were claimed in the lab channel.

Three operating points, none changed by us. The cards no longer reach 80 °C under this duty cycle (they level off at 60–64 °C), so the two cards whose clock governor is free ran at 800 MHz (aifoundry2 at 0.617 V, aifoundry1's card 0 at 0.604 V), and only aifoundry3, whose clock is pinned, ran at 600 MHz and 0.522 V. That gave the ½CV² test three voltages for free. On aifoundry3 tonight's anchors read 36.4 W on π and 52.6 W on random data, 16.2 W apart, against 16.4 W apart on 21 Sep (on another card, launched at 80 °C).

1. Same arithmetic, different order

The cleanest result needs no bit accounting at all. On the pinned card the same 4,096 multiply-adds cost +0.682 W ± 0.010 more in max-flip order than in Gray order when the four changing bits are mantissa bits 0–3, and +0.716 W ± 0.023 when they are bits 19–22 (4 passes; the RTL model predicted +0.93 W and +0.88 W). The four orders rank in the order of their flipped bits (binary indistinguishable from Gray) at bits 0–3, and in the order of their flipped bits at bits 19–22. On aifoundry2 at 800 MHz and 0.617 V, where every switching watt is about 1.9 times larger: +1.29 W ± 0.01 and +1.31 W ± 0.13 (9 passes). By the pre-registered rule, a difference of at least three standard errors, the effect is confirmed on both cards.

2. The Hamming ladder: the price of a flipped input bit

Once the first bit has flipped, power rises close to a straight line with the number of flipped B bits. On the pinned card the slope is 116.7 ± 0.6 fJ per flipped B bit with the quiet partner and 129.9 ± 0.5 fJ with the busy one (h ≥ 1, 4 passes). The residuals about the lines, 0.39 and 0.43 W over 11.7 and 13.1 W spans, pass the pre-registered linearity rule on its alternative, 5% of the span, not the absolute 0.25 W bar: the bottom of the ladder bends (the second flipped bit adds 0.25 W, each bit above h = 12 about 0.61 W), so the slope is an average along it. The first flip costs +1.95 W ± 0.01 with the busy partner, 3.3 times an ordinary bit on that ladder: a dense partner's partial-product tree sits still under a constant operand, and the first changing bit wakes it. With the quiet partner the first flip costs +0.59 W ± 0.01, 1.1 times an ordinary bit: no turn-on to speak of.

That price is everything one flipped B bit causes: the operand bus, the Booth and Wallace trees, the aligner, the adder, the result, and the accumulator reads that change along with it. The pre-registered primary measure counts every operand-bus bit the RTL says changes, accumulator reads included: 74.7 ± 0.4 fJ per bus bit (quiet) and 85.6 ± 0.3 fJ (busy). Per toggle of any counted net in the unit it is 301 aJ and 119 aJ: by the RTL's counts the busy partner wakes 2.8 times as many nets per flipped bit (1,096 against 388) and costs only 11% more, which puts most of the energy outside the tree's many small nets (a direction read from the model's own net counts, not a measured split). The RTL model, fitted on 21 Sep and frozen before tonight, predicted the shape and over-predicted the size: measured differences are 0.83 of predicted, 1.28 W rms over 26 items on the pinned card, which passes the pre-registered 1.5 W mark. It was fitted at 80 °C, and these runs launched at 58 °C.

3. One bit, one gate

On the vector integer unit a flipped bit on the two read ports costs 20.8 ± 0.1 fJ into an XOR. A flipped result bit costs 31.2 ± 0.2 fJ on the ladder whose results never touch the all-zero word, and 30.0 ± 0.2 fJ on the ladder whose results alternate with it (+4% between them here, +9% on aifoundry1's card 0): about 1.5 times an input bit. Into an AND an input bit costs 28.1 ± 0.7 fJ, 35% more than into an XOR, from the one AND rung the pre-registration kept (h = 8). The AND rung whose operands include the all-zero word (h = 32, kept out of the fit by the pre-registration) reads 25% below a straight line through h = 8, level with the XOR ladder's h = 32 rung: either the AND's price per bit falls with h, or the all-zero word is cheaper; these rungs cannot tell which. Even so, the gate is the small part of the bill: one gate of arithmetic behind a pipeline of transport. The multiply-add's 117 fJ per flipped bit is 5.6 times the read-port price: that factor is what a multiply-add does with one changed input bit.

An estimate published earlier today from the E46 instruction catalogue put the input bit at 5.0 fJ and the output and register write at 33.7 fJ, on the die. On the minion rail, the nearest thing here to a die figure, the input bit measures 16.3 fJ, about three times that, and the result bit 24.3 fJ. The catalogue's "random" operands are floats in [0.5, 2), which flip about 12 bits a word rather than 16, and its two equations do not resolve the input price from zero card by card; tonight's sources set every flip by design.

4. Not every bit costs the same

A workload in which one B bit alternates costs +0.55 W ± 0.02 above constant operands for the sign bit, +0.78 W ± 0.05 for exponent bit 23 (the only exponent bit run), +0.63 W ± 0.02 and +0.47 W ± 0.05 for mantissa bits 0 and 8, and +2.49 W ± 0.33 and +4.68 W ± 0.15 for mantissa bits 16 and 22 (aifoundry3, 600 MHz). Sign, exponent and low mantissa bits sit together around half a watt; what climbs is mantissa significance, the bits whose change runs through the whole multiplier. Mantissa bit 8 is the model's largest miss: 0.47 W measured against 2.19 W predicted. At the same Hamming weight, position matters too: two flipped bits spread across the mantissa cost +2.25 W ± 0.05 more than two low bits (aifoundry2, 9 passes), more than the low pair itself costs (+1.96 W above its zero). There is no single price per bit.

5. Does a flip cost ½CV²?

The energy of a flip should grow with the square of the supply voltage and not with the clock. On the minion rail the read-port price on the two 800 MHz cards is 1.38× and 1.43× aifoundry3's, at 1.34× and 1.40× its V²; the multiply-add ladder between h = 2 and h = 8 gives 1.45× and 1.35× on the rail. On the board meter the four ratios are 1.22, 1.38, 1.19 and 1.41. 8 of the 8 sit inside the pre-registered band for ½CV² scaling (1.18–1.59), and none near the 1.8–1.9× a price that also followed the clock would give: a flip costs ½CV². The band is wide; it rules out a clock-dependent price, not every exponent near two. One caution: on aifoundry1's card 0 the board meter reads the multiply-add ladder only 1.12× its own minion rail, against 1.33× and 1.42× on the other two cards, although the board figure carries the regulators' loss; the rail is the number to use there.

From one flip down to the floor

These energies are on the board's 12 V input, so they include the regulators' loss; the capacitances and electron counts are charge-equivalents at the minion rail's voltage, upper bounds for the die. On the minion rail itself the multiply-add price is 88 fJ and the read-port price 16.3 fJ. One flip is far below anything the meter can see; the slope over 10¹⁴–10¹⁶ flips a second is not.

What this does not measure

• A single transistor or a single wire: every price is an average over the nets one flipped bit sets switching.
• Glitches: they are inside the measured watts, not in the RTL counts.
• The die alone: board energy includes the regulators; the minion rail is reported beside it.
• Heat as such: the thermal camera's lock-in of these rungs is the next session's work.
• A second card at 600 MHz: the order and bit results have replicates only on the 800 MHz cards, at another voltage.

Method, data and how to rerun

Experiment E60 (request Q119) in the lab repository. The code is in tools/claims-v3/hl; its README holds the design, departures D1–D5 and amendments A1–A3, each committed before the data it touched (A1 added passes, the order family on the pinned card and a never-zero control; A2 starts the 800 MHz cards' window after the clock settles; A3 excludes one rung that tripped aifoundry2's power limit). Every pass's telemetry and run records, the reduced results (hl.json) and one row per run (runs.csv) are in docs/reports/data/2026-10-20-bitflip-ladder. Passes and runs kept after the gates: aifoundry3 8 passes, 220 of 220 runs; aifoundry2 9 passes (pass 5 set aside), 160 of 180; aifoundry1's card 0 10 passes, 229 of 230. To rerun a pass on a free card: bash tools/claims-v3/hl/block.sh <pass> <items>, then reduce_hl.py over the pass folders.