Synthiq Labs · Lab Notes
The research, written down.
Written companions to the channel — sources you can check, claims you can re-run.
Jul 23, 2026methodologybenchmarks
How we measure (a preview)
A plain-English tour of the Labs methodology — quality on a frozen, comparable task set; efficiency at batch size 1 with medians and error bars; provenance tags on every number; and reproducibility as the whole point.
Read the note →
Jul 23, 2026leaderboardon-devicepositioning
Why an honest on-device model leaderboard does not exist yet — and what we are building
The benchmark world splits into two camps that never meet — quality-but-cloud-only, and efficiency-but-ranks-hardware. The models people actually run on their own devices fall in the gap. Here is the gap, mapped, and the board we are building to fill it.
Read the note →
The numbers behind these notes live on the leaderboard and the method behind the numbers on the methodology page.