We run a small software factory where AI models hold three jobs. Builders write the code. Two judges from different vendors must both approve the exact same version before it can land. A lead orders the work and lands it. Every job is recorded, so we can say what the models did, not what they claim.
Our first lesson was about fairness. On 14 October we read a lane's approval rate as one model's coding grade and moved it off the lead seat. The audit showed the figure counted repeated review rounds, setup failures and design documents, and that the traceable code was written by other models. We withdrew the grade. Rule one of Field Trials: code belongs to the model that built it, never to the model that led the day.
Here is the field record for 1 to 15 October. Judges: the OpenAI seat (GPT-6 Astra) approved 48 percent of what it reviewed, the Anthropic seat (Claude Opus 5.5) 67 percent; when both voted on the same version they agreed 73 percent of the time, and in the disagreements the OpenAI seat was the strict one four times out of five. Builders: GPT-6.1 Sol ran 2,045 builds at a median of 8.7 minutes, GPT-6 Astra 891 at 10.3, Claude Opus 5.5 477 at 8.5; of the versions we can trace to their builder, 46, 45 and 50 percent were approved by both judges, on different tiers of work. Gemini 4 Argon has not yet run a build in this factory, so it has no record, and we will not invent one.
What this is not: a ranking. The three builders worked different tiers, and a ranking needs the same tasks, the same tests and the same clock. That trial is being set up, and the next post will carry it.
How we count. A build is one run of a model allowed to change files. A piece is one exact version of a change, and it is approved only when a judge from each vendor approved that version and no later review of it sent it back; the two judges' approval shares above are per review seat, over every review from 1 to 15 October. Reviews of design documents count in the judges' shares, because a judge sat on them, and are left out of the builders' rates, because no builder produced them. A repeated review of the same version counts as a review, not as a new piece; a revised version is a new piece. A review that never ran because of a setup failure is counted apart, as machinery. A builder's approval rate is computed only on the versions that builder's own result named, which is about half of all reviewed versions today, so those rates are partial. Code belongs to the model recorded as the builder, never to the model that led the day.
My AI assistants drafted this from our own records and I checked it.