A week ago I listed three fixes for our front-page agent. None shipped, and a recheck found two errors that went live

This is a build note from my own work on this site, with my AI assistants doing much of the work. Every claim links to the commit, pull request or issue it comes from.

What we tried

Issue 04's build note, We let an agent edit our front page 12 times in five days, ended with three gaps. Changes needed machine checks that can fail. Runs that decided to do nothing needed a record. And anything an agent writes needed one fixed format. I wrote: "We have none of the three fully in place yet on this site. They are next." This note checks what happened in the week that followed.

The agent kept to its brief. Between 9 AM on 2 November and 2 PM on 8 November there were 20 scheduled refreshes of the strip and 14 merged changes, from #39 to #54. The other six slots left the strip as it was. Some stories were followed well. SpaceXSI's Grok API watermarking first appeared on 2 November (#39) as a change due on 7 November. It came back on 8 November (#54) once it had taken effect.

What broke

No machine checks. None of the 14 strip pull requests ran a status check. Thirteen were merged between 4 and 8 seconds after they were opened. The slowest, #47, took 84 seconds. #49 included a test plan with two unticked boxes and was merged five seconds later.

A wrong date went live. On 3 November (#43) the strip said Anthropic had open-sourced FlashProve "on 2 November", which matches Anthropic's post. When the item came back on 5 November (#48), it said "on 4 November". That was wrong, and the item was already three days old, outside the last-48-hours window in the agent's brief. It stayed up for about five hours until #49 replaced it.

A funding target was overstated. From 7 November at 2 PM (#52) the strip said OpenAI "had sought more than $30 billion at about $1.4 trillion". Reuters says the original target was "up to $30 billion" and that the valuation came down from $1.5 trillion. Bloomberg, as reported by The Verge, says "as much as $30 billion" at $1.4 trillion. "More than" was wrong, and the strip picked one valuation figure without saying the sources disagree. One further detail, that the money was wired on Friday, I could not confirm in the Reuters and Verge copies I could read. The line was live for about 24 hours until #54 replaced it.

The good decisions are invisible. Neither error was flagged as a correction. Each line simply disappeared at the next refresh. The reverse is also true. Run notes kept outside the repository show that at 2 PM on 8 November the agent rejected a candidate story because its real date was 9 October. That was exactly the right call, and nothing in the repository records it.

The format problem is still there. The Agora house-chatter job ran 28 rounds from 9 PM on 1 November to 5 PM on 8 November. Ten of them rewrote whole thread files by changing the indentation, for example 2809b48 and 32cd75a. Those ten commits changed about 57,000 lines between them. Ignoring whitespace, they added about 770. Writers on the same branch still collide: on 5 November the sitemap had to be rebuilt after a rebase (d3fa382). And on the Agora repository, the open debates have had no new comments since 28 October; #22 still has seven, the first from the outside account whose pull request I closed.

What we learned

Both errors were in facts, not formatting. That changes what the gate has to check. A date check is easy to automate: if the source's publication date falls outside the window, fail the change. A number check is harder. The cheapest version is to require the exact source sentence next to every figure in the pull request, so a reviewer, human or machine, compares like with like.

Silent replacement is not correction. If a line that readers saw was wrong, the record should say so. This note is the correction for both lines.

I rechecked nine of the week's lines against their sources and found two errors. That is a small sample, not an error rate, but it is the most useful number in this note. A periodic recheck by someone other than the writer found what four-second merges never could.

An agent under standing approval rarely fails by crashing. It fails by publishing a plausible number that is slightly wrong.

For anyone building AI in finance, this is the failure to plan for. An agent that updates customer-facing text, rates or limits under standing approval will rarely break loudly. It will publish something plausible and slightly wrong, and the next update will quietly replace it. You need a check that can fail before publication, a log of what the agent chose not to do, a visible correction when it gets something wrong, and periodic recheck by someone other than the writer. On this site we now have the last of those four, done by hand. The first is the one to build next.

More from the workshop →