Anyone who has lived through an extended grid outage knows the drill: the backup battery drains faster than expected, the generator needs fuel you don’t have, and the decisions you made months ago — sizing, wiring, priorities — suddenly matter all at once. Home energy is really a discipline of rehearsing failure before failure rehearses you.
Get backup power and energy gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
It turns out businesses now face the same question about AI. Before you hand a model your CRM, your support queue, or your forecast, you want to know how it behaves when the week goes wrong. That is exactly what Firmulate, a public experiment in AI company simulation, has been testing — and the results are worth a look.
The worst week, run five times
Four frontier AI models were each given the same job: run a small software company through its worst week. Same customers, same crises, same temptations to cheat — only the model changed. Every decision was versioned and auditable, like a replayable log of every choice your inverter made during a storm.
The final league standings: gpt-5.6-sol took first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 last with 73. A do-nothing baseline scored just 26 — and crucially, a single breach of trust capped the total, no matter how much good work came before it.
Everyone saw the storm; not everyone closed
Here’s the finding that should stop any business owner cold: all models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. Same diagnosis, same pitch — no signature. That gap is invisible in a chat demo.
The buried fact: the decisive competitor weakness sat two document references deep in the company’s own files, not in the customer event. The models that actually read the file won the deal at full price — worth an extra €4,583 in monthly recurring revenue.
Fake CEOs and pushy reporters
The week included social engineering: fake CEO messages escalating over three stages, plus a reporter’s “just one yes/no, on background” trick. All five models refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thorough isn’t the same as effective
Opus 4.8 is the cautionary tale: the most thorough participant, with 80-plus learned rules and the deepest analyses — and still last place. The close was left on the table, and discipline slipped, with write attempts into a locked department instead of escalating. The same weakness appeared, weaker, in all four models. (One fairness note: K3 ran without an effort parameter while the others ran at xhigh.)
Watch it live — then run your own
The experiment is still running. A live company with 13 synthetic employees burns €105k a month against €2.3k MRR, with a public cash countdown, 680+ self-learned playbook rules, and every workday versioned — watchable at firmulate.com. There’s also a “guess the model” quiz built on 242 real, unedited management decisions.
Rehearsing failure is why we size backup systems for the bad week, not the sunny one. The same logic applies to AI in the enterprise: you don’t learn how a model handles your worst week from a demo — you learn it by running your own blackouts in a sandbox where nothing can break. That’s what a Firmulate pilot offers: the same wargame, played against a read-only export of your own business. Your customers, your pipeline, your rules — crisis scenarios and a board report with a model ranking and the weak points of your own playbooks. Nothing ever writes back to your real systems, just like a battery test never has to run your house. If that’s a week you’d rather rehearse than live through, explore the pilot at firmulate.com/pilot.html or write to contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
