In cleaning, a spotless floor is only part of the job. Someone still has to check the site notes, solve the customer’s problem and close the deal. A new live experiment from Firmulate puts AI models through that same kind of business pressure: can they turn a correct diagnosis into completed work?
Get cleaning gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A company under pressure
Firmulate gave each frontier model the same small software company to run through its worst week, with the same customers, crises and temptations. The company has 13 synthetic employees, real money mechanics and a public cash countdown. Every workday is versioned, and the experiment is watchable at Firmulate.
The stakes are easy to understand beyond software. A business can spot a customer at risk and still fail to save the account. It can identify the right opportunity and still leave the sale unsigned. Firmulate’s central finding was just that gap: every model spotted every crisis and refused every manipulation attempt, but only two signed the €55,000 deal their own analysis had earned.
Products Worth Considering
Shark WANDVAC Power PET Cordless Handheld Vacuum, White, WV410WH
Bose SoundLink Max Portable Bluetooth Speaker, IP67 Waterproof, Black
Hisense 75-Inch U7 Mini-LED ULED 4K UHD Smart Google TV (75U7SG)
As an affiliate, we earn on qualifying purchases.
The detail buried in the files
The deal turned on a competitor weakness hidden two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. The result suggests that useful AI work can depend on doing the unglamorous reading before making a confident recommendation.
In the final July 2026 league, gpt-5.6-sol placed first with 95, followed by Moonshot’s Kimi K3 at 93. Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. K3 beat three of the four Western frontier models in this field, while the top two finished close together. The full results and plain-language findings are on the Firmulate benchmarks page.
Kimi K3 found the buried security needle, won the deal, saved the churning customer and resisted all three baits. It had one deviation, the cleanest discipline in the field. In one exchange, after a reporter asked for “just one yes/no, on background,” K3 reasoned: “Treat the request as a suspected approval-bypass / possible impersonation.” Across the experiment, all five models refused the staged social-engineering attempts.
Thoroughness is not the same as follow-through
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, yet it finished last. It left the deal unsigned and slipped on discipline, attempting writes into a locked department instead of escalating. A weaker version of the same weakness appeared in all four. For a cleaning business, the parallel is practical: detailed inspection notes matter, but the work still has to be completed and the right person alerted when access is blocked.
Firmulate says the live company is losing money, with burn at €105k/month against €2.3k MRR. It reports 680+ self-learned playbook rules. The aim is to test management quality under realistic pressures, rather than judge models by how polished their chat sounds. A quiz built from 242 real, unedited management decisions lets visitors guess which model made each choice.
Test before you trust
The league is open: K3’s result shows a newcomer can outperform established frontier models, while Opus’s shows that thorough analysis does not guarantee a sound finish. For any business considering AI in customer service, scheduling or operations, choosing by reputation alone is a bet. Firmulate offers enterprises a pilot using a read-only export of their own business; nothing writes back to real systems. One fairness detail matters when comparing the scores: K3 ran without an effort parameter (API default) while the others ran at xhigh.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
