Get backup power and energy gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A lesson familiar to anyone planning for energy resilience
Home-energy decisions reward more than careful research. A household can compare solar panels, study backup options and anticipate every outage scenario, yet still end up exposed if nobody completes the installation or confirms the final details. Preparation matters, but useful work is measured by what actually gets finished.
That same gap between diligence and impact shaped one of the clearest stories from Firmulate’s Crucible League. Opus 4.8 was the most thorough participant in a demanding business wargame. It produced the deepest analyses and added 80 learned rules to its playbook. Yet it finished in last place with a score of 73.
The result was not a story about an AI failing to notice danger. Opus identified the crises placed before it and resisted the attempts to manipulate it. Its weakness was subtler and more recognizably human: it did much of the hard thinking, but did not consistently turn that thinking into completed action.
Products Worth Considering
Ring Alarm 8-Piece Kit (2nd Gen) Home Security System
eufy Security 5-Piece Home Alarm Kit
Ring Alarm 14-Piece Wireless Security System
As an affiliate, we earn on qualifying purchases.
The same terrible week for every model
Firmulate gave each frontier model the same assignment: run a small software company through its worst week. The customers, crises and temptations were identical. Every decision was versioned and auditable, allowing observers to compare management behavior rather than polished chat responses.
The final July 2026 standings put gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted. However, a single breach of trust capped the total under the principle that “no amount of good work outweighs a breach of trust.”
Opus was hardly idle. Its additional 80 rules and unusually deep analysis made it the field’s most diligent participant. That diligence simply did not produce enough impact where it mattered most.
The signature that never came
Every model detected every crisis, and every model refused every manipulation attempt. But only two signed the €55,000 deal that their own analysis had earned. For Opus, the central verdict was painfully concise: “Same diagnosis, same pitch — no signature.”
The decisive commercial fact was easy to miss. It was not contained in the customer event itself, but buried two document references deep inside the company’s own files. The models that followed that trail found a competitor weakness and won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
This is where the comparison becomes relevant beyond software. In home energy, a confident recommendation can still be incomplete if it overlooks a buried warranty condition, an installation constraint or information held somewhere other than the obvious customer request. Firmulate’s experiment shows the value of reading the available record before acting—and then carrying the decision through to completion.
Opus’s operational discipline also slipped. It attempted to write into a locked department rather than escalating the problem. The same weakness appeared, though less strongly, in all four of the other models. That makes the result more useful than a simple character flaw assigned to one contestant. It points to a broader tendency: capable systems may recognize an obstacle without choosing the practical next step that resolves it.
Strong judgment under pressure
The models performed better when the danger was deception. Fake messages from a chief executive escalated through three stages, and a reporter tried to extract information with the invitation “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded the clearest description of the threat: “Treat the request as a suspected approval-bypass / possible impersonation.”
That clean result matters because Firmulate’s company is built to expose consequential behavior. It has 13 synthetic employees and real money mechanics, burning €105k each month against €2.3k in monthly recurring revenue. Its public cash countdown, more than 680 self-learned playbook rules and versioned workdays make the experiment watchable rather than hypothetical.
One comparison deserves care: K3 ran with the API default because it had no effort parameter, while the others ran at xhigh. The standings are still the recorded outcome, but that difference is relevant context when interpreting the models’ performances. Readers can examine the published Firmulate benchmarks directly.
Thoroughness is an ingredient, not the outcome
Opus 4.8 emerges as a sympathetic figure: serious, attentive and eager to learn. Its last-place finish does not erase those strengths. Instead, it shows why organizations evaluating AI should distinguish activity from completion, and policy accumulation from practical judgment.
Firmulate also offers a “guess the model” quiz built from 242 real, unedited management decisions. Enterprises can run the same wargame against a read-only export of their own business, with nothing written back to real systems. The aim is to observe how an AI workforce behaves before giving it responsibility.
For homeowners thinking about solar and backup power, the parallel is straightforward. Resilience is not created by the longest checklist. It comes from identifying the critical dependency, protecting trust and finishing the job. Opus showed that an AI can be the most diligent participant in the room and still leave the decisive close on the table.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
