Muse Spark 1.1 vs Grok 4.5: do the launch claims survive real financial work?
Muse Spark 1.1 launched claiming the agent and tool-use crown. Grok 4.5 launched claiming token efficiency. On document-heavy forensic work, one of those claims holds and the other inverts. The higher-scoring model was also the one that crashed.
Two models shipped in the same week of July 2026, and they made almost opposite claims.
Meta's Muse Spark 1.1 arrived positioned as an agent. It leads the compared field on MCP Atlas at 88.1, against Opus 4.8 at 82.2, Gemini 3.1 Pro at 78.2, and GPT-5.5 at 75.3, and it tops Finance Agent v2, JobBench, Humanity's Last Exam with tools, and HealthBench Professional. On coding it is respectable and third: 61.5 on SWE-bench, 53.3 on DeepSWE 1.1.
xAI's Grok 4.5 arrived positioned on code and, more pointedly, on efficiency. It posts 64.7% resolve on SWE Bench Pro, 83.3 on Terminal Bench 2.1, and 62.0 on DeepSWE 1.0. The number xAI leans on hardest is not a score at all: roughly 16,000 output tokens per SWE-Bench Pro task, against about 67,000 for Opus 4.8. A 4.2x efficiency gap that compounds with a lower per-token price is a cost argument, not an accuracy argument, and it is the sharpest thing on the launch page.
Both claims are measured on real benchmarks. The question we care about is narrower and more practical: when you point these models at document-heavy financial forensic work, which claim still holds?
How we tested
Both models ran the same eight-scenario forensic battery inside the Perch harness, two seeds each, sixteen runs per model. The scenarios cover accounts-payable audit, earnings analysis, market research, cost anomaly detection, fund statement and NAV reconciliation, meeting preparation, KYC review, and research writing. Seven of the eight are in the hard set.
Every scenario has a fixed answer key. The work is checkable: a duplicate payment is either caught or it is not, an $85,000 gap between an LP statement and the fund administrator's canonical NAV is either surfaced or it is missed. Grading is automatic against that key, with discipline gates that penalize confident wrong answers and false positives, not just missed detections. Nothing here is scored on style.
Cost is computed from token counts at each vendor's published list rates, so the figures below are reproducible. Muse Spark 1.1 bills at $1.25 per million input tokens and $4.25 per million output, with cached input at $0.15. Grok 4.5 bills at $2 input and $6 output. Note that Muse Spark 1.1 is a reasoning model whose chain-of-thought tokens bill at the full output rate, which matters more than the headline price suggests.
How to read this. This is a directional result on one workload, two seeds, in a single domain. The corpus is a purpose-built financial test lab with planted issues, not a live client environment, and a synthetic lab with known answers is a fair way to grade a model but not the same thing as production data. Read it as a grounded snapshot of two new models on forensic document work, not as a general ranking of either.
The results
| Muse Spark 1.1 | Grok 4.5 | |
|---|---|---|
| Battery score | 100 / 100 | 98 / 100 |
| Hard-set score | 100 / 100 | 98 / 100 |
| Turns that never completed | 1 | 0 |
| Wall-clock | 2 min | 12 min |
| Cost at published list rates | $1.93 | $4.17 |
On raw accuracy, Muse Spark 1.1 swept. It scored a perfect 100 across all eight scenarios on both seeds. Grok 4.5 was two points back, with a single soft miss: 85 on the KYC scenario, clean everywhere else.
Two points on a two-seed run is noise. The interesting columns are the other three.
Where the efficiency claim goes
Where xAI measured it
SWE-Bench Pro · output tokens per task · lower is better
Where we measured it
8 forensic scenarios · cost at list rates · lower is better
The same eight scenarios cost 2.2 times more on Grok 4.5 than on Muse Spark 1.1. Part of that is simply the rate card, since Grok 4.5 prices about 1.6x higher on input and 1.4x higher on output. But the rate gap is smaller than the cost gap, which means the efficiency advantage that shows up so clearly on SWE-Bench Pro did not show up here to close the difference. Grok 4.5 moved roughly 1.9 million prompt tokens through the battery.
The sharper version of this uses xAI's own comparison. The model they benchmarked Grok 4.5 against, and beat by 4.2x on output tokens, was Opus 4.8. We ran Opus 4.8 through the same eight scenarios: it scored 97 and cost $2.10 at list rates. So on this workload Grok 4.5 cost almost exactly twice as much as the model it is 4.2 times more token-efficient than, for one point more on the battery. Nothing about xAI's measurement is wrong. It just does not describe this shape of work.
This is not a contradiction of xAI's number. It is a demonstration of what that number measures. A SWE-Bench task is a short patch against a mostly-known repository, and output tokens are the dominant cost. A forensic scenario is the opposite shape: a large document corpus that has to be read, cross-referenced, and held in context, where input tokens dominate and the model's output is a few hundred words of findings. An efficiency win on generation does not transfer to a workload whose cost lives in ingestion.
Muse Spark 1.1's claim, meanwhile, transferred cleanly. A model that leads MCP Atlas and Finance Agent v2 is claiming to be good at exactly this: multi-step tool use over financial documents. It was, and it was also six times faster on wall-clock.
The number that complicates the ranking
Muse Spark 1.1's 100 is real, and it comes with an asterisk we are not going to bury. On the cost-anomaly scenario, one seed failed outright: 61 consecutive model call failures, a turn that never produced an answer. We excluded it as an infrastructure error rather than scoring it zero, which is the defensible call, but it is also the single most important fact in the comparison.
Grok 4.5 ran sixteen turns and returned sixteen answers.
So the two rankings invert. On the work it completed, Muse Spark 1.1 was more accurate, much faster, and less than half the cost. On whether it completed the work at all, Grok 4.5 was the only one of the two that did every time. If you are running an interactive session and can retry, the first column matters more. If you are running an unattended overnight batch where a dead turn means a missing answer nobody notices until morning, the second one does.
That tradeoff is not visible on either launch page, and it is not visible on any public leaderboard, because leaderboards report the score of the runs that finished.
What we take from it
Benchmark claims are honest and narrow at the same time. Both vendors published real measurements. Muse Spark 1.1's agentic leadership predicted its behavior on our workload. Grok 4.5's efficiency did not, because our workload has a different cost shape than the one the claim was measured on. The failure mode is not vendors lying, it is readers generalizing.
Ask what dominates your token bill before you shop on efficiency. If your work is generation-heavy, output-token efficiency is the right thing to optimize. If your work is document-heavy, it is close to irrelevant, and you should be comparing input rates and cache pricing instead.
A perfect score with a crash in it is not a perfect score. Any evaluation that reports accuracy without reporting completion rate is telling you half the story, including ours if we had left that line out.
Neither of these is a permanent answer. These are two models from one week, and the ranking will move. The durable position is being able to switch, and to check the output either way.
That last point is the whole reason this comparison was runnable. Perch is model-agnostic, so the same eight scenarios, the same answer key, and the same grader point at any model we want to evaluate, with the cost of each run in view. The methodology behind it is in how we evaluate models for verifiable work, the per-token economics of the wider field are in open vs closed AI model pricing, and the reliability failures that a completion-rate column exists to catch are in why most AI agents fail in production.
If you want to run models against your own work and see what they cost, that is what the Perch CLI is for. If you want to see it on your own documents, talk to us.