Counter-evidence · Novation working paper

The case against this desk

Everything below is published, peer-reviewed where we say so, and argues against the business on this site. We went looking for it deliberately. Where we have an answer it is here; where we do not, that is here too.

A site that only publishes the evidence for itself is a brochure. This one has an open model whose published grid fails in 14 of its 30 cells, and a page headed what we don't claim. This paper is the same instinct applied properly: we commissioned a literature review across ML systems, electricity restructuring, compute market structure and commodity formation, and this is what it found that cuts against us.

Two things to know before you read it. The first is that some of this genuinely damages the case, and we say which. The second is that a reviewer who wants to argue with us should start here, because these are the strongest arguments available and we would rather you had them from us than assembled them yourself in week three of a diligence process.

1 · The workload mix is moving against us, fast

The product depends on there being work that can be cut into pieces and placed. Across the published production traces, that population has been collapsing as a share of the compute that matters.

One caveat on that table, which we owe the reader: these are different clusters with different missions and different measures — PAI reports share of GPU demand where the others report share of GPU-time, and its recommender and ads fleet was unusually granular even for 2020. This is a trend across the literature, not a measurement of one fleet, and the single-GPU series is not monotonic. What is consistent is the destination.

The single-GPU population went from 68% of GPU-time to under 2%. In the Kalos cluster, pretraining is 3.2% of jobs and 94% of GPU-time, while the decomposable evaluation work — exactly the sort of thing we call movable — is 92.9% of jobs and 0.8% of GPU-time.

And the unit is indivisible by design. Every scheduler paper in the literature treats gang scheduling as fundamental. Tiresias: “all-or-nothing resource allocation… all required resources must be allocated together.” AntMan: the scheduler “forces partially available resources to idle in reserve until the final request is satisfied.” In the Alibaba PAI trace, 85% of instances have all-or-nothing scheduling requirements and 20% of those need more than 100 GPUs at once.

If the work that can move is measured in GPU-hours rather than job counts, the literature says it is shrinking, and the part that remains is the part that cannot be cut up.

What we say to that. Two things, and only one of them is a defense. The defense is that the traces above are training-cluster traces at frontier labs, and the fleets we are built for are not that: a pharma or a mid-size lab running evaluation sweeps, fine-tunes and batch inference has a materially different mix from Shanghai AI Lab's pretraining cluster. The concession is that we cannot prove that with a published trace, because no such trace exists for the fleets we are describing. Our 30% default movable share is an assumption in a model, not a measurement, and if your fleet looks like Kalos it is wrong by a factor of ten. That is what the first two weeks of an assessment are for, and it is why the recommendation can come back “stay put.”

Jeon et al., “Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads,” USENIX ATC '19 (Philly) · Hu, Sun, Yan, Wen, Zhang, “Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters,” SC '21, arXiv:2109.01313 (Helios) · Weng et al., “MLaaS in the Wild,” NSDI '22 (Alibaba PAI) · Hu et al., NSDI '24, arXiv:2403.07648 (Acme — Shanghai AI Laboratory, not Meta, a misattribution we have seen repeated) · Gu et al., “Tiresias,” NSDI '19 · Xiao et al., “AntMan,” OSDI '20.

2 · Where the work runs is worth more than what it costs

Our whole argument is that the same silicon has a different price at different suppliers. The systems literature says the same silicon also has a different throughput at different placements, by margins that swamp the price spread.

• Philly: 114.8 → 74.1 images/second across the placement spectrum, from same-server to RDMA contention.
• Themis: VGG16 on four P100s runs at 103.6 img/s on one server against 80.4 split across two — a 22.4% loss at identical aggregate resources.
• Alibaba PAI: GPU locality is worth a 10.54× speedup for ResNet-100k.
• Gandiva: inter-server interference costs ResNet-50 up to 47%.
• Helios, operationally: “cross-RDMA-domain distributed training is not allowed.”

What we say to that. Nothing, except that it is correct and it is why the fabric acceptance test runs before the real job and why fabric drag is a deduction in the model rather than a footnote. It also bounds the product: physical placement, not the hour, is the price-determining variable, which is the same reason we do not claim mid-flight migration between clouds. If your campaign needs contiguous, rail-optimized, single-domain allocation, the number of suppliers that can serve it is small and the arbitrage is correspondingly small.

Themis, NSDI '20 · Xiao et al., “Gandiva,” OSDI '18 · Philly and PAI as above.

3 · Preemption costs minutes, not milliseconds

We had assumed faster. The correction below is the single most consequential one in the review, and it runs directly against the interruptible-capacity half of the savings bridge.

Worse, optimizing the checkpoint write does not fix it. Gemini's per-failure total is 7–12 minutes even with sub-three-second checkpoints: 162 seconds of serialization, 253 seconds of restart warm-up, 15 seconds of detection, and four to seven minutes acquiring an instance. Oobleck's comparison shows Varuna, a system with zero checkpointing overhead, still losing up to 90% of throughput when failures arrive every ten minutes — an extreme comparator case, not a typical failure tax.

And it gets worse with scale, in the wrong direction on both axes. Meta reports mean time to failure falling from 47.7 days at 8 GPUs to 7.9 hours at 1,024, with a projection of 14 minutes at 131,072. Meta's own conclusion for a 100,000-GPU run is that it would need roughly two-minute checkpoint intervals and two-minute restart overhead to hit an effective-training-time ratio of 0.9. Nobody has demonstrated that.

What we say to that. The model already deducts rerun waste and checkpoint overhead, and on the default fleet they are −$20k and −$219k against a gross of $2,849k. But a September 2026 review told us the deduction was built on the wrong quantity — that we price the checkpoint write while the literature says the cost is the restart. We checked before changing the model, and that is not what it does. rerun already carries an explicit restart term of 20 minutes plus half a checkpoint interval of lost work, so 35 minutes per preemption — longer than the 7–12 minutes per failure measured above. The write is deducted separately, and is the larger of the two lines.

The real gap is scale, and it is the one this section actually establishes. The model's default of 1.5 preemptions per 72-hour job implies a mean time to failure of 48 hours, which on Meta's curve is a gang of roughly 170 GPUs — smaller than the workloads we say we target. The rate is a single scale-independent input, and it should not be. We have not silently changed the default, because a default that moves without a reason moves every number that quotes it; instead the calculator now takes a job size, shows the preemption rate that scale implies, and flags an entered rate that falls below it. The exposure is therefore bounded rather than unknown: holding everything else at the defaults, net falls 4% at 1,024 GPUs, 16% at 4,096, and 57% at 16,384 — the last being the case where rerun saturates at the whole of interruptible spend, which is the model's way of saying that at that scale the work that can move does not belong on interruptible capacity at all.

Two failure costs remain unmodeled and we are not going to pretend otherwise: the requeue delay when a gang cannot re-form, which the literature suggests can dominate everything else in a constrained market; and hardware failures on firm capacity, since rerun and checkpoint are applied only to interruptible spend. The second is a deliberate choice — you already checkpoint on the capacity you have, so only incremental checkpointing belongs in an incremental comparison — but it is a judgement rather than a fact.

Shukla et al., “Singularity,” arXiv:2202.07848, Table 5 · Sia, SOSP '23 · Lucid, ASPLOS '23, DOI 10.1145/3575693.3575705 · Pollux, OSDI '21 · Gemini, SOSP '23 · Oobleck, SOSP '23 · Llama 3 herd-of-models paper, arXiv:2407.21783 · Meta RSC reliability study, HPCA '25, arXiv:2410.21680. None of these papers measures migration across sites or over a WAN. All assume a shared fast filesystem in one datacenter, so the cross-supplier case — ours — is unmeasured in the literature.

4 · Deadline flexibility is smaller in practice than the systems papers imply

The strongest single counter-citation to this business is a carbon paper. Sukprasert et al. study temporal and spatial workload shifting across 123 cloud regions and conclude that it is “likely not a panacea”: more than 70% of regions have low daily variation to exploit, and while the ideal saving reaches 189 gCO2eq, “practical constraints limit this to 32 gCO2eq on average.” Their observation about which jobs have slack is the one that matters here: “large jobs, which tend to have the least temporal flexibility, are a substantial fraction of cloud data centers' computing workloads.” And “sophisticated policies, in many cases, yield little additional benefit as compared to simple policies.”

Around it: Google's measured production result from the most sophisticated carbon-aware scheduler in existence is a 1–2% power reduction in the highest-carbon hours, and its own taxonomy excludes Search, Maps, YouTube and all cloud-customer VMs. Wiesner measured 20.4% of ML jobs as not shiftable at all, with the savings curve near zero below about five hours of slack. And roughly 80% of users cannot predict their own job runtime within 10% — which is a problem for any system, ours included, that asks a user to declare a deadline and then prices against it.

What we say to that. The last finding is the one that stings, because declared deadlines are an input to our scheduler and we have been treating them as reliable. They are not. The mitigation is the same one we use for the work that can move: an engineer confirms, and anything unconfirmed is treated as firm. The rest of it argues that the time dimension is thinner than the systems literature suggests — which is a reason to make the money on supplier price gap rather than on temporal shifting, and is roughly what the model already does. It is also why we do not sell a carbon story.

Sukprasert, Souza, Bashir, Irwin, Shenoy, “On the Limitations of Carbon-Aware Temporal and Spatial Workload Shifting in the Cloud,” EuroSys '24, DOI 10.1145/3627703.3650079, arXiv:2306.06502 · Radovanović et al., Google carbon-aware compute · Wiesner et al., “Let's Wait Awhile,” Middleware '21 · Chronus, on runtime prediction.

5 · The slack is real. It is not where we are looking.

The review is emphatic that unused capacity exists — Acme reports CPU memory under 50% and NICs idle more than 60% of the time; PAI reports CPU utilization of 43% against 77% allocated; Borg already runs 20% of its workload in reclaimed resources. Gandiva's 26% utilization gain and AntMan's production improvements on more than 5,000 GPUs are real, measured, deployed results.

And then the sentence we cannot answer:

The slack is real, but it lives inside allocations, inside datacenters, and inside the non-GPU resources — not in unallocated GPUs and not across suppliers.

What we say to that. That it is probably right, and that it names our competitor precisely. If the recoverable waste in your fleet is intra-allocation and intra-datacenter, the thing to buy is a better scheduler, not a procurement desk — and the alternatives page already says to fix scheduling first if utilization is low. What we sell is the supplier-price difference, which is a different pool of money, smaller, and only available to a buyer who has already done the scheduling work.

6 · Compute markets have been tried, and they failed on structure

Foster and Kesselman, writing the history of the Grid, are blunt about why the computational markets of the 2000s failed: “they have so far been proven incorrect in their assumption that a monetized Grid would feature many suppliers… Instead, today's cloud features a small number of suppliers who deliver computing resources at fixed per unit costs.” When commercial Grid started, they note, there were “no power plants, in effect.”

That is a market-structure prediction, not a mechanism failure — and it is the one condition that has visibly changed: one price aggregator now lists 53 H100 providers. So this particular objection has an answer, which is why we put it here rather than hiding it.

The 2005 HotOS paper is harder. Shneidman and co-authors give five barriers to market-based resource allocation, and the second is our fungibility problem stated twenty years early, while the third is the deadline problem from section 4: “there is a tangible penalty for misestimating resource need, since these bids are made in advance.” The review's own observation about what happened next is the sharpest line in the whole document:

AWS resolved the HotOS barriers by abolishing the market, not by fixing it.

In November 2017 AWS replaced bid-based spot pricing with provider-set algorithmic prices and decoupled termination from the buyer's bid. Khodak et al. find that 81% of spot price histories violate the condition implied by profit maximisation — “strong empirical evidence that Amazon EC2 spot prices are not set using a profit-maximizing constraint; instead… they are using the spot market only to use up excess capacity.” The largest existing compute “market” is an administered posted price whose operator's objective is utilization, not price discovery.

What we say to that. It is the reason this is a buy-side desk and not an exchange. We are not proposing to build a market; we are proposing to shop competently in a landscape of posted prices, which is a much smaller claim and does not depend on any of the mechanism design that failed. It is also why the fifth alternative on our alternatives page — buy through an auction supplier — is listed as not yet buyable rather than as a competitor.

Foster & Kesselman, “The History of the Grid,” arXiv:2204.04312, April 2022 — peer-reviewed supplier unconfirmed · Shneidman, Ng, Parkes, AuYoung, Snoeren, Vahdat, Chun, “Why Markets Could (But Don't Currently) Solve Resource Allocation Problems in Systems,” HotOS-X, USENIX 2005 · Khodak, Zheng, Lan, Joe-Wong, Chiang, INFOCOM 2018, DOI 10.1109/INFOCOM.2018.8486291 · Jeff Barr, AWS News Blog, 28 November 2017 · AWS Compute Blog, 13 March 2018.

7 · The bandwidth precedent, and the part of it that transfers

Enron's bandwidth market is the obvious analogy and it is usually dismissed as a fraud story. The only rigorous post-mortem we could find says it was not. Schwartz identifies three exposures, and the first is the one that should worry a compute desk most:

An OC-12 from New York to Los Angeles cost roughly $12,000 a month. A carrier not already co-located in Enron's building had to buy connectivity to that building from a metro provider for $8,000 a month — metro fibre running about eight times the per-mile cost of long haul. The cost of reaching the delivery point was two-thirds of the circuit price itself: $20,000 delivered for a $12,000 circuit.

If the cost of getting your workload to the delivery point exceeds the spread, the contract cannot trade — no matter how well specified it is.

The other two: incumbents had no interest in a market whose selling points were price transparency and easy provisioning, because those threatened their customer base; and bandwidth could not be standardized because “reliability, quality, and bandwidth services varied widely across service providers.”

What we say to that. We concede all three. Egress and data movement are the compute equivalent of metro fibre, they are priced at your negotiated rate in the model rather than waved away, and on a data-heavy campaign they are exactly what kills the trade — the specimen policy object on the demo page parks a slice for precisely that reason. The second exposure is why we expect no help from incumbents. The third is the fungibility problem, which has now been measured: see below.

Andrew Schwartz, “Enron's Missed Opportunity,” BRIE Working Paper 152, Berkeley Roundtable on the International Economy, Winter 2002/2003.

8 · “It fails every measure of being a commodity”

The sharpest skeptical line comes from inside the industry. Kelly Littlepage, chief executive of OneChronos — which is itself building a compute auction supplier — told Axios in August 2026: “Just from an Econ 101 stance, it actually fails every measure of being a commodity since it's not fungible. It's not storable. It's not transportable.”

He is right on all three, and the fungibility half now has a measurement behind it. A study of 3,534 physical NVIDIA GPUs across 11 cloud providers, with GPU UUIDs used to separate run noise from device and provider variation, found a 34.49% inter-provider range in FP16 throughput for the H100 PCIe and 37.82% in memcpy bandwidth for the H200 SXM. Run noise was small — 83% of devices within 2% — so noise does not explain it. Month of measurement explained under 1% of variance; GPU identity explained 32–73%. In absolute terms: H200 memory copy at 4,145 GB/s at one provider against 3,170 GB/s at another, a 31% gap, with some distributions bimodal.

The framing the authors' digest reaches is the one to keep: a cloud GPU SKU is “a promise of software compatibility, not a guarantee of delivered throughput.”

What we say to that. This is the best evidence on the site for a claim we were previously making without one, so we are grateful for it even though it is adverse. It says the unit we transact in is unstable, which raises the bar on measurement rather than lowering the value of measuring — a buyer who cannot see a 34% throughput difference between two providers is not going to notice it in an invoice either. It is also why performance adjustment is measured for your workload class rather than assumed, and why an unqualified supplier is ineligible at any price.

“Did You Win the GPU Cloud Lottery?”, Silicon Data with William & Mary researchers, GPGPU '26, ACM DL record 31 July 2026. Two caveats we are obliged to print. The authors' eight-page PDF is public (Slynko et al., downloads.silicondata.com), and the ranges above are the paper's FP16 MatrixMul figures, normalized as maximum minus minimum over the mean; each hardware subset is a fraction of the 3,534 instances studied. And the probes are microbenchmarks that omit host CPUs, PCIe paths, network fabrics, storage and collective communication — so they very likely understate cluster-level price gap. There is also an irony worth stating: Silicon Data is CME's index partner for the proposed compute futures, under CFTC review to 9 November, and is a co-author of the paper documenting that nominally identical GPUs are not fungible.

9 · What is simply not known

The review named voids as often as it named findings. These are the ones that bear on us, and we would rather publish them than let a number fill the gap.

10 · What would change our mind

The honest summary of the above is that the case for this desk rests on supplier price gap being real, large, and reachable — and that three of those four words are contested. So, specifically:

• If an assessment finds a confirmed work that can move below about 10% of spend on a typical fleet, the addressable base is too small and the product is wrong. Our default assumption of 30% would then be an artifact of who we happened to talk to.
• If restart cost at your scale is closer to Meta's numbers than to ours, the interruptible line in the bridge goes to roughly zero and the case has to survive on supplier price gap alone.
• If negotiated enterprise discounts turn out to compress the effective spread to under 1.25× for most buyers, the frontier grid says nothing clears at plausible shares actually moved, and we should stop.
• If the answer to “what did you not run last quarter, and why?” is consistently “nothing, cost was never the binding constraint,” the demand does not exist and no amount of measurement creates it.

Every one of those is still open, and the first assessment that closes any of them will be published here whichever way it goes.

Sources are named inline. This paper is a synthesis of four commissioned literature briefings compiled 4 September 2026, covering ML systems and workload scheduling, empirical evidence on electricity restructuring, compute market structure, and commodity formation and trading desks. Where a briefing flagged a figure as unverified, preprint-only, simulated or vendor-self-reported, that flag is carried above. Where two sources conflict we have said so rather than picked. Figures in this paper are other people's measurements, not ours: we have no results of our own to report, which is the point of the last section.