← Private themes

Private theme  ·  AI compute infrastructure  ·  June 2026

Re-underwriting the neocloud thesis for the inference era

In brief: the bear case on leveraged neoclouds is right, but the fashionable rebuttal, to go asset-light and broker spot GPUs, is also wrong. The durable economics sit at the two ends of a barbell.

Note on figures: the unit-economic exhibits are first-principles models with explicitly stated assumptions, calibrated to mid-2026 market anchors (cited). They illustrate the mechanics; they are not a valuation of any single named company.

1. The thesis, in brief

The 2023–25 trade, borrow against NVIDIA allocation and rent H100-hours at 60%+ gross margin, is over. But the fashionable rebuttal, “go asset-light and arbitrage other people's idle GPUs,” recreates the same commodity at a different layer. The packing software is open source, the broker owns no end demand, and it carries the basis risk of buying capacity on commitment and selling it on demand.

Leg	Economic outcome	What is actually being underwritten
A · Contracted infrastructure	Durable	Lowest cost of capital plus secured cheap power (MW under contract, low PUE) plus an investment-grade, take-or-pay contract book that transfers utilization risk to the customer. A spread business on capital cost and power, not a GPU bet.
B · Demand owners	Durable	Whoever owns inference demand at scale and converts a compounding throughput edge (tokens/sec/GPU) into margin per million tokens: model labs and inference platforms with distribution, not resellers.
C · Merchant middle	Gets repriced	Undifferentiated, debt-financed operators underwritten on training-era utilization, and pure GPU routers/brokers with no demand and no proprietary throughput. The acute risk sits in the credit/ABS stack rather than the equity.

Durable economics are not created by putting GPUs on a balance sheet, nor by a pure routing layer. They are created at Leg A or Leg B. The middle is where capital gets repriced.

2. Variant perception

Consensus bull: compute demand is insatiable, utilization stays high, and neoclouds compound into the cloud incumbents of the AI era.

Consensus bear: inference is bursty, utilization collapses to ~30%, payback stretches to 18 years, and leveraged operators default.

Our view: both misdefine the unit of analysis. The correct unit is not the dollar per GPU-hour but gross profit per million tokens delivered at the latency SLA. That number is set by three variables the market underweights: cost of capital, model throughput (a software and operations quantity), and the statistics of aggregated demand. Hardware ownership is necessary infrastructure but is not where durable economic profit accrues. The bear is directionally correct that utilization is the kill switch, but the mechanism is return on capital below the cost of capital, compounded by a financing duration mismatch, not a nominal 18-year payback.

3. First principles I: the unit is $/Mtok, not $/GPU-hour

Start from the cost of delivering compute, not the rent. Amortize node capital with a capital-recovery factor (CRF), add fully-loaded opex, and divide by the hours you actually sell:

where C is node capital, L the economic life, U sustained utilization, and G the GPUs per node. For an 8-GPU node at $350k, this is the breakeven rent you must clear per GPU-hour:

Sustained utilization	WACC 10%	WACC 12%	WACC 14%
40%	$5.19	$5.36	$5.54
60%	$3.46	$3.57	$3.69
80%	$2.59	$2.68	$2.77
90%	$2.31	$2.38	$2.46

Assumptions: 4-year economic life, $4/node-hour loaded opex. Market anchor: the 1-year H100 contract cleared ~$2.35/GPU-hr in spring 2026 (up ~40% off the Oct-2025 low of ~$1.70); spot averaged ~$3.17 with a long tail to ~$1.

Read it as a margin map, not a payback clock. At a 4-year life and 12% WACC you must hold ~80% utilization just to break even against the 1-year contract strip; below ~70% you are underwater on a fully-loaded basis at any plausible price. The kill switch is return on capital below the cost of capital, reached far earlier than a nominal payback figure suggests. On cash-on-cost, a node at $2.35 contract earns ~9% at 40% utilization (below WACC), ~18% at 60%, and ~28% at 80%, before the cost of refreshing the asset.

The real default mechanism: a duration mismatch

A headline payback number set against an accounting depreciation life conflates two unrelated clocks. The mechanism that actually defaults a leveraged operator is the gap between three durations: depreciation booked over ~6 years (the optimistic accounting life some operators adopted), an economic frontier life of ~2–3 years (the next architecture's performance per dollar, plus price erosion), and debt tenors of ~3–4 years on GPU-collateralized facilities. Every such loan embeds two assumptions: residual collateral value and sustained utilization. If the asset is displaced before the loan amortizes, the operator refinances a depreciated, lower-utilized collateral base into a tighter market. That is a balance-sheet event, not a per-node one, and it is why the stress shows up first in the credit/ABS stack of the merchant cohort.

4. First principles II: the physics of inference

4.1 The latency tax: why you cannot run inference like training

Model a server as an M/M/1 queue at offered load ρ, service rate μ. Mean time in system is:

Relative to an unloaded server the latency factor is \(1/(1-\rho)\), and the tail blows up as you approach saturation:

Offered load ρ	0.50	0.70	0.80	0.90	0.95	0.98
Latency vs unloaded	2.0×	3.3×	5.0×	10×	20×	50×

Training is a throughput job with no interactive SLA, so it runs at ρ near 1, which is exactly why the original 80–90% utilization assumption held. Interactive inference carries a p99 SLA on time-to-first-token and inter-token latency, which imposes a hard ceiling ρ* well below 1. An observed ~30% utilization in interactive inference is therefore not simply mismanagement; a large part of it is a rational SLA reserve. You do not fix it by packing to 90%; you fix it by raising throughput and by smoothing demand (section 5).

4.2 Throughput is the numerator

The lever that actually moves $/Mtok is tokens/sec/GPU. Decode is memory-bandwidth-bound (by the roofline, low arithmetic intensity means you are paying for HBM bytes moved per token, not FLOPs), so the wins come from amortizing weight and KV-cache reads across many concurrent sequences: continuous batching, PagedAttention / KV-cache paging, disaggregated prefill (compute-bound) versus decode (bandwidth-bound), speculative decoding, and FP8/INT4 quantization. Translate throughput into unit cost for the same $350k node:

Aggregate output throughput	$/Mtok @ 60% util	$/Mtok @ 80% util
3,000 tok/s	$2.40	$1.89
6,000 tok/s	$1.20	$0.95
10,000 tok/s	$0.72	$0.57
20,000 tok/s	$0.36	$0.28

Market anchor: Llama-70B-class output prices ran ~$0.32–$0.90/Mtok in 2026. Node all-in cost amortized at 12% WACC / 4-year life.

The conclusion is the whole thesis in one line: positive margin exists only where throughput is high and utilization-at-SLA is high. Both are software and operations. Neither is conferred by owning the silicon, which is precisely why the capex-heavy merchant cannot defend a spread.

5. First principles III: the real aggregator math

The intuition that you can simply “pack more onto a GPU” needs a mechanism. The correct one is variance pooling. Aggregate the demand of N uncorrelated bursty tenants and the coefficient of variation of the combined load falls as:

Independent tenants pooled (N)	1	4	16	64	256
Aggregate demand CV	1.00	0.50	0.25	0.125	0.06

Lower CV means that, for a fixed p99 SLA, you can safely run at a higher ρ*, which is higher utilization at the same latency, hence lower $/Mtok. This is an insurance business: the aggregator sells a burst-capacity option and profits from the law of large numbers.

But the edge accrues only to whoever pools enough uncorrelated demand. This is why the asset-light GPU router is the crowded middle that gets crushed, not the toll road:

• No throughput moat: the packing stack (vLLM, SGLang, TensorRT-LLM) is open source, so the underlying provider runs the same batching and keeps the margin itself.
• No demand ownership: a neutral reseller pools no proprietary, diversified demand and reduces no variance the host could not, so there is nothing to aggregate and real COGS to carry.
• Negative convexity: buying capacity on commitment and selling per-token on demand is short a burst option; the broker eats the basis whenever demand dips.

The field is already populated (Together, Fireworks, Baseten, Modal, Replicate, Anyscale, NVIDIA's Run:ai). Margins there are competitive and several are subsidized. A router is a feature factory, not a moat.

6. Who endures, who gets repriced: differentiation map

Neoclouds are not homogeneous, and the dispersion is where the analysis pays off. Score any operator on the seven dimensions that actually set $/Mtok and survival:

Dimension	Merchant-spot, levered (C)	Contracted infra (A)	Demand-owner (B)
Cost of capital / WACC	High	Low	Mixed
Secured power ($/kWh, MW, PUE)	Spot / weak	Contracted	Outsourced
Contract book (take-or-pay, IG, duration)	Merchant	IG take-or-pay	Owns the user
Demand ownership	None	Indirect	Direct
Throughput / software edge	Off-the-shelf	Bundled	Compounding
Chip-gen & residual position	Aging	Managed	Abstracted
Lease-adj. leverage vs asset life	Mismatched	Matched	Light
Outcome	Most exposed; refinances into a wall	Durable (a spread business)	Durable (a margin business)

7. The thesis, in full

7.1 Underwriting checklist: the seven questions for any neocloud

1. What is your blended cost of capital, and what fraction of the fleet is financed by GPU-collateralized debt? What are the tenors versus the asset's economic life?
2. What share of revenue is take-or-pay / prepaid, with what counterparty credit and weighted duration? (Merchant spot exposure is the risk; contracted backlog is the asset.)
3. What is delivered $/Mtok at your SLA, and what is the throughput (tokens/sec/GPU) behind it? Is the throughput stack proprietary or off-the-shelf?
4. What is your secured power position (contracted MW, $/kWh, PUE), and how much is energized versus announced?
5. At what ρ* does your SLA bind, and how much diversified, uncorrelated demand do you pool to lift it?
6. What is the residual/secondary market for your installed generation, and your refresh cadence versus depreciation schedule?
7. Do you own demand, or do you rent it? If you rent it, what stops your supplier from disintermediating you with the same open-source stack?

7.2 Catalysts and what would break the view

Supportive signals (A & B)	Stress signals (C)	What would break this view
Power becomes the binding constraint; secured-MW operators re-rate.	First GPU-backed ABS/credit event or covenant breach in the merchant cohort.	A new architecture resets performance-per-dollar so fast that even contracted infra is stranded, and power stops being the moat.
Inference volume scales and SLA-priced backlog lengthens duration.	The 1-year H100 contract price rolls back over as Blackwell / next-gen supply lands.	Throughput software stops improving and commoditizes the demand-owner edge away.
Demand owners convert throughput gains into widening $/Mtok margin.	Spot oversupply pushes merchant utilization below the ~70% return-on-capital line.	Take-or-pay contracts prove unenforceable (mass customer defaults), collapsing the A-leg thesis.

7.3 What it implies for private deployment

Legs A and B are where durable economic profit accrues, in the operators that clear the checklist above. Leg C is where capital is most likely to be repriced to its cost of capital, and the most acute risk in that cohort sits in the GPU-collateralized credit/ABS stack rather than the equity, because the failure mode is a financing duration mismatch. For new private deployment, the middle (a GPU roll-up or a pure routing layer) is the part of the stack most likely to be repriced.

Appendix: models, formulas, and glossary

Capital-recovery factor and breakeven rent:

Queue latency (M/M/1 mean sojourn) and latency factor:

Variance pooling (N i.i.d. demand streams):

Delivered unit cost:

Base assumptions

Node $350,000 (8 GPUs); economic life 4 years; WACC 10–14%; fully-loaded opex $4/node-hour; energy from first principles, roughly node IT kW × PUE × $/kWh. All exhibits recompute from these; change any input and the conclusions move, which is the point a single-point model misses.

Glossary

$/Mtok: dollars per million tokens delivered.   TTFT/TPOT: time-to-first-token / time-per-output-token (the inference SLAs).   Continuous batching, PagedAttention: techniques that raise tokens/sec/GPU by sharing weight and KV-cache reads.   Disaggregated serving: splitting compute-bound prefill from bandwidth-bound decode.   ρ*: the maximum utilization at which the latency SLA still holds.   Take-or-pay: a contract that bills reserved capacity regardless of use, shifting utilization risk to the customer.

Sources

NVIDIA FY2026 results (data-center gross margin ~72–75%); SemiAnalysis and multi-provider GPU rental indices (H100 ~$2.35 contract / ~$3.17 spot, 2026); pricepertoken.com and provider pages (Llama-70B ~$0.32–0.90/Mtok); CNBC / Quartz / Seeking Alpha on GPU depreciation life and GPU-collateralized debt (2025–26).

This research note is produced by Flywyl for informational and educational purposes only. It is not investment advice, an offer, or a solicitation, and it does not recommend buying, selling, or holding any security. The unit-economic exhibits are illustrative first-principles models with stated assumptions, not a valuation of any named company. Past performance does not guarantee future results. Flywyl is a research firm; it does not manage money or hold client assets.