Journey from Capital to Compute

· 6 min read

I always thought the hard part of getting GPUs for AI models was, well, getting them in the face of compute scarcity.

Turns out it's figuring out how to pay for the chips, and the risk involved for lenders in financing such bets.

Let's take a look at the high level process works if a startup wants to own GPUs to train or serve a video, text or audio generation model.

  1. First, you need to call the chipmaker (NVIDIA, or otherwise) to get the allocation: your place in line for chips.
  2. Then, call the OEMs (Original Equipment Manufacturers) to get the price for finished servers. In GPU land, the OEM is the company that takes NVIDIA's chips and builds them into actual servers you can plug in.
  3. Once this is done, you need colocation in a data center to house the servers. It's basically rented space with network and power.

This seems super simple on paper, but as you scale the number of GPUs, it gets fascinating.

For the purposes of illustration, all the prices in this are hypothetical.

Suppose you want 1,000 GPUs, each costing around $40,000 installed. That's $40M.

Your finance backer would get you the loan at SOFR (Secured Overnight Financing Rate, ~4% in this example) + 9% spread, making the total interest rate 13%. You also need to put down 30% ($40M × 0.3 = $12M). On the remaining $28M, that's about $3.6M in interest in year one alone, before you've trained a single model.

You might wonder why lenders are charging such a high interest rate. It's because GPUs have no determined residual value (an asset's estimated worth at the end of its useful life or lease term) yet, so they put it as zero.

Even NVIDIA doesn't put a residual value on its chips. NVIDIA was recently a backer to build data center for OpenAI in a deal involving $105B, even there, it put the RV guaranty on buildings, power and transmission, and explicitly carves out the chips.

The waiting problem

Because the wait time is so high, you might not even get your order immediately; average wait times are 30+ weeks now. Also, Your equipment is only made to order, so you'll need to put down the 30% to get the loan, which you use to confirm the order with the OEM. Only then will the OEM start making the equipment, and you often can't even test it before you sign.

If you foresee your GPU needs for training, or inference for a product launch, then you need to plan a year in advance.

The full cost stack

Chips are only the first line of the bill. Owning compute means paying a whole chain of people, most of them every month for as long as the cluster runs.

The full cost stack of owning a GPU cluster

Once you get the chips, servers and colocation, you also need to think about system availability. You need to hire infrastructure and GPU engineers to make sure that your GPUs are up, well integrated with your systems, and highly available. That's at least a few engineers maintaining traffic loads and system availability. A contract promising 95% uptime still allows about 18 days of downtime a year, and your interest and depreciation keep running on every idle hour. At scale, hardware failures are routine, and in a large training run one bad node can stall the whole job, so you also need checkpointing and spare capacity ready to swap in.

None of this shows up in the headline price per GPU, but all of it plays a big role in determining if owning the cluster is worth it, or not.

The timing dilemma

Given that the compute price fluctuates so much, your equipment might be worth less, or more, depending on market conditions. If compute prices increase a year from now, then great. Your investment becomes worth it.

But if, by some miraculous innovation, there's an abundance of GPUs hitting the market a year from now, or better chips come into the market, the compute cluster you paid for a year in advance eats away your profit margins, while your loan stays exactly the same size.

Why this is so hard for startups

The barrier to entry for small-to-medium startups is so high for owning and scaling compute: it requires a tremendous amount of capital just to get started, with no guarantee that your training runs will produce useful models. Sure, you can do a small PoC by renting a few GPUs, but scaling is where things get interesting from a compute, finance, and model behaviour perspective.

If a company chooses to do everything themselves, it requires stakeholders to do so much interdisciplinary thinking, with expertise ranging from hiring the right scientists and engineers, strategising data collection & training, and estimating compute needs, to hedging that all this WILL work and then taking the risk with the right kind of capital.

So, the capital flows from VCs and financial institutions to OEMs, chipmakers, DC operators, workers, skilled-talent, and to scientists and engineers. If all this works, THEN end users start paying you, until, of course, someone open-sources a model that is as capable as yours.

I suppose frontier labs have done this loop, which deserves massive respect in itself, just to stay relevant in cut-throat competition.

If a small startup wants to avoid it, they run to neoclouds(CoreWeave, Lambda, NScale), which take away most of this complexity and simply rent out GPUs, or to hyperscaler clouds (AWS, GCP, Azure) to take away all of the complexity. But you also end up paying a premium for it, to delegate risk to these parties.

The industry that already solved this

Power plants for Data centers face the same problem at a bigger scale: hundreds of millions in upfront cost, volatile prices, and banks that won't lend against uncertainty. They solved it with long-term contracts and hedges that turn unpredictable revenue into steady cash flow.

A plant with a solid anchor tenant can borrow at around SOFR + 2.25%, compared to the SOFR + 9% in the GPU financing example. Power is also where GPU scarcity really starts: colocation space is only as available as the electricity behind it.

The compute market is now trying to follow the suit from the power market's playbook, and the difference in borrowing costs shows how much is at stake.

How uncertainty becomes something a bank will lend against

Banks don't lend against "we'll probably make money." They lend against predictable payments. So power plant owners find someone willing to take the uncertainty off their hands.

One common trick: the plant signs a deal with a hedge fund. The fund collects whatever the plant earns from selling electricity, good days and bad, and in return pays the owner a fixed amount every month. The owner gives up the upside on great days but gets a steady paycheck.

The hedge fund is paid to take a risk it thinks it can manage. The bank lends against that fixed monthly stream, as long as it trusts the fund to keep paying. The risk hasn't disappeared, but here, it's moved to someone who wants to carry it.

GPU operators don't have this option yet, and that's a big part of why compute loans cost so much more.

What could change

This is why people are starting to trade compute the way they trade oil or electricity. CME, the world's biggest futures exchange, wants to let people lock in future GPU rental prices, and regulators will decide by November 9 whether that's allowed. Some traders and GPU companies are already signing private deals to lock in prices.

Locking in rental prices would make a GPU operator's income predictable, the same steady paycheck power plants get. Private deals could go further and guarantee what the GPUs themselves will be worth later, which is exactly what lenders are missing today.

If that happens, loans get cheaper, and a lot more teams could afford to own their compute instead of renting it.


Inspired by and building upon: