DEV Community

Damian Dixon
Damian Dixon

Posted on

"Nvidia B200 Chips, Rented by API: 31 of 33 Jobs Came Up Ready"

#ai #gpu #api #cloud

We rented Nvidia B200 GPUs through a single API call, 33 times, and measured what happened. Every job in this run was a B200. Here are the results.

What we ran

Kilawatt Cloud is an orchestration layer that routes GPU jobs across multiple suppliers. In this run we sent 38 B200 requests to the API. The router launched 33 of them: 30 five-minute jobs and 3 one-hour jobs. Across those 33 jobs we launched 39 cards.

Image 1

Image 2

The headline numbers

  1. 31 of 33 launched jobs came up ready (94%). The two that did not were refunded automatically.

  2. 37 of 39 cards came up ready.

  3. 25 of the 31 ready jobs were ready in under 60 seconds.

By supplier

  1. RunPod: 13 launched, 13 ready. Median time to ready was 8 seconds. The slowest was 14 seconds.

  2. Vast.ai: 20 launched, 18 ready. Median time to ready was 44 seconds. The slowest was 316 seconds.

Image 3

Image 4

The one-hour test

The RunPod one-hour job ran the full hour. We checked its health 87 times and all 87 checks passed. It logged 0 XID errors, 0 uncorrectable ECC errors, and a peak GPU temperature of 34C.

Both Vast.ai one-hour jobs stopped about 15 to 16 minutes in. That was at the supplier account level, not a hardware fault. We are not counting them as host failures.

What did not launch

Five requests for four cards at once could not be placed. RunPod had no machine available, and the Vast.ai quote for four cards failed our pricing check. Nothing was charged for those requests, and the router skipped them instead of forcing a bad launch.

What this does and does not show

This is a small run on one GPU type. It measures how fast and how reliably the router gets a B200 to ready. It does not yet measure what happens when a host dies mid-job. That is the next test we are building: checkpoint-aware jobs that resume on a second supplier. We will publish the timings we measure, whatever they are.

Image 5

Read the full report

The full report, with charts and the job-by-job breakdown, is on our Super Intelligence page:

kilawattcloud.dev/super-intelligence

Open the latest entry, "Nvidia B200 Chips, Rented by API: 31 of 33 Jobs Came Up Ready", to read or download the PDF. We publish each test run there, including the ones that do not go perfectly.

Questions about the method or the raw numbers are welcome in the comments.

Top comments (1)

Collapse
 
arhancanli profile image
Arhan Canli •

The counts reconcile (13 + 20 = 33 launched, 13 + 18 = 31 ready), so the question is what the denominator hides.

  • 31 of 33 is 94%, but only because the 5 requests that could not be placed are left out. Out of the 38 requests sent it is 31 of 38, 82%. Both are honest numbers; they answer different questions (will a launched job come up vs will my request get a B200), and a user cares about the second.
  • With samples this small the ranges are wide: 31/33 has a 95% Wilson interval of roughly 80-98%, RunPod's 13/13 only rules out a failure rate above about 23%, and Vast.ai's 18/20 is roughly 70-97%. So "RunPod is more reliable than Vast.ai" is suggested here, not shown. The 8 s vs 44 s medians are a much firmer gap.
  • "Ready" is not "finished". Both Vast.ai one-hour jobs stopped at 15-16 minutes, so on the only runtime test Vast.ai is 0 of 2 while RunPod is 1 of 1. I would report ready rate and survived-the-requested-duration rate side by side, since the second is what the one-hour test is for.
  • 25 of 31 under 60 seconds is 81%, and since RunPod's slowest was 14 s, the other six slow ones are all Vast.ai (up to 316 s). A p90 of time to ready per supplier would show the tail better than a median.

Do the 6 extra cards (39 across 33 jobs) all sit in the multi-card jobs, and were the 2 not-ready cards the same 2 failed jobs?