"Just call the API" is the right call for a prototype. It's frequently the wrong call once a workload is running continuously, at volume, in production — and most teams don't notice until the bill does.
Know your actual utilization curve
Self-hosting only wins when utilization is high and predictable enough to justify the fixed cost of the hardware. A bursty, unpredictable workload is usually cheaper on a hosted API, even at a worse per-token rate — you're paying for elasticity, not just compute.
The first thing we do is plot the client's actual request volume over a real month, not a projected one.
The pipeline matters more than the GPU
Batching, quantization, and request scheduling routinely matter more to the final cost-per-request than which specific card you rent. We've cut inference cost by more moving a workload from unbatched to batched serving than we have by changing hardware tiers.