1 Overview
Choosing a cloud for compute is less about picking a winner and more about matching a provider's defaults to the shape of your workload. The three largest providers converge on the same primitives: virtual machines in isolated network segments, managed container runtimes, serverless functions, and a catalog of managed data and messaging services. The differences show up in defaults, quotas, identity models, and how aggressively each platform nudges you toward its own opinionated tooling. Those differences are not cosmetic. A default that suits one team can become a recurring source of friction for another, because defaults encode assumptions about scale, tenancy, and how much operational control the platform expects you to surrender.
A practical evaluation starts with the workload rather than the brand. A steady-state API with predictable traffic behaves very differently from a bursty batch pipeline or a latency-sensitive service with strict regional requirements. Each provider can host all of these, but the operational friction, the pricing curve, and the failure modes differ enough that a generic comparison obscures the decision. The same is true of identity: how a platform models roles, service accounts, and trust boundaries shapes everything built on top of it, from deployment pipelines to incident response. Quotas and limits matter for the same reason, since they quietly define the ceiling a workload can reach before it needs re-architecture.
This article treats the three major platforms as archetypes rather than products. The goal is a durable framework: understand the workload, map it to instance families and scaling primitives, estimate relative cost, and plan for the failure modes that actually bite teams in production. Vendor-specific names change frequently; the underlying patterns do not. By reasoning about categories instead of catalog entries, the framework stays useful even as individual services are renamed, merged, or replaced.
2 Why this workload pattern matters
General-purpose compute is the substrate under most production systems. Even teams that adopt serverless for edge logic usually keep a fleet of long-running instances for databases, queues, background workers, and internal services. That means the compute decision compounds: it shapes networking, identity, observability, and deployment pipelines for years. Each of those layers is built on assumptions about how compute is provisioned, addressed, authenticated, and monitored, so changing the compute model later forces rework across all of them at once. The decision is therefore less a single purchase than a long-lived architectural commitment that quietly governs how every other component is wired together.
It also dominates spend. Compute is typically the largest line item in a cloud bill, and the gap between a well-tuned fleet and a neglected one is large in relative terms. Instance sizing, commitment discounts, and autoscaling policy interact, so a small architectural choice early can produce a persistent cost or reliability penalty later. A sizing decision that looks conservative at launch can become chronic overprovisioning once traffic patterns settle, while an aggressive autoscaling policy can trade cost for instability under bursty load. Because these levers reinforce one another, the penalty compounds rather than staying fixed, and it tends to surface as either a budget overrun or a reliability incident rather than a single clean failure.
Finally, compute choices constrain hiring and incident response. Teams develop muscle memory around a provider's console, CLI, and IAM model. Migrating later is possible but expensive, so the initial selection deserves more than a feature checklist. The right question is which platform's defaults align with how your team already operates and how your workload actually behaves under load.
3 The architecture in plain terms
At its core, cloud compute is a control plane that schedules your workloads onto physical hosts in isolated network segments. You express intent through an API: launch an instance from an image, attach storage and networking, place it in a region and zone, and grant it an identity. Managed container services add a scheduler and a load balancer on top of that. Serverless functions abstract the host entirely and bill per invocation and execution time. The control plane itself is the durable part of the picture; the hosts beneath it are interchangeable, and the API is the contract you build against. Because that contract is declarative, the same request can be replayed, versioned, and reviewed like any other engineering artifact, which is what makes automation possible at all.
Around that core sit supporting services that determine day-to-day experience: virtual networks with subnets and firewalls, identity and access management, secret storage, logging and metrics, and infrastructure-as-code tooling. Each provider ships a first-party stack for all of these, and each stack has a distinct mental model. Some emphasize resource hierarchies and policy inheritance; others emphasize project boundaries and service accounts. Those differences shape how permissions flow, how resources are grouped, and how much of the wiring you describe explicitly versus inherit implicitly. Understanding the mental model matters more than memorizing individual service names, because the model is what you reason about when something breaks.
The practical architecture pattern is similar across providers. Separate environments by account or project, define networking declaratively, bake machine images or container images, and treat infrastructure as versioned code. The provider you choose mostly changes the syntax and the guardrails, not the shape of a sound design. Teams that internalize the pattern can move between platforms with far less friction, since the underlying concerns, isolation, identity, reproducibility, and observability, remain constant even as the interfaces change.
4 Scaling and cost trade-offs
Horizontal scaling is the default for stateless services: add instances behind a load balancer as demand rises, remove them as it falls. Because each instance is interchangeable and holds no session state, capacity becomes a dial rather than a decision, and the load balancer absorbs the work of routing to whichever instances are healthy at any moment. Vertical scaling, resizing a single instance, is simpler but bounded by instance size limits and usually requires a restart. That restart is not merely an inconvenience; it is a window of reduced or absent capacity, which means vertical changes tend to be planned events rather than continuous adjustments. Event-driven scaling suits spiky or intermittent work, where paying only for execution time beats keeping idle capacity warm. In that model the platform itself decides when capacity exists, so the engineering effort shifts from managing servers to managing concurrency, timeouts, and the cost of each invocation.
Cost follows the same shape. On-demand pricing is the flexible baseline; committed-use discounts lower effective rates in exchange for a one- or three-year commitment; spot-style capacity is cheapest but can be reclaimed. The relative gap between these tiers is substantial, often a large fraction of the on-demand rate, which is why commitment planning matters more than instance-type micro-optimization for steady workloads. A slightly larger instance at a committed rate can easily cost less than a perfectly sized instance at the flexible rate, so the pricing tier frequently dominates the sizing decision. Reclaimable capacity inverts the usual assumption: it is cheap precisely because it is interruptible, and it is only useful for work that can be restarted, checkpointed, or duplicated without harm.
The trade-off is between elasticity and predictability. Autoscaling reduces waste but adds cold-start latency and operational surface. Reserved capacity lowers unit cost but risks over-provisioning if demand shifts. A common pattern is a committed baseline for steady traffic plus on-demand or spot capacity for peaks, with serverless handling the spikiest edges. The baseline covers the load that is always present, the flexible tiers absorb the variation above it, and the spiky edges are delegated to a model that scales to zero when nothing is running. Getting the proportions right is a forecasting problem as much as an engineering one, and it is worth revisiting as traffic patterns change.
5 Failure modes and what to watch
The most common compute failure is not a dramatic outage but capacity and quota exhaustion. A region or zone runs short of a specific instance family, or an account hits a service quota during a traffic spike, and scaling silently stops. Teams discover this during incidents, which is the worst time. Pre-provisioning headroom and monitoring quota usage prevent most of these events. The quiet nature of this failure is what makes it dangerous: nothing crashes, no alarm fires, and the system simply stops growing to meet demand. Capacity planning therefore has to treat headroom as a standing requirement rather than a one-time exercise, and quota consumption deserves the same visibility as any other operational metric.
Configuration drift is the second recurring failure. Environments diverge because someone made a console change that never reached the infrastructure code. The result is a staging environment that passes tests while production behaves differently. Immutable images, declarative infrastructure, and drift detection reduce this class of bug substantially. The underlying problem is that any change made outside the defined pipeline creates a second, undocumented source of truth. When those two sources disagree, the environment that was tested is no longer the environment that runs, and the gap tends to widen over time until it produces an incident that is difficult to reproduce anywhere else.
Finally, identity and networking misconfigurations cause a surprising share of incidents. Overly broad roles, missing egress rules, or a subnet without a route to a dependency can take a service down without any host failing. Treat IAM and network policy as first-class code, review changes, and test failure paths deliberately rather than assuming the platform handles them. Because these layers sit beneath the application, their faults often surface as confusing application errors rather than obvious infrastructure problems. Reviewing them with the same rigor as application code, and exercising their failure paths on purpose, keeps small policy mistakes from becoming full outages.
6 Operating it in practice
Day-to-day operations revolve around observability, deployment, and cost hygiene. Centralize logs and metrics so that a single place answers questions about what the system is doing right now and what it did during the last incident. Define service-level objectives that express what users actually care about, and alert on symptoms rather than causes, because a symptom-based alert fires when the experience degrades regardless of which internal component is responsible. Use rolling or blue-green deployments with health checks so a bad release affects a fraction of capacity and can be withdrawn before it reaches everyone. Keep runbooks current, because the person responding at 3 a.m. is rarely the person who built the system, and a runbook written for the builder is often useless to anyone else. Treat these practices as a loop rather than a checklist: what observability reveals should feed back into deployment safety and into the runbooks themselves.
Cost hygiene is an operational discipline, not a one-time project. Right-size instances based on observed utilization, delete orphaned volumes and load balancers, and revisit commitment coverage quarterly. Tag resources consistently so spend can be attributed to teams and services; untagged resources are effectively invisible until the invoice arrives. The same rhythm applies to capacity: review what is actually consumed, compare it against what was provisioned, and close the gap deliberately instead of letting it widen quietly. Attribution matters because a cost nobody owns is a cost nobody reduces.
Security and access management deserve the same routine. Rotate credentials, prefer short-lived identities over static keys, and apply least privilege at the account or project boundary. Automate compliance checks in the pipeline so that drift is caught at the point of change rather than during an audit. None of this is provider-specific, but each platform makes some of it easier and some of it harder, which is exactly why the operating model should drive the selection.
7 Frequently asked questions
Which provider is cheapest for compute? There is no universal answer. Effective cost depends on instance family, utilization, commitment level, and region. The relative ranking flips depending on whether you compare on-demand, committed, or spot pricing, and on how well your workload matches each platform's discount mechanics. A workload that runs steadily and can tolerate interruption will favor a different pricing mode than one that must stay available at all times, and the same instance family can look cheap in one region and expensive in another because of local capacity and demand. Discount mechanics themselves vary in how forgiving they are: some reward long commitments, others reward flexibility, and still others reward patience with interruptible capacity. Because these factors interact, a headline rate tells you very little about what you will actually pay once your real traffic, scaling behavior, and tolerance for interruption are taken into account. Benchmark your actual workload rather than trusting headline rates, and re-run that benchmark as your usage patterns change.
Should I use virtual machines, containers, or serverless? Use virtual machines when you need full control of the operating system or specialized hardware. Use containers when you want portability and efficient packing with a managed scheduler. Use serverless for event-driven or spiky work where operational overhead matters more than per-invocation cost. Most production systems combine all three, because different parts of an application have different control, portability, and operational requirements. The decision is rarely permanent; a component that starts on a virtual machine may later move into a container, and a background job may move to serverless once its invocation pattern is well understood.
How do I avoid lock-in? Keep business logic portable, isolate provider-specific services behind interfaces, and store state in formats you can migrate. Some lock-in is acceptable if it buys reliability or speed; the goal is deliberate coupling, not zero coupling. Revisit the decision as your workload and team evolve, because the right answer at ten services may not hold at a thousand.