We run a mixed GPU inference stack — H100s, H200s, RTX 5090s depending on availability and cost at any given time. For about a year, every time we wanted to shift workloads between providers we were effectively rebuilding deployment configs from scratch.
Not because the workloads changed. Because the configs were hardcoded to one provider’s infrastructure.
This is the actual GPU vendor lock-in problem and it took us embarrassingly long to name it correctly.

What we thought the problem was

We thought we were locked in because of which provider we were on. So we focused on making it easier to switch providers — Terraform for infrastructure provisioning, containerized workloads, documented migration runbooks.
This helped at the infrastructure layer. It didn’t help at the workload layer.
When we wanted to move a specific workload from Provider A to Provider B, we still had to update scheduling config, test on new hardware, debug provider-specific quirks, update monitoring. For a team with a growing number of inference workloads this was weeks of engineering time for what should have been an infrastructure decision.
The real problem: the workload definition was coupled to the infrastructure. Provider binding lived inside the deployment config. Portable containers on top of non-portable scheduling logic.

What actually fixed it

The fix was separating workload definition from infrastructure binding entirely.
Instead of specifying where a workload runs, specify what it needs. VRAM requirements, compute capability, container image, environment variables. Let a scheduling layer handle placement across available hardware.
We moved to Yotta Labs for this reason specifically. Their Launch Templates implement exactly this pattern. A template defines:

• Container image
• Resource requirements
• Environment variables
• Exposed ports
• Storage mounts

No provider. No region. No specific GPU SKU.
The scheduler matches requirements to available hardware across their multi-cloud provider network. When one provider’s H200s are sold out, it routes to available capacity elsewhere. Adding a new provider to the pool happens at the infrastructure layer — existing templates don’t change.

The three scenarios where this changed things for us

Capacity constraints during demand spikes

Before: provider’s RTX 5090 inventory sold out, workload queues or fails, manual intervention required.
After: scheduler routes to available compatible capacity elsewhere automatically. We find out in the logs, not in a support ticket.

Cost optimization

Before: better pricing available at a different provider, migration project to move workloads there.
After: add provider to infrastructure pool, existing workloads can route there immediately on next deployment.

Provider reliability issue

Before: provider has an outage, scramble to manually move workloads, engineering time goes into incident response.
After: automatic failure handover at the platform level. Two actual failover events in six months of production use, both invisible at the application layer.

A clarification that confused us initially

Yotta Labs Launch Templates are not the same as AWS Launch Templates.
AWS Launch Templates are EC2 instance configuration templates. They define how to launch a specific instance type. They’re infrastructure provisioning templates.
Yotta’s Launch Templates are workload-level deployment manifests for hardware-agnostic scheduling. The workload definition is the portable artifact, not the instance config.
We went down the AWS Launch Templates path initially before realizing they’re solving a completely different problem. Flagging it because the naming overlap is genuinely confusing when you’re searching for solutions to multi-provider GPU deployment.

What the migration actually looked like

Less rewriting, more removing.
The provider-specific config — scheduling constraints, node selectors, provider API integration — got replaced by a requirements declaration. The workload definition got simpler.
Container images didn’t change. Environment variables didn’t change. Application code didn’t change.
The main task was removing custom orchestration logic we’d built to compensate for provider coupling. That logic was the problem, not a feature.
Teams coming from self-managed K8s GPU clusters: the mental model shift is the bigger lift than the technical migration. Instead of telling the scheduler where to run the workload, you tell it what the workload needs. The rest is the platform’s job.

What we’d do differently

Start with hardware-agnostic workload definition from day one.
The provider-coupled configs we spent months migrating away from were never necessary. We built them because that’s the default pattern when you’re working directly with provider APIs. If we’d started with a requirements-based approach we’d have saved the migration entirely.
For anyone evaluating GPU infrastructure options early: the question worth asking is whether the portability is at the workload definition level or just at the infrastructure provisioning level. The former actually removes vendor lock-in. The latter makes it easier to rebuild your config on a new provider — which is a much weaker guarantee.

Six months in, the infrastructure incident load is close to zero. The engineering time that was going into provider-specific config maintenance is going into product.
Happy to answer questions on specifics — scheduler behavior, hardware compatibility matching, migration path from specific setups.