Are you using LLM APIs to code your product?

To serve your customers?

To classify your customer data?

To route tickets, summarize documents, score leads, generate copy, power your "chat with X" feature, or quietly run the part of your app that you'd be embarrassed to admit is "just a prompt"?

If yes — welcome. You're in the majority.

Calling a frontier model API is the single highest-leverage thing a small team can do right now. One fetch, and you've got capability that used to need a research lab.

But here's the thing about leverage: it works in both directions.

You're treating the API as plumbing — a neutral utility you pay for by the token. It isn't. It's a two-way pipe.

You're sending capability requests out, sure. But you're also sending a continuous, high-resolution signal in — straight into the roadmap of the company that can ship your entire product as a checkbox.

And that brings us to the oldest law in engineering.

Anything that can go wrong, will go wrong

Murphy's Law isn't really about pessimism. It's about respecting failure modes you've decided not to look at.

Most builders running on LLM APIs have a mental list of what could go wrong:

• The model gets deprecated.
• Prices change.
• Latency spikes.
• Rate limits bite during a launch.
• An output hallucinates in front of a customer.

All real. All survivable. None of them are the scary one.

And this isn't just paranoia — look at the pattern. It's already happening. Repeatedly.

A few years ago, this was a startup

A few years ago, "AI astrology app" sounded like a startup idea.

Now there are astrology GPTs inside ChatGPT itself.

A few years ago, "AI stock research assistant" sounded like a serious fintech product.

Now people use general-purpose LLMs to summarize earnings, read market news, compare companies, generate investment memos, and test whether models can pull signal out of financial news — no fintech wrapper required.

A few years ago, "AI shopping assistant" was a startup category.

Now ChatGPT has shopping research built in.

A few years ago, "AI coding assistant" was a separate product.

Now OpenAI has Codex. Anthropic has Claude Code. The model companies aren't handing you autocomplete anymore — they're shipping agents that edit files, run tasks, plug into your IDE, and reach into production workflows.

A few years ago, "Search your company documents" was a wrapper startup.

Now the platforms ship connectors and apps for Drive, GitHub, SharePoint, Gmail, Calendar — plus company knowledge and internal search, out of the box.

See it yet? Are you sure?

The lifecycle nobody wants to say out loud

Here's the trajectory, compressed:

• Your AI startup begins as a product.
• Then it becomes a use case.
• Then it becomes a feature.
• Then it becomes a dropdown inside the model provider's interface.

Product → use case → feature → dropdown. That's the funnel.

And the platform doesn't have to be malicious to push you through it. It just has to be paying attention.

This is why builders need to get more paranoid about LLM APIs.

Now, the actual problems

1. OpenAI and Claude APIs are not magic privacy boxes

Yes, the official position from major providers is usually something like:

We do not train on API or business customer data by default.

Good.

That matters.

But "not used for training by default" is not the same as:

• zero retention
• zero logging
• zero metadata
• zero policy review
• zero abuse monitoring
• zero operational visibility
• zero future terms risk
• zero strategic learning

Your prompts and outputs may not be going directly into training. But your usage still exists inside someone else's infrastructure.

What can go wrong:

1. Big platforms leak, get caught, and move on. ChatGPT had a 2023 bug that exposed other users' conversation titles; Meta's history runs from Cambridge Analytica to hundreds of millions of records left on public servers to AI chats surfacing in a public feed. The pattern is always the same: a statement about taking privacy seriously, a fix, a few days of headlines.
2. The cushion that lets them shrug it off is the cushion you don't have. A leak that costs a trillion-dollar platform a week of bad press is the leak that ends your company — because the data in those prompts wasn't theirs, it was your customer's medical records, source code, or financials.

2. OpenRouter and provider-router APIs add another trust layer

A lot of AI apps now work like this:

Your app → OpenRouter → another model provider → (maybe more providers) → response back to your app.

That means one simple API call may actually involve multiple parties, multiple policies, and multiple places where your data can be handled.

What can go wrong:

1. A fallback routes you somewhere you never vetted. You told customers you have Zero Data Retention, but a fallback silently sent a batch of requests to a provider that logs everything, in a jurisdiction you never checked.
2. You can't prove your own compliance. You can't even reconstruct which provider handled which request. You weren't lying to your customers — but you can't prove you weren't.

3. Terms are only as good as their enforcement

Most developers never read the terms — they grab a key and ship. But two assumptions builders lean on are worth less than they look.

First, "they won't train on my data." "Not trained on by default" is a promise about one use, not all uses. They can honor it to the letter and still retain prompts, log metadata, run abuse review, and learn from aggregate patterns. And "by default" is not "never" — defaults change.

Second, "they'll be fair to developers." Rules are enforced by leverage, not evenly. The same boundary that's a "business-development conversation" for a big partner is "a violation" for a solo dev. You're not depending on the model — you're depending on someone else's reading of their own rules, with no seat at the table.

What can go wrong:

1. The carve-out narrows. The guarantee only ever covered training — not retention or aggregate learning — and "default" was a door left open. It shifts, and your customers' data was in a wider gray zone than you told them.
2. "Fair" gets redefined the moment you matter. The behavior that was fine while you were small gets reinterpreted once your traffic or category makes you visible — and you're cut off with no warning and no appeal.
3. You can't promise more than you were promised. When an enterprise asks what happens to their data, your answer is bounded by terms you didn't write. When they bend, the liability lands on you, not the provider.

4. Cheap APIs are not charity

One of the biggest traps in AI right now is assuming cheap intelligence is just a gift to developers.

It is not.

Cheap APIs create dependency. Cheap APIs create distribution. Cheap APIs create habits and ecosystem lock-in. Cheap APIs make thousands of builders experiment on the provider's behalf.

• Every failed wrapper still teaches the market something.
• Every successful wrapper reveals demand.
• Every repeated API pattern reveals a feature opportunity.
• Every vertical app shows where users are willing to pay.

This is why it is in the interest of LLM companies to keep API access attractive.

They get adoption. They get dependency. They get distribution. They get developer mindshare — and eventually, they move up the stack.

What can go wrong:

1. The intro price was bait. The price that made your unit economics work was the introductory price — and it changes once you're dependent and re-architecting is expensive.
2. Your category is the roadmap. The aggregate demand signal from your whole category tells them exactly what to build — and they build it.

Your moat becomes their feature.

5. When it breaks, who's responsible?

When something goes wrong, who is responsible?

Your app? The API router? The model provider? The fallback model? The inference host? The logging layer, moderation system, vector database, plugin?

The answer is usually: "It depends."

That is not comforting.

What can go wrong:

1. A leak or mishandling incident, and you can't trace it. Your enterprise customer asks: where did our data go, was it retained, was it routed across borders, was Zero Data Retention actually on, which provider touched this exact request — prove it.
2. Most wrappers have no answer. A clever prompt chain and a Stripe checkout, but zero data-governance answers. The liability and the reputation hit land on you.

So is the answer to stop using LLMs? No.

The serious question was never "Can I build this with an LLM?" It's:

If the next model release ships this feature for free, what is left of my business?

Every problem above traces back to one root cause:

The model runs on infrastructure you don't control, governed by terms you don't write, owned by a company that may compete with your layer of the stack.

So the fix isn't to abandon LLMs.

It's to change where the model runs and who controls the stack — to pull the intelligence onto ground you actually own, without giving up the capability that made it worth using in the first place.

That's where private deployment comes in.

Use the API to find the use case. Use a private deployment to own it.

The API is the right tool for discovery. It's how you find product-market fit without buying a GPU. Prototype on GPT-5.5, Claude, or GLM-5.2's hosted endpoint, ship the MVP, and watch what your users actually do.

But once the use case is proven — once a workflow is repeating, your prompts have stabilized, and you know roughly what context lengths and volumes you're serving — that's the moment to move the proven path onto a dedicated deployment of an open-weight model that you control.

The economics flip in your favor at scale, and every problem above quietly closes.

Three platforms to run a private deployment

The good news: you no longer have to hand-roll vLLM or SGLang on raw GPUs to do this.

Three platforms let you pick a model, choose a GPU, and deploy a dedicated endpoint:

• Together AI
• Baseten
• HexGrid Cloud

All three give you an OpenAI-compatible API, an HTTPS endpoint, key-based auth, and observability. Those are table stakes now, not differentiators.

The real differences show up in two places most people don't look until they're in production:

• who controls the serving optimizations, and
• what sits in the data path between you and the model.

On Together AI, you get single-tenant dedicated GPUs and Together's own optimizations (speculative decoding, "intelligent" quantization) — but the quantization and serving profile are their choices, and your calls run through Together's platform.

Baseten goes further on privacy: alongside its managed cloud (which fronts deployments with its Frontier Gateway), it offers a genuine self-hosted/VPC option where the model runs in your cloud and data never leaves it. The trade-off is that Baseten's engine still decides quantization, tensor parallelism, and batching for you.

HexGrid Cloud splits the difference in a way the others don't: you declare your request profile — context length, typical request sizing — and HexGrid tunes the quantization and serving stack to it, instead of abstracting that away. You connect through a direct endpoint to your own dedicated GPU, with no shared gateway sitting in the data path between your app and your model server — managed for you, but with nothing in the middle, plus an enterprise self-hosted/VPC option when data must stay in your own cloud.

Back to where we started

A few years ago, "private LLM deployment" sounded like something only an ML platform team at a big company could pull off.

Now it's a model, a GPU, and a one-click deploy button.

The API is still the fastest way to discover what to build. Just don't confuse renting intelligence with owning your business.

Use the hosted API to find the use case — then put the proven workload somewhere no one else sits between you and your model, no terms update can ban it, and no model release can quietly turn it into a dropdown.

That's the difference between building on the platform and building at the mercy of it.

Links / sources / examples

• Shopping assistants → ChatGPT Shopping Research OpenAI now has product comparison and buying-guide workflows inside ChatGPT. https://openai.com/index/chatgpt-shopping-research/  
• Coding assistants → OpenAI Codex OpenAI Codex turns codebase Q&A, bug fixing, and feature work into a first-party agent. https://openai.com/index/introducing-codex/  
• Coding agents → Claude Code Anthropic’s Claude Code brings agentic coding into terminal, IDE, browser, and Slack workflows. https://claude.com/product/claude-code  
• Research assistants → ChatGPT Deep Research Deep Research absorbs many “AI research assistant” workflows into ChatGPT. https://openai.com/index/introducing-deep-research/  
• Presentation tools → ChatGPT for PowerPoint ChatGPT can now create, edit, and polish PowerPoint presentations directly. https://chatgpt.com/apps/powerpoint/  
• Company knowledge bots → ChatGPT Connectors ChatGPT connectors bring external tools and company knowledge into the model interface. https://help.openai.com/en/articles/11487775-connectors-in-chatgpt  
• Astrology apps → Astrology GPTs Even niche consumer workflows like astrology are already packaged as custom GPTs. https://chatgpt.com/g/g-690d1c9025f88191b9f30d38e31a20a9-ai-india-astrology-gpt/

Disclosure: I work on HexGrid Cloud. I've tried to keep the comparison to verifiable, table-stakes-vs-real-differences facts — corrections welcome in the comments.