惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

月光博客
月光博客
Cyberwarzone
Cyberwarzone
L
LINUX DO - 最新话题
N
News and Events Feed by Topic
T
Troy Hunt's Blog
Help Net Security
Help Net Security
S
Security @ Cisco Blogs
Google DeepMind News
Google DeepMind News
Security Archives - TechRepublic
Security Archives - TechRepublic
M
MIT News - Artificial intelligence
G
Google Developers Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
V2EX - 技术
V2EX - 技术
Y
Y Combinator Blog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
大猫的无限游戏
大猫的无限游戏
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Microsoft Security Blog
Microsoft Security Blog
Cisco Talos Blog
Cisco Talos Blog
T
Threatpost
Recent Commits to openclaw:main
Recent Commits to openclaw:main
S
SegmentFault 最新的问题
I
InfoQ
H
Hacker News: Front Page
D
Docker
Scott Helme
Scott Helme
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
Blog — PlanetScale
Blog — PlanetScale
人人都是产品经理
人人都是产品经理
博客园 - 叶小钗
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
N
Netflix TechBlog - Medium
AWS News Blog
AWS News Blog
Know Your Adversary
Know Your Adversary
博客园 - 【当耐特】
T
Tor Project blog
U
Unit 42
H
Heimdal Security Blog
Microsoft Azure Blog
Microsoft Azure Blog
K
KPMG report finds enterprise disconnect between AI and its ROI | CIO
P
Privacy & Cybersecurity Law Blog
PCI Perspectives
PCI Perspectives
美团技术团队
O
OpenAI News
T
Tailwind CSS Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
B
Blog
GbyAI
GbyAI
cs.CL updates on arXiv.org
cs.CL updates on arXiv.org
MyScale Blog
MyScale Blog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Zero-Downtime Crossplane v1 v2 Migration: Adopt-in-Place at Production Scale
Pavan Madduri · 2026-06-26 · via DEV Community

Crossplane v2 (released in late 2025) introduced a cleaner, namespaced resource model and removed a lot of the v1 ceremony around Claims and cluster-scoped composites. Upgrading the control plane to v2 is usually painless — if you're not using the v1 features that changed, your existing claims keep working thanks to backward compatibility.

The hard part is the next step: migrating your existing v1-style workloads onto v2-style namespaced resources. That's where there's still no cohesive, end-to-end story — and it's where I spent most of my effort taking a production EKS fleet all the way through.

This post is the field guide I wish I'd had: the adopt-in-place method, how to validate it before touching anything, and the three failure modes that will bite you in production.

Nothing here destroys or recreates cloud infrastructure. The whole point is to keep every existing AWS resource exactly where it is and just change which Crossplane resource owns it.


The setup (in generic terms)

  • A control plane running Crossplane, managing Amazon EKS clusters end to end.
  • Each cluster is represented by a Crossplane composite, which in turn owns ~90–100 managed resources (MRs): IAM roles/policies, the EKS cluster, EKS add-ons, a managed NodeGroup, a launch template, security groups and rules, an OIDC provider, and a pile of Objects managed through provider-kubernetes.
  • Several distinct cluster archetypes, each backed by its own Composition (think: general workload clusters, ingress/gateway clusters, and stateful clusters). Same migration mechanics, slightly different resource sets.
  • GitOps-driven: a Git repository is the source of truth, reconciled by a GitOps controller.

Constraints that shaped everything: no resource recreation, no node rotation, zero downtime.


Why "delete and recreate" is a non-starter

The naive migration is: delete the v1 composite, create the v2 XR, let the provider rebuild everything. In production that's a non-starter:

  • You cannot destroy a VPC, an EKS control plane, or a live NodeGroup and rebuild it under traffic.
  • Even Crossplane's Observe /import flows leave a window where the resource is briefly unmanaged or re-created.
  • Anything that recreates a NodeGroup triggers a node rotation — every pod on the cluster gets evicted and rescheduled. That's a customer-visible event you do not want as a side effect of an internal refactor.

So the goal isn't "create v2 resources." It's "make a v2 XR adopt the exact resources the v1 composite already owns, with zero observable change."


How Crossplane decides what to create vs. adopt

Two facts make adopt-in-place possible:

  1. External-name is the source of truth for the real cloud resource. Crossplane reconciles a managed resource against the actual AWS object identified by its crossplane.io/external-name annotation. If a v2-owned MR has the same external-name as the live AWS resource, Crossplane observes it instead of creating a new one.

  2. Ownership is expressed by a label + an ownerReference. A composed MR points back to its owning composite via:

    • the crossplane.io/composite label, and
    • a Kubernetes ownerReference to the owning XR (name + UID).
  3. Within a composition, each MR is keyed by a "composition-resource-name" (crn) — the crossplane.io/composition-resource-name annotation. The engine matches the desired resource the composition wants to produce against the observed MR with the same crn. Same crn + same external-name → adopt in place. Different crn → the engine thinks the desired resource is missing and creates a new one (and treats the old one as an orphan).

Adopt-in-place is just: rewrite ownership (#2) and make crn + external-name line up (#1, #3) so the v2 composition's desired output matches what's already there.


The adopt-in-place method, step by step

Step 0 — Snapshot and pre-validate (do this before touching prod)

Before any mutation, capture the live state and prove the v2 composition will adopt rather than recreate. Crossplane's render command can do this offline against observed resources:

crossplane beta render \
  xr.yaml \
  composition.yaml \
  functions.yaml \
  --observed-resources ./observed/

./observed/ holds the live MRs (exported from the cluster). The command prints the desired resources the v2 composition would produce. Diff desired vs. observed and classify every resource:

  • adopt-in-place — desired crn (after remap, see below) and external-name match an observed MR.
  • net-new — desired resource the v2 composition adds that v1 didn't have.
  • orphan — observed MR that the v2 composition doesn't produce.

Gate the migration on: zero orphans, zero unexpected net-new. This single offline check caught every surprise before it reached production.

Step 1 — Pause both the claim and the composite

kubectl annotate <claim-kind> <name> crossplane.io/paused="true"
kubectl annotate <composite-kind> <name> crossplane.io/paused="true"

Pausing only the claim is a classic mistake — the composite keeps reconciling and will fight you. Pause both.

Step 2 — Reparent every managed resource

For each MR owned by the v1 composite, repoint ownership at the new v2 XR:

  • set the crossplane.io/composite label to the v2 XR name,
  • replace the ownerReference with one pointing at the v2 XR (kind, name, UID).

Conceptually:

metadata:
  labels:
    crossplane.io/composite: <v2-xr-name>     # was: <v1-composite-name>
  ownerReferences:
    - apiVersion: <v2-xr-apiVersion>
      kind: <v2-xr-kind>
      name: <v2-xr-name>
      uid: <v2-xr-uid>                          # the new XR's uid
      controller: true
      blockOwnerDeletion: true

Script this across all ~90–100 MRs; doing it by hand is how you get an inconsistent state.

Step 3 — Point the v2 XR at the adopted resources

Patch the v2 XR so it references the composition and the exact resources it's adopting:

spec:
  crossplane:
    compositionRef:
      name: <v2-composition>
    resourceRefs:
      - { apiVersion: ..., kind: ..., name: <mr-1> }
      - { apiVersion: ..., kind: ..., name: <mr-2> }
      # ... all adopted MRs

Keep the XR paused while you do this (create it paused from the start).

Step 4 — Unpause and let it converge

kubectl annotate <v2-xr-kind> <name> crossplane.io/paused-

The engine reconciles, matches desired↔observed by crn + external-name, and adopts. Watch the XR and its MRs go Synced=True/Ready=True without any Create calls hitting AWS.


The three failure modes that will bite you

1. NodeGroup composition-resource-name drift (blue/green)

This is the one most likely to cause a real incident.

Our v1 composition emitted the managed NodeGroup with one crn (e.g. a blue/green-style nodegroup-active), while the v2 composition emits a different crn (e.g. nodegroup). Because the engine matches desired↔observed by crn, the mismatch means:

  • the v2 composition's desired nodegroup has no matching observed MR → it wants to create one, and
  • the live NodeGroup (crn nodegroup-active) has no matching desired resource → it's treated as an orphan.

The net effect would be a brand-new NodeGroup and a rotation of every node.

Fix: remap the crn annotation on the live NodeGroup to match what the v2 composition expects, and preserve the existing NodeGroup name/external-name. Don't touch external-name — that's what keeps it bound to the real AWS NodeGroup.

kubectl annotate nodegroup.<group> <existing-ng-name> \
  crossplane.io/composition-resource-name=nodegroup --overwrite

Verify no rotation after cutover by confirming the launch-template name+version and the NodeGroup version are unchanged from before:

kubectl get nodegroup.<group> <name> \
  -o jsonpath='LT={.status.atProvider.launchTemplate.name}:v{.status.atProvider.launchTemplate.version} ver={.status.atProvider.version}'

Same values before and after = the existing nodes were adopted, not replaced.

2. The cluster-auth connection-secret republish race

The subtlest one — a silent failure if you're not watching for it.

For EKS, a managed "cluster auth" resource generates the kubeconfig (a short-lived token) and writes it to a connection Secret. The provider-kubernetes ProviderConfig reads that Secret to talk to the workload cluster, and every Object on that cluster depends on it.

When the v2 XR took ownership, the connection Secret got recreated empty. If the cluster-auth resource's last token refresh happened before that recreation, it didn't immediately republish — so the Secret stayed empty. Every downstream Object then stranded with:

cannot build kube client for provider config: currentContext not set in kubeconfig

On most clusters this self-healed on the cluster-auth resource's next refresh cycle. On one, the timing left it stuck for several minutes with no sign of recovering.

Fix: force the cluster-auth resource to reconcile so it republishes the kubeconfig. A benign annotation bump does it:

kubectl annotate <clusterauth-kind> <name> \
  example.com/republish="$(date -u +%s)" --overwrite

The connection Secret repopulates, and the stranded Objects build their client and sync. The lesson: adopting an MR can re-create its connection Secret out from under downstream consumers. Put health checks on the downstream objects, not just the cluster resource, or you'll never see it.

3. GitOps source-of-truth drift

The live cutover above is imperative. Your GitOps repo still describes the v1 world. Until you reconcile it, your GitOps controller will try to "fix" the cluster back toward the manifests — unpausing the v1 claim, or having no record of the v2 XR at all.

Treat the cluster migration and the source-of-truth migration as two separate workstreams. After the live cutover, land a Git change that:

  • adds crossplane.io/paused: "true" to the v1 claim manifests, and
  • adds the v2 XR manifests without the paused annotation (so the controller manages them as the active resources).

Make sure auto-sync/self-heal won't revert your live state in the gap between cutover and merge.


A repeatable runbook

Boiled down, every cluster followed the same pipeline:

  1. snapshot — export live MRs, claim, composite, and composition.
  2. render-gatebeta render --observed-resources + diff; require zero orphans / zero unexpected net-new; confirm the NodeGroup crn remap and that launch-template/version match.
  3. reparent — script the label + ownerReference rewrite for all MRs (with a rollback script that puts them back).
  4. patch — set the v2 XR compositionRef + resourceRefs (still paused).
  5. pause v1 → unpause v2.
  6. health-gate — every MR Synced/Ready excluding a known-baseline set; NodeGroup unchanged; downstream Objects connected.
  7. reconcile Git — pause v1 manifests, add unpaused v2 manifests.

Do non-prod first, build the runbook, then prod. Keep the v1 composite paused (not deleted) for a cooldown period so rollback is a single unpause away.


What the ecosystem still needs

Most of the above was hand-rolled. A few things would turn this from "expert-only surgery" into a supported workflow:

  1. A migrate command that, given a v1 claim/composite and a target v2 composition, generates the reparent patches, the v2 XR with populated resourceRefs, and — critically — a crn remap table between the two compositions. Matching must be by external resource identity, not crn string equality.
  2. An adopt-preview/dry-run that classifies every MR as adopt-in-place / net-new / orphan and gates on zero orphans before proceeding (productizing the render --observed-resources diff).
  3. Connection-secret-aware adoption — on adoption, force a reconcile or wait on connection-secret readiness so downstream providers don't lose connectivity.

There's an active community effort around exactly this (a maintainer-run feedback discussion and a migration-tooling tracking issue, plus a community CLI for migrating composition manifests). If you've done a migration like this, your war stories are genuinely useful input — the design is still being shaped.


Takeaways

  • Adopt, don't recreate. Make a v2 XR own the exact MRs the v1 composite owned; never let external-names change.
  • Validate offline first. beta render --observed-resources + a desired/observed diff is the single highest-leverage safety check.
  • crn alignment is everything for NodeGroups — a mismatch is the difference between a silent adoption and a full node rotation.
  • Watch your connection secrets. Adoption can recreate them empty; downstream consumers fail silently until the owner republishes.
  • Two migrations, not one. The live cluster and the GitOps source of truth move separately — plan for both.

It's very doable to take a production fleet from v1 to v2 with zero downtime today — it just isn't yet a one-command experience. Hopefully this shortens the path for the next person.