惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

GbyAI
GbyAI
N
News and Events Feed by Topic
D
DataBreaches.Net
MongoDB | Blog
MongoDB | Blog
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Engineering at Meta
Engineering at Meta
T
Tailwind CSS Blog
博客园_首页
Microsoft Azure Blog
Microsoft Azure Blog
Y
Y Combinator Blog
博客园 - Franky
Hugging Face - Blog
Hugging Face - Blog
月光博客
月光博客
A
About on SuperTechFans
I
InfoQ
S
Securelist
Last Week in AI
Last Week in AI
S
Schneier on Security
C
CXSECURITY Database RSS Feed - CXSecurity.com
Hacker News: Ask HN
Hacker News: Ask HN
Schneier on Security
Schneier on Security
Know Your Adversary
Know Your Adversary
腾讯CDC
大猫的无限游戏
大猫的无限游戏
S
Security @ Cisco Blogs
博客园 - 三生石上(FineUI控件)
Simon Willison's Weblog
Simon Willison's Weblog
D
Darknet – Hacking Tools, Hacker News & Cyber Security
T
Tor Project blog
美团技术团队
aimingoo的专栏
aimingoo的专栏
G
Google Developers Blog
罗磊的独立博客
Vercel News
Vercel News
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
The Cloudflare Blog
S
Secure Thoughts
www.infosecurity-magazine.com
www.infosecurity-magazine.com
Latest news
Latest news
Recent Announcements
Recent Announcements
Exploit-DB.com RSS Feed
Exploit-DB.com RSS Feed
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
L
LINUX DO - 热门话题
Security Latest
Security Latest
TaoSecurity Blog
TaoSecurity Blog
Cyberwarzone
Cyberwarzone
有赞技术团队
有赞技术团队

Hacker News: Show HN

PurrrrrFocus: Pomodoro Timer App - App Store Workflow Engine — Multi-Step Orchestration for Bun RapidPhoto: Pro Photo Editor App - App Store GitHub - DheerG/swarms: Achieve extraordinary results with claude code across a variety of tasks SPICE simulation → oscilloscope → verification with Claude Code — Lucas Gerads Show HN: VCoding – A 5 MB native Windows IDE with no dynamic dependencies Show HN: LLMs don't hallucinate because they're bad at math, it's the format GitHub - Agent-FM/agentfm-core: AgentFM is a peer-to-peer network that turns everyday computers into a decentralized AI supercomputer. AgentFM lets you run massive AI workloads directly across a global mesh of idle CPUs and GPUs. Show HN: Tracking Top US Science Olympiad Alumni over Last 25 Years GitHub - Potarix/agent-hub: One place to talk to all your agents Show HN: Runtime security for AI agents(injection,tool abuse, data exfiltration) GitHub - dubeyKartikay/lazyspotify: Terminal Spotify client for macOS and Linux GitHub - the-banana-tool/king-louie: Easy to use GUI Personal AI Assistant. Win/Linux/Mac. Show HN I made my vacation rental bookable by AI agents–no Airbnb, 0% commission GitHub - basteez/jsf-autoreload: maven plugin to enable hot reload on jsf projects uvm32/hosts/host-gdbstub at main · ringtailsoftware/uvm32 GitHub - labsai/EDDI: Config-driven engine that turns JSON into production-grade AI agents. Multi-agent orchestration, 12+ LLM providers, MCP/A2A protocols, RAG, persistent memory, and enterprise compliance (EU AI Act, GDPR, HIPAA). Built on Quarkus. GitHub - glitchnsec/fortyone-oss: AI Executive Assistant Platform Quickstart | Alien GitHub - muxshed/shed: One stream in, or many. Every destination, simultaneously. No cloud middleman, no per-channel fees, no limits. GitHub - ocrbase-hq/ocrbase: 📄 PDF/IMG ->.MD/JSON Document OCR API for PaddleOCR and GLMOCR. Self-hostable. GitHub - impactjo/home-memory: MCP server that lets your AI assistant remember everything about your home. GitHub - Sets88/dbcls: DbCls is a powerful terminal database client that supports various databases GitHub - neptun2000/heor-agent-mcp GitHub - SeanFDZ/macmind: Single-layer transformer in HyperTalk for the classic Macintosh RollQuation: Math Puzzles - Apps on Google Play GitHub - dropbox/witchcraft Show HN: Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis GitHub - opentalon/opentalon: OpenTalon is an open-source platform built from the ground up in Go as a robust alternative to OpenClaw LinkedIn™ 职位抓取工具 - Chrome 应用商店 GitHub - EdoardoBambini/Agent-Armor-Iaga: AI agents are getting tool access — shell, file system, databases, APIs, secrets. But **nobody is governing what they actually do with it**. Frameworks like LangChain, CrewAI, AutoGen, and Claude Code give agents the power to execute. Agent Armor gives you the power to control, audit, and approve every single action before it happens. HN Vibes — Week 15, Apr 7–13 2026 GitHub - chojs23/ec: Easy terminal-native 3-way git mergetool vim-like workflow GitHub - SethPyle376/hiraeth: Local AWS emulator focused on fast integration testing, with SQS support, SQLite-backed state, and a debug-friendly web UI. GitHub - JakOb-dotcom/cloud-sandbox-security-analysis: Technical analysis and Proof of Concept (PoC) regarding environment variable exfiltration in containerized cloud sandboxes via side-channel data leaks. Springboards - Flint Alpha Show HN: A simpler coding agent harness GitHub - audiodude/sudomake-friends GitHub - 256thFission/mini-mythos: OSS clone of Anthropic’s Mythos harness to locate C/C++ memory vulnerabilities Show HN: OpenParallax: OS-level privilege separation for AI agent execution Hacker News Sorted - Chrome 应用商店 Show HN: How to Install Docker on Ubuntu 24.04 LTS: Complete 2026 Guide GitHub - himanshudongre/smriti GitHub - sverrirsig/claude-control: macOS desktop dashboard for monitoring and managing multiple Claude Code sessions GitHub - ory/dockertest: Write better integration tests! Dockertest helps you boot up ephermal docker images for your Go tests with minimal work. Chiral - Chrome 应用商店 Show HN: Two Claudes collaborating through shared memory on a $100 mini-PC GitHub - pmichaillat/latex-cv: Minimalist LaTeX template for academic CVs GitHub - oguzbilgic/posse: A web UI for Anthropic Managed Agents. GitHub - sshiraz/depsly: Dependency risk analysis tool for npm packages ABI Add safari/agent-harness — Safari browser automation via safari-mcp by achiya-automation · Pull Request #212 · HKUDS/CLI-Anything GitHub - Halfblood-Prince/trustcheck: Verify PyPI package attestations and improve Python supply-chain security GitHub - oguzbilgic/kern-ai: Agents that do the work and show it. GitHub - bruits/satteri: High-performance Markdown and MDX processing for the JavaScript ecosystem GitHub - tylergibbs1/feedstock: High-performance web crawler and scraper for TypeScript, powered by Bun and Playwright GitHub - Grimm67123/grimmbot: The self-improving sandboxed and open-source AI agent. With persistent memory and scheduling. GitHub - whitevanillaskies/whitebloom: Local whiteboard that blooms. GitHub - hwdsl2/docker-whisper: Docker image for a self-hosted Whisper speech-to-text server with speaker diarization and OpenAI-compatible transcription and translation APIs. Powered by faster-whisper. Supports all Whisper models, NVIDIA GPU (CUDA) acceleration, JSON/SRT/VTT output, SSE streaming, offline mode, and multi-arch (amd64, arm64). GitHub - yisding/reviewwiggum GitHub - MarwanAlsoltany/serrors: Structured errors for Go: sentinel hierarchies, typed data, custom formatting, and slog integration. GitHub - soatok/age-php GitHub - Luthiraa/markitme GitHub - stagas/rtdiff: realtime git diff gui and AI-assisted commits GitHub - tombedor/excalicharts GitHub - wh1le/excalidraw-edit: Open and edit .excalidraw files from the terminal. Offline, auto-saves to disk. MalExt Sentry - Malicious Extension Scanner - Chrome 应用商店 GitHub - syi0808/asciianimesvg: Generate animated ASCII art SVGs from text. CLI, Rust library, WASM, and web editor. GitHub - zaina-ml/ml_forge: A visual-based graph node editor for training computer vision models. GitHub - anakin87/llm-rl-environments-lil-course: 🌱 A little course on Reinforcement Learning Environments for evaluating and training Language Models GitHub - takaakit/superpowers-uml: Superpowers-UML modifies Superpowers to ensure a software development workflow in which AI agents design through UML modeling. AdriByte Studio - Sviluppo Web e Soluzioni Digitali GitHub - chouligi/angel-copilot: Your personalized Angel Investment Advisor Show HN: MoodSense AI (ML and FastAPI and Gradio, Deployed on Hugging Face) Moodsense Ai - a Hugging Face Space by aman179102 GitHub - agenteractai/lodmem: Level Of Detail Context Management for Agents GitHub - ostefani/subnetlens: A fast, concurrent network scanner with a TUI and plain-text CLI, built in Go. It discovers live hosts on your network, scans their open ports, resolves hostnames, and fingerprints operating systems—delivered. Cyber Pulse: Agentic Intel - Apps on Google Play Whisper API: Self-Hostable Speech to Text Transcription The Agent-Web Protocol Stack: A Research Thesis GitHub - msmarkgu/RelayFreeLLM: A restful API designed to route user prompts to various AI model providers. Show HN: Provepy – A Python decorator that proves your code using Lean and LLMs Show HN: Pardonned.com – A searchable database of US Pardons GitHub - patrickdappollonio/dux: Dux is a terminal UI that lets you run multiple AI coding agents side by side, each in its own git worktree, with full companion terminals, macros, commit generation, and a command palette that knows more tricks than you do. kMC Crystal Simulator Show HN: HyperFlow – A self-improving agent framework built on LangGraph GitHub - stef41/vibescore: 🎵 Grade your vibe-coded project. One command, instant letter grade across security, quality, dependencies, and testing. GitHub - stef41/lmscan: 🔍 Detect AI-generated text and fingerprint which LLM wrote it. Open-source GPTZero alternative. Zero dependencies, works offline. imgur.com GitHub - visionscaper/collabmem: Enabling long-term collaboration with Agentic AI - building up episodic and world model memory over time with in-context awareness 在 Steam 上购买 FriedrichAI: Offline AI 立省 10% GitHub - atripati/ark: AI Runtime Kernel — a context operating system for AI agents. Eliminates tool bloat, loads only what’s needed, and gives LLMs their reasoning space back. GitHub - nowork-studio/toprank: Open-source Claude Code skills for SEO, SEM, Google Ads GitHub - tacomanator/sash: Lightweight macOS menu bar app for reliably cycling through windows of the current application. Appents | Social Media Management for Product-First Teams GitHub - pnhoang/youtube-spam-blocker: Automatically detects and hides spam messages in YouTube Live chat. Set rate limits, keyword filters, and block repeat offenders. GitHub - decisionnode/DecisionNode: CLI + Local MCP - A shared structured memory store across Claude Code, Cursor, Windsurf, Antigravity, and every MCP client. Semantically queryable. GitHub - AvaCodeSolutions/django-email-learning: An open source Django app for creating email-based learning platforms with IMAP integration and React frontend components. The $100K Gap in Kubernetes Security Tooling Function Calling Harness: From 6.75% to 100%
GitHub - qosha1/remote-compose: Give your coding agent a full remote env with running docker composes with 1 command
quinnosha · 2026-05-09 · via Hacker News: Show HN
ChatGPT Image May 8, 2026, 11_07_11 AM

Take any docker-compose.yml and put it on AWS — as a fresh dev box with Claude already running inside, or as production infrastructure.

Two flavors of one tool:

🚀 rc dev up — disposable cloud dev environments with an agent inside

rc dev up alice \
  --repo https://github.com/owner/myapp \
  --compose docker-compose.yml \
  --gh-token "$(gh auth token)" --skip-permissions

In ~5 minutes you get a fresh EC2 box with: docker, your repo cloned, the compose stack running, and Claude Code pre-authenticated in a tmux session waiting for you to attach. rc dev attach alice drops you into it. One box per agent or per branch. Work in parallel on isolated infra. rc dev destroy when done.

Multi-repo deploys, env-file shipping, gh/bd pre-installed, shared SSH key + auth lifecycle — all built in.

🏗️ rc deploy — generate a real terraform module from your compose

rc up --from-compose docker-compose.yml

Reads your compose, asks for the handful of things compose can't express (CPU/memory, secrets, public hostname, EFS uid), generates a clean ECS terraform module, applies it. Then everyday verbs: rc deploy, rc lifecycle migrate, rc db push, rc destroy. The module is yours — cd terraform/ && terraform apply works without rc.

Status: alpha. Hand-tested against a real production Django stack. Active branch: portable-deploy. Legacy v1 (SSH/Django-app) lives below for users on main. See ARCHITECTURE.md for design + validation, AGENTS.md for workflow.


Why both modes share one tool

Same insight, two surfaces. Compose is already the spec for "how my services fit together." For dev we ship that spec to an EC2 box and start an agent inside. For production we render it to ECS terraform. You write the compose file once.

Most deploy tooling makes you choose between cloud-specific knobs (ECS task defs, k8s manifests, Helm), opinionated black-box PaaS (Heroku, Fly, Render), or hand-rolled terraform (flexible but a 500-line module per service). remote-compose takes the other bet: your compose file is the topology. The tool's job is to add the few things compose can't express and emit something clean — terraform you own, or an EC2 box that disappears when you're done with it.


Quick start

Bootstrapping a fresh machine? Run bash scripts/bootstrap-from-zero.sh instead of step 1 — it installs terraform via the platform package manager (brew/apt/dnf), creates a .venv, runs pip install -e ".[ecs]", and verifies rc doctor is all-green. Idempotent — safe to re-run.

# 1. Install (only ECS provider ships today — k8s is roadmap)
pip install -e ".[ecs]"

# 2. In your app repo (alongside docker-compose.yml)
rc init --from-compose docker-compose.yml   # scaffold a v2 rc.yml from your compose
$EDITOR rc.yml                              # tweak cpu/memory/health checks

# 3. Configure cloud creds (ECS example)
export AWS_PROFILE=myprofile

# 4. One-shot: scaffold (if missing) → deploy → push secrets → ALB URL
rc up --from-compose docker-compose.yml     # the lazy path, idempotent

# 4b. Or step through it manually
rc plan                            # show what terraform would create
rc deploy                          # build images, terraform apply, force-rolls
rc secrets push                    # upload .env files into AWS Secrets Manager
rc lifecycle migrate               # run a named hook in a live container
rc status                          # ECS service health table
rc exec django -- /bin/bash        # interactive shell
rc db push /tmp/local-dump.dump    # seed the deployed db from a local dump
rc destroy --yes                   # tear it all down

To verify the documented commands exist as advertised, run bash scripts/test-readme-quickstart.sh — it audits rc --help against this section without touching AWS.

Every command is declarative + idempotent. Re-running rc deploy after no changes prints no changes — infrastructure matches config.


First-deploy walkthrough

If you want to verify rc actually works end-to-end against real AWS before you commit to it, the repo ships a scripted acceptance trace that takes a clean account → a fully-running production-shape Django + celery + nginx stack → clean teardown. Single command, no aws-cli, no sed, no /tmp dance.

# Prereqs: terraform installed (or run scripts/bootstrap-from-zero.sh
# first), an AWS profile with creds, and a Django+celery compose to
# point at. We use start-simpli (private repo) — substitute your own
# via the START_SIMPLI / COMPOSE_FILE / REGION / AWS_PROFILE_OVERRIDE
# env vars at the top of the script.

bash scripts/test-startsimpli-end-to-end.sh

What it does, step by step:

  1. rc destroy --all-ephemeral — clean slate. Removes any prior ephemeral stacks from the local registry.
  2. rc up --from-compose docker-compose.local.yml --aws-profile X --region Y --ttl 4h — single-command full deploy. Scaffolds an rc.yml from your compose, auto-fixes nginx for ECS Cloud Map (variable-based proxy_pass + VPC resolver), imports any orphan Container Insights log group, runs terraform apply, builds + pushes images, force-rolls services, pushes file-sourced secrets into Secrets Manager, runs auto_on_deploy lifecycle hooks (e.g. python manage.py migrate --noinput).
  3. rc status polling — waits for all services to reach health=healthy.
  4. Plain curl http://<ALB>/api/v1/health/ — no Host: header rewrite, no https, no --insecure. The patient retry loop tolerates the ~60-90s window where ECS marks the task healthy but Django is still finishing migrations + collectstatic + runserver. When it returns 200, the body is checked for {"celery":"healthy"} — real workers responding to ping, not "no_workers".
  5. rc destroy --yes — clean teardown. Removes every AWS resource tagged Project=<this-stack> and unregisters the entry from the ephemeral registry.
  6. rc list --ephemeral — registry empty. No stale rows.

Scripted. Repeatable. The tracking bead is rc-e5u.46 (bd show rc-e5u.46).


What rc.yml v2 looks like

version: 2
project: my-app
compose_file: docker-compose.yml
provider: ecs

provider_config:
  ecs:
    cluster: my-app-prod
    region: us-west-1
    aws_profile: myprofile
    vpc_cidr: 10.0.0.0/16
    route53_zone: rctest.example.com   # override if zone != domain[-2:]

terraform:
  output_dir: ./terraform/${provider}
  backend:
    type: s3                           # or local
    bucket: my-app-tf-state
    key: ecs.tfstate
    region: us-west-1

# Auto-creates the backup S3 bucket via terraform with versioning + AES256
# + 14-day expiration. Set bucket_managed: false to point at an existing
# bucket you own elsewhere.
backup:
  bucket: my-app-backups
  service: postgres                    # which container hosts pg_restore
  retention_days: 14                   # or "never"

# .env files become Secrets Manager JSON blobs; provider emits one
# task-def `secrets[]` entry per KEY using arn:KEY:: selectors so each
# key arrives as its own env var (vs one giant blob).
secrets:
  - name: django
    source: file
    path: .envs/.production/.django
  - name: postgres
    source: file
    path: .envs/.production/.postgres

# Compose-driven deploy set. Default: every compose service deploys with
# sensible defaults; rc.yml services[] is for overrides. Use exclude/include
# for dev-only services (ngrok, debug profiles, etc.).
compose:
  exclude: [ngrok]                     # mutually exclusive with include

services:
  postgres:
    type: infrastructure
    cpu: 512
    memory: 1024
    volumes:
      - name: pgdata
        mount: /var/lib/postgresql/data
        # Per-service posix_user on the EFS access point so initdb
        # can chown — postgres:17 alpine = uid 70, debian = 999.
        uid: 999
        gid: 999
        mode: "0700"

  django:
    type: application
    cpu: 1024
    memory: 2048
    port: 8001
    health_check_path: /api/v1/health/
    ephemeral_storage: 21              # GiB; FARGATE 21–200
    lifecycle:
      migrate:
        command: ["python", "manage.py", "migrate", "--noinput"]
        auto_on_deploy: true           # runs after every rc deploy
      createsuperuser:
        command: ["python", "manage.py", "createsuperuser", "--noinput"]
        run_once: true                 # skips when probe exits 0
        probe:
          - python
          - -c
          - |
            import os, django, sys
            django.setup()
            from django.contrib.auth import get_user_model
            sys.exit(0 if get_user_model().objects.filter(
                email=os.environ['DJANGO_SUPERUSER_EMAIL']
            ).exists() else 1)
      shell:
        command: ["python", "manage.py", "shell"]
        interactive: true              # forwards a TTY

  nginx:
    type: proxy
    cpu: 256
    memory: 512
    port: 80
    public: true
    default_target: true               # catches anything the host rules don't
    domain: app.example.com            # primary; this name routes here
    aliases:                           # extra hostnames same service answers for
      - api.app.example.com            #   (cert SANs + R53 records, no listener rules)
    health_check_path: /health

Full schema reference: ARCHITECTURE.md § rc.yml v2 at a glance.


Feature index

What's built and live-verified on the portable-deploy branch:

Provider abstraction

  • Provider ABC — every cloud target (ECS today, K8s next) implements emit_terraform, plan, deploy, redeploy, status, logs, exec, rollback, destroy against a shared DeployContext.
  • FakeProvider for tests — every contract test runs against both ECSProvider and FakeProvider so adding a new provider is a copy-paste exercise.
  • rc-test- tag* — every project named rc-test-* gets Environment=rc-test tags + force_destroy=true on destructive resources, so test stacks always tear down clean.

ECS provider — what terraform we generate

  • VPC + 2 public + 2 private subnets, IGW, security groups, default routing
  • ECS cluster with Container Insights (log group terraform-managed)
  • Per-service: ECR repo, task def, ECS service, Cloud Map service-discovery entry
  • ALB with HTTP→HTTPS redirect (when domain is set) + ACM cert + R53 alias records
  • EFS file system + access point per stateful volume; per-service posix uid/gid/mode
  • AWS Secrets Manager: one secret per .env file, JSON-blobbed, individual keys exposed via ECS arn:KEY:: selectors
  • ECS Exec wired (task role gets ssmmessages:*); enable_execute_command = true on every service
  • ALB host-routing: per-service domain → ALB listener rule + per-service target group
  • Single fronting service: aliases: adds cert SANs + R53 records without listener rules
  • S3 backup bucket auto-created with versioning + lifecycle when backup.bucket is declared
  • Stateful services (any with EFS) auto-set deployment_minimum_healthy_percent = 0 so rolling deploys can't race-corrupt postgres data

CLI

command does
rc init scaffold a v2 rc.yml
rc migrate --in rc.yml --out rc.v2.yml convert legacy v1
rc plan terraform plan summary
rc deploy [--no-build] build, push, terraform apply, force-roll, run auto_on_deploy hooks
rc destroy --yes terraform destroy
rc status ECS service health table
rc exec <service> -- <cmd...> run a one-off command inside a live task; reliable stdout via sentinels; full TTY when stdin is a tty
rc lifecycle <hook> [<service>] run a named hook from rc.yml (resolves declarer; handles run_once probes)
rc secrets push [--rollout/--no-rollout] parse each .env file → upload as JSON to its SM secret → force-rolls every service
rc db backup / rc db restore / rc db list postgres backup round-trips through S3 (host-side presigned URLs; tasks just curl)
rc db push <file> upload a local dump → exec pg_restore inside the deployed container; auto-detects format from extension (.dump, .tar.gz, .sql); bootstraps curl + ca-certificates in containers that don't ship them
rc copilot import migrate an AWS Copilot app to rc.yml v2 + docker-compose; supports --env <name> for per-environment overrides (guide)
rc doctor preflight: terraform/docker/python/boto/AWS creds checked
rc install platform package-manager fix for missing deps

Compose feature support

  • build: with optional target: (multi-stage), args:, dockerfile: — relative dockerfile resolved against the build context (the natural compose semantic)
  • image: — pre-built image used verbatim, ECR push skipped
  • command: — overrides the container CMD
  • environment: (dict or list) AND env_file: (list or single string, paths relative to compose dir, multiple files merge in declaration order, environment: map wins on conflict)
  • ports: — when public, primary port goes to ALB target group; remaining ports become additional containerPorts in the task def, intra-VPC reachable via the existing tasks SG (use this for VNC, devtools, internal-only ports)
  • volumes: — EFS-backed when declared in rc.yml with explicit mount: and uid/gid

Lifecycle commands

Declarative one-off operations live in rc.yml as services[*].lifecycle.<hook>:

lifecycle:
  migrate:
    command: ["python", "manage.py", "migrate", "--noinput"]
    auto_on_deploy: true        # rc deploy runs this after rollout
  createsuperuser:
    command: ["python", "manage.py", "createsuperuser", "--noinput"]
    run_once: true
    probe: [python, -c, "import sys; sys.exit(0 if user_exists() else 1)"]
  shell:
    command: ["python", "manage.py", "shell"]
    interactive: true           # TTY passthrough

auto_on_deploy: true runs the hook after every successful rc deploy, in declaration order, with hook failures surfaced as warnings (not deploy failures — rerun rc lifecycle <hook> for full output).

run_once: true runs the probe: first; non-zero exit ⇒ "not yet done" ⇒ run the hook. Idempotent createsuperuser, fixture loading, schema bootstrap.

AWS Copilot migration

AWS Copilot reaches end-of-support on 2026-06-12. Every team running on Copilot needs a path off it. rc copilot import is that path — it reads any copilot/ directory tree (services, environments, addons, pipelines) and writes a working rc.yml v2 + docker-compose.yml

  • IMPORT_SUMMARY.md.
rc copilot import \
    --from ./copilot \
    --out  . \
    --env  production \
    --project my-app

What translates today:

Copilot construct rc translation
Backend Service private rc service (no public, no ALB)
Worker Service rc service type: worker
Load Balanced Web Service public rc service + port + default_target + domain (from http.alias) + aliases
image.build: { context, dockerfile, target, args } docker-compose build: block (multi-stage target honored)
image.location docker-compose image: (Copilot's ${TAG} interpolation preserved)
cpu, memory, count rc.yml cpu, memory, replicas
storage.volumes.<n>: { path, efs: {uid, gid} } rc.yml volumes with EFS access-point uid/gid
variables: { KEY: value } docker-compose environment:
secrets: { KEY: { secretsmanager: arn } } rc.yml secrets: source: aws_sm
environments.<env> overrides deep-merged when --env <env> passed
${COPILOT_ENVIRONMENT_NAME} resolved when --env is passed; left literal otherwise

What gets flagged for review (typed warnings grouped in IMPORT_SUMMARY.md):

Copilot construct warning
Request-Driven Web Service UnsupportedServiceTypeWarning — App Runner is a different runtime; best-effort translated to public ECS for review
Static Site UnsupportedServiceTypeWarning — CloudFront+S3 has no ECS analogue; emitted to compose.exclude so it's not silently dropped
count: { range, cpu_percentage } ScalingNotSupportedWarning — autoscaling not yet emitted; replicas pinned to range floor
count: 0 ScalingNotSupportedWarning — ECS doesn't scale-to-zero; replicas=1
exec: false ExecDisabledIgnoredWarning — provider always enables ECS Exec
network.vpc.placement: private PrivateSubnetUnsupportedWarning — public-subnet Fargate today (rc-e5u.25 tracks the NAT variant)
addons CFN templates listed in summary — translate to terraform manually (P3 backlog)

Tested against a corpus of real Copilot apps including:

  • aws/copilot-cli e2e fixtures (canonical LBWS, app-with-domain, static-site)
  • a public external example (ShanikaEdiriweera/aws-copilot-example)
  • a 15-service production-grade app (sentinal: backend + workers + nginx + multi-env + secretsmanager refs)

Local-data seeding (rc db push)

Spin up a test stack in a separate region, seed it with real data from a local Docker volume, validate, tear down. Repeat. The flow:

docker exec my_postgres pg_dump -Fc -U postgres my_db > /tmp/seed.dump
rc deploy
rc secrets push
rc db push /tmp/seed.dump

rc db push uploads to the configured backup bucket via host-side boto3, generates a presigned GET URL, exec's a sentinel-bracketed restore script inside the deployed postgres container that downloads with curl (or bootstraps curl via apt-get when the image doesn't ship it), runs pg_restore --no-owner --clean --if-exists, and deletes the S3 staging object on success.


Mental model in 5 lines

  1. Compose is the topology. Adding a service to docker-compose.yml deploys it (defaults: 256 CPU / 512 MB / application if it has ports, worker otherwise).
  2. rc.yml is the tuning. Override CPU, memory, port, public, domain, lifecycle, secrets, volumes, EFS uid, etc. per-service.
  3. The provider is thin. It generates a terraform module from the merged config; you can cd terraform/ && terraform apply without rc ever again.
  4. Secrets are JSON in SM. Each .env file becomes one secret; each KEY in that file becomes a separate task-def env var via ECS JSON-key selectors.
  5. Test stacks are disposable. rc-test-* projects auto-set force_destroy=true on every resource; rc destroy tears them down clean.

Codebase map

remote_compose/
├── cli.py                           # legacy v1 commands + v2 dispatch
├── cli_v2.py                        # v2 CLI: load_rc_yml, build_deploy_context, dispatch_if_v2
├── config/
│   ├── v1_schema.py                 # legacy flat schema loader
│   ├── v2_schema.py                 # ServiceV2, RcConfigV2, ComposeConfig, BackupConfig, ...
│   └── migrate.py                   # v1 → v2 with warnings on stateful services
├── envfile.py                       # standalone .env parser (used by provider + rc db push + lifecycle)
├── copilot/
│   ├── discover.py                  # walk copilot/ → typed CopilotApp model
│   └── translate.py                 # 5 focused translators + composer + warning types
├── provider/
│   ├── base.py                      # Provider ABC, ServiceSpec, DeployContext, ExecResult, ...
│   ├── fake.py                      # in-memory provider for the contract suite
│   └── ecs/
│       ├── provider.py              # ECSProvider implementation
│       ├── autosize.py              # EC2 capacity provider sizing
│       ├── ecr_auth.py              # ECR login for image push
│       └── templates/
│           ├── alb.tf.j2            # ALB + listeners + per-service target groups + host rules
│           ├── backend.tf.j2        # terraform backend
│           ├── backup.tf.j2         # S3 backup bucket + lifecycle
│           ├── capacity.tf.j2       # EC2 capacity provider
│           ├── cluster.tf.j2        # ECS cluster + container-insights log group
│           ├── domain.tf.j2         # ACM cert (with SANs) + R53 records
│           ├── efs.tf.j2            # EFS + access points (uid/gid/mode)
│           ├── iam.tf.j2            # task-execution + task roles + ssmmessages policy
│           ├── network.tf.j2        # VPC, subnets, IGW, route tables
│           ├── outputs.tf.j2        # ECR repo URLs, ALB DNS
│           ├── providers.tf.j2      # AWS provider block
│           ├── secrets.tf.j2        # SM secret placeholders
│           ├── security_groups.tf.j2
│           ├── service_discovery.tf.j2  # Cloud Map private namespace
│           ├── services.tf.j2       # ECS task def + service per compose service
│           └── variables.tf.j2
├── image/
│   ├── builder.py                   # docker build wrapper (handles relative dockerfile)
│   └── pusher.py                    # docker push to ECR/GCR/etc.
└── terraform/
    ├── backend.py                   # render_backend_block
    ├── emitter.py                   # Jinja2-based directory render
    └── runner.py                    # subprocess wrapper for terraform CLI

tests/
├── unit/                            # per-module unit tests
├── contract/test_provider_contract.py   # runs against ECSProvider + FakeProvider
├── integration/test_provider_ecs_terraform.py  # invokes real `terraform validate`
├── e2e/                             # opt-in real-AWS tests (RC_E2E=1)
└── fixtures/golden/ecs_minimal/     # byte-for-byte expected HCL output

examples/
├── demo-app/                        # FastAPI + worker + postgres + redis reference stack
└── sample-app/                      # minimal hello-world

ARCHITECTURE.md § Layers has the import-rule diagram.


Build & test

# Dev install
pip install -e ".[ecs]"
pip install -r requirements/dev.txt

# Fast (12s): unit + contract
pytest tests/unit/ tests/contract/

# Adds: real `terraform init -backend=false && terraform validate`
pytest tests/integration/

# Full opt-in real-AWS suite (~25 min, requires creds)
RC_E2E=1 pytest -m e2e tests/e2e/

# Regenerate the byte-identical golden HCL fixture
python -m tests.unit.test_provider_ecs.test_golden --regenerate

# Linters
black remote_compose/
flake8 remote_compose/

The contract suite is the heart of provider parity. Any new provider ships only when pytest tests/contract/test_provider_contract.py is green against it.


Roadmap / open work

Tracked in beads. To inspect:

bd ready                      # available work
bd show rc-e5u                # the umbrella epic
bd list --status=open         # everything still open

High-signal open items (as of this writing):

  • Kubernetes provider (rc-e5u.8) — proves the multi-cloud claim
  • Private subnets + NAT (rc-e5u.25) — currently public-subnet Fargate for cost
  • EFS encryption on fresh accounts (rc-e5u.26) — KMS key bootstrap
  • rc audit (rc-e5u.37.4) — post-destroy AWS-side cleanup verification
  • rc db dump-local (rc-e5u.37.3) — wraps docker exec pg_dump with port autodiscovery
  • rc compose import (rc-e5u.41.3) — scaffold rc.yml from a compose file
  • Framework presets (rc-e5u.35.7) — auto-default lifecycle hooks for django/rails/phoenix/laravel
  • Provider auto-import of orphan log groups (rc-e5u.37.5) — terraform import on first-run conflicts

Design principles

  1. Be a generator, not a runtime. Every piece of state we own should also be readable as plain terraform / plain JSON. Users escape rc cleanly.
  2. Compose is the contract. Don't invent parallel config; consume the file the team already maintains.
  3. Test against real clouds. Unit tests catch shape regressions; the real validator is terraform validate + a live e2e against rc-test-* projects.
  4. One-off operations get first-class commands. Lifecycle hooks, db push, exec, secrets push — all CLI verbs, not bash scripts users have to copy.
  5. Reproducible test stacks. rc-test-* namespaces auto-tear-down; isolation is a property of the project name, not user discipline.
  6. No backwards-compat ratchets in alpha. When the right shape conflicts with the old shape, file a bead, change both at once. Backward-compat shims live only as long as we're sure they don't trap us.

See AGENTS.md for the day-to-day workflow.


Related docs


Legacy v1 (pre-portable)

The content below describes the v1 SSH/Django-app deploy path on main. The portable provider work above lives on portable-deploy. v1 still ships for users on the older path; v2 is the active line.

Features (v1)

  • Docker Context Management: Create and manage Docker contexts for remote deployment targets
  • Docker Compose Deployment: Deploy docker-compose.yml files to remote hosts via SSH
  • AWS EC2 Integration: Auto-discover EC2 instances and create deployment targets
  • AWS ECS Integration: Deploy to AWS ECS (Fargate or EC2) without SSH
  • Async Deployments: Celery tasks for background deployment operations
  • Health Monitoring: Continuous health checks for targets and deployments
  • Multi-Service Orchestration: Deploy multiple services with sequential, parallel, rolling, or canary strategies
  • Rate Limiting: Protect against deployment abuse with configurable rate limits
  • Audit Logging: Track all deployment-related actions for compliance
  • Secure Credential Storage: Fernet-encrypted storage for SSH keys and AWS credentials
  • Log Sanitization: Automatic masking of sensitive data in logs
  • Webhooks & Notifications: Slack, email, and custom webhook notifications
  • Deployment History: Full deployment tracking with rollback capability

For the full v1 reference (Django settings, management commands, API viewsets, etc.) see the file history of this README in git log — the prior version is preserved at git show main:README.md.

License

MIT — see LICENSE for terms.