This is the first part of a multipart series introducing tc Cloud Functors
Sometimes I feel like the Forrest Gump of computing, having worked on or been around everything from the introduction of mini and microcomputers to starting an ISDN ISP in 1993, pioneering international server farms, and building early e-commerce sites through the earliest days of the cloud, working with AWS in 2008. (and so much more!)* Since then I've specialized in serverless infrastructure and systems.
About three years ago, I joined a startup called Informed, whose product is a very sophisticated Intelligent Document Processing pipeline for lenders. Informed had the classic situation of a startup that has found its market with a working product, but the implementation is a mirror of the random walk they took to discover their market/product fit. They were processing about 8% of all US car loans, dealing with heavy bank regulations, and had the terrifying constraint of infinite PII. Yet, the tech stack was a typical startup tangle: a massive Ruby on Rails app sitting at the center of the universe, talking to just about every AWS service imaginable through Elastic Beanstalk, and using a single Postgres database as a message bus. I don't say this with disdain. It was a working system that had lots of quality engineering and had been very successful in delivering services to some of the top banks and generating profits.
Most startups have this random wandering in the desert until they find their product/market fit, and their software tends to follow the same route. Unfortunately, it was also a nightmare of tech debt where adding a single feature felt like moving a mountain.
We knew we had to move to a modular, event-driven architecture, but we weren't just talking about a couple of functions. We were looking at a future that now consists of 107 topologies and 340 Lambdas across Python, Ruby, and Node. Managing that kind of complexity for a small team of developers is the real hurdle. Traditional tools like CloudFormation which, frankly, I find to be a collection of frustrating magic spells. Using CloudFormation felt like trying to build a high-performance engine out of low-level, brittle bones. Even Terraform, while it sucks less, didn't have the right impedance match for the high-velocity muscle of a distributed serverless system.
We knew we wanted to move to a more unified and serverless style of infrastructure. The use case matched serverless perfectly, as its API driven, machine-to-machine work primarily. It's very bursty and follows the normal sine wave pattern of the US business hours. We also found that the existing tooling did not lend itself to building such complex serverless systems.
Before we had tc our developers were spending 80% of their time manually wiring infrastructure and only 20% writing logic; we had already lost. We needed to stop managing resources and start programming the cloud as a single computer. This frustration with manual wiring led us to a breakthrough: moving away from resource lists and toward a graph-first mental model.
Amazingly, company management approved the rewrite, and we actually were successful with first customers on it after 6 months and fully switched to the new system in 9 months. It was during this time that tc was developed based on our real needs and experiences. We continue to add new features as we discover the needs.
The Aha! Moment happens when you realize most tools ask the wrong question. They focus on the implementation (the how) rather than the intent (the what). I scoured the InterTubes for something better, but everything else was still resource-centric.
| Question Asked | Primary Focus | Outcome |
|---|---|---|
Think of it like the React analogy. Before React, we manually manipulated the DOM (the infrastructure). With React, we focus on component state and relationships; the DOM manipulation is just a side effect handled by the framework. In the tc model, you describe how components connect, and the system handles the IAM roles, event rules, and subscriptions as derivative artifacts. It’s an inversion of control where the graph is the truth. To make this graph-based thinking concrete, we created a single unit of encapsulation: the Cloud Functor.
We borrowed the term functor from OCaml’s parameterized modules. In our world, a Cloud Functor is a first-class, composable unit of infrastructure. To understand it, just look at your file system as a similar model of hierarchy and other characteristics.
Namespaced: Just like a directory, every functor has a unique identifier to keep things organized and domain-specific.
Sandboxed: You can deploy isolated versions (dev, staging, feature-branch) in the same deployment account without collision.
Versioned: We use Git tags and deployment manifests to freeze implementations, making rollbacks as simple as pointing to a previous tag.
Isomorphic: The definition is abstract; the same YAML renders consistently whether it's on your laptop or in production.
The File System Analogy shows how you work through Sub-Tree Autonomy. You can cd into a deeply nested folder, say, src/loan-app/extraction/ssn-id and run tc deploy in that directory. The tool treats that subfolder as the root of the universe, ignoring the rest of the 340-Lambda monolith. This zooming allows for infinite nesting in a fractal pattern; a top-level orchestrator delegates to sub-orchestrators, and every level looks the same. Every one of these functors is built from the same set of atomic building blocks.
Breaking away from the file system analogy, the top feature of Cloud Functors is their Composability.
We identified eight core entities that are sufficient to build any sophisticated serverless topology.
| Entity | Cloud Mapping (AWS) | The So What? (Benefit) |
|---|---|---|
By defining these logical connections, the spaghetti of infrastructure glue is derived algorithmically. This isn't just a convenience; it’s the secret to killing some of the most common security headaches in the cloud.
Let's cut to the chase about the real superpower of tc: graph-based composability. Turns out, when you stop thinking in terms of disjointed infrastructure resources and start treating your cloud primitives: functions, events, queues, and routes as fundamental atoms, you can build some truly elegant, self-documenting systems.
One simply defines the logical connections between these namespaced and sandboxed entities in a clean YAML graph, and the tc composer works its magic spells to infer all the complex IAM permissions and infrastructure glue for you. Because these relationships are elevated to explicit, first-class citizens rather than being scattered as obscure implementation details, the topology definition itself becomes a highly readable architecture diagram where no low-level infrastructure configuration leaks into your business logic.
This means we can seamlessly compose these atomic entities into higher-order Cloud Functors, and then recursively compose those into massive enterprise systems, allowing my team to understand an entire topology of 30+ interconnected microservices at a glance without ever wading through the spaghetti code of traditional IaC.
In the trenches, I see developers just getting it working by stripping away SourceArn or ExternalId conditions because IAM is hard. This creates the confused deputy problem, where a service is tricked into acting on behalf of a malicious party.
The tc composer solves this using Algorithmic Security:
Tested Templates: The composer uses pre-validated, secure policy templates for every connection. It automatically adds SourceArn condition keys to trust policies. You literally cannot forget them because you don't write them.
Least Privilege by Default: Since the graph knows Function A only talks to Queue B, the compiler generates a policy scoped strictly to those ARNs. No wildcards (*) allowed.
Because you define the topology (logic) and not the policy (implementation), the compiler always regenerates the algorithmically secure version. While tc handles the muscle, we still need a skeleton to hold everything up.
You still need someone with an understanding of IAM roles/policies to create the templates, but the enforcement is done by the tooling.
We don't try to replace Terraform (at least not yet!). Instead, we use a hybrid model that respects different infrastructure velocities:
The Bones (Terraform): This is your skeleton—heavy, static stuff like VPCs, RDS, and the Event Bus. You want your bones to be rigid and reliable. We will run Terraform to set up and maintain these static elements. We do not run Terraform regularly and don't really use it for drift management. Not only that, but we expect to eventually have tc be able to create and manage all AWS elements (and maybe other clouds someday).
The Muscles (tc): These are the high-velocity parts: Lambdas, Step Functions, and the Rules/Subscriptions on that bus. They attach to the bones and provide the actual motion. tc manages this fully
The Tendons (The Resolver): This is the magic. The tc resolver queries the universe (AWS) to find the ARNs of the bones as well as the state of the deployed functors and all the previously deployed services. tc talks directly to AWS via the AWS SDKs. The resolver then uses that data to supply ARNs or other info needed to wire things up.
Zooming into Action
The workflow is designed for speed. Because the tc builder uses Docker buildx for multi-architecture support (Linux/amd64), your builds are consistent regardless of your local machine or build by CI/CD in the Cloud.
tc build # Package logic and ML models (Docker multi-arch)
tc publish # Push assets like Layers and EFS models to S3
tc create # Provision the isolated, namespaced sandbox
tc invoke # Test logic directly in the cloud via a REPL
The three pillars of this workflow are:
Sandboxing: Collision-free environments (e.g., dev-your-name). Alice’s messages never end up in Bob’s queue.
Contextual Execution: Zoom into a sub-folder and iterate in seconds, ignoring the rest of the system.
Isomorphic Rendering: The same YAML is used for your local dev sandbox and production. No more it worked on my machine.
At the end of the day, the goal is to stop being infrastructure managers and start being cloud programmers. When we moved to this graph-first model at Informed, our implementation speed just zoomed. Being spoiled by Ruby and other high-level languages, I couldn't go back to manual wiring. We are now integrating LLMs/Agents with tc both for dev and targets, so things should go even faster.
Turns out, once you stop fighting with low-level primitives and start thinking in graphs, building complex systems becomes far too easy. It’s truly a superpower. Stop thinking in resource lists. Start thinking in graphs. The cloud is your computer; it's time to start programming it properly.
tc Cloud Functors is an open source project that can be found at https://github.com/tc-functors with documentation at https://tc-functors.org/.
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。