Graph-based Code Generation Exploration

In this repo I explore the idea of generating software implementation via a structured graph decomposition. The system consists of 3 phases. The planner the plangraph and the codegenerator. The planner is a simple LLM loop to gather user requirements by prompting the user with clarifying questions until the model judges that it has adequate information about the user's intents.

In the plangraph phase, the user's plan is an input to a graph builder. Each graph is a recursive decomposition of the plan into individual self-contained components. A component can have inline responsibilities and delegated responsibilities with the latter determining whether a component is a leaf or an intermediate node in the graph.

The theory was that we can use a graph structure to better cement the LLM code generation and allow it to focus on one system at a time without requiring a "whole-project" context. The plangraph is an alternative to the typical LLM workflow of markdown planning documents which are prompt-injected into an LLM to generate the desired outcome.

Results

The plangraph module actually worked reasonably well despite my initial misgivings. The agent is able to recursively decompose problems in what seems like a reasonable manner in my tests. The repo includes an example for a simple tui calculator app plangraph. This is generated from an example plan defined in main.py.

For the actual code generation - the coder agent and orchestrator agent both work pretty well at their assigned tasks. In particular I was rather impressed by how well the coder could generate functional components from a plangraph node description. It really reinforced my thesis that structured guardrails and precise deterministic tooling is the right way to utilize agents. That being said - it does sometimes get stuck on outdated syntax and test-failing loops, though this can be chalked up to a rather haphazard implementation of a web search tool. In my tests I found that precise documentation lookup is extremely important to a successful generation, rather similarly to how humans spend time googling.

The orchestrator does its job though there isn't much fanfare with it. Its a good way to guide the graph generation process in a semi-deterministic way. It also facilitates regeneration and user feedback which is needed for a generative workflow.

The scaffolding around my code generation is also rather raw and relies on a global pyproject structure - but this isn't meant to be anything more than a research/learning experience.

Conclusion

The idea I had is "proven" in a sense - but there are some serious drawbacks. The first is that my iterative generation approach requires a lot of tokens. A single pass for the 5 component calculator app runs for over an hour. I am sure a lot of this is due to LLM latency and my usage of cheap models - but it is both time and token inefficient to run this type of decomposition.

Whether the code is that much better than a dedicated claude-code/codex session can generate, I don't know, probably not. But it could potentially be more maintainable over the long term because of the plangraph structure which was the real idea/star of the show.