An Overhead-Free Monorepo Blueprint for High-Velocity Teams
The first design decision for any serious tech startup is to decide how their
codebase is structured, and building a monorepo setup often proves to be
crucial for avoiding cross-project dependency maintenance that can stretch
a small-to-mid sized engineering team which most startups cannot afford in
pre-venture or revenue stages.
What does an "ideal" monorepo look like?
While there are multiple ways a developer or a company can structure their
monorepos and it's true that there isn't an ideal setup that fits all case
scenarios, there are universal architectural truths that separate a clean
engineering workspace from a chaotic one
There are some salient features that every staff engineer or software
architect should aim to achieve with their monorepos:
- Reproducible environment
- Domain driven development
- Module and library re-use
- Trunk-based development
- Central build cache registry
- Simpler deployment pipelines
- Package security, visibility and role-based code access
Most successful monorepos satisfy most or all of the above requirements.
I have worked with a couple of startups and have seen them all struggling with
getting most of the above requirements right.
Even established companies with
100s of engineers, sometimes have to spend hundreds of thousands of dollars,
if not millions, on platform engineers just to manage their codebase and build
pipelines.
Now, Let's go through each one of the "ideal monorepo" requirements and create
a best in class workspace for the engineering team.
Creating a Reproducible Environment
If you are a founder or someone from the tech leadership, there's a possiblity
that you hear this phrase a couple of times in a week - "it works on my machine,
I think something is wrong with the deployment". And most of the times, the
deployment turns out be just fine and it's the developer's local environment
configuration that's been acting up.
Also, there's a huge possiblity that every new engineer you onboard takes at
least 3 days to settle in and get comfortable with the codebase and all the
tools required to setup a normal working local environment.
What I have just described above is a classical design flaw from the early days
of a codebase, whether a monorepo or a polyrepo.
The medicinal anti-dote to this exact most common problem in tech companies is using
sandboxing tools that isolate the development environment and levels down the
playing field for all the engineers.
We are going to take a look at one of these tools today, that can easify your
life managing a tech team or running a tech company.
Devbox - The ideal sandbox for your dev environment
Devbox is an open-source, Nix-based package
manager that creates isolated, reproducible development environments without the
resource overhead of local Docker containers. And it is just the right tool to achieve
environment isolation that I have personally used in many of my projects.
It's not as heavy or expensive as managing docker configurations and forcing
developers to write code in docker native IDEs.
Let's start with the first steps of our desired setup:
# Install devbox
curl -fsSL https://get.jetify.com/devbox | bash
# Create directory for your codebase
mkdir monorepo
cd monorepo
# Initialise the monorepo with devbox
devbox init
Now, let's take a step back and ponder over our requirements from this virtual
environment. Most of the modern mid-level complexity full stack projects need
at least these pieces of tools to work correctly:
Essential Core Utilities:
- Git (Version Control)
Frontend Engineering Layers:
- Node.js (V8 Application Runtimes)
- Bun (High-Performance Local Testing & Package Management)
Systems & Data Processing Services:
- Go (Highly Scalable Backend Web Servers)
- Python (Machine Learning Infrastructure & Automation Pipelines)
- CMake/Make (Build tool for C/C++ used for high performance)
The list of the above tools is strictly my own choice and you can chose to go
with your own vibe of toolsets, that you want to be written on the stone for all
your team mates.
Let's install all of these and peg them against a version number to lock in the
tool of choice for everyone.
# Discover the versions of tools with
devbox search git --show-all # This gives a list of all available git versions
# Now add the desired version of the tool to your local environment
devbox add git@latest
devbox add node@latest
devbox add bun@latest
devbox add python@latest
devbox add go@latest
# Add any other tool(s) that your team needs
Now, the last step is to officially enter into the virtual environment that we have
created.
# Enter into the devbox environment
monorepo $ devbox shell
# Just when you execute the above command, your terminal prompt becomes this
(devbox) monorepo $
Now, whenever the new intern asks how he should pull the project and start the
contributing you just tell him to:
git clone <your-repository-location>
cd <your-repository>
# And...
devbox shell
# 🤯, he is all setup!!
# Now the only thing left is to execute the install scripts
# and actually run the projects.
And it just took him 2 minutes to start with the project without the weird
environment issues and it didn't matter if he is on Macbook or Linux or
Windows.
Go Task - A central registry for all commands in your project
Go Task is an open source utility that can map simple
words with complex commands, so that the junior devs never have to ping you in
the middle of the night just to figure out the command that they wrote had a
typo in it.
You can read the full article on my blog - Engineering Insights at Esence.io