foobuzz

by Valentin, February 4 2024, in tech

If you want to break up the monolith, first embrace the monolith

I've recently switched companies from a company using micro-services to a company using a monolith. My previous company had about 15 services, consisting in 1 big central legacy monolith around which gravitated 14 smaller services, which were handling more recent projects, resulting in sort of what DHH called a Citadel architecture.

This so-called Citadel architecture was not pleasing technical management, though, and it was decided that the big central monolith had to be broken down in several micro-services. The problem was approached in a top-down manner (with the help of a software architect, of course): appropriate tooling was needed to spin up new micro-services easily; architecture schemas were needed to understand what service would do what; etc.

My new company has a strong incentive to do a monolith, because they're selling self-hosted versions of their software, which is way easier to distribute as a monolith than as a galaxy of microservices. They have completely embraced the monolithic architecture, and it is interesting to see the accommodations in place so that 70 developers across various teams with distinct product ownership can develop seamlessly on the same monolith. It boils down to the following points:

  • Code ownership:
    • Have your codebase organized by features then technical components, rather than technical components then features (serviceA/controllers, serviceB/controllers rather than controllers/serviceA, controllers/serviceB). For example, Django will kind of force you to do that with their apps system.
    • Declare contributors ownership to the different features. For example, using Gitlab's CODEOWNERS file.
    • In practice, teamA will sometimes need to edit code belonging to teamB, and therefore requires a cross-team review, but this simply implies adding a reviewer to the merge request with a small explanation of the change. This is way less cumbersome than needing to open another merge request on another project, and then handling the deployment of both projects. Significant win over microservices.
    • Lack of schedule synchronization still gives an incentive to not have a cross-team reviewer on the merge request, and puts natural pressure on teams to better design their interfaces to avoid coupling to other teams. Again, this is way easier to do on the same project with function calls than on different projects with network calls.
  • System requirements / Infra separation:
    • Leverage what is possible to do at the infrastructure level to handle web requests or asynchronous workers that need special treatment.
    • Specific Kubernetes pods can be used to handle specific asynchronous tasks / HTTP routes with specific requirements (e.g. a PDF generation task running in its own pods with specific memory and restart policies).
    • Each pod runs the entirety of the monolith even though it would only be used to execute a specific part of it. It doesn't matter.
  • Deployment:
    • Schedule frequent, paced deployments of the monolith (in our case, every monday), so that every team has the opportunity to release what it wants soon enough.
    • "We have a feature to release so we need to deploy our own micro-service" becomes "We have a feature to release so we need to merge this on the trunk before the next release."
    • You need feature flags. The fact that the monolith is frequently deployed also means that you can't have a MR merged into the truck and expect it not to be released soon as well. But feature flags are really useful in any case.

All that is good and well, but my most important realization so far is the following: my new company who is not trying to do microservices has a monolith that seems easier to break up into microservices than my previous company who was trying to do microservices. When you think about it it makes sense:

  • The necessity of having teams with scopes (official or not) is invariant of the technical architecture you're using. If you want to have interfaces more precisely defined in your monolith (which is the first step of wanting to go micro), just define distinct ownerships over this monolith, and this will put natural pressure on the teams to have their interface revised accordingly.
  • You're going to deal with varying infrastructure needs depending on different parts of the application, and you don't need to wait to actually have different services to play around with that.
  • You're going to need to be good at deploying software if you have several services to deploy, and you can start by deploying your monolith often.

Those new insights have convinced me that it is a fool's errand to design microservices in a trash-the-monolith-away, top-down manner. The simple reality is that having a proper clean monolith is just the intermediate step before breaking it up. But surely the most important learning is that you might end up realizing that you don't need to break it up at all.