Wait! Don’t write your microservice … yet

Day one, we were super excited to start a new project for a huge financial institution. They seemed to know the domain and as Knoldus we understood the technology. The stakeholders were excited about the Reactive Paradigm and the client architects were all stoked up the way there would be microservices which would allow them all the benefits that other companies had been harping about. We had proposed Lagom and it was taken well.

We had an overview of the domain and we had a set of domain experts assigned to us who would be product owners on different tracks that we would work on. Great, does seem like a perfect start. But when things are too good to be true … probably they are.

The lead architect (LA) asks me, so how much logs would each microservice generate? I am like … what? And he explains “Since we are going to build it with microservices so I need to understand the logging needs so that I can size the disks”. I am like … “I don’t know … uh…mmm”.  LA ” But, we explained the domain to you so I thought this would be easy! Ok, how many microservices would we have?” I respond …” I don’t know! … yet”. And now the LA is losing it on me. I could go on how the conversation evolved or rather degraded but I would rather talk about how we finally managed to convince the stakeholders that this thought process is incorrect.

Whenever we start a new project, we start building a MONOLITH. Yes, you heard me. That is how we start. When you are starting a product, you know too little about how the intercommunication between teams, subdomains, and bounded contexts would work out. Once we have the monolith and we identify the submodules and the play between them, it is relatively trivial to break it down into microservices. The only things to remember is to design a monolith carefully. Paying attention to modularity especially at the API boundaries and data storage. If this is done right, it is simple to shift to microservices. Once we have the well-done monolith, it is easy to carve out and logically package subparts of the monolith as a microservice.

You could, of course, argue about the benefits of getting to microservices directly which are like once we have a monolith, it cannot be broken down into microservices easily. We cannot start with smaller teams which can work on their own parts and what not. But, for the teams to work on their own parts, your bounded contexts should be very well defined, which is almost never true. The boundaries and the contexts become clear(er) once we have the system reasonably in place. I would even counter-argue that a lot of times, you would want to peel off a part of the system into a microservice much later when you get feedback from production data.

In our example, one of the services which generated PDF reports for the days’ trades was bundled along with the reporting service. Over a period we realized that this service was hit a lot and especially during the closing hours or the first hour after closing. Other reports did not follow the same behavior. We separated out the PDF service as its own microservice which could expand to 2x servers on peak load. The report service continued to work the way it was.

So, the summary is that however hard you may try, you would almost always fail at separating out the microservices when you begin the project. In fact, the cost of badly carved out bounded contexts and microservices would be much higher than beginning with a monolith and break it down as and when needed.

You might be interested in this post as well “And you thought you were doing microservices“