Lack of product thinking - A rant

Many engineers believe that microservices architecture is the ideal way to develop software these days. I have had the opportunity to be part of a large project in my previous role, where we were in danger of failing. This was due to our decision to rewrite a well-tested 5-year-old legacy API into microservices, despite clear evidence that we would not be able to complete it on time.

Having challenged this view in a couple of forums at the time, I failed to get the point across and was responded by the microservice fanboys that it was our only opportunity to do such a thing.

Shiny new tech paradigms can be very seductive for engineers. The itch to try out something new can lead to tunnel vision and distract us from the real problem at hand, which, surprisingly is often, delivering a reliable product that works and solves the challenge that the customer is facing.

We clearly did not blog about this experience because we were busy resolving the issues we created by having several poorly written microservices that used to be one poorly written monolith.

At the retro, I shamelessly asked, “was it worth it?”. The answer was a unanimous NO. It was a realization that by aiming to re-write a monolith into microservices, most of us, I dare say, very bright engineers, underestimated the work involved in achieving that. Also, the consequences of having to maintain a complex distributed system with basic observability and multiple code repositories were dire.

The real concern at the time was delivering a consolidated, simplified interface to the existing monolith - which was an insurance quoting API.

The re-write was not a necessity at all. Because there was no need for it to scale to an extent that would warrant the need for scalable microservices. But engineers, being engineers, made that THE Requirement. Despite not being fully on-board with the idea, I committed to it as my argument against it was vetoed by other senior engineers. In a team, I am alright with disagreeing and committing, so with a good warning, I committed myself to it. Our focus on the fictional requirement of rewriting the monolith into microservices instead of tackling the real problem resulted in tech debt of a very different magnitude.

As engineers, we forget that we are here to make the lives of our users easier. Not focusing on the real concerns often results in us creating issues for ourselves. This is where product thinking and working backwards is absolutely critical.

As engineers, we must adopt a tool or paradigm for solving an issue that our customers are facing. Not because it is a new tool/solution that FAANG is using.

I read this blog post, and it just brought back some memories. The post doesn’t imply that the developers used the microservices approach because it was cool. Yet, it is absolutely worth reading.

Microservices and serverless components are tools that enable you to scale - when done right, especially if you have workloads that are unpredictable or happen in explosive bursts and later flat lines for a very long period. But the decision on when and where you would use one over a simple monolith has to be made on a case-by-case basis. Software engineering is a discipline of tradeoffs. Knowing what to trade over the other makes a skilled engineer.