Lack of product thinking - A rant

A lot of engineers believe that microservices architecture is the ideal way to develop software these days. Having experienced a project in an earlier life where we dangerously came close to failing because we decided that we wanted to rewrite a well tested 5 year old legacy API into microservices despite clear evidence that we would NOT make it on time, I have a lot to speak about this.

Having challenged this view in a couple of forums at the time, I failed to get the point across and was responded by the microservice fan-boys that it was our only opportunity to do such a thing.

Shiny new tech paradigms can be very seductive for engineers. The itch try out something new can lead to tunnel vision and distract us from the real problem at hand, which surprisingly is often, delivering a reliable product that works and solves the challenge that the customer is facing.

We clearly did not blog about this experience because we were busy fixing the problems we created by having several poorly written microservices that used to be one poorly written monolith.

But at the retro, I shamelessly asked “was it worth it?”, the answer was a unanimous NO. It was a realisation that by aiming to re-write a monolith into microservices, most of us, I dare say, very bright engineers, underestimated the work involved in achieving that and also the consequences of having to maintain a complex distributed system with basic observability.

The real problem then was delivering a consolidated simplified interface to the existing monolith - which was an insurance quoting API.

The re-write was not a necessity at all. Because there was no need for it to scale to an extent that would warrant the need for scalable microservices. But engineers being engineers, made that THE Requirement. Despite not being a fan, I did commit to it too. Focussed on that very fictional requirement instead of the real problem, and accumulated tech debt of a very different magnitude.

As engineers we forget that we are here to make the lives of our users easier. Not focusing on the real problems often results in us creating problems for ourselves. This is where product thinking and working backwards is absolutely critical.

As engineers, we must adopt a tool or paradigm for solving a problem that our customers are facing. Not because it is a new tool/solution that FAANG is using.

I read this blog post and it just brought back some memories. The post doesn’t imply that the developers used the microservices approach because it was cool. Yet it is absolutely worth reading.

Microservices and serverless components are tools that enable you to scale like crazy especially if you have workloads that are unpredictable and often happen like a explosive bursts that then for a very long period is flat lined. But the decision on when and where you would use one over a simple monolith has to be made on a case-by-case basis.