Feature Image by Overview of Domain-Driven Design. Source The DDD Community 2019, pp 6.
This is my understanding of the importance of DDD having read books and articles and attended courses on it and also having how projects can get really complex over time and unmaintainable with little knowledge of the domain from personal experience.
How does software development look like?
Software development is a very unique profession. You almost always create software to solve a real world problem in a very specific domain. More often than not, these problems mightnot have anything to do with software engineering until a solution is implemented.
You might now be working in a fintech, developing the next generation trading platform. Maybe a year later, you work in another one but this time in retail banking! After a couple of years, you decide to switch into e-commerce for a clothing brand. Then in another couple of years, you move onto retail insurance, where the company sells insurance to people. As you work there for a couple of years, the company gets acquired by a B2B insurance company and you move to a different team developing solutions for B2B insurance.
Do you see how the life of a software professional can be?! It can change dramatically!
One can learn about several different domains in a matter of a decade. Some even focus on a particular domain and become experts in it and switch to a career in Product ownership and business analysis or even move on to become an enterprise architect as they understand the domain really well. The point is, as you work in a domain, you start to pick up terms and understand the language of that domain. You start to visualise the journey of a customer requesting a service and the request moving through a series of components or systems and finally offering the customer something. And this can look very different in different domains. There maybe overlap in software principles used everywhere. But the language of the domain influences how you design the solution for the problem that is presented to you.
Complexity
Software development can get complex really quickly depending on the domain we are trying to develop software for. On the one hand, there maybe simple binary logic to determine certain business rules. Whereas on the other hand, business rules can be complicated, interconnected and often lead to webs of rules relying on one another. As the complexity of business logic grows, it is important to ensure that the software engineers working on the solution understand the problem in the domain well enough to design a solution to it. If you do not understand your domain well, you could be making short sighted choices that could result in hard to maintain solutions in the long run. That could slow down your business and make it very expensive to streamline and make better over time.
If you do not believe me, there has been some research into software projects and you can view them by a simple google search. One of the most famous, cited in some DDD books are Chaos Report 2015 - Standish Group. This is a research done on a small number of projects, which according to the research is more than 25000 projects, between the period of 2011-2015. And the results are astonishing! Or considering how difficult software projects can get, maybe it really isn’t that surprising.
Based on their definition of successfully resolved software projects, which is that a project was delivered on Time, within the allocated budget and had satisfactory results:
- Despite several projects meeting the budget and time criteria, in addition to meeting the goals of the software company, the clients or users of the product perceived the result as just satisfactory.
- If the customer is not satisfied with the result of the project, then what was the point making it in the first place?!
- The percentage of successful projects were a mere 29%
CHAOS research also suggests that SIZE of the project is an important factor in determining the project’s outcome. The longer it took to get it into production, the lesser the value that it returns to the organisation. Therefore, smaller projects that get to production often returns value sooner.
Managing Complexity
Ever since software systems became mainstream, there have been several development methodologies popping up that claims to solve all the problems with inefficiencies in software development. Agile Scrum and XP are some popular methods or frameworks that enable us to deliver value faster, because the primary focus is on the value delivered.
How do they all work? They all work by involving the end user in the software development process. Helping those developing the solution understand the problem better. The more the user collaborates with the development team, the better the outcome, so long as the user understands what not to do when collaborating. :D As the development team understands the problem better, they understand the terminology the importance of certain events in the domain and what depends on another and what is not the other etc. This knowledge is invaluable in designing the components that form the solution. Thereby resulting in maintainable software in the long run.
The term Domain Driven Design, is a concept coined by Eric Evans in his book Domain-Driven Design: Tackling Complexity in the heart of Software. This book, published back in 2004, is still one of the most popular books used by software architects around the world, including where I work now. The reason this is so popular is because it focuses on how software development teams can build better software by understanding the business domain better. It provides guidelines to identify certain areas in the domain and translate it into software.
Problem solving
As mentioned earlier we almost always write software to solve a problem. It is important to understand how as human beings we tend to solve problems.
According to Allen Newell and Herbert Simon in their book, The Human Problem Solving, there are two spaces: problem space and solution space. Humans solve problems by searching for a solution in the problem space.
Problem Space
This defines the initial state, intermediate states the desired final state. It can also define constraints or rules.
This is the domain, I mentioned in the beginning - fintech, ecommerce, insurance, etc. Software engineers have clients in any of these places. And as stated earlier in the Human Problem Solving book, we search for a solution in the problem space, hence if we do not understand the problem space, imagine how our solutions would be? We might invent things that are not part of the problem space to solve the problem.
Solution Space
When we search the problem space for a solution, we often come across states that we can be connected together to achieve the desired final state. This is the solution space.
Misunderstanding a problem
How many times have you assumed that the client would like something and invested additional effort in implementing that solution and later was shocked to learn that the client totally disliked your solution? Despite your intentions, the problem you solved wasn’t a problem to the customer. You invented a problem while solving the real requirement and solved that in addition to the actual problem.
I have personally had these situations, where sometimes due to lack of information or availability of someone to ask, we developers take things into our hands and assume that this button click should open a modal and display a message and must have a spinner etc and spend time building it all together. At the end of the sprint, when you demo it to the client, you finally get puzzled looks and more feedback as to why they didn’t want that.
This is the danger of solutionizing a problem too early, before you fully comprehend it.
In fact it can lead to a lot of waste in terms of money and time that Bart Barthelemy and Candace Dlmagne-Rouge, wrote in an article When you’re innovating, resist looking for solutions in Harvard Business. I have pasted a simple example from the article, that talks about this.
A military organization came to us for help because people who were being observed by pilotless drones were using techniques such as smoke screens to deceive the analyzers of the drones’ video and other data. The organization asked for assistance understanding the adversary’s deception techniques. But by framing the request that way, the client had already moved from the problem space toward the solution space — the client was specifying the type of solution that was expected.
We encouraged the client to stay in the problem space, sometimes known as the “front end,” in order to get a deeper understanding of the problem. The client soon came to see that analysts are deceived because there are limits on their ability to perceive. The real issue is understanding these limits.
This is something we software engineers get into the habit of pretty quickly. Especially when you develop a habit of competitive programming. This we might transfer to coding interviews too. We are given a short duration of time and a pretty challenging task to accomplish within that time. Some engineers get too carried away in solutionizing the problem that they forget to ask the interviewer further questions about the constraints of the problem and end up wasting time coding something that they weren’t asked to! They inadvertently fail the interview.
In Hands-On Domain-Driven Design with .NET Core, Alexey Zimarev, talks about the difference between Refinement and Exploration.
Refinment often focuses on one solution and how we can attain that solution.
Exploration on the other hand, focuses on different ways to solve the problem and not just one solution. Hence, quite often take longer but comes up with fantastic ideas to solve the problem.
Why not speak directly to the users?
Short answer: Communication is hard. Most users know what they want but they aren’t good at expressing what they want.
Althought I say that, there are times when users have clearly expressed their requirements however, before it gets to the developers it has been solutionized already by the business analysts or whoever is gathering the requirements and preparing your user stories or whatever they call requirements in your favourite software development framework.
Due to this, involving users in a feedback loop is the best way forward, which is what most agile methodologies suggest too.
However, that still doesn’t solve the domain understanding problem we were talking about. As requirements reach the developers, if an analyst translates it using very different terminologies, and sometimes even solutionizes the problem, they way they see it, then the developers would be building that instead of what the user really needs. This is surprisingly common and can cause a great deal of problems in the absense of a quick feedback loop, as the cost of reworking would drastically increase if the feedback doesn’t come soon enough.
The key is to avoid things getting lost in translation. DDD focuses on keep the domain’s language intact, such that the business, business analysts, developers and testers all mean the same thing when they say a word in the domain language. This reduces the difficulty in communication. This reduces gaps in understanding too. This is not a new programming language we are talking about. But this is the language that the domain uses.
For the sake of an example:
In the insurance world, a risk that is to be insured, requires quotes (of premium) from insurers. When the consumer purchases it, the quote is bound, a policy is generated.
That is terminology in the insurance industry. using risk and quotes in home improvements would mean something entirely different.
This use of a common language, help reduce micommunication and also helps developers understand the domain better, there by letting them create better designs for their solutions.
What is the complexity?
One of the best summaries of this I found in Alexey Zimarev’s - Hands-On Domain-Driven Design with .NET Core. He referenced a brilliant article, which I would never have known, had I not read his book. It was No Silver Bullet: Essence and Accidents of Software Engineering. The article defines:
- Essence: The difficulties inherent in the nature of software. As Zimarev calls it essential complexity. There is not much one can do about this one apart from controlling the scope of what is being developed. This is the complexity that resides solely in the problem space.
- Accidents: The difficulties that today attend its production but are not inherent. Zimarev calls this accidental complexity. This is the one you can control. This is the one that the solution brought out. It is complexity born out of the implementation. This complexity lies in the solution space.
The article goes on to describe how the maturity of the software industry has helped reduce the accidental complexity. But there is still a lot of accidental complexity that we create. Every framework or tool created, requires software developers to learn and understand its nuances before they can actually develop an efficient solution using it.
As a software developer, I understand sometimes how we have created accidental complexity. Zimarev gives an excellent example that I find very true. Paraphrasing it below:
Microsoft has made Dependency Injection built into to its platform. However, at the same time, because DI is part of the framework, we now have to rely on it even for a simple console app to initialise a logger.
Although I see that as an excellent example, I do believe that Frameworks are opinionated. So the creators of the framework, guiding developers to using DI in all their projects is quite common these days.
It is similar in the front end world too. Angular as a framework imposes several restrictions on how things can be done.
DDD focuses primarily on dealing with essential complexity. After all, delivering value is the most important thing in software development. If you write thousands of lines of code that does not do what the user needs, then you might as well have not written it at all.
That’s all for now. There is a lot to say about DDD, but this is just one in a series of posts about it. As I work through some of my projects introducing DDD and refactoring my way through the solution, I will cover more ground.