Couple week ago I started to read a book called The software architect elevator written by Gregor Hohpe. Gregor said in the book that "The more uncertain about the future I am, the more value I derive from deferring a decision.". After reading that sentence I started to think one specific very special "start-up" project.
This is a self-reflection from a project where I was involved three years ago. Overall project was a one kind of once in a life time experience. All stakeholders were highly motivated to work hard to achieve common very important goal. We all knew that stakes will be high and potential results outstanding.
I would say that we worked together like in a start-up company. Bureaucracy was minimal and everyone had an attitude and passion to get this working. Vision was a quite clear what we want to achieve but time was our biggest enemy. We realised soon that we need to work hard to get this to market as soon as possible. We knew also that we need to be agile, flexible and ready for a external pressure and tolerate high frequency of change.
I'm sure that the way how we managed to defer decisions especially from the technical and architectural point of view was one factor why this project was a successful one.
What we knew in the beginning?
We needed to change and reinvent old complex highly manual and fragile process to digital. We knew that if digitalization of this core process is successful, this new digital way of working enables a lot of new possibilities for end users.
There were a lot of more unknown things than known things in the beginning. Actually these unknow factors are not unique. I'm sure that many projects start without knowing all facts what I'll mention next.
Unknown new domain area
Like said we were in a time preasure all the time and it didn't help to understand this new domain area and it's nuances. From domain modelling point of view we tried to be as flexible as possible because we knew that there will be changes coming definitely.
Unknown lifecycle of the system
Like a start-up company we didn't know in the beginning will this be a huge success or should we find a new project after 6 months. There were also many external factors from out of our reach which will affect how long there is a need for this system.
Uncertainty of the estimated life cycle of the system caused a lot of discussion in the beginning for a reason because it will affect also to architectural and technical decisions. We balanced with this dilemma a quite a long time and eventually started make decisions based on assumption that this system will be long-lived. This was a right decision because after working with one year we saw that system has potential for other purposes as well.
Unknown user volume and usage patterns
We had some guesses about user volumes and we knew that if this sky-rockets user volumes could be substantial. We also learned during the journey that there will be significant user peeks and we need to handle those as well. During the first months it was also clear that system has national wide potential and interest as well.
Changing environment and external decision makers
It was already clear in the beginning that there will be a lot of external decision makers where we will have a small possible to impact. This will affect heavily to our work but we cannot anticipate how. We knew that we need to be flexible and ready for changes and sometimes also revert decisions. This will heavily affect also technical design and decission making.
Success requires integrations
We knew that there will be a need for multiple outbound and inbound integrations to enable all benefits of digitalizationing of the process. Our system will be the key component of orchestrating the process across other systems.
Operational reliability
It was clear that if system sky-rockets, popularity of the system will be substantial and we need to provide superior operational reliability for our end users. Reliability was one of our top priorities because we knew how crucial system was to achieve important common goals.
Security and privacy
We knew in the beginning that domain area will require special handling and focus from the security and privacy point of view. It was clear that this will increase our work and we need to integrate security deeply to our processes, practices and mindset.
What kind of deferring decisions we did?
Next I'll tell something about deferring architectural decissions what we did during during the project.
From a modular monolith to microservices
This was the one of the first deferring architectural decision what we did. We deferred moving to microservices architecture until we knew more how system will be used and what will be the amount of the people (developers) involved to development work.
There was so many unknown factors and our team was small (5 people) in the beginning so modular monolith was a good way to start. Modular monolith basically means that application is built so that code is divided to independent modules. Indepedent modules enable that modules or components can be break down to individual microservices if necessary later in the future.
It didn't make any sense to start immediately with a microservice architecture because we didn't know what is the life-cycle of the system, will this system sky-rockets and how many developers will eventually work with this (scaling teams).
During the project our knowledge, requirements and system evolved and we moved more towards microservice architecture. Microservice architecture enabled scaling of the teams as well as independent technical compontents.
We started developing one application for specific need with 5 people. Eventually system expanded rapidly so that when I leaved the project there was 7 internal UI applications and APIs, 3 outbound integrations, 5 inbound integrations. In the end there was over 20 people working with different microservices of the system.
Be cloud native and scale later
It's important to develop application to cloud native already from the beginning so you can truely benefit cloud compute scaling. You don't need to know immediately how many users will use the service or what kind of usage pattern they have. Cloud services like Azure enables scaling in horizontally and vertically so you don't need to know immediately how much you need compute power.
Like said we didn't know usage patterns of the application or user amounts. Deciding compute scaling was one of the deferring decision what we did because we had a possibility to adjust compute based on the need.
We used Azure App Service autoscaling and Event Driven pattern & post-processing to adjust high user amount peeks.
Transition from CRUD to domain driven
Everything started from a very simple use case about persisting records to database and showing data from database. Domain driven design would have been a bit too heavy for this purpose and like said earlier in the beginning we didn't know the life-cycle of the system or possible dimensions of the domain model either. These facts drive us to start with a C (create) R (read) U (update) D (delete) type of approach because it was a very quick to develope, simplistic and efficient. Our understanding about the domain was just started to increase.
Later when our use cases started to become more and more complex from the business logic point of view we started to move towards domain driven design.
This was also an one example about deferred architectural decision what we did. We started with small and changed the plan when we had more information and knowledge.
Towards API management
In the beginning our system had a small number of external API consumers. At that time our externally consumable API endpoints were easy to manage from operational point of view. Later we started to get more and more API consumers so we decided to start using centralized API management to help management. API management provided also a developer portal functionality for external developers and improved also governance of our APIs.
We did this architectural change when time was ready and we had a true need for this functionality.
From Synchronous to Asynchronous processing
When we learned more about user amounts and usage patterns we started to move more to asynchronous processing in a background. Our application contained many functionalities which didn't require an immediate response back to user after submit so transition to asynchronous processing was a clear choice.
It was easy to start with synchronous processing and move to asynchronous when we had more information about the peeks and patterns.
What we achieved by applying deferred decisions?
This wasn't an easy journey but overall project was a succesful. I think deferred decision making process enabled that we always started with a small and maybe simpler solution which required less work. I believe that this guided also us a direction to avoid over-engineering. Of course this didn't happen without pain and problems but we managed to solve them because we were allowed to refactor and change the direction. Time to market was the most important KPI for us and I believe that deferred decision making process helped to achieve this.
Modular monolith architecture in the beginning
Microservices based architecture after evolution
Comments