The primary drivers for adopting agile methodologies, DevOps, Microservices, containerization, cloud adoption is to provide value to our end-users and customers at the earliest. Time to market is critical for the business to stay on top of the competition, retain customers and onboard new customers.
In recent years, our industry has seen a rapid pace of technology changes, entry of new products, tools, architectural patterns for solving various challenges.
Continuous Integration (CI), Continuous Delivery(CD) have contributed to the speed that we do our business, reducing operational cost, amplifying return on investments. Enterprise Agility is the new mantra for enterprises now to shake up the entire cultural change, embrace New ways of working (NWOW), aligning wheels of various business units, departments, teams of the entire organization to achieve the meet the desired outcomes.
In the series of multi-post, I would like to share my experiences in architecture, design and operationalizing Microservices. I am not an expert but I am influenced by various thought leaders, practitioners, architects, and developers. Every organization is unique, they have their own people, process, challenges, opportunities and cost associated with the way of doing their business. In this series of posts, I am reflecting some of the “Microservices challenges in traditional enterprises” my own experiences, hope this helps someone like me the other side to watch out for these symptoms, think about the challenges well in advance, care about the architectural decisions, manage the operational complexities.
There is a great post from Martin Fowler on Microservices prerequisites, please read through this before you begin: You must be this tall to use Microservices.
Monolith or Microservice
The great dilemma for architects, designer, and developers when it comes to making the Architectural decision is this, understanding and appreciating the concerns are important before you embark on this journey;
Shiny new toys are always attractive, sometimes they come with the hefty price tag. Microservices is one among that category, asking the right questions upfront and finding answers will help you, your team and the entire organization. It is important to document these concerns, weighing them, recording the recommendations, outcomes and sharing with wider enterprise will help you save money, efforts in long run.
Monoliths are not bad and Microservices are not that great; Microservices adoption needs great level of engineering maturity, that is the reason “You must be this tall to use Microservices”.
Understanding how the architectural decision impacts our operational concerns is important quality for architect these days e.g. Single process vs many processes, single vs polyglot, centralized vs distributed architecture system etc.,
Infrastructure support
This is fundamental to organizations Microservices adoption, without automated provisioning your infrastructure, your team will be not able to get the goodness of Microservices; Enterprise needs whole new thinking around platforms provisioning.
Assuming traditional enterprises are either on-prem or hybrid, careful selection of Infrastructure automation / PaaS is mandatory to be successful; Microservices scaling requirements, resiliency and fault tolerant requirements need some significant infrastructure/platform uplifting in some organizations. If you are on the public cloud, managed containers platform it is easy to achieve that, but for on-prem you need careful planning in selection of tools;
Questions like Open Source or Vendor supported, what level abstraction you need, all these depends on enterprise level planning including your existing people, their skillset, training requirements, security, governance and compliance requirements, business interests/needs, preparing for operational complexities.
Engineering – Architecture, Design, Build
Agile been there for a while, some enterprises are well ahead in their plan regards to Agile adoption, few started, some still finding and fighting over challenges like funding, project management, test automation, continuous integration etc.,
There are plenty of materials available around scaling issues, problems, challenges in large enterprises and you have frameworks and answers to those challenges. First, you need to be an “Agile” organization, regardless of where you stand and how mature you are.
Microservices architecture needs Architect to understand the business problems with the lens of Domain Driven Design (DDD), Domains and Subdomains, Bounded Context and Context maps, Aggregates and Aggregate roots etc., This DDD practice is a journey, you will refine as you go like other adoptions like Agile, DevOps; Eric Evan’s book Domain-Driven-Design-Tackling-Complexity-Software will be great help for someone who is starting with DDD.
Understanding your core domain (problem space) that your team working on, the existing legacy systems in the enterprise, integration with external and third-party products & services, and compliance & regulatory requirements etc., to get your domain modelling correct.
The next bit is designing Microservices itself, you need some guidance, if you are new and finding where to find more relevant answers, you need to then read Sam Newman’s Building-Microservices.
We will discuss some of the architecture and practices like
- Database design for Microservices
- Communication between services
- Continuous Integration
- Test automation
- Automated deployments
Preparing for operational challenges
Architecture is abstract unless it is operational; Hence a well architected and design systems/ services assessed for its quality and characteristics only when it put into operation.
Productionizing Microservices comes with its own challenges. When something goes wrong how to trace the request,
• Handling downstream failures,
• Scaling in-out based on-demand,
• Rolling out new features,
• Service observability,
• How your services are writing logs,
• Where these logs get aggregated,
• How this aggregated information surfaced to the product teams/ squads/operations team.
“Operation is the Business”. The crux of the whole purpose of what we architecture, design, build is all about operationalizing the services. The beauty of design is in the reality of running these services.
It is important to prepare the enterprise for what it is getting into, these requirements might need some significant investments and effort for new tools, log aggregation and dashboard etc.,
Ops team challenges:
- Instead of looking at 4 virtual machines, now they may need to manage 100s to even 1000s of containers running,
- Instead of monitoring one system / one process they might need to observe now 100s of services,
- In some cases the same service instance might be running on ten different hosts/containers.
Enterprises need to understand that manual ops activities do not scale for these kinds of Microservices landscape demands, having good automation practices, right tools, skilled engineers help them to adapt to Microservices.
Conclusion
Right, it seems complicated? No, the fact it is simple.
Answers to the following questions will further help you in your Microservices journey:
- What does the service look like?
- In enterprise big picture, Where does it fit into your ecosystem of services?
- What to measure and what metrics are important?
Let us explore each of these categories and we will deep dive where I have relevant experience. Finally, Microservices challenges in traditional enterprises are real but let us take the bold step.
BMK 