Decentralizing Your Microservices Organization
Adaptability — the ability to quickly and easily change — has become a primary goal for modern businesses and has put pressure on technology teams to build platforms that are easier and less costly to change. Working in such environments, these teams have been attracted more and more to the microservices style of software architecture. What attracts them is the promise of a method for expediting changes to software, without introducing unnecessary danger to the business.
The microservices way of doing things is made possible in large part by favoring decentralization of software components and data — more specifically, by breaking up “monolithic” elements into smaller, easier to change pieces, and deploying those pieces on the network. Making this architecture work well requires a change to the way work is done and how work is governed. The organization that adopts microservices is one that “gets out of the developer’s way,” and provides the freedom and autonomy to make the magic happen.
In organizational design speak, their goal is to decentralize the decision authority. Rather than having a few people make the architectural and software decisions for everyone in the organization, decentralization would enable them to distribute decision making power amongst the people who do the work.
Pushing decision-making authority directly to workers allows them to produce with more freedom and autonomy. Under the right circumstances, this will lead to better and faster changes. However, if your organization gets it wrong, a series of bad decisions may slow down the rate of change — or worse, end up damaging your business.
The trick is to decentralize only those things that help you expedite, without sacrificing your system’s security.
Finding the right decentralization strategy is an evolutionary process that requires you to adjust, analyze, and adapt. To help you get started in the right direction, here are the three most important questions you should consider:
Which Decisions Should We Target?
Our goal in all of this is to augment the changeability of your platform. So you should start by finding the bottlenecks that prevent change from happening. Take a close look at how features and changes move from idea to implementation and find the parts of the process where the people who need to do work can’t because they are waiting for someone else to make a decision.
Decentralization is not a silver bullet for unblocking bottlenecks. In an organization that leans toward centralized decision making, however, decentralization will probably help. On the other hand, if you can’t find many situations where employees are getting stuck because of centralized decision processes, decentralization shouldn’t be your biggest concern.
Making a microservices system work takes more than changing ideas about the size of components. All of the areas of your organization that involves creating and changing services have a part to play. Here is a non-exhaustive list of the types of decisions that could be candidates for decentralization in a microservices world:
- Service lifecycle — When are services created or retired? What are they called? When do we need to break them apart?
- Service implementation — Which tools, languages and architectures should we use inside each service?
- System architecture — How do the services talk to teach other? How do developers learn about them?
- Data architecture — How is data shared between services?
- Change process — When can services be changed? What are the tools and process for deployment and QA?
- Team management — Who serves on which team? What is each team responsible for? What do team members do?
- People management — How are people hired and fired? How are employees incentivized and rewarded? What is the vacation policy?
- Security management — How do we reduce the risk of security incidents? What needs to be done to improve security for the overall system?
- Procurement — What software can be purchased? Which protections are required for using open source software?
It’s worthwhile to analyze how the decisions in these spaces are made. How do they impact the way that the system is changed? Which decision processes are holding you back? Which ones stop people from being innovative? And finally, in which cases would more freedom and autonomy be beneficial?
Netflix is a great example of a company that has been able to decentralize in innovative ways. Its policy of empowering employees to decide how much time off they need decentralizes a decision that has traditionally been closely controlled.
Giving employees the authority to designate their own holiday allocations might sound odd. But the exact approach you should take if you want to kick your organization’s changeability into a higher gear is by ridding it of redundant and unnecessary coordination work, while at the same time refraining from introducing new risks. Teams that are pursuing the microservices style of building software should be thinking about these kinds of optimizations throughout their organizations.
This doesn’t mean that the policy of distributing holiday decisions should be a principle of microservices architecture. Netflix can do it because its culture and workforce make such a policy easier to implement. Its employees are trusted with making decisions that are best for the entire system, and the company is known for being selective in whom it employs.
Not every organization looks like Netflix, and very few operate in Netflix’s niche of online video content delivery. Each company has its own unique set of constraints and goals. You’ll need to discover your goals and constraints for yourself. Analysis, insight, and experimentation will help you prioritize your decentralization efforts, and steer your system towards the goal of adaptability.
Who are the People Involved?
Some of the decisions people make can be incredibly impactful to their company. A decision to change the way that a bank account transaction is implemented would be risky for a traditional bank. A decision to change an application’s user experience would be a risk for a software company with a large user base familiar with the old interface.
Organizations try to increase control over these types of decisions so that this risk is minimized, leading to a centralization of decision making power. For example, in past years, Apple was known for having a highly centralized design group, consisting of a relatively small number of people who make most of the decisions about the design of their products.
Centralization happens because the right people need to make the decisions that matter most. Usually, the “right people” are the ones who have the mix of talent, expertise and experience that allows us to trust them enough to make the best decisions. We could call these people our “star” decision makers.
If an organization was only filled with stars, then all its decisions could be decentralized. If we trusted more of our people to make the best decisions, we would distribute more decisions to more people.
In practice, companies have a limited amount of star decision makers. In reality, most teams have a few stars combined with a larger number of decision makers who are competent but lack some of the required experience or talent to make perfect decisions.
The good news is, you don’t need an all-star team to employ a decentralization strategy. You just need to be thoughtful about how you put your teams together and where you deploy your best decision makers.
The microservices style makes all this easier because the impact of decisions can be constrained at the same time the speed of implementing individual changes is increased. If a team makes a wrong decision when working on a microservice, the blast radius of the mistake should be small and contained. When changes to the system are made cheap and easy, the team can quickly improve on previous decisions, allowing them to get to the best decision faster.
In this type of environment, you aren’t limited by your star power — when the goal is to reach the best decision, you only need to provide a system that gives competent workers the freedom and autonomy to get there.
Who Owns Which Part?
No decision is made instantaneously. It’s based on choices, which in turn is based on domain knowledge. Decisions should never be implemented immediately. Sometimes it may require someone’s blessing, while for other times, it may need highly specialized skills or knowledge to implement.
Management expert Henry Mintzberg gives us a nice model that outlines the steps of a decision process:
- Research and information gathering
- Generating choices
- Selection (making a choice)
- Authorization of the choice
- Execution and implementation
The key to all of this is that you don’t need to be absolute when employing decision decentralization. Each of Mintzberg’s steps can be independently centralized or decentralized, allowing for much greater flexibility when balancing the speed and safety of decision-based changes to a system.
Consider the case of the typical hiring process in a big company: When it comes time to find new people, it’s the centralized Human Resources division that broadcasts the open position and invites people to apply for the job. The same centralized team screens the candidates and generates a list of the best from that group. The list is then handed over to the actual hiring manager, who selects the best candidate based on further scrutiny. From there, the hiring manager hands the ball back to the central Human Resources team, who finish the paperwork and complete the process.
This pattern of centralized choice generation combined with de-centralized choice selection is a common one in big companies. In fact, most of the organizations who have incorporated the microservices style use it in some form.
For example, a centralized enterprise team can identify the three flavors of databases that all microservices teams should be using. It’s up to the individual teams to make the choice selection, but they are expected to choose from the menu that’s been provided. If they deviate from the approved list, they’ll need to justify their decision, thus providing the centralized team with a feedback mechanism with which they may re-evaluate the menu.
Decentralizing the selection, authorization and execution parts of the process allows the individual teams to move fast and at scale. Centralizing the research and choice generation steps, hurts innovation overall, but reduces the risk that decisions will be made that negatively impact the overall system. It’s a popular pattern because it strikes the right type of compromise for most organizations.
Decentralization comes up a lot when people talk about microservices organizations because it’s an effective way of improving the speed of change. But don’t forget that it is just one part of the equation. Who your people are, how your teams coordinate, and all the systems, tools and context they work in, are equally important.
It’s essential for you to understand that thinking about how decisions are made — and more importantly, how the process can be improved — is a great way of moving toward a change-friendly organization.
CA Technologies is a sponsor of The New Stack.
Title image of people building a pyramid — specifically, gymnasts lifting each other into a pyramid, circa 1909 — in the public domain, courtesy of the U.S. Library of Congress.
_(14784186253).jpg){kind=link}
