Day 22 - Simplicity in Complex Systems

Written by: Joshua Zimmerman (@thejewberwocky)
Edited by: Michelle Carroll (@miiiiiche)

Simplicity as a reaction to complexity

Complexity is inherent in IT. We work with tools and technology that are objectively complex, in organizations made up of complex structures of complex people, and can only achieve the goals of the organization with complex interactions. We build this complexity into the systems that we architect. We justify most complexity, believing that it may be needed to accomplish our goals. We disregard this complexity until it is sufficiently difficult to accomplish our goals. We talk about complexity a lot.

However, we often don’t talk about simplicity until the complexity in our systems causes problems that frustrate us or block our goals. Engineers often talk about simplicity only as it relates to complexity. We’ll use simplicity to sell someone on a product or idea, but we only adopt it because the tool lessens our complexity. We lack a common understanding of simplicity, and forget to talk about simplicity on its own merits.

Let’s change that.

Before diving into a real discussion of simplicity and complexity, I need to reiterate the tools we use in the IT landscape are inherently complex, but not as complex as the systems of people that create and maintain these technologies. There is no true dichotomy between simple and complex: in the simplest cases, things exist on a spectrum of the two. Therefore, it’s impossible to use basic definitions of simplicity and complexity. If we were constrained to those definitions, we would never be able to call the tools we use in IT simple, because tools like hammers and levers exist.

This leaves us with the question, how do we define simplicity for our complex technological and cultural systems? By discussing simplicity in a more abstract way, I hope it becomes easier to apply simplicity to our applications, systems, and the organizations and communities that support them. To frame our conversation, let's explore John Maeda’s *Laws of Simplicity. *

The Laws of Simplicity and modern IT

Laws of Simplicity was published in 2006, when Maeda was still a professor at MIT’s Design Lab. Simplicity was already a trend in the design community. After a couple years of iteration, the iPod was in what we now call its "classic" design. This simple interface design stayed largely the same for a decade. Google, with its minimalist front page, had emerged as the dominant search engine, and the term “googling” was already proof of its popularity. Maeda had been working on distilling his ideas about simplicity in design into laws that would help people discuss, understand, and achieve the concept, in design and beyond. He believed these laws were applicable outside of the realms of design. The result of that effort is Laws of Simplicity. Maeda treats the book as the start of an ongoing discussion — one need not follow all the laws, nor need to consider them a complete set.

Law 1 - Reduce: The simplest way to achieve simplicity is through thoughtful reduction.

The first step in simplifying something is to remove what is not necessary. What do we do when we have gotten rid of all the obviously unnecessary or unused pieces, and still have something complex? There are other ways to reduce a system. Maeda suggests a few, with the acronym SHE: Shrink, Hide, Embody.

We have a strong reaction when something is physically or conceptually large. When we shrink things, they seem simpler. A desktop may seem more complex than a tablet, despite having very similar underlying technology. This type of comparison works with more abstract concepts. We use terms like "lightweight" and “portable” to describe containers, and before Docker became popular, we used those same terms to describe VMs. We conflate simplicity and smallness; we use the word “lightweight” even though a container has no mass. By shrinking the conceptual size of a system, it seems simpler.

Hiding the complexity serves a similar purpose as shrinking. We see this often in IT, through menus in a UI, or interacting with a service through the use of an API. When we implement these things, we usually are not removing any of the complexity of the system. However, we simplify working with that system by only calling upon the complexity when needed.

However, you can’t just reduce, shrink, and hide your systems and applications. The final product must embody value and quality equal to what you are removing. When organizations first implement a central logging system, some people will still want to go to the original host and grep the log file. They see more value in the old way of doing things than in the new system. You need to communicate the value of the change, or risk sprawling complexity.

Law 2 - Organize: Organization makes a system of many appear fewer.

Human beings tend towards organization as a way to simplify things, and it’s easy to see this tendency in our attempts to organize our applications and technical systems. We organize sets of functions into libraries. Service-oriented architecture and microservices are employed to break monoliths into smaller, more sensible buckets. Service discovery tools were created to easily find, organize, and manage like systems.

As an industry, we’re actually pretty good about reorganizing systems when the complexity causes us pain. We sometimes forget that organization is primarily about grouping. The goal is to simplify the set of things you’re organizing by combining things into related, sensible categories. If you have too many categories, you haven’t adequately simplified. If you end up with too few categories, you may have oversimplified and added complexity to the system.

Maeda relates organization to Gestalt psychology. Our brains allow us to extrapolate and form patterns from partial evidence. If you see three lines connected at 90 degree angles, your brain will fill in the fourth line to create a square. Maeda argues that you can create Gestalt groupings to increase simplicity, but you may make that group more abstract. For example, authentication and authorization solutions are often coupled together. This coupling can allow you to store information on users in one place across many applications that need to consume it. However, the individual applications lose the ability to collect user information specific to it.

Law 3 - Time: Savings in time feel like simplicity.

Maeda says that waiting for things makes life feel needlessly complex. It’s hard to disagree. Spending time in meetings where you learn nothing new and make no decisions feels like added complexity in your day. An alert going off for something you can’t control feels like an unnecessary distraction. Time that you spend babysitting a process to completion in front of a screen feels wasted. A developer waiting days — or worse, weeks — for an environment reveals needless complexity in your organization. Automation has been making the lives of IT professionals easier for decades. Even if it did not produce more consistent results, we would still implement it because it reduces the time and cognitive load needed to do repetitive tasks.

Automation can't reduce all of our waiting time, but we can make that waiting time more tolerable. For example, we allow CI to run our tests for us so that we can do other things while they run. We can find ways to imbue the waiting time with more value so it feels less wasted. Many people feel that writing documentation when it’s not their primary role is a waste of their time. You can retroactively ascribe more value to the time people spend writing by making sure to thank the writer when a document they wrote is useful to you. That impact makes their next writing session feel more valuable.

There is another component to the law of time that is more humanitarian in nature: You should be doing what you can to save time for other people. You can simplify the lives of others by saving them time. Some of the most common ways to do this are:

• Automate your systems
• Help others automate the systems that affect them
• Ask people for help finding bottlenecks
• Create and send out meeting agendas in advance, and keep the meeting on topic
• Make sure your meetings do not go over the allotted time

When you work towards saving time for others, they have more time and mental energy, and are more likely to want to help you reduce the wasted time in your day.

Law 4 - Learn: Knowledge makes everything simpler.

We have all been in the situation where we learn how something works, and say, "Oh, that’s so simple! Why didn’t I understand it before?" The trick is it probably wasn’t that simple— you’ve just gained the requisite knowledge to understand the subject. Even more confusing, once you understand something, it is often difficult to view its complexity from the eyes of a beginner. Pamela Vickers’ talk, “Crossing The Canyon Of Cognizance: A Shared Adventure,” is a great reminder of what getting started in our industry can be like.

Nothing adds complexity to an application or system quite like somebody who does not understand it, yet has been tasked with working on it. The fault does not lie with them! Knowledge and learning are everyone’s responsibility. When we can’t make a system simple enough for people to intuit, we must take responsibility for explaining it. It is our responsibility to become better teachers.

Maeda, who was a professor at the time that he wrote The Laws of Simplicity, stresses learning and teaching the basics. In tech, we have a tendency to undervalue our teaching processes by trying to be efficient and quick. Whether you are teaching people new to the profession, training juniors, or onboarding new employees, your focus should be on helping someone else become a valuable member of your organization or community. On an organizational level, you should be iterating on your processes and learning how to help people build these foundational skills.

Maeda also stresses repetition. Most people don’t fully understand something after doing it only once. That’s why math teachers have students do similar problems over and over. When learning something new — whether a tool, a language or the way applications and systems are architected at a new job — be prepared to not understand things immediately, and take the time to go over things again and again until you’ve learned them. When you are teaching someone, expect to repeat yourself a lot. Have patience. Your student may not ask for help when they need it if you appear frustrated by repeating information.

Maeda recommends we avoid creating desperation for the learner. In the tech industry, we are awful at this. Organizations throw new employees into the on-call rotations before they feel confident in their abilities to fix issues. People tell new folks to RTFM, regardless of the quality of the documentation. We have created a culture that encourages impostor syndrome. People fear being outed as an impostor, as someone who doesn’t really know enough to belong, and may not ask questions for fear of being found out. When we create an environment in which someone is both desperate and without knowledge, we intimidate that person and make it more difficult for them to ask questions and to learn. Consider what you can do to create a more supportive learning environment. Run game days with new employees before putting them on call. Read the manual with the person you’re training, and if the information is difficult to find or outdated, teach them and fix the documentation. Understand impostor syndrome and empathize with those that have it.

Law 5 - Differences: Simplicity and complexity need each other.

I started this article lamenting how we only talk about simplicity in the context of complexity. The concepts of simplicity and complexity have a… complex relationship. It is easy to think of them as two ends of a spectrum, but something can be simple in some dimensions and complex in others. It is rare that any system is purely simple or complex, and you should design your systems with that in mind.

We need to be cognizant of the symbiotic relationship between the two concepts. Sometimes, to make something more simple, you must make something else more complex. Making a change to simplify the UI may make the codebase more complex, as you have to intuit more about what the user wants. You may wish to simplify that codebase by using a library for that language, but that library may have dependencies, and all of those libraries need to be deployed and maintained. Microservices and service-oriented architecture make every individual application simpler and easier to maintain, but create more complexity when it comes to managing the interactions between services and the people who maintain and manage them.

Often, we simplify things for ourselves at the cost of complicating things for others. We need to have empathy for those to whom we are pushing this complexity. How you communicate the transfer of complexity is important, and predicts how the application functions in the long term. You should feel comfortable asking and discussing questions like the following:

• Does the other person or team have the time or knowledge to deal with the additional complexity?
• Can you help them learn?
• Can you help simplify the complexity through any other means?
• Can you take on additional complexity elsewhere to save them the time required to take on this new complexity?

One of the most important things that Maeda says about this law is the reminder that we do desire complexity from time to time. If our jobs were always simple, many of us would quickly grow bored. Too much simplicity is boring and may not help us accomplish our goals, yet too much complexity can waste time and cause frustration. Balance is important.

Law 6 - Context: What lies in the periphery of simplicity is definitely not simple.

The law of context reminds us that focusing on what we feel is relevant allows us to simplify, but there could be important things in the periphery that should not be removed.We tend to approach simplification of an existing system with all of the hindsight and correspondence bias that we can muster, usually because the system is causing us frustration or wasting our time. We think, "Clearly what the people before me did is wrong, and they did it because they were lazy or bad at their job!" Gaining the knowledge of why a system is the way it is, and how it came to be that way can be as important to simplifying the system as learning it in the first place.

Maeda points out that the absence of something can provide context and meaning. Google.com is a good example. They made the active choice to make their website primarily whitespace, to focus the user on the few things they put on the page. When a decision was made not to do something, it is always worth finding out why.

Law 7 - Emotion: More emotions are better than fewer.

Emotions are complex. They are one of the primary ways that people interact with the world around them. They don’t necessarily follow logic or reason. Trying to simplify or ignore emotions — whether your own or someone else’s— is an anti-pattern. Rather, endeavor to understand and accept them.

Empathy and caring for the individual has been a theme in software development for over a decade. It is a fundamental idea of DevOps. The first value listed in the Agile Manifesto is, "Individuals and interactions over processes and tools." You, and the individuals in your organization, are more important than simplicity, and more important than software. Disregarding how somebody feels will almost always result in a worse product, because you’ll never get a chance to include their knowledge and opinions in the process.

Empathy is one of the most powerful tools we have. It allows us to share and understand emotions. This understanding simplifies our interactions with those around us, and improves how we communicate, enabling us to create better products. When working in a complex system of people trying to produce a shared product, it is okay to sacrifice the simplicity you see in a product for the emotions of others.

Law 8 - Trust: In simplicity we trust.

It is easy to trust something that is simple. Take an application like Etsy’s Deployinator. The UI of this app basically amounts to: click a button corresponding to the environment you would like to deploy to and watch it go. It’s immediately clear how to choose the application you want to deploy, and you can trust that it will just work. It goes the other way as well: Imagine asking someone who had never seen Nagios before to write a check for something without any documentation. Are they going to trust their solution? People will trust your output if you can make the process simple and approachable — whether it’s building an environment, building a tool, designing an architecture, or creating a process.

There are other ways to build in simplicity to facilitate this trust. When you log in to Slack for the first time, you see pop-ups that explain all the functionality that you may wish to know about. They build knowledge that makes you confident you are using the tool correctly. You can reduce the complexity needed to interface with your system. Luke Francl gave a great talk at DevOps Days Minneapolis about helping developers monitor their own applications. They enabled their developers to write Nagios checks in the same language they were writing their application and in a manner that was simple and straightforward to them. Building simplicity into your systems will help people trust them.

Trust also helps to simplify things. Trust allows us to use APIs to applications. It allows us to download and use modules to extend languages and applications. It allows us to use open source software. We make the active choice to use these things because we trust them to work and continue working, and we do it because they simplify the way we write our software and create our systems. We also make the choice to trust the people that we work with. Trusting others on your team and other teams in your organization saves you time, and simplifies what you have to do. When you trust others, you can ask for help when you lack the knowledge to do things, you feel enabled to share your emotions, you have the ability to provide context to your problems and explore the differences of complexity and simplicity in your system together. When others trust you, you feel empowered to reduce and reorganize the complexity you know about as needed and you save time because they are enabling you to do what you think is best. Trust simplifies the way people work together and communicate in a complex organization. By Conway’s law, the simplifications that you see in your organization should become apparent by simplifications in the systems that you create.

Law 9 - Failure: Some things can never be made simple.

Not everything can be simplified. You should expect to fail at simplifying things from time to time. Even if you do succeed in simplifying something, as was talked about in Law 5, you may just create complexity elsewhere. Due to the complexity of the organizations that many of us work in, it is always important to remember that success for some can be failure for others.

Tech organizations have been learning to talk about failure in more constructive ways in recent years. Borrowing from other fields such as safety engineering, we’ve learned how to learn from and be accepting of failure. If you fail at creating the simplicity that you set out to create, as long as you learn from the process, you succeed in creating the simplicity that comes from having a better understanding of your systems and organization, and your next endeavors at creating simplicity will be more informed.

Law 10 - The One: Simplicity is about subtracting the obvious, and adding the meaningful.

The final law needs no significant explanation, but it deserves repeating. When in doubt, simplicity is about subtracting the obvious, and adding the meaningful. You know where the pain points caused by complexity are in your applications, in your operational infrastructure and in your organizational culture. Remove the things that are clearly not needed and work at improving the areas that would benefit your users, your colleagues and yourself.

The Three Keys

After organizing his thoughts into the ten laws, Maeda was left with three ideas that did not fit neatly into the laws and were all related to technology. These ideas became the three keys.

Key 1 - Away: More seems like less by simply moving it far away.

While it is clear that simplifying things sometimes just moves complexity around, sometimes you can break that complexity away from your systems. SaaS, PaaS, and IaaS solutions exist in many respects to take complexity away from our organizations.

Of course, there are many things to consider with these services. We are removing something, so we should be thoughtful about if we are adding value equal to what we remove. For an organization with a large, seasoned Ops team with automation in place, moving to a PaaS may not provide enough value. Does this service save us time? What do we need to learn to use this service? How do we and our colleagues feel about this change? Do we trust the vendor?

Key 2 - Open: Openness simplifies complexity.

Openness runs through all of the laws. Being open enables us to share in easier, more meaningful ways. Through sharing knowledge and emotions, we learn, reduce time spent doing the same thing multiple times, and we can build trust.

Maeda uses open source as his primary example, showing that sharing our code across our communities simplifies our lives. More interesting to me are the tools and ideas that we are building around openness. Chatops simplifies the communication in your organization by using chatrooms to manage your automated tools. It helps build and retain context for events going on in your system. But it is not enough to just leverage the automation. Your organization needs to welcome a wider group of people to more conversations. Most of your chatrooms should be open to anyone in the organization. Direct messages should be discouraged unless necessary.

Key 3 - Power: Use less, gain more.

Maeda uses this key to talk about our reliance on electricity and our devices. While I think it is important to discuss our industry’s reliance on massive amounts of electricity, this article is not the place. However, I’d like us to focus on the time we spend connected to our devices.

We know that it’s common for people in tech to work more than 40 hours a week, to not take (or receive) much vacation, to have on-call rotations that border on unhealthy. We force ourselves to be available by being connected to our devices. However, research and experimentation have shown that these are all anti-patterns. The 40 hour work week was based off research finding the maximum amount of time someone could work before the quality of their work decreased. Organizations have begun experimenting with minimum vacation policies that by all accounts are healthier for both the organization and employees. By disconnecting our employees from their devices appropriately, we improve the organization.

By creating simpler systems, we lessen the load on people in our organizations.. Planning to not have people, or a specific person, around forces us to simplify our systems. We focus on sharing knowledge so more people can fix an issue. We learn to trust a wider range of our colleagues. The emotions people have toward work are more positive. We learn to better cope with failure. We gain our personal time back, making the time we spend at work more simple.

Conclusion

You’ve likely noticed that I haven’t given a lot of specific advice on how to simplify a complex system. Every server, every application, every organization is a unique system of extraordinary complexity. Each deserves its own conversation on simplicity. We need to build up our ability to discuss and think critically about simplicity. We should prioritize simplicity in planning, instead of accidentally creating complexity.

You may have also noticed that the laws reflect more upon the people who create and maintain these systems than the systems themselves. Culture and communication are inextricably linked to the things an organization creates. When looking to simplify something, consider whether the complexity is a reflection of the culture of your organization. Solving cultural issues in your organization will decrease unnecessary complexity.

Thanks

A big thanks to the organizers of SysAdvent for running this awesome event, my editor Michelle Carroll for being a wonderful collaborator on this article and making it far more concise and coherent, John Maeda for his book that allowed me to develop these ideas and my older brother for telling me to read it in the first place.