December 20, 2017

Day 20 - Regarding the Responsibility of Systems Administrators

By: Ben Rockwood (@benr) Edited By: Joshua Zimmerman (@TheJewberwocky)

Anyone who has worked for me will tell you I’m a big fan of reflection. At the end of every year I utilize a simple Compact Calendar created by David Seah to look back at the year and sketch out, week by week, what happened. By looking back, we can chart the direction that we’ve been traveling, with or, more often, without our conscious knowledge. In DevOps we often say that it’s all about “velocity” and “velocity is speed with direction”, thus our reflections allow us an opportunity to course correct when we determine that the direction we’ve been traveling on doesn’t actually align with our desired course.

SysAdvent is, to me, like a SysAdmin year book. We can see the topics that were important to us during a given time, allowing for reflections in the years to come. There is, therefore, a certain responsibility upon us as we contribute.

Many great things have happened this year in tech for us to reflect on. I strongly feel that that 2017 was the year in which containers finally proved that they are indeed here to stay. The technology itself was never really in question, but a massive ecosystem needed to come around the fundamental technology to make it truly feasible. Kubernetes, Service Meshes powered by intelligent and lightweight reverse proxies like Envoy, Serverless, application centric monitoring and metrics empowered by Prometheus, major changes in SD-WAN and NFV that will power Hybrid Cloud and reliance on telecom networks in a way never before seen, and on and on, so many technologies that have matured this year and begun to shape a new era in the way we operate services. And while all these things are important, there is a topic far more important, and indeed less enjoyable, which must be addressed as we reflect.

The biggest event of the year in tech was arguably the massive Equifax breach. Equifax made a lot of mistakes in a variety of ways, but if you are reading this you likely know how the IT and Operations functions are managed and run inside most companies. I doubt it would take much effort for you to examine the few facts we have about what happened at Equifax to build your own mental picture of what it was like before and during the breach. Tell me honestly, could this have been prevented? We all know that it could have been. Tell me honestly, is your organization any better? I’m willing to bet you are not. Yes, Equifax Inc. took the blame and certainly there is a very real systemic problem there. But while the ultimate responsibility lies in the leadership of the company, the problem occurred because the SysAdmins, our brothers and sisters, failed to prevent it.

If we’re not stopping to reflect on our part in these massive breaches then we’re frankly unqualified to do this type of work. When you hear of these breaches do you get a chill down your spine with fear that you too are vulnerable? That you could be next? And what do you do in response? My guess is very little. There may be a short, focused attention to security but after we’ve worked off enough of our guilt we tend back towards business as usual.

In 2011 when I delivered my LISA Keynote “The DevOps Transformation”, I tried to make a case for a new level of professionalism within Systems Administration by, among other things, casting DevOps as not any one specific thing but rather a “banner for change”, dispelling a general fear of standards and best practices, such as ITIL and COBIT, and connecting Systems Administration with the greater tradition of Operations Management over the previous 100 years. I have also predicted elsewhere that tech will become a regulated industry by 2030. We are seeing events nearly every month which make me more and more convinced of these positions. The level of professionalism and accountability in our trade must rise significantly. With no credible industry governing bodies to drive this transformation, it must happen culturally. DevOps was that cultural change, but the sheer magnitude of changes we have needed to take place has largely trumped this meta-theme.

I firmly believe that that every operations team, of any size, should be externally auditing for SOC2 Type II compliance with, at a minimum, the Security Principle. Furthermore, the European Unions General Data Protection Regulation (GDPR), while it may have its faults, is a quantum leap forward for both the rights of individuals and accountability of corporations. We can no longer afford to remain cynical geeks belittling such reasonable advances as a PITA. If your unsure of where to start, consider the Cloud Security Alliances STAR Self Assessment.

Above all, Systems Administrators must step up and take a more active role in systems engineering and architecture. SysAdmin’s don’t create business software, we assemble software together to provide value. That is, SysAdmin’s engineer and architect solutions as integrators; creating, maintaining, and protecting business value. All too often we feel lucky just to keep up on last years trends and handle the day to day incidents and maintenance tasks.

The solution isn’t to hit the brakes on innovation, lock everything down and focus just on security and privacy. Rather, we have a variety of innovations that give administrators increasing levels of sophistication and capability. Docker and Kubernetes allow for very small surface areas and a more testable and deployable solution set than we’ve ever had before. Hashicorp’s Vault and Consul allow for security capabilities that were only a dream just a couple of years ago.

One concept that particularly excites me is Kubernetes Operators which “represents human operational knowledge in software to reliably manage an application.” While this isn’t a terribly new concept, its never looked so achievable as it does with Kubernetes. A It drives towards the old famous SysAdmin mantra of “automate yourself out of a job.” Clearly, that never happens, but extending Operators with an increasing body of knowledge rather than creating text documents and procedures clearly the way of the future, particularly when Kubernetes give us a rich API and consistent deployment model.

And so, as we come to the end of 2017, I’d strongly encourage you to grab a Compact Calendar and reflect on your year to examine your velocity. How fast did you really go? What direction did you really go? And, above all, did you make the most important things the most important things? As we close out the decade lets use all these amazing new innovations serve the goal of providing real value to real people. We are in the best possible positions to advocate on their behalf and create real change to safeguard the rights to privacy and security that all people should enjoy.

No comments :