December 19, 2008

Day 19 - Visibility and Communication

A friend said to me tonight, "Once again, at a company function, the CEO has forgotten operations exists." This is a visibility problem that often hits operations and support teams.

Good systems administration is about hidden work and effort nobody ever sees. You are probably accustomed to only being highly visible when there is a problem - interrupts asking if something is broken, when it will be fixed, etc. If this is the only visibility you receive, how can you expect to be loved and adored by the world? You need to help yourself and your teammates with visibility. Your manager should help you, too.

Improving local visibility, that of your teammates and manager, is just as important as external visibility to your customers (employees or otherwise).

Keep track of your work: code commits, ticket resolutions, etc. A habit I developed while working at Google was to maintain a weekly report of things done. I've sinced modified this habit to include not only finished tasks, but things not done yet, progress blockers, and future todos. At the end of each week, send this data to your manager. Have the rest of your team do the same. For bonus points, send it to your team, so your coworkers will know what you did last week.

This weekly tracking will help you do two things: first, to maintain a high quality stream of communication to your manager and your coworkers, and second, to help you better track things that need to get done. If you're lucky (and you probably are), a fellow coworker will see that you have something on your todo list that he or she would like to do and offer to relieve you of this burden.

Tracking this data will also help you show how you can or can't take on that new project for time management reasons. Further, it's a huge help to have a document of accomplishments for career advancement.

To enhance visibility and communication with your manager, have periodic (weekly, etc) one-on-one meetings. Email is good for status reports, like above, but face to face contact is best for discussion. It's a two-way street, so use this time to make sure your visibility and perceived performance is what you expect it to be. Additionally, make requests of your manager if you have any. If you submit status reports that include things that are blocking you, your manager should ask how he or she can help remove these blocks.

Visibility to your customers and to your management should be handled differently. Your manager should be the funnel of information up (and down!) the management stack. Make sure he or she is performing this task. A good time to ask about this is in your one-on-one meetings.

Your customers are very important. Your work will directly, positively or negatively, affect their work. This is power and responsibility that can lead to resentment and anger if not handled properly. Creating interaction policies and informed expectations is critical to customer visibility and happiness.

Your interaction policy should explain how to contact your team, and be sure it's accessibly documented. I find bug systems to be great for tracking customer requests or problem reports, so require usage of this system for such things. Define escalation criteria, such as "if a critical problem is not responded to in X minutes, please email this pager address." You need to define "critical" in the previous statement, too. Don't use email alone for problem reporting, as it doesn't easily lend itself to historical tracking.

Set expectations! Planned changes that will cause outages should be announced ahead of time, at the start of the work, and at the end of the work. Any changes in planned change should be announced in a clear way. Announce known issues to anyone affected as soon as you are aware of the problem and include a contact (if not you), a time estimate on repair, and a description of the scope of the outage. Define an SLA for any service you support. An SLA is a common form of expectation declaration.

Additionally, don't waste someone's time. If you send an email about an upgrade to a specific component that only a subset of your customers use, then put a very clear header at the top, such as:

This is regarding an upgrade to the internal mysql servers. If you don't know what this is or don't use these systems, you can stop reading now.

Lastly, visibility and related communication does not have to be manually generated. Automation is sexy, and automatically informing customers about information important to them is a great way to avoid getting 15 tickets filed for the same problem. Have a web-based dashboard that includes a list of known problems and links to related trouble tickets, a list of upcoming planned changes, perhaps a "tip of the day," and any other useful information you see fit. There's plenty of content management systems available for free to help you get this dashboard site up and running in a very short time.

Healthy visibility is about good communication. Systems can go down and customers still be happy because you've involved them in the process by telling them and setting appropriate expectations. Your manager will be happy knowing your team is working effectively by knowing what everyone is working on without having to ask. Happy customers and happy managers means happy and appreciated sysadmins, even when things are on fire.

No comments :