DevOps Means Don't Be an A-hole
"If the company's doing well and people don't hate each other, you're probably doing ok."
– John Allspaw, speaking at Devopsdays 2011 MV.
This year I had the pleasure of being a panelist at Devopsdays 2011. This is a two day mini conference where people interested in the DevOps movement get together to discuss the state of our world. The general theme of the conference was the technical and philosophical questions involved in building, deploying, and managing complex software systems.
I come to the world of DevOps from the operations side, as I’ve written about before. I currently work at Yahoo on very large software installations and I am keenly interested in whether DevOps can result in more efficient practices and happier people. These days I’m cautiously optimistic about DevOps, but the reasons for my optimism about DevOps have shifted.
The above quote was a casual remark that John Allspaw made as he participated in one of the Devopsdays panels, but it really got me thinking. Originally I (like many people) was attracted to DevOps because of the focus on infrastructure as code. Like any technical person, I’m always looking for a better tool to solve my problems. In my case that seems to lead to writing more and more complex perl scripts, and playing around with things like Test Driven Development and sprint planning.
Those are good tools, but you can’t define a movement by the tools it uses. Movements are about philosophy and group consensus. That’s what started me pondering the context of John’s comment. My realization was that DevOps isn’t about tools, it’s about working together. The problem with our existing development vs. operations divide is that people don’t get along. For various perfectly good reasons, the traditional software development and deployment paradigm results in distrust and fear on both sides of the fence. Development create change, Operations is penalized for change leading to outages, soon everyone is pissed off.
So, what’s the alternative to Devs and Ops being at loggerheads? Communication. As John said, it’s about people not hating each other. That hate comes from fear. We all deserve to work in an environment of mutual respect, and the only way to get to that is to overcome our anger and fear and talk to each other.
Almost all the roadblocks in my own career have been due to communication failures. Many (heck, most) of those failures were probably my fault. Here’s a couple of recent examples:
-
I attended a meeting with a development team to discuss the design of their software. They wanted to design their component to protect their servers from misconfiguration at all cost. The problem is the way they do this leads to extended software install times due to all the extra configuration checks. My Release Management team pushes this software to tens of thousands of production servers, so a 15 minute delay on every host has some pretty serious consequences. In the meeting, I told the development lead that they are doing things wrong. Tempers flared and everyone in the room got pretty tense during a heated ‘discussion’.
-
My operations team has been working with a very difficult development team. The developers don’t seem to know what they are doing, and it felt like the project manager isn’t on top of the schedule. This lead to a lot of internal grumbling in my team about how this other team is a bunch of idiots. On a team conference call, I spend some time complaining about how that development team has yet again missed an important deadline and caused our team to have to work harder.
What do these two examples have in common? In both cases, I failed to communicate properly. Let’s look at the first case: what was the benefit of how I handled that interaction? Almost nothing. Maybe I felt a little better after I told the developers they were wrong, but that’s not going to really fix the situation long-term. Negativity just leads to resentment on both sides. I guarantee that in the future interactions between my team and that development team will continue to be negative if we keep following this path.
In the second situation, I thought I was just blowing off some steam in the privacy of my team. However, my manager pointed out this sort of thing can have some far-reaching effects. I’m a senior member of the team, so my opinions for good or bad carry a lot of weight with my team members. This is something everyone who cares about DevOps should think a lot about - many of us are very senior members of our respective technical teams, and thus we are probably in positions of leadership.
After I thought a bit more about this, I realized my boss was right on the mark. Negative talk is poisonous, especially when it comes from senior team members. Think about it: my negativity about the problematic development group sends a clear message to the rest of the group that I think that other team is a bunch of goddamn idiots. That’s a lesson that everyone else on the team is going to consciously or unconsciously emulate. Again, my lack of proper communication negatively affected the interaction between a development team and my operation team.
Maybe all of this falls under the general umbrella of ’leadership training’, DevOps is really about communication, and the best way to make that communication happen is to lead by example. I want to encourage everyone to spend some time thinking about how your actions influence how others behave. It’s so incredibly easy to get annoyed and say something negative. Trust me, I know, I do it all the time. But is that really how you want to be perceived, as that grumpy operations neckbeard or that curmudgeonly developer? You can’t directly change how other people behave - all you can do is try to set a good example. That’s what I’m trying to do and I encourage everyone else to strive to do the same. Making meaningful improvements to improve the level of group communication is how DevOps can really make a difference.
So putting that all together, here’s my tl;dr for this blog post: Don’t be an a-hole.
I’m philiph on twitter, by the way.
Comments
No, Philip, I don't think you failed to communicate properly. You communicated in the way that your limbic system determined was the best possible way to do. If you had overridden this, the only person with problems would've been you, leading to frustration and ultimately ulcers and a heart attack at age 57.
Negativity may be necessary, at first, to get things going in the right direction in the same way that introducing a magnet into a container of metal shavings gets those haphazardly strewn/disorganized shavings aligned. AFTER this vital operation, after the "corrective" realignment, can it be possible to get some next/action items going. Negative talk is ALSO poisonous if suppressed, leading to that heart attack of which I spoke earlier.
There are many reasons why the other team may be lacking in progress and as long as there's some interdependence, the onus is on everyone involved to call things to order and if that necessarily involves some negativity, as long as it's not violent and malicious, have at it.
Even Jesus brought out the pummeling cudgel.
Disagreeing, opposing arguments, constructive criticism, and making persuasive comments are all part of good engineering practice; it results in comprehensive design. But there's a difference between being practicing those things, and practicing those things as an asshole, which is what Philip is trying to get at.
Engineers want to work with people who will make persuasive arguments about engineering. They don't want to work with people who are jerks, and there's a difference. :)
IMHO, it is all about being assertive and understanding where the others come from when we discuss.
Being patronising or condescending are behaviours that lead to frustration.
There is a lot of ego in the world of IT and software development. I think the word "idiot" was tossed around almost daily at my old work place, oftentimes in reference to the people at MS, but just as often for customers or consultants, or vender tech support. It just generally adds to a negative atmosphere.
I think it's safe to say (and pretty transparent) how all this negative talk is used to make oneself look better. After all, everything is relative, and the more you degrade others the better you look. Unless other people are smart enough to see it for what it is, then it has the opposite effect. I think all geeks would benefit from a class on primate behavior. It would help them understand the workplace better, and I don't mean that sarcastically. People are constantly re-enforcing a pecking order, making mild threat displays, or trying to safe face.
I remember once my boss called one of the Sys Admins a "technician" and it was so obviously aimed at bringing him down a notch or two.
And, of course, if you are an operations guy, there is the constant never ending stream of put downs from developers, harping on your job as being "easy", and just generally attempting to make you less satisfied with your work and life, because you are not as awesome as them.