IT is War : DevOps & ITIL through the lens of military history

Climbing into the DevOps vs. ITIL debate is like stepping into a minefield, as least from my vantage point. Both have serious-minded proponents, and engender the kind of passion that lesser methodologies only dream of.  But, after this very issue has come up multiple times recently, I really felt compelled to think more about it. Most of the attempts at reconciling the two haven’t resonated with me. So, I have tackled this issue the way that appeals the most to me – using military analogies.

Military history is probably my favorite part of the huge topic of history. I particularly enjoy understanding how advances in weapons, tactics, and organization have influenced the course of events. The evolution of IT over the years has been a lot about adjusting process and tactics to the available tools and competitors. The evolution of military strategy boils down to the same thing, so I think a quick look at the evolution of military tactics over time can shed some real light on the DevOps & ITIL debate.

Discipline and Organization beats Chaos

Success in war usually favors the bold – those who embrace new technology and changemacedonian pike phalanx their tactics to deal with new situations and threats. A few days ago I watched a documentary on the massive defeat of the invading Persian army by the Greek city states in 490 B.C.. What many people don’t know is that it was the technological advantage of iron lances and large shields, and the highly coordinated group maneuver called the Phalanx that gave the Greeks the edge. Moving as one unit, with shield interlocked, spears pointed front, the Greek hoplites were an unstoppable force cutting a swath through the Persians

Fast forward a thousand years. The Europeans, with their endless wars of the 17th, 18th, and 19th centuries made an art napoleon-army-salamanca-spainform of the highly trained foot soldier with a musket. Napoleon represents the pinnacle of that art. Coordinated volleys of musket fire from trained soldiers, supported by well-placed cannon, and fast moving light cavalry, could wipe out less well-equipped and trained troops. Napoleon also worked at a scale unheard before of his time, mastering logistics for hundreds of thousands of troops in the field. With his armies, Napoleon was able to dominate the Europe for 20 years, and his ideas lasted longer.

Tools can obsolete Tactics

The effectiveness of these large scale maneuvers was upset by the advances of the 19th century, particularly during the U.S. Civil War. The vastly increased accuracy of rifled muskets and cannon, and the high rate of fire afforded by caplock (percussion cap), meant that the magnificent infantry charges of the 18th century only made for easier targets and mind-blowing casualty rates. This reached a peak with the mindless violence of trench warfare in World War I.

MarineRaiders

And again, armies adapted. Along with the introduction of the tank, it was small group tactics that won the day and broke the stalemate in World War I France. Instead of ordering breathlessly stupid charges into the face of machine gun fire, small squads of soldiers could adapt to the circumstances and more rapidly advance.  World War II continued the refinement of those tactics. This didn’t mean that large scale coordination wasn’t still necessary. Artillery and air support still needed to be coordinated with the soldiers on the ground. Tanks and mobile infantry could move quickly and pack a powerful punch (which the Germans perfected with Blitzkrieg).

Very interesting, but what’s the point

Other than indulging my need to geek out with talk of weaponry, this rapid flyby of history does have a point. Successful armies over time have adapted their tactics and tools to meet the threats at hand. The ancient Greeks and Napoleon used organization and discipline to overwhelm their enemies. In the face of the devastating weapons of the 21st century, successful armies used more flexible and fast-moving tactics to dominate their slower moving enemies. All of these militaries adapted to their circumstances and made the most of what tools they had.

I don’t think IT is all that different. ITIL made a lot of sense when confronted with the chaos of IT operations, and the need to provide stable services for a business questioning the value being derived from their investment. With well-documented processes and coordination, IT departments could confront and conquer the chaos.

Conversely, DevOps has arisen in the wake of the pressures exerted by a hyper-competitive business environment and hard-to-please users with no end of choices. Just like the soldiers facing rifled carbines at Gettysburg and those facing machine guns nests in French trenches, IT operations teams trying to please the 21st century Internet user can’t march into battle with the highly coordinated, but rigid, maneuvers of ITIL. By the time they perform the service management equivalent of a pivot, the business has lost customers and revenue. In the word of U.S. General George S. Patton of World War II fame –

“A good plan violently executed now is better than a perfect plan next week”.

On the other hand, DevOps teams need a backdrop of coordinated services (e.g. cloud services or automation) to enable their agile methods, just like the U.S. Marines in World War II needed artillery and aerial support.

So, what’s the takeaway? We should never compare methodologies in a vacuum. Any methodology needs to solve today’s problems, not yesterday’s. The whole point of methodology is to provide a way to repeat the successes of the past. So, you need to find the Von Clausewitz that has succeeded where you want to succeed, and follow their lead. The methodology that best helps you meet your goals today is the right one every time.

Advertisements

Training others in the dark arts of DevOps Automation

If you are the Automation hero, why would you EVER share that stage? You are basically reducing your value to the organization by sharing your secrets. Right? Wrong! You are actually doing yourself a lot of harm, as I discussed in the the first blog post. How can you move on to other exciting challenges if you have to maintain your work of automation genius?

That is why the the IT Automation Curator’s job description has training as a core requirement:

  • Collect existing automation, and then Catalog it where others can find it (See Part 2)
  • Develop new automation based on requirements from IT (See Part 3)
  • Train others on how to use the automated processes
  • Maintain the existing automation

I had a great comment from jamesmarcus in the first blog post. Here is what he said:

“As a Director of IT I look to tools that promote easy automation, documentation, andbest practices. I try to design networks and setups with the “if I disappear” rule in mind. Meaning another sys admin of lesser knowledge should be able to look at my work and understand how why we did something in a certain way”.

I think this is a great perspective at the core of why I included training in my job description. Very few programmers, sysadmins, and other IT techies enjoy documenting their work. I don’t either – when it is after the fact. It is so mind-numbing to document your automation after you are already done and want to move on. So, that brings us to our first post.

Build your Automation to be well-documented and re-usuable

While performing amazing feats of scripting judo can impress your colleagues and get you kudos online, it is not a good long-term objective. One thing I learned early as a programmer is that creating incredibly efficient and elegant code seemed great, but it was really bad if even I couldn’t figure out what I had done a year later. That all comes down to great comments while you are writing the code. I know this may seem basic, but I have seen too many IT organizations with automation scripts, packages, etc. that no-one understands anymore. This is essentially a guarantee that the automation in question will be left alone to become outdated, brittle, and even “dangerous”. And if you are the only one that understands it, then it is your burden to bear.

So, basic to our train function of the IT Automation curator. How can you possible train people if your automation is over-complicated, un-documented, and impenetrable to mere mortals? Only with difficulty, and no one (especially you) will enjoy the experience. By documenting your automation very well as you write it, and building it to be as straightforward and simple as possible, you increase your chance of handing it off successfully. Writing well-documented and straight-forward automation has to be part of your process – bottom line.

Find automation disciples, and train them in the dark arts

While I see no reason why you can’t “teach” a class on automation as part of this role, I don’t think that is optimal, or even desirable for most people (stage fright anyone?). I have always envisioned a much more personal approach to automation training. Not every sysadmin, administrator, or IT techie extraordinaire will have an aptitude for, or interest in, designing automation. The right person has a somewhat rare combination of programming know-how, patience, troubleshooting skills, and IT systems knowledge. Obviously, everyone will use the automation, but only a few will write it.

The IT Automation Curator should be a mature, senior IT operator that has an eye for spotting talent. Like Mr. Miyagi in Karate Kid, you can watch for the young IT admin with lots of promise and fire in their belly, but unable to conquer the IT problems with their lousy karate skills. In all seriousness, I think mentoring promising candidates on automation best practices is more enjoyable and effective than the typical shotgun approach. The best part is that you can let the young upstart take care of the boring automation bits, while you save the best for yourself!

So, in summary:

  1. You can’t pass on automation that impenetrable to anyone but yourself
  2. One-on-one mentoring is a much more effective way to pass on your automation skills and knowledge

So, here is a parting challenge for all of you out there that actually remember Karate Kid. What might the IT Automation equivalent be of Mr. Myagi’s “catch a fly with chopsticks” trick?