The Zen of Deployment Automation Retrospective

Every now and then, I’m asked by one of our Octopus users: “Sometimes we need to change one or two files. Can I just deploy one or two files?” It’s at this point, I place my elbows on my desk and put my head in my hands. And I thought I was bad at letting go. Let me backtrack a little.
Several weeks ago, we had a team meeting as we were inundated with numerous requests and it’s been one of our operations sore spots. When you have 200+ project teams on your platform, there’s going to be maintenance issues however, we also have several initiatives that has divided our attention. Over time these requests essentially led us into doing the work for a project team who has no idea what they’re doing. Not even in regards to Octopus Deploy, mind you, but even within TFS, builds, dependencies, etc. Our “DevOps” team became more of a tech support team for enterprise deployments.
During our discussion, our senior manager brought up an interesting point which I had not considered: ownership. Ever since these teams were creating applications, they never owned the deployment side of the work. They simply had a checklist of things to do remotely via RDP and did the prescribed tasks without question. When it came time to do production deployments, the checklists were simply passed onto the team who did the production deployments. Sounds familiar, right? Well now, with Octopus Deploy (and other tools as well), the old passing the buck scenario is gone. You now own your deployments – which theoretically sounds awesome. Except in practice, the ugly side of deployments starts to rear its ugly head – aka the devilish details. Many people come and go and the task of deployments sometimes shuffle around because no one wants to do it – thereby making the set of instructions on how to deploy become too important and passively allows people to know less and less about the details of the system, networks, components and so forth. At this point, the mindless steps become ingrained into the process and to a greater extent, the operational side of a project. That’s why people stick to the old routine: no one really had to own the process from beginning to end.
I’m somewhat surprised that ownership never really occurred to me before, especially considering the numerous teams I’ve coached. It’s a small revelation (to me at least) of why people do what they do and in spite of newer and better tooling, if people don’t own their deployments nothing really improves as far as I can tell. But how’s that possible? Aren’t they’re automating their deployments? They’re saving time, doing deployments more rapidly, etc. Yes by definition automating things can improve except if you don’t own the responsibility for understanding all the components of the system you really end up missing out on so much more. You may improve your deployments somewhat, but by passing on crafting and exploiting more (appropriately applied) automation, you miss out on a world of potential improvements and the sum of all the little tweaks.
We have many successful teams. Some teams knew right away what they wanted and how to do it very well. Our DevOps team had little to no interaction with them as they finally got the control over their deployments that they wanted. For other teams it was a learning experience, but I could gradually see the transformation of their deployment process, their variables and their deployment history and it’s a rewarding experience to see the improvement. Not only as a mentor/coach but to see teams shed the old manual ways that have plagued .NET development for ten years too long was a reward in and of itself. For the teams that have embraced, experimented and continually use Octopus Deploy the fruits of their labor are clearly evident. I’m not talking about theses teams. I’m talking about the teams that scrape the bottom of the barrel. They get from point A to point B with minimum effort or worse, too much effort. There’s redundant steps, poorly configured IIS sites, too few (or many) variables, no refactoring or redesigning their deployment processes – the list goes on. They do everything as they did before but just in Octopus Deploy. The results are astonishingly similar to their understanding of their system.
I’m going to try to stop ranting about this topic because as much as I love automation, I see people struggle with it or worse – ignore it altogether. Automation can be complex but the dividends are totally worth the energy, effort and then some. Automation’s dividends yield so much more than people realize, I feel it needs to be highlighted in front of people to get it into the forefront of people’s consciousness. In my first rant about the Zen of Deployment Automation, I had four rules or tenants to moving towards faster, more reliable deployments. And now, looking back with ownership in mind, every single one of my points from my first Zen of Deployments rant is a direct conclusion of ownership. If you want mastery at anything, ownership is essential and automation of any kind is no different. I wish I could prescribe a list of steps to do, things to read and ideas to take to heart, but I can’t make anyone claim ownership of their own deployment automation. There’s no shortcuts. Ownership is a conscious, autonomous and genuine choice. You cannot fake it – you have to own it to truly embrace and leverage automation.