The case for engineering automation

Why automation is important

When you survey developers on how to improve engineering practices and their daily job experience, their answers invariably include getting rid of little annoying things - what's called toil.

Toil is manual and repetitive tasks that waste your time. Toil is arguably worse than crisis, because a crisis is temporary and firefighting can feel rewarding when it's over. Toil is more like a death march - an insidious force that eventually leads to burnout.

In fact, the software industry moves itself forward by constantly automating away higher and higher degrees of toil.

Remember the war over tabs versus spaces? It was a thing. Today, we’ve mostly worked around it by delegating menial formatting work to automated linters.

Once you’ve done something three or four times, good software development teams automate it. That is in line with the nature of what we do as developers: we write software to allow people to do things more easily that were previously hard.

Choosing to automate

When it comes to building automations, you don’t want to reinvent the wheel. Ask the following questions each time:

• What have teams done manually?
• What have teams done in a bespoke way?
• What investments have they made to build their own solutions?
• What will people recognize as useful or impactful?

When I worked on Jira at Atlassian, we had written lengthy tests that took three or four hours to run. If something failed, we had to restart the tests and wait for them to complete all over again. We decided to invest a couple of months of developer time to split our tests so we could run them in parallel. That got our test time down to 45 minutes.

The process improvement was worth it, but it was expensive, and it was a continuous investment: we built it ourselves, so it took a lot of time to maintain. Many folks back in those days made this investment too, and the industry recognized it. These days, you can click a button in CircleCI to do that logic for you.

The point is, you know you got an automation right when it gives you a capability you had always wanted but was always too much trouble to invest in.

Four categories of automations

Choosing action is the way to improve, and automation is the best way to get there. We see four categories of automations teams can use to improve:

Guardrails: Automations that ensure agreed upon boundaries for teams to work within are codified into processes

Notifications: Automations that ensure user awareness of conditions that might require their attention, so they don’t have to proactively detect such conditions themselves

Actions: Automations that execute complex actions

Workflows: Multiple automations that together form a new capability to achieve advanced developer workflows

In the next articles on this series we’ll dive into each category of automation and how they can help engineering teams improve.