Engineering Improvement Runbook | Engineering Automations
Don Brown
November 20th, 2023
Ship code faster via Slack approvals
Use a Slack-based approval process to ship code faster
In this post, we'll show you how to set up a Slack-based deployment approval process that has helped the Sleuth team shorten lead time and ship to production faster.
With this process, developers can approve production deploys from within Slack simply by giving a thumbs-up emoji 👍.
As a bonus option, we'll also show you how to automate this approval process entirely.
You can do this in three steps:
- Step 1: Set up a build that includes a manual approval step
- Step 2: Install an Automation to enable Slack-based approvals
- Step 3 (optional): Install another Automation to enable automated approvals
Set up a build that includes a manual approval step
You do this in your CI/CD tool. This is how our test and deploy process look like in CircleCI:
Checks: These comprise running tests, formatting, building images, creating assets, and JavaScript tests.
Staging: Here, you'll deploy the code to an environment where you can test and verify if the code is functioning as expected.
Approval: This step pauses the pipeline until you manually approve it.
Deployment: Once the approval is granted, the final step of deploying to production environment is executed.
Install Slack-based deployment approvals Automation
In the Sleuth Automations Marketplace, there is a workflow Automation called Slack-based deployment approvals.
The configuration is straightforward:
- Set it to trigger when a deployment goes out to a certain environment - in our case staging.
- Choose the repository you're installing this on - in our case the whole application.
- Select the build where you have that approval step - in our case, test and deploy.
- Select a Slack channel where the message will be sent to - in this example, tmp because we've already got this set up in another channel.
What it does is it tracks when a deployment has gone to an environment like staging, and send a message to Slack that says, "do you want to promote this to production or not?"
If you answer yes, then it'll go back to that circle CI approval step and approve it automatically. This way, the developer can live in Slack and not have to go back to Circle CI.
This is how the Automation looks like in action, in Slack:
Install another Automation to enable automatic approvals
Now for a bonus step, what you'll notice in this example is that I didn't actually click the approve button, and yet it was deployed to production!
This is because we have another Automation that's active called Auto-promote soaked deploy if environment is healthy:
So once a deploy goes to staging, it sits there for 5 minutes. And what Sleuth does is it tracks the health of that change. If it's still healthy after 5 minutes, then it clicks the approval on your behalf and promotes the deploy to production. And that's actually what happened here.
Now how do we know it was healthy? In our case, we've set it up for Sleuth to look at error rates. Sleuth is tracking the number of errors in Sentry, and then it's trying to figure out what's normal and what's not. If it's abnormal, then it calls the deployment unhealthy.
Manual vs. automatic deploy approvals
As you can see in Step 2, you can choose to have a human do the health check where they look at the graphs, make sure things are healthy, and then manually approve it to production.
Another option as we laid out in step 3 is to use Sleuth to automatically track the health of that deployment to staging, and after a period of soaking, have it automatically be promoted to production.
In our case, we actually use both of them, but one or both might be applicable to you.
The important thing to remember is you don't want to be automatically shipping code from staging to production without any kind of health check. That's what proper continuous deployment looks like.