Slack as a searchable chat-ops sink
I have been using Slack quite a lot this last year for my day-to-day work.
Late adopter as I am, I have been using it both in a geographically distributed startup as well as a tightly-knit mature company.
In both cases, it is a force multiplier.
What I will cover is how to easily utilize Slack’s API for historical data processing.
The use case
Photo by Franck V. on Unsplash
We have a long-running business-critical process. Let’s say an automatic apple tree care-taker and harvester.
Upon hitting an important “milestone” or completing a task, it sends a message to Slack channel
Weed removal: Starting at orchard A23
Apple harvesting: Box of 30 Gala apples loaded on truck 26
The same happens for failures which need immediate operator attention.
@here Failure of harvester John Deere 25, orchard B5. Operator attention required
Having the process running for a period of time, we have the questions:
- Are failures happening more often?
- Is there a correlation with the harvester type?
- Is there a correlation with the orchard location?
Let’s get cracking!
Photo by Taskin Ashiq on Unsplash
The API endpoint for searching messages is… umm, the
What a surprise! :-)
There are 3 things to note in this method:
This is the text you are looking for, plus any other filters (e.g. dates).
Check the extensive docs on theSlack query language.
Results will come in pages, to save bandwidth.
If you want to iterate through them, you need to utilize the pagination handles.
Last but not least, decide how you want to scroll through the messages.
Here is the anatomy of the JSON search result
…with a token
Photo by Scott Webb on Unsplash
To call the API you need to
- create a Slack application
- install it in your workspace
- …and generate the OAuth token.
To create the application, go to Slack’s apps page and click
Create New App.
Pick the right workspace where it lives in, if you belong to more than one.
This is the workspace from where you will be able to make changes to the app's settings. This is important if you plan to share the same app between multiple workspaces.
After it has been created, you edit its features.
The only thing we are interested here are the permissioned OAuth scopes, which will allow us to call the different endpoints. In our case, the scope in question is
Finally, you need to install the app in your workspace.
If you are not the workspace’s admin, then a request is generated for them to approve. You do not need to worry about distribution, unless you plan to use it from multiple workspaces.
Once the app is added, you can access the OAuth token.
…and some code
Photo by Chris Ried on Unsplash
Tempted as you might be to write one yourself, Slack has published a number of client libraries in various languages.
I will use the Python library in a simple script.
The code can be found on Github.
We are going to generate a CSV with 5 columns:
- UTC timestamp in ISO format
- machinery type, e.g. ‘harvester’
- machinery make, e.g. ‘John Deere 25’
- farm type, e.g. ‘orchard’
- farm id, e.g. ‘B5’
First we need the query string.
From the problem description, something like
"Operator attention required" in:farm-ops will do just fine.
Then once we have located the messages in question we need to extract the fields.
A regexp pattern will work for now:
\<\!here\> Failure of (\w+) (\w\s)+\, (\w+) (\w+)\. Operator attention required. Note how
@here needs to change in the search string.
We need to initialize the Slack client passing the API token.
Reading it from an env var will do in this simple example.
Calling an API endpoint requires passing the name of it to the
api_call method, along with the URL parameter key/value pairs.
We can use the size of the
messages.matches array as an indication of whether we have reached the end of the resultset.
page_count is helpful in case we want to display progress.
Once we have our messages (the items in the
matches.messages array), we can process them.
And that’s pretty much it!
When the script runs, we will have our results in a nice CSV, ready for analysis in the tool of our choice.
You can view the full script and clone here.
Photo by NeONBRAND on Unsplash
Slack is a versatile and mature communication tool.
Its open API and large number of apps and integrations allow for some extremely interesting use cases even on the free plan.
Originally published at https://sgerogia.github.io on May 18, 2019.