Editor's Note: The following article is a guest post by Kevin Schmidt, an executive analyst at Gartner, where he supports GTP's Secure Infrastructure team in security operations. He also explores artificial intelligence and its uses in security operations.
Despite organization-wide interest in implementing security automation, security leaders often struggle to identify common security automation scenarios and implement custom automation.
Security leaders often face common roadblocks that include automating the wrong things, improperly prioritizing use cases, and misunderstanding what to automate or where to automate.
In order to create a framework for identifying security automation use cases, security leaders should implement a four-phase approach that allows them to identify high-priority areas of interest for automation and gather requirements for technical professionals to build and deploy automation strategies.
Stage 1: Preliminary work
At this stage, security leaders should first gather automation requirements—without good requirements, security leaders may end up automating just for the sake of automation.
It may be “fun” to see what can be automated, but often the results will not justify the cost and effort of automation.
This phase can be started by conducting a needs assessment of the processes to identify the best candidates for automation and the amount of work required to automate these candidates. The data collected in this preliminary work phase will guide subsequent selections for use case identification, development, and implementation.
While selecting use cases, security leaders will analyze potential gains, identify success metrics, and rank candidates based on a scoring methodology.
One goal while selecting a use case is to identify security automation that can help save time, provide better predictability regarding response, speed up response/containment time, and act as a force multiplier for existing staff.
Ultimately, we hope this will lead to long-term results for the SOC and the organization. Think of these as strategic outcomes, while automation goals are more about tactical outcomes.
Step 2: Choose a use case
Using the list of automation use case candidates outlined from the pre-work phase, security leaders can move on to selecting use cases.
Security leaders will need to keep in mind that they may not be able to automate all the candidates they identify in the pre-work phase—and that’s okay. The goal of the gain analysis is to identify the most critical tasks that will help them achieve their automation goals.
First, security leaders can conduct a benefit analysis for their use cases. One of the most effective ways to communicate potential benefits from automation is improvement over the baseline. The first step should be to conduct a benefit analysis only for the best candidates for automation, as it can be time-consuming.
One way to identify the best candidates for automation is to sort the list by total time spent. Second, security leaders can look at frequency. For example, are there tasks that occur multiple times a day rather than just a few times a week?
There are six steps to gain analysis that security leaders can take:
Start with the candidates identified in the pre-work phase. If there are a large number of candidates, it may not be possible to evaluate them all using this gain analysis. You may need to identify the top 5-10 candidates who will provide the best results. These are likely to be activities that are performed daily and frequently. Don’t focus on the less frequent activities because they may not provide the best gains. Record the actual work that needs to be done on each tool at the lowest level, and estimate the average time that would be required to perform each task. Take the candidates you identified and deduce the time savings (or other gains) for each case. The method for inferring time savings varies but generally falls into two options: estimated time savings and actual time savings. Calculate the total time savings for all tasks, and create a prioritized list of the activities for which automation provides the greatest benefit. Note that this is about determining the time savings for each task. Estimate the total gains per month based on how frequently these tasks are performed, or the known time spent performing them manually. A task may be able to be automated in as little as an hour of manual work to as little as five minutes. But if it is only done once a month, will there be any real benefits from automating this task? Prepare a profit analysis report where all the captured data is consolidated so that decisions can be made about which automation to implement, which to suspend, and which to leave untouched.
Stage 3: Develop Automation/Operation Manual
Exploring whether and how to automate it is an interesting engineering challenge. However, it is highly recommended to identify the development requirements for automation through gain analysis, as discussed above.
The outcomes of the gains analysis should give security leaders a good idea of the processes and tasks needed to drive the development of the playbook.
The gains analysis should also yield success metrics, which can be used to validate the effort required to develop the automation. Can the published operating manual and implementation strategy yield the type of gains expected? Some of this work may already have been completed as part of the scoring methodology.
One advantage of breaking goals down into processes and tasks is that while the goal may be unique to an organization, the activities required to reach the goal are common. Common activities are likely to have already been developed by a group of domain experts, which can save time when automating goals.
As with any development effort, testing and validation are important steps to ensure that design requirements are met.
During testing, make sure that the task functions are working as expected. Are the APIs responsive? Are there delays in delivery? Are there delivery errors? These are all important tactical testing steps to ensure that your playbooks work as designed.
Stage Four: Implementation
At this stage, the operation manual must be working according to the requirements and verified that it is working in the operations according to the plan. During this stage, it is time to put the operation manual into the production processes so that the gains can be realized.
Operational processes should be updated to reflect the use of the operation manual. Since operation manuals can vary in size and type, document how and when to use which ones. The use of the operation manual should be documented at the target level in the process documents. This level of documentation will help not only provide guidance on the usage guidelines, but also track dependencies and breakpoints at the process level.
Security leaders also need to communicate the metrics used in reporting, which should be updated regularly to include actual gains achieved through automation. Some automation providers include playbook-level usage tracking in their tools, which can help with reporting.
If there is no such possibility, it is best to incorporate a way to track the usage of the playbook and record the gains of each instance for regular reporting.
The type of activities that any worker performs should not change much, but the individual's involvement in activities and tasks will change. The type of decision assigned to each role should be consistent—such as direction, classification, and procedures—and the worker should be trained in how to make such decisions.
As with any code, operating manuals have a shelf life. They will require maintenance and will eventually need to be disposed of. For each operating manual, identify who is responsible for keeping the operating manual up to date.
Maintaining custom playbooks is an internal responsibility. During development, document any dependencies in the playbooks (such as third-party APIs, log format, or OS or application version).
If security leaders follow these steps, they will be on their way to achieving an enterprise-wide goal of staying on top of the latest security trends to stay one step ahead of any attacks.
