Increasingly, senior business executives and boards of directors are scrutinizing cybersecurity capabilities to ensure they are appropriate and delivering the results expected. Outcome-driven metrics (ODMs) have a direct line of sight to the operational outcomes of investment and to the level of protection delivered in a business context. They help CIOs balance the need to protect the business with the need to run the business more effectively.
Good ODMs possess specific properties for new types of governance to be supported. They measure protection levels, and support direct investment to change protection levels, and they are explainable to executives with no technical background. Business leaders must follow four steps to ensure that new ODMs embody these properties.
Step 1: Define the Control’s Protection-Level Outcome
Protected-level outcomes describe both operational performance as well as desired protection benefits in a simple manner. Define a measurable outcome that reflects higher and lower levels of protection and one in which direct investment can be made to alter the outcome.
• Phishing training helps shape people’s behaviour related to clicking on links that can lead to security incidents. This level of protection can be measured by tracking the percentage of people who click on phishing training emails over time as part of an organization’s security, behaviour, and culture program.
• Threat and vulnerability management (TVM) manages the ongoing flow of vulnerabilities that can be exploited. The protection level is measured by the time it takes to deploy patches.
• Third-party risk engagement manages the assessment and governance of third parties to inform business decision making over which third parties the organization should engage with. This level of protection is measured by the percentage of third parties engaged that did not pass their assessments.
Step 2: Describe the Value
For each control, explain the correlation between cost (investment) and value (protection level). Typically, higher levels of protection (lower risk) will come at a greater expense, and vice versa. The cost includes not only the budget for implementation and operation, but also potential hindrances to business operations such as employee and customer satisfaction, restricted business services, or other associated expenses related to tightening controls.
For instance, higher click-through rates for phishing indicate that employees are more likely to click on malicious links in emails. Lowering phishing rates requires investing in anti-phishing tools and can cause business friction.
In terms of TVM, the sooner known vulnerabilities are fixed, the less time they are vulnerable to attacks. Fixing them quickly is usually more expensive, not just in terms of direct expenses, but also in terms of disrupting business processes. On the other hand, fixing them slowly is less expensive, but also riskier.
The main objective is to comprehend the levels of protection and associated costs throughout the organization as you adjust controls. These benefits and costs serve as crucial factors in decisions regarding priorities and investments for implementing or modifying a control. They also establish a clear understanding of the level of protection when applied in a business context.
Step 3: Define Benefit Outcomes
Benefits are a way to measure impact. In a business setting, this typically refers to the impact on the business caused by security threats or incidents. These results should be clearly defined to guide the connection to the desired benefits of the control outlined in Step 1.
Security incidents should not always be considered as a control failure. For instance, if an organization has invested in a 20-day patching policy and a vulnerability is exploited within 15 days of the patch being released, this is a result of a business decision to accept any hacks that occur within 20 days. This should not be viewed as a control failure.
Step 4: Sharpen Outcomes in a Business Context
ODMs should be evaluated based on the technologies that support discrete business units, operating functions, or departments that contribute to the organization’s overall business outcomes.
Ultimately, metrics are most effective and beneficial when they aid in decision-making within a formal governance process. This can be achieved through protection-level agreements (PLAs), which are contracts between executives and CIOs/CISOs to ensure a desired level of protection for a planned cybersecurity investment.
Additional analysis on cybersecurity ODMs will be presented during the Gartner IT Symposium/Xpo 2023 in Kochi, November 28-30.
(This article is authored by Paul Proctor, Distinguished VP Analyst, Gartner)
