The Risk of Measuring Risk
Automated measuring of control effectiveness is a very good idea conceptually. When you can combine control gaps with relevant threat information, you get a very good picture about the actual technical cyber risks your business faces. If done correctly, it provides you with an end-to-end view of what is going on in your organization.
Unfortunately, organisations can’t confidently say their controls are really deployed everywhere they’re expected. As you know, hope is not a strategy. In many organizations, asset and service (as well as API) inventories are neither complete nor actual. Useful inventories require continuous triangulation and reconciliation between various data sources to assure organisations have accurate and complete control effectiveness.
The Road to Risk Mitigation: Measuring Security Control Coverage
This is why measuring risks and relying on their results only makes sense if you have a firm grasp on your security control coverage. Otherwise, you make decisions on faulty risk information. The security controls coverage metric lets you see just how broadly your controls have been deployed across your environment. This visibility is essential to the success of your overall cyber risk measurement programme.
The only way you can have true confidence in your overall security programme is to measure not only the operating effectiveness of your controls, but also measure the coverage of your controls. As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about, that get you into trouble.
Compromises typically occur in the absence of a control, or when a control has failed. We all live in a highly dynamic world and the ongoing digital transformation continues to disrupt the status quo. These changes can also disrupt your controls; some may not deploy, some may be removed, or some may fail. Every security organization must be able to capture these deficiencies as soon as possible.
“As a security professional, I want and need to know where I have gaps. It’s the things you don’t know about that get you into trouble.”
A proper look at control coverage can deliver even more value. Controls coverage is an essential data point in risk quantification. Methodologies like FAIR (Factual Analysis of Information Risk) and Cyber VaR (Cyber Value at Risk) allow organisations to quantify risk. CyberVaR, in particular is very data-driven. It looks at a wide variety of aspects of security, risk and controls, including external threat landscape, internal events, threat scenarios, security capability, security controls coverage, and your overall security posture. It brings all of these together to give you a view of overall residual risk which can then be quantified into a value that’s meaningful to the business.
In order to provide a high level of confidence in your overall security posture, you need to know:
your controls are working effectively, and
you have 100% coverage, defined by your policies.
You must understand where your controls gaps are in order to address and remediate those gaps. If you don’t know where the gaps are, that’s where the compromises are most likely going to happen.
Your Key to Success: Automation
The route to success here is automation. When a process is automated, you get accurate results time and time again. You don’t have to question the data or the validity of the results.
Automation also lets you reduce your operational costs. Like it or not, every security function must find ways to reduce their operational costs and maximize their productivity. When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster. Industry benchmark studies show security teams often spend 36% of their time on reporting. Automating this process allows security people to focus more doingsecurity rather than reporting it.
“When you automate the processes around not just your controls coverage metrics, but all your security measurements, you reduce your cost of operations and scale faster.”
If automation is not an option, you will suffer the fate of creating quality controls coverage metrics manually. You will have to go to each tool individually, compile all the data together, then clean, aggregate, normalise, deduplicate and correlate all that disparate data. And by the time you’ve done all that and you’re ready to use the data, it might be out of date already. As a result, questions will arise around data integrity, and discussions about reducing risk devolve.
Don’t Forget to Communicate Your Controls
There are numerous stakeholders within an organisation who need to see your security metrics all the way up to board level. This applies to your controls coverage in particular.
The primary audience is the control owners, whether they are within the security function, infrastructure team, application development, or front-line staff. It’s important for the control owner to understand control coverage as well as how those controls are performing so that they can address any deficiencies or exposure. This is especially important for the front-line team, because they are responsible for managing the risk and they need to take action to address any gaps.
Some other stakeholder audiences are people in the compliance, audit and regulatory functions. These stakeholders must be able to rely on the controls data in order to make informed decisions, measure compliance to policy, and identify any gaps or risks within that environment. With complete, accurate data, these people can drive risk-based conversations and take actions as needed.
To wrap up, a common theme emerges here: trust in the data. When we all use the same set of data, we understand where and how it was derived and we have a high confidence in the data’s accuracy because it’s been automated. When everyone uses the same data set, and trusts it, discussions focus on risk and the right trade-offs and prioritisations, not about the accuracy of the reporting.
