Don't aim for certainty -- aim for less uncertainty

According to the FAIR Institute, most FAIR analyses start with incomplete and imperfect data, which CISOs should not view as a barrier to success. Even without much or any empirical data, CRQ results can still be highly credible, useful and defensible -- if practitioners transparently and consistently document their sources, assumptions, estimations and confidence levels.

The organization also notes that the goal of CRQ is not to predict the future with certainty, but "to reduce uncertainty to a level that supports informed decision-making." With that in mind, informed, calibrated estimates -- based on structured interviews with internal or external subject matter experts (SMEs), for example -- can be as useful as empirical data.

In identifying data for a FAIR analysis, the goal is often to arrive at a reasonable range rather than a single data point. "There is literally nothing we will likely ever need to measure where our only bounds are negative infinity to positive infinity," CRQ expert Douglas Hubbard wrote in his book How to Measure Anything: Finding the Value of "Intangibles" in Business.

There is literally nothing we will likely ever need to measure where our only bounds are negative infinity to positive infinity.

In a FAIR Institute blog post, Jack Jones, creator of the FAIR methodology, offered the following tips for estimating an accurate range:

• Start with an absurd estimate -- e.g, the person is likely taller than an inch and shorter than 10 feet.
• Use references and logical reasoning to continually narrow the range.
• Challenge your team's reasoning throughout the calibration process.
• Remember that the goal is accuracy, not precision.

Where to find data for a FAIR analysis

Every risk calculation depends on the following fundamental pieces of data:

1. The likelihood of an event occurring. The FAIR model uses the term loss event frequency.
2. The severity or impact of the event if it does occur. The FAIR model uses the term loss event magnitude.

Where to find data for loss event frequency

Loss event frequency represents the number of times a disruptive operational event is likely to occur in a designated timeframe, typically a year.

Practitioners can either estimate loss event frequency using empirical data or derive it by multiplying the following factors:

• Threat event frequency. The statistical likelihood of an event. For example, the odds of a home in a particular ZIP code being robbed, based on recent crime data.
• Susceptibility. Vulnerabilities that increase the event's likelihood. For example, how often residents of the home leave doors unlocked.

The FAIR Institute suggests practitioners use the following data sources to inform loss event frequency, as well as its contributing factors, threat event frequency and susceptibility.

Data sources for loss event frequency:

Internal data sources:

• Incident response (IR) logs from past security events.
• Security operations center logs detailing successful exploits.
• Historical loss event logs from risk registers or ticketing systems.

External data sources:

• Industry-specific information sharing and analysis centers (ISACs).
• Verizon's annual Data Breach Investigations Report (DBIR).
• Sector-specific breach reports.

Threat event frequency data sources:

Internal data sources:

• Intrusion detection system and intrusion prevention system logs.
• Security information and event management alerts.
• Auth logs.
• Firewall logs.
• Access records.
• Identity and access management systems.
• Internal threat profiling.

External data sources:

• Threat intel feeds -- e.g., Mandiant, now part of Google; Recorded Future; and CrowdStrike.
• Verizon DBIR.
• Architecture models.
• Mitre ATT&CK mappings.
• Threat profiling.
• Adversary behavior reports.

Susceptibility data sources:

Internal data sources:

• Red team results.
• Incident forensics.
• Pen test results.
• Patch management metrics.
• Vulnerability scan outputs.
• Third-party risk assessments.

External data sources:

• Industry breach reports.
• Mitre ATT&CK.
• Threat intel feeds -- e.g., Mandiant; Recorded Future; and CrowdStrike.
• InfraGard bulletins.
• Industry-specific ISACs.
• Security control maturity benchmarks.
• Audit reports.

Where to find data for loss event magnitude

Loss event magnitude reflects the operational and financial effects of a given event. It might factor in both direct or primary losses, such as ransomware payments and lost productivity, and indirect or secondary losses, such as regulatory fines and reputational damage.

The loss event magnitude value should be computed in financial terms -- e.g., lost revenue.

The FAIR Institute suggests practitioners use the following data sources to inform loss event magnitude.

Data sources for loss event magnitude:

Internal data sources:

• Financial and accounting records related to past security incidents.
• Business impact assessments from business continuity planning.
• IR case management or time-tracking records.
• Ticketing logs indicating resource hours and resolution times.
• Asset valuation.
• Impact logs.
• Legal case records and cost tracking.
• Compliance records.
• Legal settlements.
• Customer support communication logs.
• PR response history.
• PR and media spending.
• Customer churn models.
• Reputational damage assessments.
• Insurance claims documentation.
• SME interviews with PR, media, legal, finance and compliance leaders.

External data sources:

• IBM's annual "Cost of a Data Breach" report.
• Cyentia's annual "Information Risk Insights Study."
• Ponemon Institute.
• FAIR Institute's "How Material Is That Hack" website.
• Securities and Exchange Commission (SEC) disclosures.
• Crisis reports.
• Regulatory disclosures and enforcement databases -- e.g., General Data Protection Regulation and the SEC.
• Public breach databases.
• Breach follow-on reports from Cyentia, Deloitte and legal analysis firms.
• Industry loss studies from Ponemon, Cyentia and Forrester.
• Publicly disclosed fines or class-action settlements.
• Market research on brand impact and consumer trust.
• SME interviews with PR, crisis management, law and insurance firms.

Alissa Irei is senior site editor of Informa TechTarget Security.

Paul Kirvan, FBCI, CISA, is an independent consultant and technical writer with more than 35 years of experience in business continuity, disaster recovery, resilience, cybersecurity, GRC, telecom and technical writing.