Security incident and vulnerability metrics
Security is part of delivering and running a service continuously, so measure how long people stay exposed, how fast you contain what goes wrong, and how willing colleagues are to raise a hand, and never turn a mistake into a reason for blame.
Overview and motivation
Every service carries weaknesses, and some of them will be used against it. A vulnerability is a weakness in software, hardware, configuration, or process that can be exploited to harm the confidentiality, integrity, or availability of a system. A security incident is an event in which that harm occurs or is credibly threatened. Metrics in this area try to answer two questions. How exposed are we, and how quickly do we find, fix, and recover from problems?
The agile and DevOps tradition treats security as a property of continuous delivery, not a gate at the end of it. DevSecOps extends the DevOps idea by building security checks into the pipeline and sharing responsibility among development, security, and operations. ITIL, through its information security management practice, contributes policy, risk-based thinking, and the connection to incident, change, and asset management. The two meet in the daily work: dependency updates arrive as ordinary changes, vulnerability findings become backlog items with owners, and incidents are handled with the same blameless reviews as any other (see topic 4.2, Postmortem and learning metrics).
Security metrics are exposed to particular distortions.
- Counting findings rewards not looking. A team that scans less finds less. A raw count of vulnerabilities can fall because a scanner lost access, not because anything improved.
- Severity scores are inputs, not truths. The Common Vulnerability Scoring System (CVSS) gives a severity score for a vulnerability in the abstract, and registries such as Common Vulnerabilities and Exposures (CVE) give them identifiers. Whether a vulnerability matters for you depends on whether it is reachable in your environment, whether an exploit is in use, and how valuable the affected asset is. A raw severity score used alone sends effort to the wrong places.
- Blame suppresses reports. If people are punished for clicking a link or for reporting an incident they caused, reports stop, and exposure grows unseen. Psychological safety is a security control.
- Time measures hide the tail. A median time to patch can look healthy while a handful of critical systems sit exposed for months.
This topic gives the main measures, with their definitions, their gaming vectors, and guardrails. It builds on the asset and configuration records from the previous part, because you cannot patch or scan what you cannot find (see topic 7.1, IT asset management metrics, and topic 7.4, Configuration and CMDB quality metrics). A caution: this is a measurement guide, not a security manual. Choose controls with qualified security professionals, and follow the standards and legal duties that apply to you, such as ISO/IEC 27001 and the guidance of national agencies.
The guardrail from Goodhart's law applies sharply to security. When a metric becomes a target, people reclassify findings, exclude assets from scans, and close tickets without fixing the cause (see topic 1.2, Goodhart's law and the psychology of metrics).
Key principles
- Measure exposure time as well as counts. How long a weakness stays open matters more than how many are found.
- Prioritize by context. Combine severity, exploitability, reachability, and asset value.
- Check coverage first. A clean report from a scan that missed half the estate is a false comfort.
- Build security into the pipeline. Make secure choices the easy default, and give fast feedback.
- Collaborate across disciplines. Security, development, and operations own outcomes together.
- Make reporting safe. Blameless handling of mistakes keeps the signals flowing.
- Aggregate human-behaviour data. Report group patterns and never name or rank individuals.
Metric definitions
| Metric | Definition | Formula | Unit | Data source | ITIL practice | Agile counterpart | Cadence |
|---|---|---|---|---|---|---|---|
| Exposure window | Time a known vulnerability remains open on in-scope assets, by priority band | median and 85th percentile of days from the date it was known to apply to the asset to the date a fix or accepted mitigation is verified | days | Vulnerability scanner, ticket system, change records | Information security management; problem management | Time to fix a defect in the backlog | Weekly |
| Patch compliance | Share of in-scope assets that meet the patch policy window for their class | assets patched within the policy window / assets in scope that required a patch in the period | percent | Endpoint and server management tools, asset register | Information security management; release and deployment management | Dependencies kept current in the pipeline | Weekly |
| Overdue critical findings | Count and age of critical-priority findings past their policy deadline | number of critical findings beyond deadline; age of the oldest | count; days | Vulnerability management system | Information security management; risk management | Security defects exceeding the agreed limit | Weekly |
| Scan coverage | Share of in-scope assets assessed within the required interval | assets scanned successfully in the last interval / assets in scope | percent | Scanner reports joined to the asset register | Information security management; IT asset management | Pipeline checks run on every build | Weekly |
| Time to contain | Time from detection of a confirmed security incident to containment | median and 85th percentile of hours from detection to containment | hours | Incident records | Incident management; information security management | Time to restore service after a failure | Per incident, reviewed monthly |
| Risk exception age | Age of accepted-risk exceptions, and the share past their review date | days since approval; exceptions past review date / open exceptions | days; percent | Exception register | Risk management; information security management | Technical debt review | Monthly |
| Security report rate | Share of staff-reported suspicious events relative to confirmed security events, as an indicator of reporting culture | reports from staff in the period / suspicious events confirmed in the period, with group-level trend | ratio | Security mailbox, incident records | Information security management; incident management | Raising impediments openly | Monthly |
State the priority bands and the policy windows in the register. Exposure window starts when the vulnerability is known to apply to the asset, which is the later of disclosure and the time the asset is known to run the affected component, and it stops at verified remediation or a documented, time-limited mitigation. Time to contain starts at detection, not at the first alert that nobody read. Exclude test findings and false positives only after review. Security report rate describes groups and trends, never individuals.
Recommendations
Know your estate and your exposure
1. Start from the asset and dependency records. Scans, patching, and ownership all depend on knowing what exists. Link every finding to an asset, an owner, and a service, and measure scan coverage against the register. Where coverage is low, fix that before reading the findings. An unscanned asset is an unknown, which is worse than a known weakness.
2. Reduce the attack surface first. Remove what is not needed: unused services, open ports, old accounts, forgotten environments. Everything you retire is something you do not have to patch. Track the count of internet-facing services and of dormant accounts as simple indicators, and review them with owners.
3. Include your software supply chain. Modern services depend on many components. The software supply chain includes libraries, build tools, container images, and the suppliers behind them. Use the software bills of materials described earlier to find where a vulnerable component is used (see topic 7.2, Software asset and licence metrics), and include dependency findings in the same measures as infrastructure findings.
Prioritize and fix
4. Rank by context, not by severity score alone. Combine the CVSS severity with whether an exploit is known to be in use, whether the vulnerable component is reachable, and the importance of the asset. Public lists of vulnerabilities known to be exploited help with the first factor. Write the ranking rules, review them, and record the reason when a high-severity item is deprioritized.
5. Set policy windows by band and report against them. For each priority band, agree a target time to remediate or mitigate. Report the share within the window, the median and 85th percentile of exposure, and the oldest open item. Windows are targets for conversation, and the oldest items deserve attention because they show where the process fails (see topic 1.6, Statistical literacy for service metrics).
6. Treat patching as ordinary, frequent change. The safest patch process is one that runs often and is boring. Small, frequent updates are easier to test and reverse than large rare ones, and automated pipelines can apply them continuously. Use the change enablement approach for risk-based approval, and keep an emergency route for urgent fixes (see topic 4.3, Change enablement metrics).
7. Make exceptions visible and short-lived. Sometimes a fix is not possible. Record an exception with a reason, a compensating control, an owner, and a review date, and report the age of exceptions. Exceptions without review dates become permanent, and permanent exceptions become the real risk register.
Respond and learn
8. Practice incident response. Computer security incident management covers preparation, detection, containment, eradication, recovery, and lessons learned. Rehearse with exercises, record time to contain from detection, and review each significant incident blamelessly. The structure of security incident handling is similar to that of major incident handling (see topic 3.4, Major incident metrics).
9. Measure containment, not just detection. Detection tools produce alerts, and the value lies in what happens next. Track how long confirmed incidents take to contain and recover, and what share were found by your own monitoring versus reported by outsiders. A rising share of internally detected incidents is a sign of improving visibility.
10. Learn from near misses. A phishing email that was reported and blocked, or a vulnerability found by a colleague before an attacker, is a free lesson. Record and review these as you would an incident, and thank the person who reported. The security report rate is a health indicator of that culture, and it should rise with trust.
Build the culture and the pipeline
11. Put checks where developers work. Run dependency and static analysis in the build, give feedback in the pull request, and fail builds only on issues the team agrees are blockers. Fast, relevant feedback gets fixed. A flood of low-value findings gets ignored. Track the fix time for pipeline findings, and tune the rules with the teams (see topic 2.7, Pull request and code review metrics).
12. Train and test without shaming. Awareness exercises, such as simulated phishing, can build skills, and they can also damage trust when results are used to name or discipline individuals. Report only aggregate results, focus on how many report suspicious messages, and explain the purpose. If an exercise makes staff feel tricked, change it.
Gaming vectors and guardrails
| Metric | How it gets gamed | Paired guardrail metric or practice |
|---|---|---|
| Exposure window | Closing findings without verification; changing the start date; reclassifying severity | Verification step by someone outside the fixing team; start-date rule in the register; severity change log |
| Patch compliance | Excluding hard-to-patch assets from scope; counting a patch as applied when the system was not restarted | Scope changes logged; verify installed version from the asset, not the ticket |
| Overdue critical findings | Splitting findings or merging them; downgrading severity | Count by unique vulnerability and asset; review all severity changes |
| Scan coverage | Scanning a narrower set; counting a scan that failed to authenticate | Coverage against the asset register; scan quality (authenticated, complete) |
| Time to contain | Starting the clock later; declaring containment early | Clock starts at first detection signal; verify containment with evidence |
| Risk exception age | Renewing exceptions automatically; splitting exceptions to reduce age | Review exception renewals with a reason; cap renewals without new evidence |
| Security report rate | Encouraging low-value reports; discouraging reports to improve other numbers | Quality of reports; read the comments; survey on whether people feel safe reporting |
A healthy reading is a high scan coverage, a short and shrinking tail of exposure, honest exceptions with review dates, incidents contained quickly and found internally more often, and staff who tell you about suspicious events without fear.
Trade-offs: pros and cons
| Approach | Strength | Weakness |
|---|---|---|
| Severity-score-only prioritization | Simple, consistent | Ignores context; sends effort to the wrong places |
| Context-based prioritization | Focuses on real risk | Needs data and judgement |
| Fixed patch windows | Clear expectations | Can force risky rushed changes |
| Risk-based windows | Fits the situation | Harder to audit |
| Frequent automated patching | Small changes, less exposure | Needs good tests and rollback |
| Rare large patch cycles | Familiar | Longer exposure, bigger risk per change |
| Phishing simulation | Builds awareness | Can erode trust if misused |
| Reporting culture focus | Surfaces problems early | Hard to measure directly |
The prose answer is to rank by context, patch often in small steps with good rollback, record exceptions with deadlines, and invest in reporting culture as a control.
Questions to discuss with your team
1. How much of our estate do our scans actually see? Compare the scanned assets with the asset register. Ask what is missing and why. The facilitator should look for categories that are systematically unscanned, such as containers, remote devices, or third-party systems. Agree one gap to close this quarter.
2. What is our oldest open critical finding, and why is it still open? Look at the tail, not the average. Trace the oldest few: who owns them, what blocks them, and what the compensating control is. The facilitator should ask whether the exception is recorded. Agree who will unblock the oldest one.
3. When did a security fix last cause an outage, and what changed afterwards? Recall an example. Was the cause a rushed fix or a missing test? Discuss how to make patching routine and safe. Link the answer to change and release practice.
4. If a colleague reported a suspicious email today, what would happen to them? Describe the real path from report to response, including how the reporter is thanked. Ask whether anyone has been embarrassed for reporting or for clicking. The conversation is about trust. Agree a small change that makes reporting easier or safer.
5. Which of our security numbers could go up while we get less secure? List the measures and think about how each could be improved by narrowing scope or relabelling. Assign a guardrail to each. This exercise builds the register entry for the next review.
6. How fast can we contain a confirmed incident, and how do we know? Review the last two incidents and trace detection to containment. Note delays. Ask what exercises we have run. Agree to run one before the next review.
Four perspectives
Product and delivery teams
Delivery teams make most of the fixes and introduce many of the weaknesses. They should keep dependencies current as part of normal flow, treat security findings as backlog items with owners and dates, and use pipeline checks that give fast, relevant feedback. They need clear priorities and the time to act on them, and they should be able to raise concerns freely.
Service and operations teams
Operations teams run the scanners, apply patches, and respond to incidents. They should keep the asset links current, report coverage honestly, and handle exceptions with deadlines. They also need protected time to rehearse incident response, and a blameless culture when mistakes happen.
Leadership and governance
Leaders set the risk appetite and decide whether security is funded as a continuous activity or as a project. They should ask for exposure windows and the oldest open items rather than totals, protect the reporting culture, and accept that numbers will rise when visibility improves. Governance should require evidence for exceptions and for incident learning.
Customers and users
Users are both beneficiaries and a line of defence. They want services that protect their data, and they want clear, fair guidance. They should be told promptly when an incident affects them, treated respectfully when they report problems, and given security measures that do not make their work harder than necessary (see topic 6.4, User and employee experience).
In practice: a service management example
A regional credit union runs online banking, a mobile app, and branch systems with a small engineering team and an outsourced security operations service. The numbers below are illustrative.
The board asks for a security dashboard. The first draft from the security analyst shows 4,200 open vulnerabilities and falling, and 96 percent patch compliance. The head of engineering asks about scan coverage. Joining the scanner output to the asset register, the analyst finds that 31 percent of servers and nearly all containers were not scanned in the last 30 days, because the scanner lost credentials after a migration. Among the assets that were scanned, the median exposure window for critical vulnerabilities is 9 days, and the 85th percentile is 41 days. The oldest critical finding is 212 days old, on a legacy system with an exception that expired four months earlier. Patch compliance of 96 percent was calculated only on the assets that the patch tool reported.
The team makes four changes with engineering and operations. Scan coverage becomes the first line of the dashboard, with a target to reach full coverage of the register, and the scan authentication is repaired and monitored. Prioritization adds exploitation status and asset importance to the severity score, so the 4,200 findings reduce to a short list of about 60 that matter now. Dependency updates move into the pipeline as frequent small changes with automated tests. Exceptions get an owner, a compensating control, and a review date, and the expired one is reviewed by the risk committee. The security analyst also changes how simulated phishing results are reported: only the group-level report rate is shown, and the analyst personally thanks the colleagues who report.
Two months later scan coverage is 97 percent, the 85th percentile exposure window for critical items is 14 days, and the oldest critical finding is a mitigated exception with 60 days to its review. A suspicious payment-request email is reported by a branch employee within minutes, and the incident is contained in under two hours. The board asks for the open vulnerability count to be halved by year end. The head of engineering suggests reporting exposure window and overdue critical findings instead, explaining that a count can fall by scanning less. The decision is recorded, and the board agrees to review the new measures at the next meeting.
Agile, AI, automation, and analytics
Agile
Treat security as part of continuous delivery, with fixes flowing through the same pipeline and review as other changes. The caution is that targets on vulnerability counts encourage reclassification.
AI
AI adds new risks, such as prompt injection and data leakage, and AI tools can help triage findings. Track incidents involving AI systems and the share of AI-triaged findings that a person confirmed. The caution is that an AI that dismisses a real finding gives false assurance.
Automation
Automate scanning and patching where safe, and measure scan coverage and failed patches. The caution is that automatic patching can break services, so use staged rollouts and rollback.
Analytics
Report exposure window as a distribution by severity with counts. The decision informed is where to cut exposure first. The caution is that vulnerability scores are inputs, not outcomes, so combine them with exploitability and asset importance.
Sector lens
Startup. There is usually no dedicated security team, so build the basics into the pipeline: dependency updates, secrets scanning, multi-factor authentication, and least privilege. Track exposure window for a short list of critical systems and have a simple incident plan. Resist buying a large toolset before you know your estate.
Small business. Outsourced services and cloud tools carry most of the risk. Keep an inventory, turn on multi-factor authentication everywhere, keep systems updated automatically, and know whom to call in an incident. Measure patch compliance and time to restore backups. Avoid shaming staff who report mistakes.
Enterprise. Scale needs consistent data across many teams and tools. Define priority bands and windows centrally, let teams own their findings, report exposure by business service, and run regular exercises. Coordinate with suppliers' security teams and test the hand-offs.
Government and public sector. Sectors often face mandatory directives with deadlines for specific vulnerabilities, formal incident reporting duties, and audit requirements. Treat those as design inputs: track compliance with directive deadlines, keep evidence, and report incidents as required while protecting individuals' privacy (see topic 8.5, Regulated and government reporting).
Business case: motivations, ROI, and TCO
The costs are scanning and monitoring tooling, which may already be owned; the time to maintain the asset links; engineering time to fix findings and update dependencies; incident response preparation and exercises; and training. Pipeline checks cost setup time and some build time. Outsourced monitoring has a service fee.
The return is a lower likelihood and lower impact of incidents, shorter exposure, and fewer rushed emergency changes. These benefits are real and difficult to quantify, because the incident that did not happen leaves no record. Avoid quoting industry average breach costs or incident frequencies unless you can trace the source and its relevance. Build the case from your own data: the exposure window trend, the number of emergency changes caused by late patching, the time spent on manual patch work that automation removed, and the cost of your past incidents including staff time and customer impact. Describe risk reduction in terms of specific scenarios, as the earlier topic on business cases suggests (see topic 7.7, Return on investment and business case).
Anti-patterns and pitfalls
- Reporting counts of vulnerabilities. Counts fall when scans miss things. Report exposure window, overdue items, and coverage.
- Prioritizing by severity score alone. It ignores reachability and exploitation. Add context.
- Treating the scan as the truth. Check coverage and scan quality against the asset register.
- Permanent exceptions. They become hidden risk. Give every exception an owner and a review date.
- Shaming people for mistakes. Reports stop. Use blameless review and aggregate results.
- Security as a gate at the end. It slows delivery and invites workarounds. Build checks into the pipeline.
- Big-bang patch cycles. They are risky and slow. Patch often in small changes.
- Ignoring suppliers. Their weaknesses are yours. Track supplier security obligations and incidents.
Maturity model
| Dimension | 1 Initiate | 2 Develop | 3 Standardize | 4 Manage | 5 Orchestrate |
|---|---|---|---|---|---|
| Visibility | Occasional scans of some systems | Regular scans of main systems | Scan coverage tracked against the asset register; findings linked to owners | Coverage, exposure windows, and exceptions managed and trended | Shared view across teams and suppliers, including dependencies |
| Prioritization and fixing | Fix by loudest request | Severity-based lists | Context-based ranking; policy windows by band; frequent small patches | Windows and the tail reviewed; pipeline checks tuned with teams | Automated, risk-based remediation with human oversight |
| Response and culture | Ad hoc response; blame | Basic plan, rarely tested | Rehearsed plan; blameless review; reporting valued | Containment time and internal detection rate managed; reporting culture measured | Joint exercises with suppliers and customers; learning shared widely |
Checklist
- Scan coverage is measured against the asset register.
- Findings link to an asset, an owner, and a service.
- Priority bands combine severity, exploitation, reachability, and asset importance.
- Policy windows by band are agreed, and exposure window and the oldest open item are reported.
- Dependency and pipeline checks run on every build, with agreed blockers.
- Patching runs as frequent, small, tested changes with an emergency route.
- Exceptions have an owner, a compensating control, and a review date.
- Incident response is rehearsed, and time to contain is measured from detection.
- Human-behaviour data is reported only in aggregate, and reporting is thanked.
- Supplier security obligations and incidents are tracked.
AI prompts
Review a vulnerability prioritization rule.
Our current rule for prioritizing vulnerabilities is: [describe]. Our data
fields include: [list, for example severity score, exploit status, asset
importance, internet exposure]. Review the rule for blind spots and ways it
could be gamed, and propose a context-based ranking with clear bands and policy
windows. Do not state specific numbers as industry standards. List your
assumptions and what a security specialist should confirm.
Check scan coverage.
Here is a summary of our asset register by class and the assets seen by our
scanner in the last [interval]: [paste counts or sanitized lists]. Identify
classes with low coverage, likely reasons for gaps, and the quickest checks to
confirm each reason. Suggest how to report coverage so that a failed or
partial scan does not look like success. List your assumptions.
Plan a tabletop exercise.
Design a 90-minute tabletop exercise for [type of organization] on this
scenario: [describe, for example a compromised supplier credential]. Include
roles, a timeline of injects, decisions to prompt, what to record (including
time to detect and contain), and questions for a blameless debrief. Do not
include real names or systems. List your assumptions and what to adapt to our
context.
Write a blameless message for a reported mistake.
A colleague reported that they [describe, for example clicked a link in a
phishing email]. Draft a short, kind reply from the security team that thanks
them, explains what we will do, and asks for details we need. Then draft an
aggregate note for the weekly update that shares the lesson without naming
anyone. List your assumptions and any tone choices I should review.
AI skills
- Service Management Metrics skill: use to find the paired guardrail for a security or vulnerability metric.
- Service Management Metrics maintainer skill: use to check that a security metric entry carries its gaming vector, guardrail, and privacy rules.
Key takeaways
- Measure exposure time, the oldest open items, and scan coverage, not raw counts of findings.
- Prioritize by context: severity, exploitation, reachability, and asset importance.
- Patch often in small, tested changes, and give every exception an owner and a review date.
- Contain incidents quickly, learn blamelessly, and track internal detection.
- Treat reporting culture as a control: thank people who report, and never rank individuals.
- Build security checks into the pipeline so secure choices are the easy ones.
References and further reading
- National Institute of Standards and Technology, Guide to Enterprise Patch Management Planning: Preventive Maintenance for Technology, NIST Special Publication 800-40 Revision 4, 2022.
- National Institute of Standards and Technology, Computer Security Incident Handling Guide, NIST Special Publication 800-61 (latest revision).
- Forum of Incident Response and Security Teams, Common Vulnerability Scoring System, specification (latest version), https://www.first.org/cvss/.
- Cybersecurity and Infrastructure Security Agency, Known Exploited Vulnerabilities Catalog, https://www.cisa.gov/known-exploited-vulnerabilities-catalog.
- ISO/IEC 27001, Information security, cybersecurity and privacy protection: Information security management systems: Requirements, International Organization for Standardization.
- Forsgren, N., Humble, J., and Kim, G., Accelerate: The Science of Lean Software and DevOps, IT Revolution, 2018.
- Beck, K., et al., Principles behind the Agile Manifesto, 2001, https://agilemanifesto.org/principles.html.
- Wikipedia, Vulnerability (computing).
- Wikipedia, DevSecOps.
- Wikipedia, Common Vulnerability Scoring System.
- Wikipedia, Common Vulnerabilities and Exposures.
- Wikipedia, Goodhart's law.
- Wikipedia, Attack surface.
- Wikipedia, Software supply chain.
- Wikipedia, Computer security incident management.