AI is finding security holes faster.
Cool. So are the attackers. Apparently nearly 37,000 software vulnerabilities were reported in Q3. Roughly triple last year's number. And now companies are leaning harder into AI tool…
そらのtab渋滞@sora_tabtraffic
I would like scores to include details on 'what date' and 'how many days' they cover.
On screens where AI displays vulnerability scores, I want it to show 'which date and what period the estimate is for' along with the numerical value. Even if two numbers are displayed side-by-side, it doesn't necessarily mean they are answering the same question.
The NIST 2025 report includes an example for CVE-2023-29373 generated on January 22, 2025. It shows an LEV of 0.54350 and a final EPSS of 0.00. This final window is adjusted to the 19-day period starting from January 4.
Meanwhile, checking the original 30-day value for the same CVE on January 4 via the FIRST historical API, the EPSS was 0.001270000. While rounding the probability to two decimal places results in 0.00, the original value is not zero. It is necessary to read the display adjusted for the report's shorter window and the original 30-day value separately.
As NIST explains, EPSS is an estimate of the probability of exploitation being observed within the next 30 days, while LEV deals with the likelihood of past exploitation. Looking at a low future forecast and a high past estimate, it's too early to conclude that one of them is wrong. The LEV of 0.54350 is also not a record that the organization was actually compromised.
Therefore, I propose a small information panel on the confirmation screen that summarizes these five items: target CVE, reference date, target period, model version, and original value with display precision. In this example, it could show: 'CVE-2023-29373 / Ref Date 2025-01-04 / Next 30 days / EPSS v3 / 0.00127 (0.00 when displayed at two decimal places)'. Rather than just overwriting current values, this would make it easier to review the judgment later. This is my suggested display format and is not meant to imply it is currently implemented in any specific product.
Of course, adding more fields won't solve everything. For screens that need to be viewed in a hurry, it seems best to provide a summary and allow for opening detailed info. Aligning periods does not prove the model's estimate is correct, and this past example cannot be taken as an indicator of the current model's performance. Before considering how much to automate, I want to first ensure the scores are displayed in a way that minimizes misinterpretation.
References: NIST §6 (printed page 15), FIRST historical values and data guide.