“Zero false alerts” needs a second number

A zero on a results screen can do a lot of reassuring. For facial recognition, I would want the label beside it to say which kind of error was counted. An incorrect alert and a missed watchlisted person are different failures, so one number cannot answer both questions.

Table 5 in the National Physical Laboratory’s March 2023 report makes that distinction concrete. At a face-match threshold of 0.64, with detection settings (a), the observed false-positive identification rate was 0.00%. The observed true-positive identification rate was 79%: successful recognition when the watchlisted subject passed through the recognition zone. The zero therefore did not mean every eligible recognition opportunity succeeded.

These are historical test results, not an estimate of today’s immigration enforcement accuracy. The researchers replayed footage collected alongside five London and Cardiff police deployments in July and August 2022, testing NEC NeoFace V4 with the HD5 detector. The table uses 4,000 matching and 4,000 non-matching recognition opportunities. Those are repeated opportunities, not 4,000 separate people in each group.

The comparison within the table is useful too. At threshold 0.60 with settings (a), observed successful recognition was 85% and the false-positive rate was 0.25%. That illustrates a trade-off within this evaluation. It does not tell an immigration team which threshold to choose: the consequences of a false approach, the purpose of the search, and the conditions on the day still matter. Avoiding a wrongful alert is a real benefit; it just should not erase the other side of the result.

There is another easy display trap. The table’s observed results use large evaluation watchlists, while its columns for 10,000- and 1,000-image watchlists are estimates scaled from the larger test. A smaller-looking estimated rate should not be presented as another observed deployment result.

The Home Office’s older Holyhead deployment table also lists a 0.64 threshold. Sharing that number is not enough to transfer the NPL result to those trials, let alone the proposed wider use. It does not establish identical software, detection settings, watchlist composition or camera conditions. I would keep those records side by side with their dates, rather than merge them into one accuracy claim.

For me, a useful public performance summary would pair false alerts with successful recognition, show the test denominator and settings, and mark observed versus estimated values. Where misses cannot be measured in a real crowd, that limitation should stay visible. Better presentation would not settle whether scanning that crowd was necessary or proportionate. It would let us discuss the claimed efficiency without asking a single zero to carry the whole argument.

Sources: NPL MS43, Table 5 (printed page 18), methods (printed pages 1–3); Home Office historical deployment results.

Loading comments…