Face Recognition Trials

Results

1/12/2025

Equal Error Rate by Matching Engine

(Bag Drop vs Kiosk - matched results)

Rank 1 Identification Rate by Engine

(Bag Drop vs Kiosk - matched results)

ROC by Matching Engine

(Bag Drop vs Kiosk - matched results)

Quality of Passport Image

 

Quality of Kisok vs Bagdrop

 

Quality by Country

non-NFC Passport Images

 

NFC Passport Images

Outliers Review

Hats and Head Coverings

  • Partial eye occlusion
  • Distorted face shape
  • Appeared in >50% of outlier cases

Blur / Motion

  • Bag‑drop motion blur
  • Poor focus at kiosk
  • Major contributor to low match scores

Lighting Issues

  • Shadows
  • Uneven contrast
  • Poor indoor illumination

Occlusions

  • Face masks
  • Glasses with glare
  • Hands partially covering the face

Similar‑Looking Individuals

  • • Extremely similar‑looking different persons explain the top two highest non‑matches in every engine.

Cropped or Low‑Quality Reference Images

  • Non‑NFC passport scans with artifacts
  • Half‑cropped upper facial region
  • Incorrect bounding boxes

 

Match Score vs. Quality (Amadeus)

(Bag Drop vs Kiosk - matched results)

Bag Drop Quality

Kisok Quality (enrolment)

Bona-Fida

Presentation Attacks

Presentation Attack

Amadeus

Violin Plots of Score Distributions

(Bag Drop vs Kiosk - matched results)

Amadeus

Cognitec

Amadeus

Cognitec

Idemia

Amadeus

Cognitec

NEC

Idemia

Amadeus

Cognitec

NEC

Idemia

Thales

Amadeus

Cognitec

NEC

Idemia

Thales

Amadeus

Cognitec

NEC

Idemia

Thales

Live vs Reference

Mean transaction time (ms)

Bona fide transaction tests (10 tests) completed successfully in all cases, with no failures recorded. Presentation attack results varied by PAI species: 

  • Printed photo attacks on paper were unsuccessful in 9 of 10 cases. 

  • Digital display attacks using mobile devices were unsuccessful in all ten cases. 

  • Balaclava-style wearable artefacts resulted in successful bypass in all ten test cases. 

  • T-shirt presentation attacks resulted in 9 successful bypasses and 9 rejections (18 tests). 

  • Silicone prosthetic attacks were successful in 5 cases and unsuccessful in 4 (9 tests). 

  • Silicone mask tests, conducted end-to-end, resulted in 4 successful and 1 unsuccessful outcome (5 tests). 

PAD Trials at Amadus

PAD Trials at Amadus

Next Steps

  • Approval of PAD and Accuracy Reports
    • with BNE
  • Completion of Quality Analysis Paper
    • by 7th Dec
  • DTC Validation Paper
    • awaiting Amadeus input