Face Recognition Trials
Results
1/12/2025
Equal Error Rate by Matching Engine
(Bag Drop vs Kiosk - matched results)
Rank 1 Identification Rate by Engine
(Bag Drop vs Kiosk - matched results)
ROC by Matching Engine
(Bag Drop vs Kiosk - matched results)
Quality of Passport Image
Quality of Kisok vs Bagdrop
Quality by Country
non-NFC Passport Images
NFC Passport Images
Outliers Review
Hats and Head Coverings
- Partial eye occlusion
- Distorted face shape
- Appeared in >50% of outlier cases
Blur / Motion
- Bag‑drop motion blur
- Poor focus at kiosk
- Major contributor to low match scores
Lighting Issues
- Shadows
- Uneven contrast
- Poor indoor illumination
Occlusions
- Face masks
- Glasses with glare
- Hands partially covering the face
Similar‑Looking Individuals
- • Extremely similar‑looking different persons explain the top two highest non‑matches in every engine.
Cropped or Low‑Quality Reference Images
- Non‑NFC passport scans with artifacts
- Half‑cropped upper facial region
- Incorrect bounding boxes
Match Score vs. Quality (Amadeus)
(Bag Drop vs Kiosk - matched results)
Bag Drop Quality
Kisok Quality (enrolment)
Bona-Fida
Presentation Attacks
Presentation Attack
Amadeus
Violin Plots of Score Distributions
(Bag Drop vs Kiosk - matched results)
Amadeus
Cognitec
Amadeus
Cognitec
Idemia
Amadeus
Cognitec
NEC
Idemia
Amadeus
Cognitec
NEC
Idemia
Thales
Amadeus
Cognitec
NEC
Idemia
Thales
Amadeus
Cognitec
NEC
Idemia
Thales
Live vs Reference
Mean transaction time (ms)
Bona fide transaction tests (10 tests) completed successfully in all cases, with no failures recorded. Presentation attack results varied by PAI species:
-
Printed photo attacks on paper were unsuccessful in 9 of 10 cases.
-
Digital display attacks using mobile devices were unsuccessful in all ten cases.
-
Balaclava-style wearable artefacts resulted in successful bypass in all ten test cases.
-
T-shirt presentation attacks resulted in 9 successful bypasses and 9 rejections (18 tests).
-
Silicone prosthetic attacks were successful in 5 cases and unsuccessful in 4 (9 tests).
-
Silicone mask tests, conducted end-to-end, resulted in 4 successful and 1 unsuccessful outcome (5 tests).
PAD Trials at Amadus
PAD Trials at Amadus
Next Steps
- Approval of PAD and Accuracy Reports
- with BNE
- Completion of Quality Analysis Paper
- by 7th Dec
- DTC Validation Paper
- awaiting Amadeus input
Copy of Deepfakes
By Ted Dunstone
