Hard CopyDocument IDV · Eval

Results

Run 05bf6677 · 30/06/2026, 21:03:06 · cli · $1.9060 total

Data Minimisation & Retention — GDPR framing

Document field extraction manufactures structured PII (name, DOB, document number) that did not exist as structured data before. Under GDPR/UK GDPR: you need a lawful basis (e.g. contractual necessity for KYC) to extract; data minimisation requires keeping only what's needed for the verified purpose; and retention limits mean full document images should not be stored indefinitely — extract the fields needed, delete the image. This prototype uses specimen/synthetic data only and retains no real PII. In a production flow, the consent and retention schedule would be defined before extraction begins.

VLM CER
0.0017
vs OCR 0.709
VLM Field Acc
98.3%
VLM ACER
0.0%
auth accuracy
Doc Threshold
0.2519
selfie: 0.28
Reading
VLM CER0.0017
OCR CER0.709
VLM Field F10.983
Cost / doc$0.0377

VLM beats OCR decisively. OCR fails on messy synthetic renders; VLM reads context.

Authenticity
VLM APCER0.0%
VLM AUC1.000
ELA APCER100.0%
ELA AUC0.500

VLM catches all fakes (AUC 1.0). ELA baseline accepts every forged doc (APCER 100%).

Face-on-Document
Doc threshold0.2519
Selfie threshold0.28
Doc FAR3.0%
Doc FRR3.0%

Printed portrait degrades ArcFace signal. Doc threshold shifts lower to compensate.

VLM wins on reading, not matching

The specialist model (ArcFace) dominates face matching. On document reading the roles reverse: layout complexity and multilingual text beat OCR's regex assumptions.

Cross-domain threshold shift

Document photo → selfie cosine similarity is lower than selfie → selfie. Operating threshold shifts from 0.28 to 0.2519. Every looser threshold widens the FAR/FRR trade-off.

ELA is a random baseline

Error Level Analysis flags JPEG re-compression artefacts. It accepted every forged document in this corpus (APCER 100%), making it indistinguishable from a coin flip. Reasoning beats forensics here.

Drill-down
Dataset: Synthetic (no MIDV-2020) · 10 documents
VLM CER=0.002, OCR CER=0.709. VLM outperforms Tesseract on document field extraction, as hypothesised.
VLM CER
0.002
lower is better
OCR CER
0.709
Tesseract baseline
VLM Field Acc
98.3%
VLM F1
0.983
Per-Field CER
vlm_extract
CER
0.002
Field Accuracy
98.3%
Precision
0.983
Recall
0.983
F1
0.983
Cost / doc
$0.0377
Latency p50
7100ms
Latency p95
13833ms
ocr_baseline
CER
0.709
Field Accuracy
21.7%
Precision
0.500
Recall
0.217
F1
0.302
Cost / doc
$0.0000
Latency p50
158ms
Latency p95
273ms