VaaniNetra
Benchmark Results
Annexure III Part A — Technical Performance Metrics
Annexure I — Required Metrics Scorecard
All metrics specifically required by Hackathon Evaluation Criteria (Technical Robustness, Section 5)
Entity F1
0.9622
FUNSD / Indian Financial (primary)
Target: > 0.85 ✓
CER (Digital)
0.00%
SROIE — lossless text extraction
Target: < 5% ✓
CER (Scanned)
5.37%
Gemini Vision OCR
Target: < 12% ✓
mIoU
0.9322
DocLayNet equivalent
Target: > 0.85 ✓
ROUGE-1
0.4042
CNN/DailyMail + Indian Financial
Target: > 0.35 ✓
BERTScore
0.8889
roberta-large semantic
Target: > 0.85 ✓
Compliance Macro-F1
0.8575
110 adversarial samples
Target: > 0.80 ✓
MCC
0.8080
Matthews Correlation Coefficient
Target: > 0.60 ✓
Metrics marked ★ Annexure I in the tables below correspond to the IndiaAI Financial Reporting Compliance Challenge evaluation rubric (Annexure I, Technical Robustness).
Loading benchmark data…