BMVC 2026

DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering

Luca De Grandis1 Silvia Cappelletti1 William Raccagni1,2 Marcella Cornia1 Lorenzo Baraldi1 Rita Cucchiara1
University of Modena and Reggio Emilia, Italy1 University of Pisa, Italy2
arXiv Dataset and models Code

What is Visual Answer Grounding?

Visual Answer Grounding requires a model to not only answer a question, but also localize the document region that supports its answer.

Visual Answer Grounding example showing a question, answer, and supporting document region.

Why does it matter?

Document VQA models can answer correctly for the wrong reasons. Grounding makes predictions more verifiable and interpretable.

Answer correctly

The model should understand the document content.

Show the evidence

The model should identify where the answer comes from.

Introducing DocAttriBench

DocAttriBench is a large-scale benchmark for Visual Answer Grounding in Document VQA, linking answers to precise supporting regions across diverse document types.

DocAttriBench overview showing source-attributed document visual question answering data.
237K document images
296K grounded QA pairs
12 evidence types
8 source datasets

How is DocAttriBench constructed?

We introduce MAPPET, an automatic attribution method that identifies the document layout element supporting each answer.

MAPPET: Mask-based Perplexity-Derived Attribution

MAPPET masks document layout elements one at a time and measures how answer perplexity changes. If removing an element causes a strong increase in perplexity, that element is likely to contain information needed to answer the question. Ambiguous or low-confidence attributions are discarded.

MAPPET attribution method overview.

Grounded Answer Generation

How well can models answer a question and localize the supporting evidence at the same time?

Zero-shot MLLMs often achieve reasonable answer accuracy but struggle with evidence localization. Fine-tuning on DocAttriBench substantially improves grounding performance.

MAPPET-7B

Fine-tuned Qwen2.5-VL-7B Avg: 33.8
+30.8 vs. backbone +9.5 vs. VISA

MAPPET-8B

Fine-tuned Qwen3-VL-8B Avg: 37.0
+27.6 vs. backbone +12.7 vs. VISA

MAPPET-I-8B

Fine-tuned InternVL3.5-8B Avg: 33.4
+28.1 vs. backbone +9.1 vs. VISA
Grounded Answer Generation benchmark table.

Grounding-only Tasks

How well can models localize supporting evidence, independently from answer generation?

We evaluate two complementary settings: answer locating, where the model receives the document and question, and post-hoc grounding, where the answer is also provided. Fine-tuning on DocAttriBench yields large improvements over the corresponding zero-shot backbones in both tasks.

MAPPET-7B

Fine-tuned Qwen2.5-VL-7B Answer Locating: 49.2
+31.3 vs. backbone −0.4 vs. VISA
Post-hoc: 53.1
+31.1 vs. backbone +1.1 vs. VISA

MAPPET-8B

Fine-tuned Qwen3-VL-8B Answer Locating: 53.6
+32.5 vs. backbone +4.0 vs. VISA
Post-hoc: 52.2
+23.1 vs. backbone +0.2 vs. VISA

MAPPET-I-8B

Fine-tuned InternVL3.5-8B Answer Locating: 61.7
+46.4 vs. backbone +12.1 vs. VISA
Post-hoc: 68.1
+49.7 vs. backbone +16.1 vs. VISA
Grounding-only benchmark table.

Qualitative Examples

Qualitative examples for seven of the eight datasets that make up DocAttriBench;
SlideVQA is excluded due to licensing restrictions.

DoclingMatix
DoclingMatix qualitative example 1. DoclingMatix qualitative example 2. DoclingMatix qualitative example 3.
DocVQA
DocVQA qualitative example 1. DocVQA qualitative example 2. DocVQA qualitative example 3.
VisualMRC
VisualMRC qualitative example 1. VisualMRC qualitative example 2. VisualMRC qualitative example 3.
VISA
VISA qualitative example 1. VISA qualitative example 2. VISA qualitative example 3.
VisualWebBench
VisualWebBench qualitative example 1. VisualWebBench qualitative example 2. VisualWebBench qualitative example 3.
LongDocURL
LongDocURL qualitative example 1. LongDocURL qualitative example 2. LongDocURL qualitative example 3.
MMLongBench-DOC
MMLongBench-DOC qualitative example 1. MMLongBench-DOC qualitative example 2. MMLongBench-DOC qualitative example 3.

Abstract

Answer grounding in document visual question answering remains an open challenge: most benchmarks lack grounding annotations or provide limited-quality labels, while constructing grounded datasets still requires costly manual effort. We introduce DocAttriBench (DAB), a large-scale benchmark for fine-grained, element-level source attribution in Document VQA, grounding answers to specific layout elements such as text blocks, tables, and images. To build DAB, we propose a Mask-based Perplexity-Derived Attribution method (MAPPET) that combines document layout and language modeling to identify the most informative element for each answer. MAPPET measures the increase in perplexity after masking candidate elements and attributes the answer to the element contributing most to model confidence. Applying MAPPET to multiple existing Document VQA datasets yields DAB, with 237k documents and 296k question-answer pairs with element-level grounding. We benchmark grounding-capable multimodal LLMs on DAB, evaluating answer accuracy, attribution accuracy, and overall answer quality. Results show that while larger models generally achieve higher answer accuracy, even the strongest models often fail to localize the supporting elements. DAB provides a scalable benchmark for developing grounded, verifiable, and trustworthy Document VQA models.

Enlarged DoclingMatix qualitative example 1. Enlarged DoclingMatix qualitative example 2. Enlarged DoclingMatix qualitative example 3. Enlarged DocVQA qualitative example 1. Enlarged DocVQA qualitative example 2. Enlarged DocVQA qualitative example 3. Enlarged VisualMRC qualitative example 1. Enlarged VisualMRC qualitative example 2. Enlarged VisualMRC qualitative example 3. Enlarged VISA qualitative example 1. Enlarged VISA qualitative example 2. Enlarged VISA qualitative example 3. Enlarged VisualWebBench qualitative example 1. Enlarged VisualWebBench qualitative example 2. Enlarged VisualWebBench qualitative example 3. Enlarged LongDocURL qualitative example 1. Enlarged LongDocURL qualitative example 2. Enlarged LongDocURL qualitative example 3. Enlarged MMLongBench-DOC qualitative example 1. Enlarged MMLongBench-DOC qualitative example 2. Enlarged MMLongBench-DOC qualitative example 3.