Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks

Frontier Multimodal Large Language Models (MLLMs) exhibit remarkable capabilities in Visual-Language Comprehension (VLC) tasks. However, they are often deployed as zero-shot solution to new tasks in a black-box manner. Validating and understanding the behavior of these models become important for application to new task. We propose an Explicit Logic Channel, in parallel with the black-box model channel, to perform explicit logical reasoning for model validation, selection and enhancement. The frontier MLLM, encapsulating latent vision-language knowledge, can be considered as an Implicit Logic Channel. The proposed Explicit Logic Channel, mimicking human logical reasoning, incorporates a LLM, a VFM, and logical reasoning with probabilistic inference for factual, counterfactual, and relational reasoning over the explicit visual evidence. A Consistency Rate (CR) is proposed for cross-channel validation and model selection, even without ground-truth annotations. Additionally, cross-channel integration further improves performance in zero-shot tasks over MLLMs, grounded with explicit visual evidence to enhance trustworthiness. Comprehensive experiments conducted for two representative VLC tasks, i.e., MC-VQA and HC-REC, on three challenging benchmarks, with 11 recent open-source MLLMs from 4 frontier families. Our systematic evaluations demonstrate the effectiveness of proposed ELC and CR for model validation, selection and improvement on MLLMs with enhanced explainability and trustworthiness.

💡 Research Summary

The paper addresses a critical gap in the deployment of multimodal large language models (MLLMs) for visual‑language comprehension (VLC) tasks: the lack of transparency and reliability when these models are used as black‑box zero‑shot solvers. While recent MLLMs achieve impressive performance, they often suffer from factual errors, hallucinations, and inconsistent reasoning, especially in novel or out‑of‑distribution scenarios where ground‑truth annotations are unavailable.

To mitigate these issues, the authors propose an Explicit Logic Channel (ELC) that runs in parallel with the conventional MLLM, which they term the Implicit Logic Channel (ILC). The ELC mimics human logical reasoning by combining three components:

1. Large Language Model (LLM) – prompted to extract task‑relevant concepts (facts) and logical relations from the textual input.
2. Vision Foundation Model (VFM) – used to ground each extracted concept in the image and to produce confidence scores (probabilities) for the presence of those visual entities.
3. Logic Reasoning (LR) – a probabilistic inference engine that integrates the grounded visual evidence with the extracted relations to compute a posterior distribution over possible decisions.

The authors define a Consistency Rate (CR) as the proportion of test instances where the ILC’s prediction matches the ELC’s prediction. Formally, CR = (1/|Q|) Σ_{q∈Q} I