Orchestrator-Agent Trust: A Modular Agentic AI Visual Classification
  System with Trust-Aware Orchestration and RAG-Based Reasoning
Modern Artificial Intelligence (AI) increasingly relies on multi-agent
architectures that blend visual and language understanding. Yet, a pressing
challenge remains: How can we trust these agents especially in zero-shot
settings with no fine-tuning? We introduce a novel modular Agentic AI visual
classification framework that integrates generalist multimodal agents with a
non-visual reasoning orchestrator and a Retrieval-Augmented Generation (RAG)
module. Applied to apple leaf disease diagnosis, we benchmark three
configurations: (I) zero-shot with confidence-based orchestration, (II)
fine-tuned agents with improved performance, and (III) trust-calibrated
orchestration enhanced by CLIP-based image retrieval and re-evaluation loops.
Using confidence calibration metrics (ECE, OCR, CCC), the orchestrator
modulates trust across agents. Our results demonstrate a 77.94\% accuracy
improvement in the zero-shot setting using trust-aware orchestration and RAG,
achieving 85.63\% overall. GPT-4o showed better calibration, while Qwen-2.5-VL
displayed overconfidence. Furthermore, image-RAG grounded predictions with
visually similar cases, enabling correction of agent overconfidence via
iterative re-evaluation. The proposed system separates perception (vision
agents) from meta-reasoning (orchestrator), enabling scalable and interpretable
multi-agent AI. This blueprint is extensible to diagnostics, biology, and other
trust-critical domains. All models, prompts, results, and system components
including the complete software source code are openly released to support
reproducibility, transparency, and community benchmarking at Github:
https://github.com/Applied-AI-Research-Lab/Orchestrator-Agent-Trust
6877b026ff8f47a7f8644314
Konstantinos I. Roumeliotis
claimed_verified
2025-07-17T08:21:36.699Z
64bcce94f8f28a19b0ba6c86
/avatars/45316b6fd19886eb18ff669f08bdd311.svg
Dr. Konstantinos I. Roumeliotis
user
rkonstadinos
6877b026ff8f47a7f8644315
Ranjan Sapkota
claimed_verified
2025-07-16T15:05:10.193Z
67ddd80896ac367438d400a6
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/C1NY6Nv5i0erwLnzCrTUM.png
Ranjan Sapkota
user
RanjanSapkota
6877b026ff8f47a7f8644316
Manoj Karkee
6877b026ff8f47a7f8644317
Nikolaos D. Tselikas
Orchestrator-Agent Trust: A Modular Agentic AI Visual Classification
  System with Trust-Aware Orchestration and RAG-Based Reasoning
Modern Artificial Intelligence (AI) increasingly relies on multi-agent
architectures that blend visual and language understanding. Yet, a pressing
challenge remains: How can we trust these agents especially in zero-shot
settings with no fine-tuning? We introduce a novel modular Agentic AI visual
classification framework that integrates generalist multimodal agents with a
non-visual reasoning orchestrator and a Retrieval-Augmented Generation (RAG)
module. Applied to apple leaf disease diagnosis, we benchmark three
configurations: (I) zero-shot with confidence-based orchestration, (II)
fine-tuned agents with improved performance, and (III) trust-calibrated
orchestration enhanced by CLIP-based image retrieval and re-evaluation loops.
Using confidence calibration metrics (ECE, OCR, CCC), the orchestrator
modulates trust across agents. Our results demonstrate a 77.94\% accuracy
improvement in the zero-shot setting using trust-aware orchestration and RAG,
achieving 85.63\% overall. GPT-4o showed better calibration, while Qwen-2.5-VL
displayed overconfidence. Furthermore, image-RAG grounded predictions with
visually similar cases, enabling correction of agent overconfidence via
iterative re-evaluation. The proposed system separates perception (vision
agents) from meta-reasoning (orchestrator), enabling scalable and interpretable
multi-agent AI. This blueprint is extensible to diagnostics, biology, and other
trust-critical domains. All models, prompts, results, and system components
including the complete software source code are openly released to support
reproducibility, transparency, and community benchmarking at Github:
https://github.com/Applied-AI-Research-Lab/Orchestrator-Agent-Trust
A modular Agentic AI framework integrates multimodal agents with a reasoning orchestrator and RAG module to improve trust and accuracy in zero-shot visual classification tasks like apple leaf disease diagnosis.
multi-agent architectures visual classification zero-shot settings generalist multimodal agents non-visual reasoning orchestrator Retrieval-Augmented Generation (RAG) confidence-based orchestration trust-calibrated orchestration CLIP-based image retrieval confidence calibration metrics (ECE OCR CCC) trust-aware orchestration image-RAG iterative re-evaluation perception meta-reasoning scalable multi-agent AI