Abstract
Fact-checking systems have gained traction as scal-able solutions, yet they often face challenges such as handling diverse evidence sources, integrating multimodal data, and presenting comprehensive narratives. In this work, we propose CRAVE (Cluster-based Retrieval Augmented Verification with Explanation), a novel framework that integrates retrieval-augmented Large Language Models (LLMs) with clustering techniques to address multimodal misinformation on social media. The framework is designed to process multi-modal inputs (text and images) and iteratively refine evidence through agent-based mechanisms. We validated the framework on multiple real-world and synthetic datasets, showing that breaking up evidence into narrative clusters improves both retrieval precision, clustering quality, and judgment accuracy, showcasing its potential as a robust decision-support tool for fact-checkers.