Logo image
Trustworthy Classification for Complex Social Surveys: A Memory-Enhanced Hierarchical Framework with Calibrated Uncertainty
Conference proceeding   Peer reviewed

Trustworthy Classification for Complex Social Surveys: A Memory-Enhanced Hierarchical Framework with Calibrated Uncertainty

Zeqiang Wang, Rebecca Oldroyd, Yuqi Wang, Jiageng Wu, Jie Yang, Nishanth Ramakrishna Sastry, Jon Johnson and Suparna De
Proceedings of the ... AAAI Conference on Artificial Intelligence, Vol.40(46), pp.39415-39422
The 40th Annual AAAI Conference on Artificial Intelligence (Singapore, 20/01/2026–27/01/2026)
17/03/2026

Abstract

natural language processing text classification

Automated classification of complex social survey questionnaires is crucial for large-scale social science research but faces significant reliability challenges due to intricate hierarchical label structures, severe class imbalance, semantic ambiguity, and incomplete data coverage. Conventional classification methods often struggle with these combined complexities, yielding results that lack trustworthiness. We introduce HOCM, a framework designed for trustworthy classification in complex, real-world taxonomies. It features two synergistic components: (1) memory-enhanced contrastive learning, tailored to learn robust representations from noisy, imbalanced data by leveraging quality-aware category memory banks; and (2) hierarchical uncertainty calibration, which enforces taxonomic consistency while providing reliable confidence estimates and identifying inputs falling outside well-represented known categories. Our evaluation on a large-scale, real-world social survey dataset—a challenging exemplar of our target problem class—demonstrates that HOCM maintains strong accuracy on known classes while effectively identifying uncertain cases, significantly boosting accuracy on confident predictions. Furthermore, it adeptly detects low-resource/unknown categories. HOCM provides a more reliable automated classification tool, enabling efficient expert review and enhancing the trustworthiness of analysis in domains with complex, hierarchical data.

url
https://doi.org/10.1609/aaai.v40i46.41291View
Published (Version of record)
url
https://aaai.org/conference/aaai/aaai-26/View
Event Website Conference website

Metrics

1 Record Views

Details

Logo image

Usage Policy