Abstract
Meta-learning is an effective method to handle imbalanced and noisy-label
learning, but it depends on a validation set containing randomly selected,
manually labelled and balanced distributed samples. The random selection and
manual labelling and balancing of this validation set is not only sub-optimal
for meta-learning, but it also scales poorly with the number of classes. Hence,
recent meta-learning papers have proposed ad-hoc heuristics to automatically
build and label this validation set, but these heuristics are still sub-optimal
for meta-learning. In this paper, we analyse the meta-learning algorithm and
propose new criteria to characterise the utility of the validation set, based
on: 1) the informativeness of the validation set; 2) the class distribution
balance of the set; and 3) the correctness of the labels of the set.
Furthermore, we propose a new imbalanced noisy-label meta-learning (INOLML)
algorithm that automatically builds a validation set by maximising its utility
using the criteria above. Our method shows significant improvements over
previous meta-learning approaches and sets the new state-of-the-art on several
benchmarks.