Towards an Unsupervised Method for Model Selection in Few-Shot Learning

Our paper presented at the ICML 2020 - Fourth Lifelong Learning Workshop.

Link to paper: https://openreview.net/pdf?id=wGO2NgC-ua9

TL;DR: Do we need labeled data from held-out classes for model selection in Few-shot Meta-Learning? Our method doesn’t. Based on the learned space of representation and using all available data for training, results show it can outperform meta-validation.

Abstract:
The study of generalization of neural networks in gradient-based meta-learning has recently generated great research interest. Previous work on the study of the objective landscapes within the scope of fewshot classification empirically demonstrated that generalization to new tasks might be linked to the average inner product between their respective gradients vectors (Guiroy et al., 2019). Following that work, we study the effect that meta-training has on the learned space of representation of the network. Notably, we demonstrate that the global similarity in the space of representation, measured by the average inner product between the embeddings of meta-test examples, also correlates to generalization. Based on these observations, we propose a novel model-selection criterion for gradient-based meta-learning and experimentally validate its effectiveness. Our paper presented at the 4th Lifelong Learning Workshop of ICML 2020.