Older adults are particularly vulnerable to phishing attacks. Gamification has been shown to be less effective to develop confidence in distinguishing between genuine and phishing emails in this demographic. To overcome this, we present our novel, open source interactive training platform, Phish&Tips, based on a simulated inbox. Our multi-analysis approach provides comprehensive data that enables us to compare participant's self-assessed competence with their performance on the training platform. We present results based on pre-and post-training surveys, focus groups and the analysis of the training platform data (N = 37). Over half the participants demonstrated an improved understanding of various detection strategies and an increase in confidence in being able to interpret emails. However, these results were not evident in the analysis of the platform data. This disparity between participants' perceived knowledge and their performance on the platform highlights the challenges of applying their knowledge effectively.