Abstract
This research tested a criterion-centric approach to validation (Bartram, 2004) for the criterion of leadership effectiveness involving: i) establishment of the criterion domain, and ii) measurement thereof, based on a-priori hypotheses of the predictors and criteria for the validation. The study proposed a refined model and concept for the implementation of this approach, which included the usage of different types and structures of predictor and criteria measures as matched to a specific criterion framework. This approach allowed the examination of the effectiveness of this approach of validation including key considerations for actual implementation. Using a process model with a two-phased mixed method approach, studies were conducted in Malaysia in an organisation with national level mandates. The study devised a trait based theoretical criterion framework from a qualitative inductive approach encompassing critical incident interviews, card sort tasks and a focus group. The six components of this framework were in line with current leadership literature (e.g., SHL Great Eight Competency Model), with the card sort task revealing Asian participants placing more emphasis on people-related criteria than Europeans. These components then determined the predictors and criteria measures for the subsequent validation study including a logical judgment exercise and collecting data using predictor measures for cognitive ability, trait/personality and motivation-to-lead (Chan & Drasgow, 2001), while the criteria measures consisted of the organisation competencies and job performance ratings and also a 360 work-performance evaluation. There were statistically significant results for cognitive thinking criteria only (though the relationship was not the strongest with the hypothesized component of the framework). The emphasis on people-related criteria found in the qualitative study was not corroborated by the quantitative data, which indicated that leaders are more effective if they are bright. Based on the findings, refinements to the process model were proposed, including a hierarchical structure criterion domain and differentiation of both criterion framework and measures for more effective validation using a criterion-centric approach. The research demonstrated the challenges of using ‘real life’ data in determining clarity for predictor-criterion relationships for this approach. Future research needs to develop better-articulated models of performance to address the need for differentiated measurement criteria which are vital for both practitioners and researchers.