The N-back task is used extensively in literature as a working memory (WM) paradigm and it is increasingly used as a measure of individual differences. However, not much is known about the psychometric properties of this task and the current study aims to shed more light on this issue. We first review the current literature on the psychometric properties of the N-back task. With three experiments using task variants with different stimuli and load levels, we then investigate the nature of the N-back task by investigating its relationship to WM, and its role as an inter-individual difference measure. Consistent with previous literature, our data suggest that the N-back task is not a useful measure of individual differences in WM, partly because of its insufficient reliability. Nevertheless, the task seems to be useful for experimental research in WM and also well predicts inter-individual differences in other higher cognitive functions, such as fluid intelligence, especially when used at higher levels of load.