Face cognition is a crucial skill for social interaction and shows large individual differences in healthy adults, suggesting a possibility for improvement in some. We developed and tested specific training procedures for the accuracy of face memory and the speed of face cognition. Two groups each of 20 healthy middle-aged trainees practiced for 29 daily sessions of 15 minutes duration with different computerized home-based training procedures. In addition, 20 matched and 59 non-matched controls were included. Face cognition speed training enhanced performance during the training and transferred to the latent factor level as measured in a pre-post comparison. Persistence of the training effect was evidenced at the manifest level after three months. However, the training procedure influenced the speed of processing object stimuli to the same extent as face stimuli and therefore seems to have affected a more general ability of processing complex visual stimuli and not only faces. No effects of training on the accuracy of face memory were found. This study demonstrates that face-specific abilities may be hard to improve but also shows the plasticity of the speed of processing complex visual stimuli - for the first time in middle-aged, normal adults.