Episodic recognition memory for odors, faces, and unfamiliar symbols was assessed in apolipoprotein episilon4 positive and epsilon4 negative individuals with pathologically-confirmed Alzheimer's disease (AD), individuals diagnosed with probable AD, and healthy age- and gender-matched controls. The analyses revealed significant differences in performance between the AD patients and nondemented elderly controls based on the presence of the epsilon4 allele and the modality of the stimulus to be remembered. Measures of hits did not discriminate among groups but an analysis of false positive errors revealed significant differences, particularly for olfactory stimuli. Nondemented epsilon4 positive controls committed fewer false positive errors for olfactory stimuli compared to all other groups. In contrast, nondemented epsilon4 positive controls committed false positive errors for olfactory stimuli at rates similar to the AD patients. The performance of epsilon4 positive and epsilon4 negative controls was indistinguishable on the visual recognition memory tasks suggesting the particular vulnerability of olfactory memory to the early neuropathology of AD resulting in false positive errors in recognition memory for olfactory stimuli. The results of the present study offer new insight into how recognition memory is affected by AD, the epsilon4 allele, and the modality of the stimulus to be remembered.