Apolipoprotein E E4 (APOE4) is a risk factor for cognitive decline. A high blood vitamin C (VC) level reduces APOE4-associated risk of developing cognitive decline in women. In the present study, we aimed to examine the effects of functional variants of VC transporter genes expressed in the brain (SLC2A1, SLC2A3, and SLC23A2) on APOE4-associated risk of developing cognitive decline. This case-control study involved 393 Japanese subjects: 252 cognitively normal and 141 cognitively impaired individuals (87 mild cognitive impairment and 54 dementia). Database searches revealed that rs1279683 of SLC23A2, and rs710218 and rs841851 of SLC2A1 are functional variants that are significantly associated with the altered expression of the respective genes and genotyped as three single nucleotide variants (SNVs). When stratified by SNV genotype, we found a significant association between APOE4 and cognitive decline in minor allele carriers of rs1279683 (odds ratio [OR] 2.02, 95% CI, 1.05-3.87, p = 0.035) but not in the homozygote carriers of the major allele. Significant associations between APOE4 and cognitive decline were also observed in participants with major allele homozygotes of rs710218 (OR 2.35, 95% CI, 1.05-5.23, p = 0.037) and rs841851 (OR 3.2, 95% CI, 1.58-6.46, p = 0.0012), but not in minor allele carriers of the respective SNVs. In contrast, the three functional SNVs showed no significant effect on cognitive decline. Our results imply that functional SNVs of VC transporter genes can affect APOE4-associated risk of developing cognitive decline via altered VC levels in the brain.