Breast cancer is the most common malignancy in women in the Western world. Except for the high breast cancer risk in BRCA1 and BRCA2 mutation carriers as well as the risk for breast cancer in certain rare syndromes caused by mutations in TP53, STK11, PTEN, CDH1, NF1 or NBN, familial clustering of breast cancer remains largely unexplained. Despite significant efforts, BRCA3 could not be identified, but several reports have recently been published on genes involved in DNA repair and single nucleotide polymorphisms (SNPs) associated with an increased breast cancer risk. Although candidate gene approaches demonstrated moderately increased breast cancer risks for rare mutations in genes involved in DNA repair (ATM, CHEK2, BRIP1, PALB2 and RAD50), genome-wide association studies identified several SNPs as low-penetrance breast cancer susceptibility polymorphisms within genes as well as in chromosomal loci with no known genes (FGFR2, TOX3, LSP1, MAP3K1, TGFB1, 2q35 and 8q). Some of these low-penetrance breast cancer susceptibility polymorphisms also act as modifier genes in BRCA1/BRCA2 mutation carriers. This review not only outlines the recent key developments and potential clinical benefit for preventive management and therapy but also discusses the current limitations of genetic testing of variants associated with intermediate and low breast cancer risk.