Naturally occurring modification of the canonical A, G, C, and T bases can be found in the DNA of cellular organisms and viruses from all domains of life. Bacterial viruses (bacteriophages) are a particularly rich but still underexploited source of such modified variant nucleotides. The modifications conserve the coding and base-pairing functions of DNA, but add regulatory and protective functions. In prokaryotes, modified bases appear primarily to be part of an arms race between bacteriophages (and other genomic parasites) and their hosts, although, as in eukaryotes, some modifications have been adapted to convey epigenetic information. The first half of this review catalogs the identification and diversity of DNA modifications found in bacteria and bacteriophages. What is known about the biogenesis, context, and function of these modifications are also described. The second part of the review places these DNA modifications in the context of the arms race between bacteria and bacteriophages. It focuses particularly on the defense and counter-defense strategies that turn on direct recognition of the presence of a modified base. Where modification has been shown to affect other DNA transactions, such as expression and chromosome segregation, that is summarized, with reference to recent reviews.