Gene expression in bacteria is traditionally studied from the average behaviour of cells in a population, which has led to the assumption that under a particular set of conditions all cells express genes in an approximately uniform manner. The advent of methods for visualizing gene expression in individual cells reveals, however, that populations of genetically identical bacteria are sometimes heterogeneous, with certain genes being expressed in a non-uniform manner across the population. In some cases, heterogeneity is manifested by the bifurcation into distinct subpopulations, and we adopt the common usage, referring to this phenomenon as bistability. Here we consider four cases of bistability, three from Bacillus subtilis and one from Escherichia coli, with an emphasis on random switching mechanisms that generate alternative cell states and the biological significance of phenotypic heterogeneity. A review describing additional examples of bistability in bacteria has been published recently.