We study three-dimensional generalizations of the quantum spin Hall (QSH) effect. Unlike two dimensions, where a single Z2 topological invariant governs the effect, in three dimensions there are 4 invariants distinguishing 16 phases with two general classes: weak (WTI) and strong (STI) topological insulators. The WTI are like layered 2D QSH states, but are destroyed by disorder. The STI are robust and lead to novel "topological metal" surface states. We introduce a tight binding model which realizes the WTI and STI phases, and we discuss its relevance to real materials, including bismuth.