Mounting evidence suggested that histone H4K20-specific methyltransferase SET8 is required to maintain the malignant phenotype of various cancer types; however, the role of SET8 in mediating tumor metastasis in prostate cancer (PCa) has remained elusive. The present study demonstrated that small interfering RNA-mediated knockdown of SET8 inhibited the invasive potential of the PCa cell line PC-3 in vitro. Knockdown of SET8 reduced sphere formation, downregulated E-cadherin and α-catenin, and upregulated N-cadherin and vimentin expression in CaP cells, while upregulation of SET8 expression with a recombinant plasmid had the opposite effect. Furthermore, SET8 was shown to be physically associated with the epithelial-mesenchymal transition (EMT) inducer zinc finger E-box-binding homeobox 1 (ZEB1) in PCa cell lines. Chromatin immunoprecipitation suggested that SET8 binds to the promoter of cell adhesion molecule E-cadherin and vimentin. Luciferase reporter assays suggested that E-cadherin and vimentin are direct targets of SET8; furthermore, loss- and gain-of function studies of SET8 and ZEB1 indicated that suppression of downstream E-cadherin and activation of vimentin are important mechanisms by which SET8 and ZEB1 cooperatively trigger metastasis. Furthermore, SET8-induced methylated H4K20 was indicated to exert a dual function in ZEB1-regulated gene expression. In conclusion, the present study revealed that SET8 and ZEB1 are functionally interdependent in promoting the EMT and enhancing the invasive potential of PCa cells in vitro.