In this review article, we provide an introduction to ferrogels, i.e. polymeric gels with embedded magnetic particles. Due to the interplay between magnetic and elastic properties of these materials, they are promising candidates for engineering and biomedical applications such as actuation and controlled drug release. Particular emphasis will be put on the polymer architecture of magnetic gels since it controls the degrees of freedom of the magnetic particles in the gel, and it is important for the particle-polymer coupling determining the mechanisms available for the gel deformation in magnetic fields. We report on the different polymer architectures that have been realized so far, and provide an overview of synthesis strategies and experimental techniques for the characterization of these materials. We further focus on theoretical and simulational studies carried out on magnetic gels, and highlight their contributions towards understanding the influence of the gels' polymer architecture.