In this study, we aim to reconstruct single-photon emission computed tomography images using anatomical information from magnetic resonance imaging as a priori knowledge about the activity distribution. The trade-off between anatomical and emission data is one of the main concerns for such studies. In this work, we propose an anatomically driven anisotropic diffusion filter (ADADF) as a penalized maximum likelihood expectation maximization optimization framework. The ADADF method has improved edge-preserving denoising characteristics compared to other smoothing penalty terms based on quadratic and non-quadratic functions. The proposed method has an important ability to retain information which is absent in the anatomy. To make our approach more stable to the noise-edge classification problem, robust statistics have been employed. Comparison of the ADADF method is performed with a successful anatomically driven technique, namely, the Bowsher prior (BP). Quantitative assessment using simulated and clinical neuroreceptor volumetric data show the advantage of the ADADF over the BP. For the modelled data, the overall image resolution, the contrast, the signal-to-noise ratio and the ability to preserve important features in the data are all improved by using the proposed method. For clinical data, the contrast in the region of interest is significantly improved using the ADADF compared to the BP, while successfully eliminating noise.