Due to their intrinsic physical properties potentially useful for imaging and therapy as well as their highly engineerable surface, biocompatible inorganic nanoparticles offer novel platforms to develop advanced diagnostic and therapeutic agents for improved detection and more efficacious treatment of major diseases. The in vivo application of inorganic nanoparticles was demonstrated more than two decades ago, however it turns out to be very complicated as nanomaterials exhibit much more sophisticated pharmacokinetic properties than conventional drugs. In this review, we first discuss the in vivo behavior of inorganic nanoparticles after systematic administration, including the basic requirements for nanoparticles to be used in vivo, the impact of the particles' physicochemical properties on their pharmacokinetics, and the effects of the protein corona formed across the nano-bio interface. Next, we summarize the state-of-the-art of the preparation of biocompatible inorganic nanoparticles and bioconjugation strategies for obtaining target-specific nanoprobes. Then, the advancements in sensitive tumor imaging towards diagnosis and visualization of the abnormal signatures in the tumor microenvironment, together with recent studies on atherosclerosis imaging are highlighted. Finally, the future challenges and the potential for inorganic nanoparticles to be translated into clinical applications are discussed.