A taper-based Mach-Zehnder interferometer (MZI) embedded in a thinned optical fiber is demonstrated as a highly sensitive refractive index (RI) sensor. A RI sensitivity of 2210.84 nm/RIU (refractive index unit) is obtained at the external RI of 1.40, which is ten times higher than that of normal taper- and long-period fiber grating (LPFG)-based sensors. The sensitivity can be further improved by decreasing the diameter of the thinned fiber and increasing the interferometer length of the MZI. The proposed MZIs have lower temperature sensitivities compared with normal fiber sensors, which is a desirable merit for RI sensors to reduce the cross sensitivity caused by thermal drift.