We present an experimental demonstration of fast all-optical switching on a silicon photonic integrated device by employing a strong light-confinement structure to enhance sensitivity to small changes in the refractive index. By use of a control light pulse with energy as low as 40 pJ, the optical transmission of the structure is modulated by more than 97% with a time response of 450 ps.