Fast steering mirror (FSM) is an efficient and reliable mechanical device in aerial optical image systems for controlling the beam direction with high precision. With the advantages of compact size, high speed, simple structure, and long linear stroke, voice coil motors are ideal actuators for FSM systems. However, model uncertainty can lead to poor performance or even system divergence, especially in environments with temperature variations, electromagnetic environment changes, etc. This paper proposes a novel finite-time adaptive control (FAC) algorithm for an FSM system to obtain high performance, i.e., positioning accuracy, dynamic performance, and robustness. In addition, the finite-time convergence of the controller is analyzed. In the experiments, the controller is implemented in a DSP-based microprocessor. The step response results show that the proposed algorithm has a shorter setting time, smaller overshoot, and smaller steady-state error compared to classical sliding mode control (SMC). The sinusoidal signal tracking accuracy of FAC + SMC has been improved by 19.8%. In addition, as the model uncertainty increases 10%, the root mean square errors (RMSEs) are 1.73″ and 1.18″ for SMC and FAC + SMC, respectively. With 20% model uncertainty, the RMSEs increase to 2.56″ and 1.85″, respectively. Extensive experiments demonstrate the general effectiveness of the proposed algorithm.