The surface figure precision, weight, and dynamic performance of spaceborne primary mirrors depend on mirror structure parameters, which are usually optimized to improve the overall performance. To realize rapid multi-objective design optimization of a primary mirror with multiple apertures, a fully parameterized primary mirror structure is established. A surrogate model based on a hybrid of improved particle swarm optimization (IPSO), adaptive genetic algorithm (IAGA), and optimized back propagation neural network (IPSO-IAGA-BPNN) is developed to replace optomechanical simulation with its high computational cost. In this model, a self-adaptive inertia weight and a modified genetic operator are introduced into the particle swarm optimization (PSO) and adaptive genetic algorithm (AGA), respectively. The connection parameters of BPNN are optimized by the IPSO-IAGA algorithm for global searching capability. Further, the proposed IPSO-IAGA-BPNN, based on a rapid multi-objective optimization framework for a fully parameterized primary mirror structure, is established. Moreover, in addition to the proposed IPSO-IAGA-BPNN model, the Kriging, RSM, BPNN, GA-BPNN, PSO-BPNN, and PSO-GA-BPNN models are also analyzed as contrast models. The comparison results indicate that the predicted value obtained by IPSO-IAGA-BPNN is superior to the six other surrogate models since its mean absolute percentage error is less than 3% and its R2 is more than 0.99. Finally, we present a Pareto-optimal primary mirror design and implement it through three optimization methods. The verification results show that the proposed method predicts mirror structural performance more accurately than existing surrogate-based methods, and promotes significantly superior computational efficiency compared to the conventional integration-based method.