The phase diversity (PD) algorithm is an image-based co-phasing error detection method for segmented mirror synthetic aperture optical systems. Particle swarm optimization (PSO) is suitable for PD for its fast convergence speed and simple structure. However, with the increase of the numbers of subapertures, optimization of cost function formed by the PD algorithm becomes a high-dimension non-linear optimization problem, which would lead PSO to result in a premature solution. Regarding the problem above, this paper presented a modified PSO in the PD algorithm to co-phase the segmented primary mirror, which overcame drawbacks of the traditional PSO, such as premature convergence and deactivation of particles, and enhanced the dig ability of the algorithm. Vast simulation results show that the modified PSO in the PD algorithm can effectively restore the segmented primary mirror, reducing the peak-to-valley (PV) value to 0.0012λ and the root mean square error to 0.0007λ, and raising the Strehl ratio to over 0.999. A comparison was also conducted to show that the modified PSO has advantages over two existing algorithms in higher accuracy and faster convergence speed.