Metamaterial absorbers, consisting of assembling arrays of optical resonators with subwavelength dimensions and spacing, allow efficiently absorption electromagnetic radiation by leveraging the strong electrical and magnetic resonances. Beyond the enhanced absorption, there is a growing interest to realize multi-functional absorbers, for example, absorbers with extended bandwidth, strong polarization extinction ratio, to name a few. Traditionally, designing multi-functional absorbers require complex brute-force optimizations with sizable parameter space, which turn out to be rather inefficient. Here, using the particle swarm optimization algorithm, we design and experimentally demonstrate broadband and highly polarization selective mid-IR metal-insulator-metal absorbers, covering the technologically important 3-5 μm atmospheric transparency band. With spectrally averaged absorption exceeding 70%, a high polarization extinction ratio of 40.6 is concurrently achieved by the algorithm. We also investigate the incident angle dependence of the spectral absorption and clarify the origin of optical losses. By integrating with the growing range of mid-IR detectors and imagers, our devices can enable new applications such as mid-IR full Stokes imaging polarimetry for remote sensing.