Dispersion is one of the most important issues in see-through near eye displays with waveguide technology. In particular, the nanophotonics design is challenging but demanding. In this paper, we propose a design method for a multilayer achromatic metasurface structure for near eye display application by a physics-driven generative neural network. Two in-coupling metagratings under different projector illuminations are presented and numerically verified with the absolute diffraction efficiency over 89%. A beam splitter, which provides a balance between compactness and visual comfort in a single-projector-binocular display, is also designed. Finally, we apply this method to an out-coupling metasurface with the capability of enlarging the visible region by threefold.