We consider a Josephson junction bilayer consisting of two tunnel-coupled two-dimensional electron gas layers with Rashba spin-orbit interaction, proximitized by a top and bottom s-wave superconductor with phase difference ϕ close to π. We show that, in the presence of a finite weak in-plane Zeeman field, the bilayer can be driven into a second order topological superconducting phase, hosting two Majorana corner states (MCSs). If ϕ=π, in a rectangular geometry, these zero-energy bound states are located at two opposite corners determined by the direction of the Zeeman field. If the phase difference ϕ deviates from π by a critical value, one of the two MCSs gets relocated to an adjacent corner. As the phase difference ϕ increases further, the system becomes trivially gapped. The obtained MCSs are robust against static and magnetic disorder. We propose two setups that could realize such a model: one is based on controlling ϕ by magnetic flux, the other involves an additional layer of randomly oriented magnetic impurities responsible for the phase shift of π in the proximity-induced superconducting pairing.