Remote sensing techniques offer a unique solution for mapping stress and monitoring its time-course. This article reviews the main issues to be addressed for quantifying stress level from remote sensing observations, and to mitigate its impact on crop production by managing cultural practices. The case of nitrogen fertilization is used here as a paradigm. The derivation of canopy state variables such as the leaf area index (LAI) and chlorophyll content (C(ab)) is first addressed. It is demonstrated that the inversion of radiative transfer models leads to useful estimates of these variables. However, because of the ill-posed nature of the inverse problem, better accuracy is achieved when using prior information on the distribution of the variables and when multiplying LAI by C(ab) to get canopy level chlorophyll content. This variable, LAIxC(ab) is well suited for quantifying canopy level nitrogen content. It is used for nitrogen stress evaluation by comparison with a reference unstressed situation which is, however, not easy to get in practice. The combination of remote sensing observations with crop models provides an elegant solution for stress quantification through assimilation approaches. It fuses several sources of information within our knowledge of the processes involved and accounts for the environmental budget which can be integrated when making decisions about cultural practices. Conclusions are drawn on the issues related to the retrieval of canopy state variables from remote sensing data, to the link between these observables and crop models, and to the assimilation approaches. Avenues for further research are finally discussed along with the required observation system.