Development of palliative care educational programmes continues in Japan. An instrument to evaluate a wider range of palliative care knowledge among general physicians and nurses is needed. However, such an instrument does not currently exist. The aim of this study was to develop an assessment to measure the efficacy of palliative care educational programmes. The questionnaire survey was validated with a group of 940 nurses at two facilities. The response rate was 85 % (n = 797). This study used psychometric methods such as item response theory and intraclass correlation coefficients. Ultimately, 20 items in 5 domains including 'philosophy', 'pain', 'dyspnoea', 'psychiatric problems' and 'gastrointestinal problems' were selected. For these items, the intraclass correlation was 0.88 overall and 0.61-0.82 in each domain; the Kuder-Richardson formula 20 in internal consistency was 0.81. Validity and reliability of the instrument were established. This tool is designed to evaluate a wider range of palliative care knowledge than currently available assessments and can be used for general physicians and nurses. The evaluation of educational programmes and the clarification of actual knowledge acquired are possible using this instrument.