Main findings
Previous work has shown an increase in the number of unscheduled medical contacts by children in autumn months (September to December), which may be a result of the start of the new school term.13 By sending a letter in July to remind children (and their parents) of the importance of using their inhaler, it was hypothesised that the increase may be averted. More specifically, the hope was that a reminder letter would lead to a greater uptake of inhaler prescriptions in August, that this, in turn, would also lead to an increased adherence and, finally, that fewer unscheduled medical appointments would be required.
There is evidence of an impact on the first part of this pathway, as the intervention group demonstrated a higher uptake of prescriptions in August 2013. There was also an increase in scheduled contacts in the same month in this group. The data are not available to confirm actual medicine use (as quantified by the medicine possession ratio), and so it is unclear whether or not the increased uptake also translated into increased use.
The primary end point was unscheduled medical contacts in September 2013, which coincided with the start of the new school term. There was no evidence of a reduction in the intervention group, but the finding of a greater number of unscheduled medical contacts (albeit not statistically significant) is unexpected. We can offer three potential explanations for this.
First, a repeat prescription request may not be dispensed without a review in situations in which the child has not received a prescription for several months, or in which the parent wishes to discuss the advantages and/or disadvantages of recommencing treatment that the child has stopped some months before. This in turn may be classified as an unscheduled contact in the coding algorithm that we used to define contacts as unscheduled. The evidence to support this is the large increase in unscheduled contacts in the intervention group in August (relative to the control group). In addition, it seems the longer the time since a patient last collected a prescription, the higher the likelihood of an unscheduled contact in September in the intervention group. This implies that patients with more troublesome asthma may be more likely to have collected a prescription recently; conversely, patients who have not collected a prescription recently may have stopped their preventative medication and may be seeing their GP to check whether or not it is still necessary.
Second, the letter may have acted inadvertently as a trigger to contact the practice in relation to an unrelated medical issue they had been meaning to discuss, which may increase the number of contacts in the short term.
Third, and finally, the data collected at the time may be equivocal in the coding of the contact, leading us to incorrectly adjudicate certain contacts as unscheduled when the contact was scheduled, a limitation of routine data that we will return to in the next section; this is a factor that is important for this intervention, which did increase scheduled contacts in the first instance.
Despite the increase in unscheduled contacts in September, both the total number of contacts per child (i.e. scheduled plus unscheduled) and unscheduled contacts were lower in the intervention group than in the control over the extended study period (September–December 2013) and the full year (September 2013–August 2014). Although the effects were not statistically significantly, the minimal cost associated with the intervention meant that the intervention was found to have a high probability of being cost-saving overall. The economic analysis (which used data over a 12-month period from August 2013 to July 2014) estimated a mean cost saving across the base case of £36.07 per child and 96.3% probability that the intervention is cost-saving. By contrast, the cost-effectiveness results are suggestive of an intervention that would be generally deemed not cost-effective if effectiveness (QALY gain) was important for decision-makers, as it also resulted in a QALY loss in 82.9% of samples and a mean loss of 0.00017 QALYs.
The small effects observed could be because of the strength of the link between prescription uptake and unscheduled medical contacts. Around 5–6% more children received a prescription for asthma medication in August 2013, a difference that, while substantial, may not be sufficient to achieve the postulated 5% reduction in unscheduled medical contacts that the study was planned to detect. On the other hand, it could be that the increased prescription uptake could have reduced the severity (and days off school), which we could not detect from routine health-care data.
Strengths and weaknesses
Strengths and weaknesses of the trial
The primary strength of the trial is its simple and generalisable design, incorporating an intervention that could be delivered within general practices with minimal cost. The intervention comprised a short letter, delivered at the start of the school summer holidays, reminding children of the importance of adhering to their asthma medication. The trial demonstrated that doing so increased the number of children both requesting a repeat asthma prescription and having a scheduled medical contact (such as an asthma review) in August without an associated increase in cost. Nevertheless, as noted above [see Secondary outcomes, Contacts over 12 months (September 2013 to August 2014)], there was (at best) limited evidence that this translated into an overall reduction of medical contacts.
We believe the risk of methodological bias is low in this study. The designation of contacts as ‘scheduled’, ‘unscheduled’ and ‘irrelevant’ was based on an independent adjudication panel comprising experienced GPs who were blinded to treatment group.
The main limitations of the trial were those imposed by the use of routinely collected data as the sole data source. The issues with using routine data are fairly obvious; data that are collected primarily as a record of medical care may not contain the information needed for a subsequent research question. For the outcomes evaluated within this trial, there was considerable uncertainty around the adjudication of some of the contacts as scheduled, unscheduled or irrelevant. Many of the contacts were coded ‘other’ or otherwise ambiguous, and the GP Adjudication Panel advised that this probably reflected the limited time available to GPs when summarising the appointment. There were 34,947 such contacts in the data set, out of 440,429 contacts (109,352 of which were deemed irrelevant). Even if adjudication of all contacts was feasible, however, some of more detailed fields are withheld as per CPRD policy, with the (understandable) reason being to safeguard potentially identifiable details contained therein. As a general issue, this highlights the tension between research ethics and individual rights, which arises in the use of routine data for research. On the advice of the GP Adjudication Panel, these contacts were coded as unscheduled.
We note that the use of routine data was a strength in some regards. First and foremost, using routine data substantially reduced the cost of the trial and allowed us to study a relatively large cohort of children. It is also the only appropriate way to assess the impact of a population-level intervention. If patients were recruited to a trial of the effect of receiving (or not) a letter, the very process of recruitment would have been an intervention. It should also be noted that this is the first large-scale trial of its type using routine CPRD data as the sole source data: issues that arose in the course of this trial may be abated on future trials as researchers and data providers become more familiar with the practical considerations involved in the process of data collation, transfer, recoding and analysis. Some specific examples encountered here are that practices may withdraw from being with CPRD; that prespecified coding for contacts are often not used; and that some fields are withheld for reasons of data confidentiality. The extent of these had not been appreciated or accounted for at the design stage of this trial.
The push for the use of routine data in clinical evaluation seems likely to continue, and it is important that researchers have appropriate expectations of what is, and what is not, realistic and achievable, when using these repositories. Practices leaving the CPRD is a particular issue for studies that have a long-term follow-up as the primary analysis.
Over the course of the study period, 28 practices stopped contributing data to the CPRD as a result of switching practice computer systems. At the time of the conclusion of the study period, the CPRD was only able to collect research-usable data from practices using the Vision IT computer system. It should be noted, however, that the CPRD is working towards being able to collect research data from GP practices using computer systems other than Vision. Once this work is complete, it should be the case that practices switching from one computer system to another will be able to continue their participation in trials and studies.
In retrospect, qualitative interviews with key stakeholders, including practice nurses and GPs, as well as a larger group of children with asthma, may have added a different dimension in both the development and (suggested) implementation of the intervention.
Strengths and weaknesses of the economic analysis
As with the clinical evaluation, the main limitation of the economic analysis is that it relied solely on routine data available within the CPRD database. We therefore had to infer the number of exacerbations experienced by patients, as well as the duration and severity of those exacerbations, from data on health-care resource use, which required several assumptions. For example, we assumed that any week including one or more unscheduled health care contacts was an exacerbation week. Under this assumption, two unscheduled contacts occurring 2 days apart may count as 1 or 2 weeks of exacerbation depending on whether or not they fall within the same calendar week. This adds further uncertainty to the QALY estimates that is not quantified within the CIs provided by the bootstrap analysis.
The use in the study of routine data also meant that we had to rely on published estimates for the impact of asthma exacerbations on children rather than measuring HRQoL in the patients themselves. This was problematic, as there was a lack of relevant and high-quality data on the impact of exacerbations on HRQoL for school-aged children. As a result, we decided to use data from adults in the base-case analysis, but this may not accurately reflect the impact of exacerbations on HRQoL in children, as their experiences of asthma may differ from those of adults.
Although the CPRD provides comprehensive data on resource use for the costing analysis, a number of assumptions were needed to classify all the health-care contacts as either scheduled or unscheduled. We also had difficulty classifying contacts as respiratory related or not, with a large proportion (38%) remaining unclassified. For this reason, in the costing analysis we included all contacts, whether they could be classified as respiratory related or not. As the intervention is not expected to have any effect on non-respiratory-related contacts, our analysis assumes that any difference between the intervention and comparator arms was a result of a change in respiratory-related contacts. The inclusion of these unrelated contacts in the costing analysis is likely to have made it harder to detect whether or not there was a true difference in respiratory-related contacts. We also found that a significant proportion of contacts (10%) were coded as consultation type ‘other’, which does not provide a clear indication of the activity involved. We therefore had to make an assumption regarding the type of activity that might be coded this way. Our sensitivity analysis found that making alternative assumptions regarding contacts coded as ‘other’ made some difference to the likelihood that the intervention was cost-effective, further supporting our conclusions that the cost-effectiveness of the letter intervention remains uncertain.
The data recorded in the CPRD on the duration of the consultation and the staff members involved in each consultation were not considered to be robust enough to use for calculating unit costs. Therefore, in calculating the unit costs for different types of consultation, we had to make assumptions using advice from our clinical experts regarding the likely staff mix and duration of contact. We also had to make assumptions regarding the likely severity of asthma exacerbations presenting in primary and secondary care.
The costing analysis for prescriptions was also problematic, as many of the drugs used in the management of asthma are available as a large number of different preparations, each with a unique product code. For example, for salbutamol inhalers alone, 17 unique products were prescribed within the data set. To keep the prescription cost analysis manageable, we estimated the cost per prescription for the 10 most commonly prescribed products for each drug. However, this approximation is not expected to have significantly biased the cost-effectiveness analysis because the absolute cost of most products prescribed in the management of asthma is low. We did find that the cost of prescriptions was significantly higher in the year overall for the letter arm, but this was more than offset by cost savings for other activity, resulting in a statistically non-significant cost-saving for letter compared with no letter overall.
The analysis takes an NHS and PSS perspective for costs and considers the benefits falling on patients themselves. Although this is the perspective preferred by NICE,35 it excludes the broader impact of asthma exacerbations on the children themselves, such as reduced educational opportunities due to missed schooling. It also excludes any impact on parents and carers, such as time off work when children are not fit enough to attend school.
A strength of this analysis was the ability to control for baseline costs in the BA economic analysis, which is an important aspect for consideration based on the allocation of patients (and their associated resource use patterns) to the control and intervention arms of the trial. Although it was shown in Table 17 that there was no statistically significant difference in overall resource use or costs between the trial arms for the 12 months post intervention, there was a statistically significant (p < 0.1) difference in overall costs at baseline (12 months before intervention) between the trial arms (these baseline results are presented in Appendix 10, Table 40). There are four reasons why the baseline resource use and costs are important for the BA economic analysis results and their interpretation:
A strong predictor of future resource use is past resource use, and it may be more difficult to influence the resource use habits of high resource users (who are also referred to as frequent attenders in the empirical literature) and, therefore, the patients in the letter group may have been less influenced by the intervention than those in the no-letter group may have been if they were allocated to receive the letter.
The letter intervention was allocated to a group of higher resource users, and this is not accounted for in the unadjusted economic analysis, which accounts only for the incremental difference in costs at the 12-month follow-up. Therefore, the costs of the letter group were already naturally higher in the letter group, which has an effect on the assessment of the incremental 12-month follow-up costs in this economic evaluation.
In relation to points 1 and 2, high resource users are, by nature, able to have larger changes in resource use and costs than low resource users, which will influence the incremental difference between groups when focusing on incremental differences at follow-up (in terms of costs) if these high resource users are allocated more to one arm of the trial than the other (e.g. reducing the resource use patterns of healthier patients with low resource use using an intervention will potentially have a much smaller change than the potential change if the intervention successfully altered the resource use patterns of unhealthier patients who are high resource users).
The higher resource use and costs could actually have been a result of improved (or just more) resource use recording at the practices allocated to the letter intervention group (note that allocation was at the practice level, rather than the patient level), which would have resulted in artificially higher costs and resource use in the letter group.
For the purpose of discussion, it is unclear which of the aforementioned points may have contributed to the statistically significantly higher resource use and costs for the letter group at baseline; however, whatever the reason for the difference in costs and resource use between the letter and no-letter group at baseline and post intervention, this aspect was controlled for in the BA economic analysis. Therefore, there is reason to consider that the results from the BA economic analysis may be a better representation of the potential economic benefit of the letter intervention than the unadjusted (observed) economic analysis. In this report, both results have been presented in order to allow decision-makers to account for both sets of results when judging the cost-effectiveness of this intervention; however, without the BA results, the intervention would seem less cost-effective than it actually may be in practice.
Patient and public involvement input on the trial results
The children and parents fed back that they felt that the saving per individual as a result of the intervention was an important finding from the study.27 Their feedback was that this should be emphasised more in the reporting of the result. We had the children and parents involved throughout, but this input was from October 2015.
A key point that came from the feedback was the question of whether the results would have been different if the study had been conducted over a 3-year period rather than 1 year. The parents felt that if the letter was something they expected each summer it could help in their planning for the start of the school year.
The trial in context: other studies and differences in results
We have not identified any other studies that have examined the economic benefits of a simple postal intervention in asthma patients and, therefore, it is difficult to compare our results with those of existing published studies. Yong and Shafie43 have published a systematic review that looked more broadly at non-pharmacological interventions aiming to improve asthma management. The interventions included by Yong and Shafie varied from educational and self-management interventions to environmental interventions. While the PLEASANT study intervention letter could be considered a simple form of patient education, the educational interventions included by Yong and Shafie were all more intensive, and the population was not restricted to school-aged children, making comparisons difficult. However, the broader evidence reviewed by Yong and Shafie suggests that non-pharmacological interventions that aim to improve individuals’ management of their asthma have the potential to be cost-effective.
Meaning of the study and implications for clinicians or policy-makers
The intervention in the PLEASANT study caused an increase in prescription collection in August, as well as an increase in scheduled medical contacts. It also had the effect of increasing medical contacts in August and September. After September, there was evidence of a fall in medical contacts, which, although not statistically significant, did follow through in the economic analysis to give a high probability of the intervention being cost-saving.
The increase in prescriptions and scheduled contacts in August could lead to individual GP practices wishing to implement the intervention. Evidence from the trial suggests that this would not increase overall costs associated with the asthma management, and may improve scheduled care. However, the evidence from the PLEASANT study is not sufficient to support a general recommendation for all GP practices.
Recommendations for future research
An objective of the study was to increase the take-up of prescriptions in August, as well as the adherence of children with asthma to taking their medications. The study was not able to assess the latter aspect.
Using routine data has many advantages in terms of trial efficiency in assessing public health or population-level interventions or in the assessment of proven interventions in real-world settings. Our analysis of the PLEASANT study data set suggests that further work is required to determine how to assess adherence using such data.
A suggested refinement for future trials using routine data could be the inclusion of a prompt for clinicians to answer study-related questions for patients in the study. For example, if patients enrolled in the study were to be given a specific study code, then clinicians, when having a consultation with such patients, could be automatically presented with a template for reporting key data during the study period. For the PLEASANT study, this would involve asking the clinician one simple question: whether the appointment was scheduled or not (and perhaps, second, if this was a respiratory-related consultation).
An additional point would be to emphasise to clinicians the importance of ensuring that the routinely collected patient data needed for the trial are complete, for example, in the PLEASANT study, prescription data to assess adherence. This could be done through a system prompt.
An investigation of the intervention on emergency contacts (such as out of hours, walk-in centres and emergency departments) would be of interest.
Future research in assessing interventions to improve adherence in school-aged children with asthma could include additional qualitative interviews with key stakeholders such as practice nurses, GPs and a wider group of children with asthma.
A study estimating the impact of asthma exacerbations on school-aged children, using a preference-based measure of HRQoL that has been validated for use in children, would be useful to inform future cost-effectiveness analyses.
