Introduction
To manage demand, focus services on need and ensure that patients are seen in the correct setting, a gatekeeper function has been introduced by services within both oral surgery pathways and the wider NHS, for example in dermatology services.17,39 Such gatekeeper services vary in their design and implementation in terms of what is assessed, and by whom, and how it is delivered, but all can be considered a form of clinical triage. The concept of primary care services underpinned by effective clinical triage has been advocated by NHS England in its commissioning advice to ATs, although there is little detail on how this might be achieved.1,6
The provision of Level 2 services within primary care is predicated on the safe and efficient diversion of suitable patients to such services. This can be achieved in several ways, such as by:
undertaking a face-to-face clinical assessment of the patient
enabling referrers to select determine the case complexity
using machine learning or algorithms to classify the case complexity
clinical assessment of the referral – remote clinical triage by consultant staff.
There are advantages and disadvantages to each of these approaches. The use of face-to-face clinical assessments would be expensive, would require significant expansion of the clinical workforce, estate and support staff and would delay the patient journey by introducing an additional step. These factors seem at odds with the primary drivers for introducing Level 2 services and, hence, such an approach could not be recommended.40
Referrer-based triage makes broad assumptions that the population of referring dentists can adequately assess case complexity, has a good knowledge of the various complicating factors that can affect patient care and understands the local directory of services available for individual procedures. However, such an approach is relatively inexpensive and ensures rapid referral to a provider without a delay to the patient journey. It preserves and enhances clinical autonomy, and it can be argued that dentists know their patients best, as they have had the benefit of making a full clinical examination. However, GDP decision-making may be biased and influenced by patient demands (e.g. for GA or sedation) or by a wish to continue to send patients to ‘known’ consultants or to support colleagues in primary care. Of course, the patient could be allocated to an inappropriate service, leading to either a failure to reduce the burden in secondary care or the need for an onward referral if inappropriately assigned to a primary care service. However, this approach (peer assessment) is endorsed by The King’s Fund20 and is explored in phase 3 of this study (see Chapters 4 and 5).
The use of artificial intelligence, or algorithmic triage, is in its infancy. A number of systems have been assessed in emergency medicine departments and for trauma management.41 Such systems generally support clinical decision-making rather than taking full control over the process of triage. In dentistry there have been some studies that have examined machine learning for treatment planning.42 The development of an algorithm that captured, via the referral process, key elements of the pathway would be a simple matter – that is, identify those ‘red flags’ that would indicate a Level 3 patient. However, GDPs may be become familiar with the process and seek to circumvent it to get their patient to their preferred provider. Automated triage does have several benefits, as it:
could almost instantly send a referral to the provider, ensuring no delay to a patient’s journey
would be workforce neutral and, hence, capacity to deliver triage is limitless
would be more economical as there are no costs associated with the number of referrals
can be audited and assessed by appropriate clinicians to determine accuracy and appropriateness.
In the current work, it was decided not to deploy an algorithmic solution. The use of primary care Level 2 providers was in its infancy at the start of the study, and it was felt that the evaluation of algorithms would be more appropriate with mature services for which data were available to help support the development and assessment of automated decision-making.
It has been proposed that a hybrid of these two systems can be adopted: a system that provides the governance, clinical leadership and independence of the face-to-face approach while mitigating the costs, time and estate requirements of it. Remote clinical triage involves the assessment of standardised referral forms and appropriate attachments (most frequently dental radiographs) to undertake a case complexity assessment. Triagers are asked not to recommend a treatment plan, direct to a specific service or otherwise assess the referral, but to assign a Level 1, 2 or 3 complexity score to each referral. Once scored, an algorithm is applied to the referral to send it to an appropriate provider, and in most systems this is determined by patient choice for secondary care and geographical proximity for primary care Level 2 services (usually based on home postcode).
Remote clinical triage has been utilised in several settings43 and for various clinical disciplines, but the efficiency of this approach for oral surgery referrals based on the described case complexity assignments has not been assessed.44,45
The use of consultant-level triage was considered as a ‘reference standard’, and designating an experienced clinician to lead the service was thought, by many, to ensure safe and appropriate triage in the absence of established pathways. The use of consultant-led (if not actually delivered) triage is also recommended in the current NHS oral surgery commissioning guides.1
Consultants involved in the triage process will be aware of appropriate clinical guidelines (e.g. NICE guidance on the removal of third molars35) and the type of procedure and anaesthetic requested. This guidance, along with further obligatory information, such as medical history, social demographic information and levels of anxiety, will be common factors driving decision-making. The referral trajectory, however, involves GDPs initially carrying out a consultation and examination with the patient face to face, deciding there is a need to refer the patient for specialist treatment, then entering the appropriate referral data. Consultants then interpret the referral data and decide on the appropriate level of specialist care.
There are a number interactions taking place among individuals and organisations that may be subject to other influences and drivers, additional to official guidance and system parameters. For example, ‘intuition’ of clinicians is frequently cited as a factor involved in triage decision-making;46 however, this is a phenomenon that is difficult to define.47 Considine et al.48 state that knowledge and experience influence triage nurses’ decision-making; the integration of factual knowledge (a series of facts relating to a patient), procedural knowledge (decision rules, clinical guidance) and conceptual knowledge (assimilation of prior knowledge and new information) result in a unified comprehension that is applicable to a range of situations. Clinician experience is defined as a combination of the passing of time and gaining skills and exposure to an event. Together, the combination of knowledge and experience may form the elusive ‘intuition’ that is observed in experienced consultants, particularly as focusing on a specialty affords more opportunity to develop conceptual knowledge through exposure to events in a specific area.
Clinicians undertaking triage may consider several factors, in addition to clinical information and guidelines, in assessing a referral. Most literature around triage decision-making is based in acute emergency settings when the patient is present and available for examination.41,46,49 Edwards44 investigated decision-making using telephone triage in accident and emergency departments and concluded that experienced nurses considered several additional factors, such as contextual information and risk minimisation, in addition to purely medical information, when making decisions around treatment. In addition, lifestyle factors were found to be important in orthopaedics when deciding whether or not patients should be recommended for planned total joint replacement.47,50 There is, however, a dearth of research examining the complexities of remote clinical decision-making in the absence of the patient for elective treatment.
Although many consultants in oral surgery would have ‘triaged’ their own referrals – for example, a desk-based exercise to prioritise referrals or determine staff allocation – there were no data on the efficiency of a formal triage process where decision-making was based on case complexity and the diversion of patients to primary care. This study was therefore undertaken to determine the use of such triage in a diagnostic test accuracy study.
Aims
The aim of this research was to undertake a diagnostic test accuracy study of the accuracy of remote clinical triage performed by both GDPs and consultants, compared with a reference standard of face-to-face clinical consultation performed by an experienced consultant.
Recognising that clinical decision-making is a complex area, and that agreement levels between and within clinicians will vary, a qualitative element of the study sought to understand the reasons for this by examining the impact of variation on the feasibility of such services, and suggesting how remote clinical triage may be improved or enhanced.
More specifically, we sought to answer the following research questions:
How do remote clinical triage outcomes conducted by an experienced consultant compare with outcomes of face-to-face examination (reference test) performed by the same consultant?
How do remote clinical triage outcomes performed by GDPs and different consultants compare with outcomes of face-to-face examination performed by an experienced consultant (reference test)?
What are the views of triagers on the benefits and problems of a remote clinical triage system and how can the system be improved based on their experiences?
Methods
Ethics approval was sought and gained from NHS National Research Ethics Service, London Fulham Committee, approval number 12/LO/1912. The research was divided into two main stages:
A qualitative component featured in both stages. Patients recruited to the study were > 18 years of age, able to consent and had been referred to a secondary care facility by their GDP for an oral surgery procedure. The setting where the study took place was Greater Manchester, as NHS Manchester had recently implemented remote clinical triage as part of a centralised referral management system.
Stage 1: assessment of remote triage versus face-to-face clinical assessment by a single consultant
A total of 282 referrals to the NHS Manchester referral system were assessed (based on the level of acceptable precision at a 95% confidence level, a minimum sample size of 279 participants was calculated) to investigate sensitivity and specificity of referrals triaged to secondary or primary care. It is important that there is high precision when comparing triage methods and that erroneous referrals should ideally to be sent to secondary care rather than to primary care, to ensure patient safety and service quality. Therefore, our sample size was calculated based on a sensitivity of 0.98 and a specificity of 0.88.51
Sample size calculation
A total of 279 referrals were required, if taking the most conservative estimate, given a primary care prevalence of 30% and a sensitivity of 0.98 based on people referred to secondary care via assessment of standardised referral form only out of people referred to secondary care from face-to-face triage:
Process
A single consultant in oral surgery (PC) first examined all the referral forms. These were supplied on the agreed oral surgery pro forma that requires a minimum data set to be provided and adequate radiographs supplied. To reduce incorporation bias, all non-relevant patient-identifiable information, such as patient name and address, was removed from the e-referral form that incorporated the pro forma. A decision was rendered in each case from the following options:
suitable for secondary care consultant-led services (Level 3)
suitable for primary care advanced services, such as those offered by DwSpIs or those on the oral surgery specialist list (Level 2)
suitable for primary care – any competent GDP should be able to provide this treatment safely and effectively within general dental practice (Level 1)
rejected – sent back to original GDP as a result of incomplete form or missing radiographs.
Following a washout period of at least 3 weeks, the same consultant (PC) clinically examined the same patients ‘face to face’ (blinded to his previous remote triage decision) to determine a reference standard clinical triage. At the face-to-face assessment, decisions were made and noted regarding the most suitable hypothetical setting for treatment (although all patients in the study were ultimately treated in hospital). Study triage examinations took place as part of standard initial consultation prior to oral surgery procedures.
For the qualitative element of stage 1, an experienced qualitative researcher carried out detailed observations of a purposive sample (n = 30) of the consultant’s clinical (face-to-face) sessions. Cases were selected to be representative of types of clinical diagnoses, medical complexity and patient demographics, such as gender and age. During paper-based decision-making, the consultant was asked to articulate decision-making processes in real time (thinking aloud). Both procedures were audio recorded and transcribed prior to analysis and data were scrutinised to consider key factors that simplify or complicate decision-making. This approach was utilised to illuminate the processes of clinical decision-making.
Stage 2: examining the use of different examiners in the assessment and triage of referrals
In the second stage of the diagnostic accuracy and ‘workability’ test, the paper referrals of the cases that had been assessed face to face were provided to four other clinicians (one further consultant in oral surgery, one consultant in OMFS and two GDPs with experience in oral surgery). Each was asked to undertake a triage with the same options as described above. The decisions from these groups were compared with the reference decision (face to face) performed by a single consultant in oral surgery (PC). Although it was intended that this study would utilise consultant-led triage, we felt that it was important to explore if there are any major discrepancies between consultants and other dental professionals in where they believe certain cases should be referred to.
The sensitivity and specificity of the clinicians’ decisions were tested against the original consultant’s face-to-face examination decision. Additionally, tests for paired sensitivities were undertaken to determine the equality between the additional examiners using the median sensitivity score. The participant flow diagram is shown in .
Participant flow in the diagnostic test accuracy study.
For the qualitative element, where discrepancies in the decision-making between examiners were identified, referrals were selected for case presentations to be discussed in two focus groups. For the composition of the focus groups, see .
Cases were selected if there was a discrepancies existed between decision-making following the reference standard face-to-face examinations and the reference consultant’s (PC) decision-making following examination of the corresponding paper referral.
Cases were selected based on referrals resulting in differences in decision-making between different clinicians (GDPs/consultants).
Qualitative data analysis
Observations and the consultant’s narration for the triage process in stage 1 and the discussion of the focus groups in stage 2 were digitally recorded and transcribed. All transcripts were anonymised and checked for accuracy. Analysis drew upon some common techniques of grounded theory approaches (after Glaser and Strauss52), including the technique of constant comparison, whereby analysis was carried out concurrently with data collection so that emerging issues could be explored iteratively. Stages of coding consistent with a grounded theory approach, comprising initial coding of text segments, followed by recoding and memo writing to generate conceptual themes, were carried out. Themes were constantly compared within and across cases, paying attention to negative cases and possible reasons for differences. The data were organised with the aid of qualitative data software package NVivo 11 (QSR International, Melbourne, VIC, Australia). Two researchers discussed emerging themes regularly to enable refinement of conceptual categories and to identify common threads or differences across the different respondent groups. The team ensured that an audit trail of all stages of the analysis, to maximise credibility, dependability, confirmability and transferability,53,54 was made available.
Results
A total of 551 eligible referrals were considered, of which 460 were booked into study clinics and, following consent, accounting for non-attendance and clinic cancellations, a total of 282 participants were recruited to the study. All participants required an oral surgery procedure. The mean age of participants was 42 years (± 10.2 years) and 53% were female. No adverse events arising from the face-to-face assessments were reported.
Stage 1: assessment of remote triage versus face-to-face clinical assessment by a single consultant
Quantitative results
The decisions made by the reference triager (experienced oral surgery consultant – PC) are shown in . demonstrates these decisions based on their efficiency. For example, if a Level 2 case is directed to Level 3, it is considered efficient as care can be provided (although there is potential loss to primary care and any associated savings). A Level 3 case sent to Level 2 is considered as inefficient, as this will require an onwards referral to secondary care and this impacts on the patient’s journey and may incur charges in both settings.
Comparison of face-to-face and paper-based triage decisions
Assessment of efficiency of the paper-based triage compared with the reference standard
Data collected on the benefit of face-to-face assessment showed that there was a consistent collection of metrics that contributed to the change in decision. For those Level 3 cases sent to Level 2, this was a result of the treatment plan being different from that originally indicated on the referral (13 cases), additional radiography demonstrating increased case complexity (13 cases), discrepancies in the medical history (six cases) and the patient’s anxiety being higher than expected (four cases).
For sensitivity and specificity assessment, the decisions were dichotomised. Given the very small numbers of Level 1 referrals, these were combined with Level 2 referrals to provide an indication of primary care, and Level 3 referrals became the positive diagnosis. Therefore, a test with high specificity (88%) in this example will correctly identify those cases suitable for Level 2 care, but will send some Level 2 cases into Level 3, hence efficient triage error. High specificity was seen throughout the experiment.
Stage 2: examining the use of different examiners in the assessment and triage of referrals
demonstrates the comparison of the primary examiner’s (PC) remote and face-to-face triage sensitivity and specificity results, and then demonstrates the results from additional examiners. The results of a comparative analysis show that there were no differences in the specificity scores of any examiner, although the primary examiner had consistently and significantly higher sensitivities than the other examiners.
Sensitivities and specificities of all examiners using clinical face to face as the reference decision, with prevalence levels based on reference decision
Qualitative results
The participant codes are shown in . The qualitative results of stage 1 [observation and think aloud by the reference triager (PC)] and stage 2 (focus groups) are presented together because of the complementary nature of the outputs. Focus groups revealed several processes involved in triage decision-making. Where the discrepancy between remote triage and face-to-face consultation decisions resulted in the face-to-face patient being ‘diverted’ to secondary care, there was often missing or inaccurate information on the referral form or the radiograph was inadequate. Patients also disclosed additional clinical details at their face-to-face consultation, often around sensitive but relevant subjects, such as alcohol intake and mental health. The main themes that arose from the data were as follows:
Quality of information: quality of referral information was discussed frequently in the focus groups, where issues were raised around the minimum data set needed to carry out remote triage accurately. Some pragmatic suggestions were made for minor alterations to the online pro forma.
Holistic view of patient: clinicians habitually attempted to construct a narrative and context for the patient in the referral in the absence of the individual, resulting in attempts to form a holistic view of the patient. At times, it was reminiscent of the treatment planning process rather than the simple assessment of case complexity.
Organisational context: treatment decision-making must be made pragmatically, in the context of available resources. This is especially relevant when considering the capacity for primary care to deliver certain information about patients, such as sedation assessments, and availability of panoramic radiographs.
Quality of information: accurate information surrounding diagnoses and medical history of the patient is vital for case complexity assessment, both for remote triage and during face-to-face consultations. When the patient is present, questions can be asked, an examination can be carried out and the patient can influence the consultation by expressing preferences and fears. However, to decide on case complexity based on referral information given by a GDP in the absence of the patient, it is essential that the clinical detail and information relating to medical history is thorough and accurate. In addition, good-quality radiographs are an important element in the assessment of case complexity, especially in relation to more complex tooth extractions.
A combination of an inadequately completed medical history and either no or a poor-quality radiograph can result in a referral that is difficult to triage. The following describes a case in which a consultant had triaged the patient to primary care based on the information on the referral form. However, following examination of a good-quality radiograph, he was sure the patient should be treated in hospital because of the position of the inferior alveolar nerve in relation to the tooth (with an attendant risk of postoperative paraesthesia):
. . . we’re saying that he needed the X-ray to make a decision and that showed [after] he did get an X-ray but I changed this because it’s close to the nerve . . .
C1 (focus group 2)
When information is missing or inaccurate, an inappropriate referral decision can be made. The example below describes a patient who was correctly reported as having epilepsy. However, during consultation it was found that the epilepsy was controlled to a much lower extent than reported in the referral information and it was decided the patient should be treated in a hospital. This resulted in an adjustment to the previous decision that the case was suitable to be seen in primary care:
But added to it but probably the main thing was the fact that his epilepsy was not controlled . . .
C1
Aha, because the difference [evidence in the transcript of the consultation] . . . consultant has changed his mind from primary to secondary because the epilepsy was not reported accurately.
C1 (focus group 1)
Supplementary information contained on the forms, such as the Index of Sedation Need (IOSN), also influenced decision-making for triage, as the patient needs to be diverted to the provider with the facilities to administer the appropriate adjunct sedation.9,10,55 For example, simple surgery may be Level 1, but a request for sedation and supporting information to justify that sedation is required may move a referral to Level 2:
So I have assessed to the primary care specialist because it’s multiple surgicals, more than a GDP could handle. In terms of additional information, the patient is anxious and needs sedation, but again it could be primary care, but it would have to be primary care with sedation service – that wasn’t clear from the referral letter – the box hadn’t been ticked to indicate either way and again there is no IOSN or mention of anxiety.
C1 (C1 decision-making transcript)
Holistic view of the patient
Clinicians attempted to construct a narrative and context for the patient, looking for cues that might indicate an individual’s lifestyle and social circumstances, and using this to create a holistic view of the patient upon which they based their decision-making. Experienced clinicians described attempting to go beyond descriptive clinical information to form a holistic view of the patient behind the referral. One consultant describes this succinctly as:
The difference between treating the picture in the X-ray and the patient.
C2 (focus group 1)
The following triager described an attempt to form a holistic picture of the patient from the information provided on the referral form. It appears that he appraised the information available to attribute causes for the patient’s carious teeth. In the absence of this information, he exercised caution and selected secondary care treatment for this patient:
Incomplete information, because it doesn’t tell us whether they want that [IV sedation] or not. It just said, ‘difficult extraction’, great. There’s no selected choice for sedation and no indication of sedation needs or the patient’s anxiety . . . But how come they have got to that state, why have they got multiple teeth that are like, grossly carious and requiring surgical extraction? It could be, anxiety could be a reason, it could be economic reasons or education. You don’t know. But again, we haven’t got enough information to make a decision, so again I was moving up [to secondary care] rather than down [ to primary care] because I wasn’t sure.
C1 (focus group 1)
The process of decision-making when information regarding patients’ lifestyle, behaviours and anxiety levels was absent or ambiguous appears difficult for consultants who may be used to a shared approach. The process was articulated well in the discussion below around two key factors: the clinical information from the referral and the hypothetical patient. Again, when in doubt, the default approach was to refer to secondary care:
C1 (focus group 1):
You have, it’s got here, GA justification for access medical.
F1:
But, they’re case complexities but what I think I’m hearing is . . .
C1:
It’s patient complexity.
F1:
. . . it’s very difficult to disassociated these two things and that you can’t just . . . because that case complexity is level one isn’t it?
C1:
. . . complexity of procedure is one, but the whole patient.
C2:
But, the information you do have, they’ve had a bad experience at the dentist before it may be better to have someone that’s a bit slicker when they go to hospital . . .
Organisational context
Organisational considerations, such as costs, resources and efficiencies for both patients and health services, influenced treatment decision-making. Clinicians making triage decisions had local knowledge regarding the services and estate available with different providers and wanted to minimise the need for multiple appointments. Cost and time implications for patients were considered, even if clinical indications suggested that a procedure could be carried out in primary care. For example, if it is likely that a detailed (panoramic) radiograph will be needed, and possibly a general anaesthetic because the patient is anxious, a clinician may be more likely to refer to secondary care, where these services are available, thus reducing the potential number of appointments needed. In addition, where medical history indicates potential complications, it may be not only safer, but also more efficient, for the patient to be referred to secondary care because of access to other services and specialties:
C1 (focus group 1):
Patient is on warfarin has a history of alcoholism and problems with his liver. I think that’s high risk to do that. I wouldn’t even want to do restorative with IV block on somebody who has got…I wouldn’t want an IV block you could end up killing them . . .
C1:
So, again I’m basing it on the inconvenience to the patient.
C2:
And, how to get rid of that inconvenience, where they could just go to hospital have the test, the next day have the tooth out . . .
GDP:
Yes, inconvenience, also cost effectiveness, you know, going to the doctor, getting the blood test. They have got to take the blood, send the blood to the hospital. Get the hospital or lab, get the blood results back . . .
F1:
So, what’s actually driving this here, is not necessarily the case complexity around surgery again, it’s the facilities that are available in the hospital.
Clinicians argued that good-quality radiographs are vital for accurate triage, and this was an issue highlighted consistently in the discourse. However, since the introduction of the 2006 dental contract removed the financial incentive for investing in panoramic imaging machines (there is no longer a specific fee for item for these larger, more expensive radiographs) there has been a reduction in the number of primary care practices offering this radiographic service. This type of radiograph is particularly helpful for diagnosing third molar (wisdom teeth) problems, particularly the shape of the root and the proximity to the inferior alveolar nerve (damage to which is a common risk factor for such surgery). Radiographs that are of poor quality or otherwise inadequate could affect decision-making, as triagers were instructed to take a default position of referring to the higher level of care when in doubt:
C1 (focus group 1):
. . . Just as R’s saying, because there’s no decent X-ray you can’t actually make the decision. So whatever you write is right and wrong, isn’t it? So we just don’t know. We’re just guessing.
C2:
So it needs a referral just from an imaging point of view.
C1:
There were more DPT [dental panoramic tomography] machines around and now there are fewer than ever.
Often, GDP practices have access only to bitewing and periapical radiographs, which show only the first six teeth of each quadrant. This is insufficient to make some diagnoses, as demonstrated by the following observations from a hospital consultation:
We’ll need to get another X-ray today as well – they knew that when they sent you, just with those ones in your mouth there is a limit to how far back you can hold that, but we need to see the whole tooth to see the shape of the root – we’ve just got the very front of it on the X-ray. So we do a different sort that’s from outside your mouth so you haven’t got that problem of holding it steady
C1 (C1 P1 98–115)
The following extract from focus group 2 highlights how the issue of access to high-quality radiographs has been a long-standing problem. Clinicians have considered the situation to be problematic to the extent that possible solutions were debated:
C2 (focus group 2 p12 525–550):
There is a common theme here that, especially for the last ones. For the vast majority of wisdom teeth actually the imaging is inadequate and actually . . . so you wonder whether a default would be, you know, if you haven’t got a DTP [dental panoramic tomography] in your . . . you know, that should be a reason for referral because the general consensus that the vast majority of imaging is pretty rubbish and then you . . .
GDP:
I think . . . and we also . . . I mean we, at the [name of] practice, take local practices’ referrals for DTPs [dental panoramic tomographs] . . .
C1:
It’s tricky, isn’t it, because what the ideal is the NHS to be supporting general practice and us to all have DPT machine so they can send that image digitally so you can make the right decision, but that would be very expensive. Your way would be cheaper but would a dentist want to do that? So the patient might come to your place and say, oh this is impressive, because you’ve got this machine, why has my dentist not got this machine, and they start coming here.
Local practices may be unable to provide radiographs of adequate quality to support triage if they do not have the appropriate equipment, and this can affect triage decisions. Sometimes a pragmatic decision needed to be made around convenience and cost for the patient:
SPD (focus group 2):
See the trouble I have at the moment is I don’t have a practice. I don’t have an OPG machine in the practice. Now I can do . . . being a DwSpI I can do the surgery but because I don’t have an OPG machine means I’m having to refer them to [a district general hospital]. Do I then muck around and get the patient to go into [a district general hospital], do an OPG and then come back to me for the treatment or do I . . . ? I might as well just make the referral, get their OPG done and have it done there.
C1:
I know. It’s not as slick as it should be.
SPD:
No.
C1:
It’s messing the patients around at the moment, isn’t it, because it’s not easy to image.
Currently, records held by GDPs are not linked to other NHS records, such as those held by GP practices. Therefore, obtaining accurate information for the referral form is reliant on patients disclosing accurate information, particularly about their medical history. The following discussion related to a consultant describing a discrepancy between a decision made during paper triage (decision to refer to specialist primary care) and following a face-to-face consultation, during which the patient disclosed a higher quantity of alcohol consumption:
. . . So, again, that’s . . . it’s often the case there, I guess sometimes that happens between a consultation thing I’ll have something like [alcohol consumption] and the patient might deny something. Then they go through to see the nurse for a pre-op assessment for a GA or something and they confess to lots more because they’ve had a few minutes to think about it. So it could be that they do deny it to the general dentist and then somebody asks the same questions and they start thinking, oh well maybe there is something I need to tell them.
C1 (focus group 2)
A possible driver for an initial referral to specialist or secondary care from general dental practice may be the time and effort taken to carry out a procedure versus remuneration available in primary care. The following quotation is from a GDP who described possible reasons for a referral for a procedure that he felt was uncomplicated:
One of the drivers for multiple quadrants being referred out and the situation for even cons [fillings] for sedation is the fact that GDPs can only claim three UDAs for it. So, that’s the driver from the primary care side. But, if you look at each, what you can see on the radiograph, if it was just that tooth in isolation, it’s not technically difficult to take that out.
GDP (focus group 1)
Clinician specialty and experience
Clinician experience is defined as a combination of the passing of time and gaining skills and exposure to an event. Focusing on a medical specialty, such as oral surgery, affords more opportunity to develop conceptual knowledge and increases exposure to events specific to this area of expertise.48 Where differences were found in the decision-making of GDPs and consultants, GDPs tended to focus on the clinical information, whereas consultants tended to try to form a holistic view of the patient. It seems that their experience had taught them to focus on the severity and management of existing illnesses and to not assume that existing conditions are necessarily well controlled.
Although experienced GDPs undertaking triage of referrals might consider themselves to be more than capable of carrying out certain procedures, newly qualified GDPs and consultants were more likely to triage referrals to a specialty or secondary care service:
NQD2 (focus group 2):
I think that if I was a patient that sore and someone brushed me off and sent me for a referral waiting 8/12 weeks . . . I’d be annoyed. But what if you fractured it and it was very sub gingival and you haven’t got the surgical [skill]? You’re then having to refer to have the rest of it retrieved out.
NQD1:
I think you’ve got to start off knowing that it is going to be surgical.
GDP:
. . . Clinical experience is something that is accumulative isn’t it? As your decision-making evolves, your ability to carry out procedures evolves?
Consultants were more likely to consider the patient’s medical history and draw on their experience of complications that can result during surgery:
GDP (focus group 1):
And, if you look at them, I mean you can’t really see what’s on the right, but if you look at the molar, what does that look like? It doesn’t look like it’s going to be a particularly traumatic extraction . . .
C1:
But, I would argue even if it is someone with more experience it’s risky to do that in practice. Just like people don’t get caught out very often, but sometimes they do, they end up sending them to A&E [accident and emergency] because the patient . . . or they don’t even know, because three days later the patient is still bleeding, goes to A&E and get transfused, and it happens. They get transfused and all sorts of problems.
C2:
I think where maybe in . . . sometimes the practice you get a biased view and that’s because of nine times out ten you don’t have a problem. But, we’ve also had a biased view, because we see the ones that don’t get away with it in our chequered history.
Discussion
Triage is described as:
The medical screening of patients to determine their priority for treatment; the separation of a large number of casualties, in military or civilian disaster medical care, into three groups: those who cannot be expected to survive even with treatment, those who will recover without treatment, and the priority group of those who need treatment in order to survive.
Stedman’s Medical Dictionary56
This definition of triage still permeates the medical literature today (with findings of variation not dissimilar to those reported here), with examples from emergency medicine and military field operations.57 However, the picture is changing, with the implementation of telephone triage on helplines such as 11144 and remote dermatology services.39,58 Triage is not merely the selection of those who need care to survive but is becoming a tool by which the complex map of medical services can be navigated to ensure that patients receive the right type of care, in the right setting, delivered by the most appropriate clinical team with the most efficient use of resources.
This mixed-methods element of the project assessed the efficiency of remote clinical triage compared with a reference standard of face-to-face clinical assessment by an experienced oral surgery consultant. The qualitative results from the reference examiner (stage 1) suggest that consultants are willing to triage referrals to primary care, and the quantitative findings suggest the proportion of Level 2 cases is similar to that seen in the work of Kendall14 – around 70% for both remote (76%) and face-to-face (71%) assessment. The specificity of remote triage was high, and those cases that are suitable for primary care were identified as such. The rates of efficient and inefficient triage error were broadly similar, at 11.7% and 12.8%, respectively. The lead consultant (reference triager) had superior triage outcomes to the other examiners (with respect to sensitivity). However, as they had responsibility for face-to-face assessments this afforded a clear continuation of clinical decision-making processes, which was not available to other examiners. The results showed that there were no appreciable differences between the decisions of the other examiners.
The rate of inefficient triage calls is higher than that reported from the Croydon service, where the proportion of patients initially triaged to primary care (Level 2) who were subesquently referred to secondary care (Level 3) was reported to 0.5%, much lower than the 11% rate we report. However, it is important to note that the reference assessment of these patients took place in a secondary care rather than a primary care setting. It may be that an environmental impact facilitated an increase in the assessed complexity of these patients. For example, during an assessment in a primary care, treatment under GA would rarely be discussed, as it cannot be provided. Without the option of GA, the patient may elect for sedation or even treatment under local anaesthetic.9 However, when offered the possibility of GA in a hospital they may request this option and, hence, create a demand-led uplift to their case complexity.
The rate of Level 1 cases is consistent with that in Croydon: 1% in both settings. This suggests that the majority of referrals from GDPs are appropriate.17 This suggests that anecdotal explanations for the increase in referrals, for example that it is the result of financial incentives, are incorrect; rather, it may suggest that the increasing age and complexity of dentate patients could be a more important driver. Despite this, it should also be recognised that many GDPs are capable of delivering, and frequently deliver, Level 2 procedures, and the availability of a service may encourage such clinicians to refer rather than treat in practice. Another possibility is the presence of a structure-centralised referral system has changed GDP behaviour and resulted in a reduction in inappropriate (Level 1) referrals. This issue will be discussed in later chapters.
It is appropriate to consider the triage decisions in the context of the likely commissioning framework. It would be unusual to remove 70% of activity from an acute trust provider without causing significant destabilisation. In Greater Manchester it was the stated ambition of commissioners to reduce oral surgery flows into secondary care by 30–40%. A reduction of this size not only creates a stable system in the hospitals but also permits a range of Level 2 referrals to be seen in Level 3 environments, where they can be used for training purposes, and provides the possibility of reducing costs for the local commissioners. The costs of Level 2 care are borne by the primary care budget and not by the local commissioners’ hospital budget. If the triage service is directed to reduce inefficient triage errors (i.e. the incorrect assignment of Level 3 cases to Level 2) at the expense of efficient triage errors (the assignment of Level 2 cases to Level 3), then patient safety, service stability and training opportunities are maximised. The thresholds for such a triage service can be adjusted over time to ensure that there is a balance between capacity in each service area and that all stakeholders’ interests are recognised. The adage of caution first is appropriate, and if in doubt referrals should be ‘moved up’ to the next tier of care.
Given the complexities of patients, their presentation and the fact that a referral form is merely a summary and snapshot of a complex picture, it is not surprising that remote triage is associated with some inefficiencies. However, it should be recognised59 that treatment planning variation is seen across medical disciplines even when clinicians can physically examine patients, and it is not a phenomenon purely of remote assessment. Bader and Shugars60 said of dentists’ clinical decision-making:
Even when differences in patients are controlled, variation in dentists’ clinical decisions is ubiquitous. While its consequences remain undetermined, the variation in basic clinical decisions such as caries diagnosis signals the need to consider the extent to which the appropriateness of care is affected.
Bader and Shugars60
The qualitative assessment sheds useful light on the process of identifying elements that are either intrinsic to clinical behaviour or can be changed and improved to enhance the triage system’s performance. There is a clear recognition that triage can only be as good as the information provided by the referrer, and the clinicians involved in the study were often frustrated by the lack of information or, when information was provided, its quality. Ambiguous terms such as ‘bleeding problems’ or poor social histories caused the triagers to send more cases to secondary care based on lack of confidence rather than a firm decision. Although not optimising the system for identification of all Level 2 cases, the results of the study support the safe operation of a triage service and places patients’ interests first. Given the potential high percentage of cases that are available for diversion, it seems reasonable that some marginal Level 2/3 cases are sent to Level 3 – if only to support training.
The qualitative data make it clear that radiographs, accurate clinical data and medical history are vital to accurate decision-making. The findings highlight potential issues with patients not disclosing sensitive information to GDPs, for example around alcohol and mental health, or GDPs not asking for fear of alienating patients. Patients attending the face-to-face assessment in a hospital environment may have disclosed these issues more readily – or it could simply be they are seeing a new clinician, and one they do not have an enduring and continuing relationship with. Evidence suggests that clinicians vary in their ability to get full disclosure from patients.61 Consideration should be given to GDPs having access (with appropriate permission to view) to GP summary care records for the purposes of medical and drug history confirmation. Triagers found the IOSN to be helpful in their decision-making process. The IOSN combines medical, treatment and patient factors into a single, consistent one-page referral form. When completed correctly, the IOSN has been shown to be predictive of patients’ needs for sedation.62
Individual differences in triagers’ experience, training and setting inevitably impact on triage decision-making. Although the variances are not surprising, the underlying reasons behind them seem easier to address for triage than those for full treatment-planning variances. The triager’s role is to assess case complexity, not to design a treatment plan for the patient. The desire for a holistic evaluation of the patient is to be commended, but may also undermine the triage process as clinicians try to second guess the patient’s likely presentation following triage. It is also clear that, although many hospital staff will undertake internal triage, this form of case assessment is different. Consideration should be given to calibration of triagers and the formation of triage groups that can be internally benchmarked and externally audited. The use of triage guidelines and case examples would be helpful.
It should also be noted that these referrals were all seen and accepted by secondary care (to ensure a consistent face-to-face reference assessment was undertaken by PC). There was, therefore, no opportunity for the triage assessors to reject the referral requesting further information or clarification. The use of such a system, where additional information can easily be requested and added, would seem to address many of the concerns expressed by the triagers who felt that their accuracy in decision-making might have benefited from opportunities to ask further questions of the referrer.
Summary of main findings
In this section of the project, we posed three research questions:
- 1.
How do remote clinical triage outcomes conducted by an experienced consultant compare with outcomes of face-to-face examination (reference test) performed by the same consultant?
There was substantial variance between triage decisions and face-to-face examination decisions. The default position was to refer to secondary care if there was a query or uncertainty over a referral.
- 2.
How do remote clinical triage outcomes performed by GDPs and different consultants compare with outcomes of face-to-face examination performed by an experienced consultant (reference test)?
The performance of the various clinicians was very similar, and differed from the reference triager. Experienced GDPs would be more likely to refer to primary care (Level 2) than consultants.
- 3.
What are the views of triagers on the benefits and problems of a remote clinical triage system and how can the system be improved based on their experiences?
The qualitative work highlighted the complexity of the decision-making process important of providing complete information in the referral forms and high-quality radiographs to aid decision-making. The availability of specialised equipment in primary care was an impediment to referral to Level 2 services. Experienced consultants and newly qualified GDPs were more likely than an experienced GDP to err on the side of caution and select secondary care as an appropriate treatment destination. Clinicians seemed to find it difficult to make objective decisions solely on the information provided without attempting to create a holistic picture of the patient. Suggested improvements to the system based on the findings are set out below.
Conclusions and implications
The clinical triage system is imperfect but fit for purpose, recognising that a pragmatic clinical decision-making process is utilised. Although consultants and experienced GDPs performed at a similar level, the use of consultants to triage ensures that there is an acceptable level of governance in the approach, especially in embryonic schemes if there has not been sufficient time or experience to develop cohesive guidelines and to refine and reflect on processes to ensure that they are delivering the objectives of the system. There is a clear emphasis on patient safety embedded within the approach.
This part of the project identified several elements that were incorporated into the electronic referral system. The learning from this part of the project may help to inform the development and design of any oral surgery referral service. Key elements are described below.
The use of an electronic referral form enables a number of issues to be addressed. For example, forms cannot be submitted until mandatory elements are completed. Although it would be impossible to assess the value or accuracy of the submitted information, it does prevent the ‘blank box’ issue described by the triagers. Electronic referrals can also be quickly rejected by triagers and returned to the referring dentist for correction, amendment or the addition of further information. Such two-way, rapid communication between triagers and referrers could also promote the provision of clinical advice, which may be of value when interpreting the impact of medical and drug history on a patient’s outcomes. The increase in patients on polypharmacy makes such a system important for the future. Electronic referral forms enable conditional input to be facilitated, ensuring that referrers enter only information relevant to their patient. Electronic referral systems also lend themselves to aggregation of the information in electronic referral forms into databases to provide a health needs assessment tool for commissioners to understand the needs of the population referred for care.
The provision of high-quality and diagnostic radiographs should be a mandatory part of the referral process. Such images not only facilitate triage, but should also be deployed in the treatment-planning process. The accurate assessment of treatment complexity increases efficiencies of services by enabling ‘see and treat’ appointments. High-quality radiographs provided at referral stage reduce the need for repeat imaging in referral treatment centres, sustainably decreasing appointment times and complying with radiology guidelines.63
The incorporation of the IOSN tool into referrals seems to add benefit. Although the IOSN tool is not designed to either restrict or promote access to sedation, the data provided were viewed as useful not only by triagers but also by treating clinicians.62 The IOSN enables assessors to determine if the drive for sedation is based purely on patient anxiety or if it is a combination of medical complexities or treatment burden.55 This promotes the concept that sedation is not simply for those who are anxious, but may add quality to a service if a procedure is time-consuming or unpleasant.9
Referral management systems typically end at the disposal of the referral to the indicated provider. Perhaps the inclusion of whole-journey monitoring would assist in refining the triage process – that is, the provider could provide outcomes on the quality of the referral, the accuracy of the triage and whether or not additional information (and of what type) may have helped achieve better care. Such data could inform the development of referral forms and guidance, monitor the performance of individual triagers and identify individual practices or practitioners from which or from whom poor-quality referrals are frequently sent.
Chapter 3 describes how these recommendations have been implemented in an all-electronic referral management system for oral surgery referrals for a defined population.
