Learning objectives: You will learn about the use of information for health service planning and evaluation.
Read the resource text below.
Commissioning or planning health care begins with an assessment of need .
This requires information on:
- Size and structure of underlying population
- Areas of met and unmet need
- Incidence and / or prevalence of disease(s) of interest
- Effectiveness of interventions available
- Relative cost-effectiveness of those interventions
- Current services, including capacity, quality, effectiveness, efficiency
- Prioritisation processes, including political priorities such as waiting times, service targets etc
Typically this would be in the context of an assessment of the population's overall health which requires identifying and reviewing data sources such as:
- Local data, such as a local specialised survey
- Routine local and / or national statistics
- Ad hoc data
- Relevant published surveys
- Qualitative data as well as quantitative sources
- Trends in incidence and outcome
When data are not available directly, such as when future demand needs to be predicted or when estimating the impact of service re-design, the available information may be used to formulate the assumptions of a model, and to calibrate it.
Issues to consider when using information for local health service planning include:
- The applicability of national data to the local population and conditions. If only national data are available, when applied locally the data may need to be adjusted for the composition of the population such as age, ethnicity and deprivation.
- Many data sources cannot give any indication of who has the health condition of interest but is not accessing services.
- National or regional data may mask inequalities at smaller geographical levels such as wards.
- Data are subject to random fluctuations. Small area data may have very small numbers. Indicators of precision, such as confidence intervals, should always be used in these cases.
- Confidentiality issues with small numbers may mean the information is suppressed to prevent disclosure when the data source is used in combination with other sources.
- How the data have been obtained.
- For what purpose the data were originally collected. If different from the new purpose, there may be biases within the data that hamper a correct assessment of the new purpose. For example, a clinician records patient information to help treat that individual, not for population based analyses.
- This could result in inconsistent coding. Also, there are legal restrictions on using personal data for purposes other than those for which it was collected.
- How timely the data are.
- How complete the data are.
The process of evaluation and monitoring should be a continuous one, and lessons learned should inform future developments . A crucial part of evaluating health care is assessing the quality of both health and social care and public health interventions. In the UK, at least 10% of inpatient episodes lead to unintended harm. Around half of these cases are preventable .
Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. (Lohr, 1990)
Services can be evaluated using the Donabedian framework of Structure-Process-Outcome; all three elements are required to give an overall picture of the quality of a service .
Structure: examine the provision of facilities and staff available. This could include:
- numbers of hospital beds
- ratio of doctors, nurses and other staff to patients and to each other
- comparisons across different geographical regions
- ease of access and opening hours to a clinic or screening service
Process: what is done for and to a patient or a population, and how well. This could include:
- methods by which patients are identified as suitable to be recorded on a disease register
- how thoroughly the diagnostic criteria had been determined
- validity and reliability of diagnostic tests
- time taken from diagnosis to treatment
- frequency of patient follow up
- how people are recruited and involved in a process, e.g. a screening or immunisation programme or substance abuse treatment
- Did the patient get better?
- Were there complications?
- Did the patient feel satisfied with the service?
- Has there been a recurrence?
- Was there a reduction in incidence in a population?
- Has coverage improved?
- Has life expectancy increased?
Maxwell's dimensions of quality take account of the population as well as patient care . The dimensions include:
- Access to services (for example, taking a population based approach, do some sub-groups find services easier to access than other sub-groups? This could be about the physical location of services as well as different attitudes on seeking care)
- Relevance to need (for the whole community)
- Effectiveness (on an individual patient basis)
- Equity (could the service in any way be made more fair?)
- Acceptability (is this a procedure which many find too uncomfortable/embarrassing/painful to undertake, and so avoid treatment?)
- Efficiency and economy
Example of where structure, process, and outcome framework could be applied:
Death of a patient due to maladministration of an anti-cancer drug :
In this case a drug which should only ever be administered into a vein was inserted into the spine. The patient died and the investigation found 40 system failures, including:
The syringes for the two different methods of injecting were labelled in a confusing manner. A senior doctor was not present.
Two junior doctors were allowed to dispense these drugs without adequate training. No formal induction programme was in place for starting junior doctors.
Death of a teenager.
Evaluations can lead to service re-design which then needs to be monitored regularly, such as:
- Clearer protocols
- Formal training
- Induction programmes
- Redesign of equipment
- Re-balancing of junior and senior staff
Information sources for evaluating health services
Evaluation involves comparison of the actual outcome of an intervention with the intended outcome. While it often involves an economic component, this is by no means the whole of evaluation.
Depending on what needs to be evaluated, the information sources listed in the planning section above will be useful to varying degrees. For example, if patients' post-operative mortality rates are being measured and compared with other areas, Hospital Episode Statistics (HES) will be required with linkage to ONS mortality data, since HES only captures mortality within hospital stays.
Patient satisfaction may be determined by means of a survey which captures qualitative information.
If access to a service is under-represented from one ethnic group compared with what would be expected for that population, then ethnicity needs to be captured consistently as part of an appropriate process (e.g. when it is asked for, how it is asked for, which ethnic group categories are used). However, there are legal restrictions on collecting these data: hospitals may only collect them for admitted patients, not for outpatients or patients awaiting services.
Nationally, any health care system must have robust mechanisms in place for assuring the quality and safety of services provided to patients that enable comparisons across health care institutions and geographical regions. This requires consistent methods of capturing and coding data. Within the NHS, a framework for ensuring health care quality includes:
- Clear national standards (e.g. from organisations that issue clinical guideline such as NICE or the medical Royal Colleges)
- Local clinical governance mechanisms for implementing quality assurance programmes as well as patient safety and quality improvements
- Inspection and audit programmes (Healthcare Commission)
- National audits such as the confidential enquiries into Perioperative Deaths, Maternal deaths, Suicide and Homicide by People with Mental Illness
In addition to clinical audits and inspections, ongoing monitoring of public health issues can be undertaken using a variety of websites and databases.
NHS providers routinely send data to commissioners through the Connecting for Health Secondary Uses Service (formerly via the NHS-wide clearing service).
Independent providers of services, such as terminations of pregnancy, should supply datasets detailing their activities to support their invoices, and these data are used to monitor progress against plans. These data include clinical, personal, and financial data, but patients are anonymised before transmission.
Incidence of communicable disease is monitoring continuously by the Health Protection Agency, who are informed through the notifiable diseases process.
The London Health Observatory, alongside the other nine Public Health Observatories in England, produces public health performance management reports on a quarterly basis for each PCT in London, which monitor a range of indicators such as take up of screening services, immunisation uptake, and participants in smoking cessation services. It takes data from the Department of Health, Health Protection Agency and Compendium websites, as well as data from the UNIFY database via NHS Net. This database stores data on local delivery plan targets, and all PCTs need to upload data to this database on a regular basis.
The National Drug Treatment Agency collates and analyses data about substance misuse on a monthly basis, data that local drug action teams use to monitor their services against target.
Healthcare needs assessment
Health needs assessment (HNA) is "a systematic method of identifying the health and healthcare needs of a population and making recommendations for changes to meet these needs." (Wright J. Assessing health needs. In Pencheon D, Guest C, Melzer D, Gray JAM editors. The Oxford handbook of public health practice. Oxford: Oxford University Press 2001.)
In this context "need" is defined as the ability to benefit from a healthcare intervention. The aim of HNA is to maximise the appropriate delivery of effective healthcare, and minimise both the provision of ineffective care and the existence of unmet need for healthcare. HNA provides a systematic framework for undertaking a complex and important task in an evidence-based way. The principles are widely used by public health practitioners. Assessing the health needs of the population is one of the functions of PCTs.
The practice of HNA first became widespread in the early 1990s, at the time of the introduction of the NHS 'internal market'. HNA was used to improve efficiency-aiming to identify which cost-effective services should be provided to meet population health needs, and which ineffective services should not. More recently, with the rise of the inequalities agenda, the emphasis of HNA has shifted more towards achieving equity, i.e. fair user access to health services and fair allocation of resources by providers to services. The shift to a primary-care led NHS has seen Primary Care Trusts (PCTs) take on the function of assessing the health needs of the population. Primary care is seen as well placed to assess needs for health services at a local level.
Stevens and Raftery (Stevens A, Raftery J editors. Health care needs assessment: the epidemiologically based needs assessment reviews. Oxford: Radcliffe Medical Press; 1994) have described the common approaches to assessing population needs for healthcare. These are characterised as the corporate, comparative and epidemiological approaches to HNA.
- The corporate approach "involves the structured collection of the knowledge and views of informants on health care services and needs". Informants are the stakeholders in the issue being addressed by the needs assessment. They might include clinical workers in both primary and secondary care, health service managers, commissioners of services, experts in the field and service users. Advantages of this approach include making the needs assessment responsive to local concerns and fostering "local ownership" of the issues. Disadvantages of this approach (if carried out in isolation) are that it determines demands rather than needs, and stakeholder concerns may be influenced by political agendas.
- The comparative approach involves the comparison of levels of service provision between different localities. These could be cross-national comparisons of the levels of service provision (e.g. comparing England with other countries in Western Europe), or could be at a more local level (e.g. comparing the service provision in one town with another that has a similar demography). This approach is usually fairly quick to achieve and inexpensive, but the disadvantages include the problems of finding a sufficiently similar locality for an accurate comparison.
- The epidemiological approach to HNA has three elements: (i) determining the incidence and/or prevalence of the health problem, (ii) identifying the effectiveness (and cost-effectiveness) of existing interventions for the problem, and (iii) identifying the current level of service provision. This combination of epidemiology (health status assessments) and evidence (effectiveness/cost-effectiveness) has also been described as the evidence-based approach to HNA. While this is a systematic and objective approach, its disadvantages lie in the frequent lack of existing local epidemiological data, and the lack of evidence for certain interventions - particularly for population subgroups such as the elderly (NB lack of evidence does not imply evidence of lack!). Carrying out new epidemiological work is also costly and time consuming.
See the Healthcare Needs Assessment
Health Impact Assessment (HIA)
Paragraph taken from APHO: The HIA Gateway
Health impact assessment is intended to help make decisions by predicting the health consequences if a proposal is implemented. In addition to assessing the health consequences, it also produces recommendations as to how the good consequences for health could be maximised and how the bad consequences could be avoided or minimised. It aims to predict not only the overall consequences for a population, but also the distribution of health impacts in that population, that is, which groups benefit and which groups lose, or at least benefit less. HIA may be used to assess policies, programmes or projects. Those undertaking the HIA should always carefully consider the requirements and the concerns of the decision makers whom they intend to inform. The principles and methods of HIA can be used to assess health consequences as part of another impact assessment (such as SEA or EIA). Whether the health consequences of a proposal are assessed with an HIA or as part of some other assessment is unimportant. The important thing is that they are thoroughly assessed.
There are three types of HIA:
Prospective - conducted before the implementation of a proposal
Concurrent - conducted during the implementation of a proposal
Retrospective - conducted after the implementation of a proposal
For further information see the Health Management Specialist Library.
Public Health Observatories
There is a network of 12 public health observatories (PHOs) working across the five nations of England, Scotland, Wales, Northern Ireland and the Republic of Ireland. They produce information, data and intelligence on people's health and health care for practitioners, policy makers and the wider community. They were established in 2000. Among other activities they produce health profiles reports. There are also three topic-based observatories: on obesity, injury, and child and maternal health.
Since 2005, the Healthcare Commission has run an annual health check to assess and rate the performance of every NHS Trust. It scores Trusts on 4-point scales for quality of services and for use of resources (scored as excellent or good or fair or weak). In 2007/8 the Health Protection Agency, NHS Direct, and the NHS Blood and Transplant service were included for the first time.
 Pencheon D, Guest C, Melzer D, Gray JAM Oxford Handbook of Public Health Oxford University Press, 2003
 Donaldson LJ, Donaldson RJ Essential Public Health 2nd Edition (Revised) , Petroc Press 2003
 Vincent C, Neale G, Woloshynowych M Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001; 322:517-19
 Donabedian A. (1988) The quality of care: how can it be assessed? Journal of the American Medical Association 260: 1743-8.
 R. Maxwell, Quality Assessment in Health, British Medical Journal, Vol. 288, l984.