Reading H

Clinicians and their patients have been monitoring symptoms and medical conditions for hundreds of years – some would go so far to suggest since the time of Hippocrates, who emphasized observation and documentation in 400 BC. Monitoring and evaluation are integral components of all evidence-based psychotherapies and most brief psychological therapies, and they are the foundation on which cognitive behavioural therapy (CBT) and low intensity (LI) interventions sit. Evaluation before treatment allows LI practitioners to assess the full range and impact of patients’ mental health problems, and to identify targets for intervention. Ongoing monitoring during treatment fulfils three further functions:

• Safety. Regular evaluation of patients’ risk of suicide, harm to, or neglect of self or others ensures that emergency action can be taken immediately if the need arises
• Individual progress. Treatment progress can be checked and the intervention adjusted to a patient's changing needs
• Service evaluation. Information about the quality of the service can be gained.

In this chapter we will outline different types and purposes of monitoring and evaluation in LI interventions, discuss some of the challenges that may occur and offer some practical suggestions for overcoming obstacles.

Monitoring to aid treatment selection and assessment of progress

It is advisable for LI practitioners, before commencing an intervention, to evaluate a patient’s mental health symptoms and functional impairment, the triggers, stressors and controlling variables associated with the person’s problems, as well as their coping resources and supports. At this stage, the purpose of the evaluation is to ensure that the least burdensome evidence-based treatment is recommended so as to maximize the patient’s recovery. Evaluation should also take place routinely and regularly throughout the intervention. Information about patients’ symptoms and functioning, as well as care pathway and contact level information is regularly collected. The purpose of the monitoring is to assist patients and their LI practitioner to assess progress, and to adjust the intervention where necessary. Research findings have shown that when therapists receive information about patients’ progress, there is a significant and substantial effect on patients’ outcomes, especially for those who show a poor initial response to treatment (Lambert et al. 2001). After 1 month (or less if indicated), it is advisable to assess whether an adjustment to the intensity of the intervention is warranted, for example, whether the patient needs to move up a step in the stepped care model when improvement is limited. However, ongoing monitoring of symptoms and routine evaluation should continue during the next level of stepped care. Administration of measures at the conclusion of the intervention is also necessary to check outcomes and patients’ satisfaction with the service. In some services, a limited number of key measures are also administered at each treatment session to ensure that a clinical end point is available in case patients withdraw prematurely from treatment (e.g. National Institute for Mental Health in England 2006; see Richards, Chapter 2).

In addition to the regular forms of LI evaluation, such as questionnaires, it is common for LI practitioners to ask patients to complete self-monitoring forms between the sessions and/or at the commencement of each session. Some CBT programs delivered by internet, CD-ROM, telephone, or correspondence have self-monitoring tools built into them. It is recommended that patients monitor their thoughts, emotions, behaviour, and physical reactions, as well as the frequency, intensity, and duration of them on a regular basis. Examples of CBT self-monitoring include, inter alia:

• Behavioural activation forms and automatic thought records for patients with depression
• Sleep diaries, and self-reported pre-sleep arousal for insomnia
• Daily food record in CBT for anorexia
• Self-monitoring forms for intrusive thoughts in post-traumatic stress disorder.

Alongside the patient’s core problems, monitoring often extends to associated triggers, lifestyle factors, and coping strategies. In addition, the patient’s level of functional impairment and disability, and the level of risk to self and others should also be monitored routinely.

Patient self-monitoring has several distinct advantages:

• It can occur in real time and is therefore potentially more accurate than assessment methods requiring retrospective recall
• Clinically relevant events, thoughts, behaviour, emotions, and physiological reactions that may not be accessible to external observation nor occur in the therapy session can be monitored in the person’s day-to-day life
• It is inexpensive and requires few clinician resources
• It can provide a continuous measure of patients’ progress as treatment proceeds
• It provides patients with continuous immediate feedback outside of the therapy session
• It supports engagement and encourages patient self-control.

Monitoring as a Treatment Intervention

Self-monitoring not only serves as an evaluation tool, it also operates as a treatment intervention in itself. Reactive effects, where the problem improves as a result of self-monitoring, have been documented for a wide range of clinical conditions including post-traumatic stress disorder (Ehlers et al. 2003), alcohol abuse problems (Kavanagh et al. 1999), binge eating (Latner and Wilson 2002), and insomnia (Jason 1975). Examples of therapeutic change include a reduction in binge eating following self-monitoring of food intake and decrease in time taken to fall asleep in insomnia.

Reactive effects from self-monitoring are reported to be small, but reliable. They also have the advantage of occurring immediately and persisting as long as the monitoring continues. Goal-setting and recording each occurrence of the target behaviour have been shown to enhance reactive effects, but such effects reduce when multiple responses are monitored concurrently (Korotitsch and Nelson-Gray 1999).

When assessing a patient, it is necessary for therapists to use valid and reliable evaluation tools. Valid tools are those that accurately measure what they purport to measure, while reliability is a yardstick of the measure’s consistency, either internally or on each occasion it is used. There are a number of valid and reliable tools available to measure common mental disorders within an LI intervention (see Donker et al., Chapter 23).

If objective tests are not available, however, the LI practitioner can teach the patient to rate her/his mood, behaviour or physical reaction on a 0-10 scale (e.g. ‘Thinking back over the last week, how has your depression been on a 0-10 scale, 0 meaning no depression, 10 meaning the most depressed you have ever felt’).

The use of retrospective self-reports, usually collected at the start of each session, is popular, primarily for expediency. However, caution needs to be exercised as they can be subject to recall bias and are not well suited to show how behaviour changes over time and across different contexts. Research has shown that even answering a simple factual question ‘How many alcoholic beverages did you have last week’ may be distorted by memory processes, as well as the patient’s current state (current pain, for example, influences recall). Requests to summarize past experience (‘How many times did it happen?’, ‘How severe was it, on average?’) can also lead to biases such as giving more weight to salient or recent experiences (Piasecki et al. 2007; Stone and Shiffman 2002).

Self-monitoring is a more sensitive measure than retrospective self-reports, questionnaires or self-rating scales, and it is extensively used in CBT. Patients should be encouraged to self-monitor throughout the day to gain the most benefit and to achieve valid and reliable data. Monitoring may be either at regular pre-determined times or whenever a behaviour, event, or subjective experience occurs (such as a mood change is noticed, exposure practice is carried out, etc.). However, it is not always feasible or convenient to monitor throughout the day, and patients may not be motivated to do so, particularly when paper monitoring forms need to be carried to conduct the self-monitoring. Another limitation of paper and pencil diaries is that they are often back-filled, and on occasions even forward-filled. A novel study by Stone and Shiffman (2002) illustrates the problem of non-compliance in monitoring. The researchers embedded a photosensor in the diary binders of 40 patients, which recorded when the binder was opened and closed. Over a 3-week monitoring period, 90% of the diaries were completed, but the monitoring took place at the scheduled time on only 11% of occasions. The LI practitioner has no reliable way of knowing whether back-filling has occurred and what items may have been affected.

Educating patients about the benefits of monitoring can assist in motivating them to commence monitoring and to persist with it in a valid and reliable manner (e.g. Ritterband et al. 2001). Another technique to encourage patients to continue with their self-monitoring is to graph patients’ scores and self-rated information, and feed it back visually to them as progress reports. Many electronic CBT programs have in-built graphing and reporting functions (see Fig. 7.1).

Example of a simple progress report

A solution to the problem of recall bias is to use Ecological Momentary Assessment (EMA). This involves patients assessing symptoms, behaviour, or events in real time, in their natural environment. A pre-programmed wristwatch, mobile phone, or hand-held computer (which the patient can set him/herself) signals to the person to report how they are feeling or what they are doing at the moment, i.e. during normal daily activity. Data can be collected by paper-and-pencil, telephone, electronic diaries, and physiological sensors. EMA is used to monitor particular events in a patient’s life, such as panic attacks, or to assess patients’ symptoms or functioning at periodic intervals. An example of the latter is the evaluation of mood in the morning, afternoon and evening to capture diurnal variation. EMA significantly reduces recall bias, maximises ecological validity and allows patients and their LI practitioners to study the processes associated with their functioning in real-world contexts (Shiffman, et al. 2008)

Another suggestion for assisting patients to persist with self-monitoring is to incorporate electronic technologies in the monitoring process.

Internet and CD-ROM Monitoring Systems

Internet and CD-ROM-based therapy programs regularly include a monitoring component to assist patients to evaluate their mood and behaviour. There are also electronic programs available that are purpose-built as stand-alone monitoring tools. In addition to recording variables associated with symptoms and functioning, these tools offer a wide range of other monitoring variables from which people can select, such as daily events, stressors or triggers, medication, and other treatments, coping strategies employed. Feedback and reporting tends to be sophisticated and user-friendly, with capacity to superimpose several key dimensions on a graph, along with a historical view, to assist users to see patterns in their mood and functioning. Online monitoring tools can be found by entering search terms such as ‘mood tracker’ and ‘symptom tracker’ into a web browser. It is important to check the authorship, affiliation, currency, and security of a tracking program before recommending it to patients.

Mobile Phone Monitoring Systems

Mobile phones offer significant advantages for monitoring and evaluation in LI interventions. They are carried on the individual, they are usually turned on, and their population penetration is high and increasing. Monitoring can be via text messaging which is relatively inexpensive (see Shapiro and Bauer, Chapter 27) or mobile web. Using either technology, mobile phones are ideally suited to gather EMA data. SMS messages can also be sent to prompt patients to report their monitoring data. The messages can be either delivered at the time of writing, or pre-programmed for delivery at a certain date and time. Another advantage of using mobile technology for monitoring and data collection is that the validity of the data can be enhanced by, for example, the system being programmed to send an error message if the patient attempts to enter a response out of the range of responses (e.g. entering 11 in a 0-10 rating scale).

Disadvantages of the use of mobile phones in monitoring include the cost of the text messages or of Internet access, which may present a challenge to some patients, the variety of mobile phone applications that limits interoperability, and the fact that current network coverage, although improving is not 100%. Nonetheless, mobile phones have been successfully used to monitor problem gambling behaviour, alcohol consumption, migraines, smoking, and with adolescents to gather mood data (Boschen and Casey 2008; Reid et al. 2009). A Mobile Mood Tracker service is currently being developed at the Black Dog Institute in Australia to assist people with depression, anxiety, or stress, and those at risk of the conditions to recognize, self-monitor, and self-manage their problems. This public health service will be available in 2010.

Monitoring and evaluation are critical components of LI interventions and should take place routinely before, during the intervention, and at its conclusion. It is incumbent on LI practitioners to use valid and reliable assessment tools, and to feedback the results to their patients to ensure they remain engaged as partners in the care process.

In addition to providing information about the progress of an LI intervention, monitoring has been shown to bring about improvement in symptoms and functioning as a reactive effect. Of course, LI practitioners should continually check their patients’ self-monitoring activities and ask their patients to cease self-monitoring if it is not helping, or when the patient has improved sufficiently.

Take Home Messages

• Monitoring is integral to CBT and LI interventions
• Assessment should take place before, during and at the completion of an intervention
• Self-monitoring not only serves as an evaluation tool, it also operates as a treatment intervention in itself
• Retrospective self-reports can be subject to recall bias
• EMA involves patients assessing symptoms, behaviour, or events in real time and in their natural environment
• There are numerous internet and CD-ROM-based programs available to assist patients to monitor their mood and behaviour
• Mobile phone monitoring programs offer further convenience for EMA.