Collecting patient-reported outcomes electronically
Moving a patient-reported outcome questionnaire from paper to a screen changes how many questions a patient is asked, what happens to the answers and who else can see them. Electronic collection brings gains and costs of its own. Computer-adaptive testing and a common metric build on it, and hospitals that collect these outcomes routinely through a patient app show what the approach changes in follow-up care, what cautions it carries and what it depends on.
Electronic and paper collection
An instrument can be administered on paper or electronically, through electronic diaries, computers, telephones and other portable devices.
Most patient-reported outcome measurement tools were designed for paper-based collection, and question fatigue can be a problem in that format, especially because patients are often asked to complete more than one measure at a follow-up visit. Collecting and analysing paper questionnaires also presents logistical and cost problems to researchers. Paper-based questionnaires are in widespread use, but they can be an inefficient way of collecting patient data, and electronic collection has the potential to drastically change patient data collection for the better.
The ISPOR PRO Mixed Modes Task Force listed the advantages of electronic data collection as less subject burden, avoidance of secondary data entry errors, easier implementation of skip patterns, date and time stamping, reminders and alerts, edit checks, and more accurate and complete data. Compared with paper, electronic collection generally reduces missing information and avoids the data entry errors that arise from an intermediate source, is immediately accessible, triggers alerts and notifications, increases a patient's willingness to report sensitive information, and gives real-time tracking of survey compliance. A systematic review and meta-analysis of studies conducted between 2007 and 2013 found that measures administered on paper are quantitatively comparable with measures administered on an electronic device.
Electronic collection has potential disadvantages too, including the costs of a custom-built platform. Important barriers to its use are increased expense, some cultural resistance, and limited time for training patients. Critics also point to the difficulty of reaching the right patient population: a 95-year-old patient, for example, may not be tech-savvy. Collecting patient data also immediately brings issues of security, privacy and confidentiality to the fore. Electronic collection is still relatively new, and as with all new technologies not everyone will be an early adopter.
One study examined the introduction of electronic systems in two orthopaedic clinical practices, where patient completion rates were 93% and 95%. For comparison, annual paper-based completion rates were as low as 30.6% for patients undergoing total joint arthroplasty at a single academic medical centre in San Francisco. The study's authors concluded that an electronic system to capture these outcomes in real time is feasible without any major disruption to the clinical workflow. An author at Smith & Nephew held in 2018 that only time would tell whether electronic collection delivers on its promise to transform clinical research, but that early indications were positive.
PROMIS and computer-adaptive tests
Government-funded organisations worldwide have invested in developing standardised and usable patient-reported outcome instruments. In 2004 the US National Institutes of Health began developing the Patient-Reported Outcome Measurement Information System, PROMIS, which aims to substantially improve the standards for assessing self-reported health status. More than 300 measures of physical, mental and social health are available for use in the general population, adults and children, and in individuals with chronic conditions, and they have been tested and validated in large reference populations, which makes them suitable for research on different health conditions. The technology is being employed in many disciplines, including orthopaedic research.
These outcomes can be delivered using computer-adaptive tests, individually tailored electronic questionnaires that focus on a single domain and use item response theory, so that the next question drawn from the question bank depends on the answers the patient has already given. Questions continue until the patient's score for that domain has been identified or the maximum number of questions has been reached. The PROMIS Physical Function test contains a maximum of 12 questions, but typically fewer are needed to identify the patient's score, often just five to seven. For pain interference, if a person says they cannot walk 15 metres without pain, it is clear that pain interferes with their life and there is no need to ask about hiking or contact sports, so all subsequent questioning calibrates how bad the interference is: can they walk 5 metres without pain, are they able to get out of bed? It is then possible to rapidly pinpoint where the patient is on the pain interference scale, whereas a traditional instrument requires every question to be asked and answered to arrive at a final score. Computer-adaptive tests can reduce the question burden for everyone involved.
The PROMIS initiative has generated a reliable and often more sensitive system than traditional patient-reported outcomes, customised to the patient, which poses fewer questions than traditional paper-based measures do. A systematic review comparing legacy measures with PROMIS in an orthopaedic setting stated that PROMIS measures can be administered more quickly and applied to a broader patient population while remaining highly reliable. A common PROMIS metric enables the results of different measures to be compared and simplifies interpretation of the score, and a patient's PROMIS score is correlated to a specific level of ability, such as lifting a cup to the mouth or running 10 miles. Using computer-adaptive tests instead of traditional measures, which may contain numerous questions, may help increase patient compliance. Because a respondent's current state of health and satisfaction is recorded quickly and precisely, surgeons can track their patients' progress interactively and more regularly than scheduled clinical visits alone allow, which may also help improve the patient's experience and satisfaction.
Smith & Nephew ran a four-month prospective cohort study to determine the usability, reliability and validity of PROMIS computer-adaptive tests for patients undergoing total knee arthroplasty, in which patients completed tests on pain behaviour, pain interference, physical function and depression before and after surgery. The study also examined user experience and clinician satisfaction with the digital platform. Eighty-seven patients were enrolled from five UK sites and one US site between January and April 2018. In late 2018 the results had not yet been published, but the company reported preliminary findings indicating high levels of patient engagement and satisfaction with the app and high levels of completion of the surveys. One of the study's clinical investigators holds that traditional collection is time-consuming and often burdensome for both patients and healthcare professionals, that mobile technology is a significant improvement over standard care, giving the patient an easy-to-use tool to report progress and letting surgeons track recovery closely, and that the data collection is seamless and lets the hospital evaluate its own performance.
The PROMIS common metric and its domains
The chair of PROMIS Germany holds that PROMIS started because new methods such as computer-adaptive tests could be employed for more precise measurement, and because the initiative had the political clout and the funding to make it happen, but that its bigger achievement is that it creates a framework of health, with the potential to set scales independent of the tools, which is indispensable for standardisation. With it, he says, there would for the first time be definitions and scales in this field as easy to understand as the Celsius scale is for temperature. He likens most instruments today to thermometers using different scales, which makes comparing their results highly complicated even when they measure the same construct, and holds that scoring an instrument on the PROMIS metric lets the scores from different assessment tools be compared instantly and meaningfully.
He estimates that there are probably over 4,000 different instruments, most of them carefully developed and validated, but believes this plethora actually hinders their acceptance: for them to enjoy the same acceptance as biomarkers, the field needs much greater standardisation and less confusion. In his view there will never be complete agreement on which instrument to use, since people never want just one type of car, but the advance PROMIS brings moves the field closer to the status of biomarkers.
He holds that PROMIS also addresses the old dispute over whether to favour generic or disease-specific tools. Disease-specific tools are typically more responsive in demonstrating treatment effects, whereas generic tools allow comparisons between different clinical populations, and disease-specific tools are essentially a compilation of health domains combined in one composite score. PROMIS domains are generic but can at the same time act as building bricks providing a disease-specific score, so the compilation of domains is specific, not the assessment itself. PROMIS identifies elements such as physical function, pain and anxiety that are relevant to everyone, from which the domains relevant to a disease are chosen: physical function is relevant to both heart disease and musculoskeletal disease, while dyspnoea is relevant only to heart disease. The domains are liberated from being tied to specific instruments and diseases, and the combination chosen is what is disease-specific.
Routine collection in Danish hospitals
Medical writers are increasingly likely to come across patient apps in their work. Patient-reported outcome measures have been used for some time in the Danish healthcare sector, mostly through research projects, with questionnaires about health and treatment experiences traditionally on paper and posted or completed at the clinic. The years up to 2018 brought major changes, primarily in the way the questionnaires are completed, the breadth of their use and the ambition for using the data collected. The change is partly a result of developments in the Danish Healthcare Quality Programme, introduced in 2004. The programme aims to ensure continuous development of the quality of care so as to create better patient pathways and prevent errors and unintended events, and its two fundamental objectives are to involve patients and their relatives in healthcare decisions and to ensure a patient-centred culture within hospital departments. A main strategy of the 2015 to 2018 national quality programme is to work towards systematic use of cross-sector measures, covering patient-reported symptoms, self-assessed health and treatment experiences, with the aim of directly influencing treatment decisions and ensuring quality of care. That will make it essential to collect valid, reliable data to guide decision-making and the development and evaluation of health services. In 2018 systematic use was generally seen as a positive step but was still a developing concept, and there was as yet no national consensus on which measures should be used or in what format.
The availability of electronic platforms has significantly changed collection. Instead of developing paper questionnaires for a single purpose and discarding them after the requisite number of years, data can now be collected via platforms such as RedCap or apps on mobile devices that facilitate storage, analysis and feedback.
Odense University Hospital and Svendborg Hospital together provide local and acute services for the Danish island of Funen and are the largest referral hospital in Southern Denmark, with all medical specialties, about 1,000 beds and about 1 million outpatient visits a year, and Odense also takes patients from all over Denmark for its highly specialised services. Patient-reported data have been key aspects of clinical research for some time, for example EQ-5D data collected alongside measures of functional independence and physical mobility for patients undergoing orthopaedic surgery, the development of a quality of life questionnaire for thyroid disease, and quality of life in toddlers with middle ear disease. The two hospitals have prioritised systematic collection in the expectation that patients will benefit, primarily through faster and more appropriate diagnostic pathways and fewer, more targeted follow-up visits. Systematic collection is not without challenges, including lower completion rates for elderly patients with impaired cognitive skills and the need to reassure healthcare staff that the data make a meaningful contribution to the individual patient's care. Many of the measures to be used for systematic collection at Odense were to be implemented through the regional app, "My Patient Journey".
Odense has established a networking group in which departments using these measures systematically can help each other and draw on each other's experience. The hospital's consultant on these measures reports that the departments are very positive: most use measures that have already been developed, which is the easiest solution, while others will have to develop new ones, which is a challenge, but they can draw on existing experience and use existing measures as a starting point.
The My Patient Journey app
"My Patient Journey" was developed in 2014 at Odense University Hospital's Centre for Innovative Medical Technology for easier digital communication between patients, medical staff and hospital departments. It helps patients find and keep track of information from the hospital and aims to give them a better overview and experience in communicating with it. In 2018 it was in use across the Region of Southern Denmark by approximately 44,000 patients and 2,000 clinicians, and although it is primarily for patients it can be downloaded from Google Play or the App Store as "Mit forløb". It is the user interface for patients at Odense: they can send text messages to medical staff, access information about their own treatment as text, video and images in place of the more general paper pamphlets, enter data such as weight or blood pressure, and answer questionnaires uploaded by the department. Nursing staff are its main operators in the hospital, and it has been integrated into the electronic medical journal already used daily for recording patients' visits and information and for exchanging data with other hospital departments, general practitioners and the municipalities. The app can be adapted to the needs of a department and to selected patient groups, so departments are free to choose which measures and other questions to include, typically with patient involvement to ensure relevant data collection and minimal respondent burden, and the emphasis is on existing validated health and quality of life questionnaires. When questionnaires are applied within the app, an algorithm can be created that calculates a score from the individual patient's answers, and the score can then determine how staff should follow up with the patient.
Cardiology follow-up at Svendborg Hospital
The cardiology clinic at Svendborg Hospital has been running a pilot in partnership with the municipal rehabilitation services, a cross-sector collaboration in which the patient's answers to staff questions and to the measures are made available to the hospital department and the municipal rehabilitation services at the same time. In the old follow-up system the patient visits the hospital and the municipal rehabilitation service in person; in the new one the patient sends health status data to both, who then advise on the form and frequency of follow-up. The two measures used are the Hospital Anxiety and Depression Scale, which can identify people at risk of clinical anxiety or depression, and the HeartQol, developed by the European Association of Preventive Cardiology for patients with ischaemic heart disease; both have 14 items, and the department had been collecting them on paper for some time. Loaded into the app, they are completed by patients who have undergone a cardiology intervention at the first nurse consultation, about two weeks after the intervention, and again after three and six months, while the patient is referred to a rehabilitation programme of exercise training and education on healthy living with heart disease. The objective is to see whether the data can enrich the patient's contacts with hospital and municipal services through more relevant discussion and treatment, which may take the form of a more individualised rehabilitation programme or interventions for anxiety, depression, smoking cessation, weight loss and the like.
In an example from that project, questions from the Hospital Anxiety and Depression Scale are combined with blood test results, here cholesterol level, to stratify patients at follow-up as red for a current problem needing immediate attention, yellow for a current problem needing follow-up, or green for no current problem, and the patient in the example moves from several red and yellow issues at the first visit to green over time. The focus points tracked across the initial, midway and final consultations are cholesterol, blood pressure, smoking, diabetes, diet, exercise, weight, sleep, alcohol, anxiety and depression, and stress.
The 30 patients who had taken part by 2018 gave positive feedback. They like the easy interactive format in which they can send a text message to a nurse they know about something in daily life, knowing that the answer is based on their current health and status, and they feel better prepared to talk to the nurse at the next consultation and to discuss health and emotional issues. Nurses found it easier to prepare for consultations and could focus the discussion on the issues the individual patient currently faces. The data let staff stratify patients earlier, so that patients who are doing well and do not need close follow-up can avoid unnecessary hospital visits. For rehabilitation staff, the patient-reported data on problems and challenges help them plan the programme in advance, provide a way of following the patient's progress, and can identify areas that need more focus.
Prostate cancer follow-up at Odense
Following a two-year project that the hospital's own staff describe as successful, an app was by 2018 used routinely with patients undergoing prostate cancer surgery at the urological ward at Odense, who answer electronically from home before surgery and again at 3, 6 and 12 months after it. The questionnaire was developed by a national working group under the Danish Health Authority and is aimed at all patients with prostate cancer, whatever their treatment. At the same time points the patient has a blood test taken by the general practitioner, and the app's score together with the blood test results give a colour code for whether telephone or outpatient follow-up is necessary, orange or red, or not necessary, green. The objective is to use the data to reduce unnecessary hospital visits, and the hospital's staff report that the use of the data has eliminated two-thirds of the follow-up visits in this patient group, giving staff more time for patients with more complicated problems and reducing the waiting lists.
Next steps include broadening the app's use, for example to patients with prostate cancer treated with medicines rather than surgery, which may be more challenging in terms of resources, because the app is not expected to reduce visits there and it takes staff time to go through the patient's answers and discuss them as appropriate.
Cautions and open issues
One caution is the tendency to focus on the specific health issues that emerge from the patient's data and so potentially miss important information that was not asked about. A similar problem can arise with telephone and email consultations and from telemedicine, because the full picture of a person's health is only achieved through a well-performed traditional consultation, and the issue requires further research, as it could have negative effects on patient treatment.
A further issue is how far measures should be piloted and validated before a hospital initiates large projects or implements routine data collection. In many countries questionnaires are typically put through a long testing process before use, whereas Odense decided to test the measures in real life, by putting them into use and correcting errors as they are identified.
Denmark is known for a comprehensive system of health registers that can be linked through the individual personal identifier, and it is recognised that sharing these data between patient and staff and between hospital sectors has to be done with care and attention to individual privacy. Written informed consent to share the data is typically given through the app: the patient is presented with the relevant information, which must be registered in the app as read before consent is possible, and consent can also be withdrawn through the app. The issues around data privacy and informed consent may become more prominent as sharing personal health data becomes more widespread.
The response rate to measures completed via an app still requires attention, as some patients were less accepting of the approach, especially older patients less familiar with smartphones and tablets, and this is one of the aspects to be evaluated in the next phase of the app project.
What routine app collection requires and achieves
Effective clinical use of systematic patient-reported outcome data requires the data to be easily accessed and visualised at the clinical contact, and both patients and staff need to know that the data provided will make a meaningful contribution to improved health care. An important element is that patients can see that staff have made active use of the data, and that the data are not just collected and then stored. Patients report that the app helps them to be more active and involved in their treatment and in discussions with hospital staff. The app approach appears to be especially beneficial for elderly patients, who can send a picture or a video instead of making the trip to the hospital.
The hospital staff who introduced the apps report that experience so far shows that systematic collection via patient apps can help staff focus on the individual patient's needs and can improve communication between patients and staff, and some patients have been able to avoid unnecessary hospital visits. Odense's medical director holds that the question is not whether to use these measures: they should be used, and much more than today, in every department in the hospital. In his view they provide possibilities for better and faster diagnostics as well as fewer and better follow-ups, and they are useful for collecting research data, which is especially valuable for a university hospital. He believes that, depending on the patient's diagnosis, many outpatient visits can be replaced by surveys. Another potential advantage of the app approach is in clinical research projects, where patients with baseline data can easily be block-randomised into intervention and control groups.
Those staff conclude that there is still much to learn about systematic collection via apps as an aid to optimising treatment and care, but that it appears a promising approach for focusing on the individual patient's needs and current status and for improving communication between patients and healthcare staff. An important next step is more formal evaluation, and in 2018 the cardiology project was being evaluated using the Model for Assessment of Telemedicine approach, which assesses aspects such as clinical effects, patient safety, patients' perspectives and experiences, financial aspects and organisational aspects.