What are PROs?

In the context of this evidence review, we focus on brief PRO assessments useful for clinical care, typically but not always collected on touch-screen tablets at the start of clinic appointments to improve care with summary results made available to providers for the clinic visit. Other approaches are also possible such as interviewer-based collection or electronic health record patient portals; however, we emphasize these much less due to the high costs, staff burden, and under-reporting of risk behaviors with interviewer-based approaches, and the exclusion of so many potentially high-risk patients with patient portals that require a level of technological sophistication and access that is not inclusive of entire HIV clinic populations.

Patient reported outcome measures, or PROs, gather patients’ assessments and experiences of clinically important issues, typically through brief, validated questionnaires. PROs offer insight beyond what may be directly observable by the provider in several dimensions of care.

PROs may include one or more dimensions of care: physical symptoms; mental health (e.g, depression, anxiety); health behaviors (e.g., medication adherence, substance use, sexual risk behavior); functioning (e.g., cognitive, physical); life circumstances (e.g., current housing, nutrition, family history of chronic illness, intimate partner violence); identity (e.g., sexual orientation, gender identity); social well-being (e.g., social support, HIV stigma); health-related quality of life (HRQL); and patient satisfaction with care.

On-site PRO collection prior to routine clinical care appointments, via hand-held computer tablets with real-time results available to providers during clinic visits, has improved provider ability to detect and address depression/suicidal ideation, inadequate antiretroviral (ART) adherence, and substance use in HIV care 1,2 . Integrating PROs into clinical care of patients with chronic conditions, such as HIV, cancer and rheumatoid arthritis, have also been shown to be acceptable to patients and providers and valuable in clinical care 2-4 ; they have improved patient-provider communication 5-9 and increased patient satisfaction with care 8,10-12.

The Value For HIV Providers

Improve identification of health issues and behaviors through improved patient-provider communication

PROs have been valuable in helping providers detect previously unrecognized problems 1,2. Controlled trials have found PROs improve provider ability to identify highly personal or stigmatizing symptoms or behaviors, including suicidal ideation, depression 1,13 , substance use 1 ,inadequate ART adherence 1 , and sexual behavior 14 . PROs have also improved detection of a broad range of symptoms in several clinical populations 7,15-20 as well as less-observable quality-of-life issues, such as social functioning 5,8,14,21.

Improve patient health outcomes

Evidence suggests the use of PROs has contributed to improved health outcomes including fewer emergency department visits and hospitalizations 22,23 , increased survival rates among cancer patients 22,23 , improved outcomes for mental health conditions 24 , lower symptom distress 16,17,25 and better health-related quality of life 7 .

Workflow impact

Providers reported PROs allowed for more comprehensive identification of issues and concerns that was generally regarded as an additional but manageable impact on workflow that was regarded as a valuable tradeoff. Both patients and providers described high acceptability of PROs in routine care and high added value.

Value For Patients

Optimize your care

Patients living with HIV have found the use of PROs improved communication with their provider in a number of ways. Specifically, it has 1) offered an opportunity to take inventory of current health needs, symptoms, health behaviors, and quality of life, including mental health and social well-being, improving their ability to advocate for their needs; 2) offered a preview in advance of the appointment of potential topics for discussion; and 3) helped patients to initiate, structure and prioritize discussion topics with their provider. 4) enabled easier discussion of sensitive or highly personal topics, such as mental health, sexual health, and substance use.28 .

Improve health outcomes

Evidence suggests the use of PROs has contributed to improved health outcomes including fewer emergency department visits and hospitalizations 22,23 , increased survival rates among cancer patients 22,23 , improved outcomes for mental health conditions 24 , lower symptom distress 16,17,25 and better health-related quality of life 7 .

Honestly, I was surprised at how useful I was going to find it….I figured my patients and I have a pretty good relationship…they're gonna tell me all these things. [But I was] hearing things I hadn't heard before. You're almost glancing behind a curtain.
-HIV Physician

It really did a lot for me because you had a lot of questions that I probably wouldn't have asked the doctor, or she probably wouldn't have asked me.
-Patient living with HIV

From HIV Providers and Patients Living with HIV

Clinical Topics Measured by PROs

PROs cover a broad range of topics clinically relevant to patients living with HIV, including symptoms, mental health, health behaviors, basic needs, social context, and quality of life. A resulting report summarizes results and displays topics of interest in order of clinical priority, and can depict changes over time. Alerts for critical issues such as significant anxiety or distress, interference with function, or suicidal ideation, may be sent to an appropriate provider or displayed to the patient, depending on whether the patient is within or outside a clinical setting when using the PROs. The following are just a sample of PRO measures, and is non-exhaustive.

Mental Health

Common PRO measures of mental health include the PHQ-9 29,30 for depression, the PHQ-5 29,30 and GAD-7 31 for anxiety.

Substance Use

Common PRO measures of substance use include the AUDIT-C/AUDIT 32,33 (alcohol use), and the ASSIST 34,35 (drug use). A measure by Bruneck et al 36 measures tobacco use. Past substance use treatment may be assessed via the Treatment Services Review. 37

Antiretroviral Adherence

PRO measures available include the adherence Self-Rating Scale, 30-day visual analog scale, AACTG adherence instruments (7 day missed dose, last missed dose, weekend missed dose). 38-40

Sexual Risk Behavior

Sexual risk behavior is measurable via the Sexual Risk Behavior Inventory 41 , which includes perceived partner attributes such as gender, HIV status, and ART/PrEP use, as well as concern for recent STI exposure.

Basic Needs

Basic needs include housing, and nutrition. Housing status is assessable via the CNICS Housing Measure 42 ; an option for measuring nutrition is the Canadian Nutrition Screening Tool 43 .

Social Environment

Measures to evaluate social environment and circumstance include Multifactorial Assessment of Perceived Social Support-Short Form (MAPSS-SF) 44 and the HIV Stigma Mechanism Measure 45 (adapted)

Quality of Life

The EQ-5D 46 is a commonly-used measure of health-related quality of life and PozQoL is an emerging option. Specific questions from the HATQOL are also used in the PROgress Study.

Symptom Tracking

Symptoms and level of severity is measurable via the HIV Symptom Index. 47

Exposure to Violence

The Intimate Partner Violence 4-item measure (IPV-4) 48 is a brief measure assessing physical, sexual, and psychological violence. The Adverse Childhood Experiences-International Questionnaire (ACE-IQ) 49 (adapted) assesses childhood household violence prior to age 18.

Sample PRO Results Page

PRO results reporting can be tailored to suit the needs of any clinic. Modifiable aspects of PRO results reporting include highlighting/prioritization of the most clinically urgent topics through use of placement, bold font, or color; the use of multiple time points to note change over time; and style and degree of scoring and interpretation needed. In addition, responses to clinically urgent questions, such as suicidal ideation, can be programmed to trigger pager alerts in real time for relevant staff.

PROgress Implementation Toolkit

The PROgress Implementation Toolkit provides practical advice to support the introduction of clinical PRO assessments into routine HIV care. These insights draw from a range of sources, including practical experience integrating PROs into HIV clinical care, published literature, and primary interviews with stakeholders with experience integrating PROs into HIV clinical care. Its focus is on implementation of tablet-based patient self-administered PRO assessments in clinical care yet many of the lessons are applicable across different types of applications. The toolkit includes information on:

  • Assessing clinic readiness and preparing for PRO implementation
  • Stakeholder engagement strategies
  • Building and/or selecting technical infrastructure for electronic data collection
  • Creating a PRO assessment that best suits the needs of your clinic and its patients, including an online time-budget tool to minimize patient response burden and impact on clinic flow
  • PRO integration and sustainability success
  • Staff training needs
  • Monitoring and evaluating successful PRO integration

PROgress Evidence Review and Summary

An evidence summary document has been collated to raise awareness of the potential value of well implemented PRO instruments within routine HIV care. It draws upon evidence from published literature characterizing the impact of PROs in routine clinical care for patients with chronic comorbidities, including HIV-related literature as well as other fields, particularly oncology. It includes real-world evidence surrounding the impact on patient outcomes, care processes, and the usability and acceptability of PRO platforms.

PROgress North American Site Study Summary

The PROgress study implemented patient-reported measures and outcomes (PROs) in two HIV outpatient clinical care settings to answer two overarching research questions:

  1. What is the added value of implementing PROs into routine HIV care for the salient stakeholders, including the patient, the providers, and other clinic staff?
  2. What are the essential program elements that can improve the sustainable adoption and implementation of PROs into routine HIV care across a range of differently resourced clinics?

The two North American outpatient HIV care clinics were:

  • Midway Specialty Care Center, Fort Pierce, Florida, USA.
  • St. Michael’s Hospital, Toronto, Ontario, Canada.

The PRO assessments included measures of symptoms (e.g., depression, anxiety), behaviors (e.g. adherence to antiretroviral therapy (ART), substance use), and other circumstances and perspectives (e.g., housing status, intimate partner violence, HIV medication satisfaction).

Using an implementation science framework (RE-AIM) and a mixed methodology quantitative and qualitative approach, we sought effectiveness and implementation insights. Data collection included 1761 completed patient PROs, 600 chart reviews, 47 qualitative interviews, 200 patient questionnaires, 17 provider questionnaires, training evaluations and a significant volume of implementation data and insights.

RESULTS:
Reporting of the results will be made available on this website as we publish our findings in peer reviewed journals and at conferences. These will also be integrated into the PROgress Evidence Review and Summary

Published Posters from 2020

Fredericksen RJ. Short D. Fitzsimmons E. Musten A. Jacobs B. Suri S. Korlipara D. Hodge V. Gough K. Ramgopal M. Bacon J. McReynolds J. Lober W. Crane HM. Patient perceptions of the utility and impact of a same-day self-administered routine electronic patient-reported outcomes (PRO) assessment in HIV care in two North American clinics. Virtual poster presentation, ISOQOL 27th Annual Conference, October 2019.

Fredericksen RJ. Short D. Fitzsimmons E. Musten A. Jacobs B. Suri S. Korlipara D. Hodge V. Gough K. Ramgopal M. Bacon J. McReynolds J. Lober W. Crane HM. Provider perceptions of utility and impact of a same-day routine electronic patient reported outcomes (PRO) assessment in clinical HIV care. Virtual poster presentation, ISOQOL 27th Annual Conference, October 2020.

References and Acknowledgements

This website was made possible by funding from ViiV Healthcare, a global specialist HIV company established in 2009, majority owned by GSK, with Pfizer and Shionogi as shareholders. We thank the patients, providers, and research staff from two PROgress Study clinics, Midway Specialty Care in Ft. Pierce, FL, and St. Michael’s Hospital in Toronto, ON, for their rich insights and commitment to this work. We thank the staff of the Ontario HIV Treatment Network and affiliated clinics for their continued collaborative partnership in advancing the role of PROs in HIV care. We also thank the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS), and the University of Washington Clinical Informatics Research Group (CIRG), for provision of resources and expertise in development of this website. Finally, we thank the PROgress Study Steering Committee, comprised of an expert panel of patients living with HIV, HIV care providers, clinic directors, and health care researchers, for their oversight and guidance.

References

  1. Crane HM, Crane PK, Tufano JT, et al. HIV Provider Documentation and Actions Following Patient Reports of At-risk Behaviors and Conditions When Identified by a Web-Based Point-of-Care Assessment. AIDS Behav. 2017;21(11):3111-3121.
  2. Fredericksen RJ, Tufano J, Ralston J, et al. Provider perceptions of the value of same-day, electronic patient-reported measures for use in clinical HIV care. AIDS Care. 2016:1-6.
  3. Wolfe F, Pincus T, Thompson AK, Doyle J. The assessment of rheumatoid arthritis and the acceptability of self-report questionnaires in clinical practice. Arthritis Rheum. 2003;49(1):59-63.
  4. Stover A, Irwin DE, Chen RC, et al. Integrating Patient-Reported Outcome Measures into Routine Cancer Care: Cancer Patients' and Clinicians' Perceptions of Acceptability and Value. EGEMS (Wash DC). 2015;3(1):1169.
  5. Detmar SB, Muller MJ, Schornagel JH, Wever LD, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. Jama. 2002;288(23):3027-3034.
  6. Wagner AK, Ehrenberg BL, Tran TA, Bungay KM, Cynn DJ, Rogers WH. Patient-based health status measurement in clinical practice: a study of its impact on epilepsy patients' care. Qual Life Res. 1997;6(4):329-341.
  7. Velikova G, Booth L, Smith AB, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22(4):714-724.
  8. Taenzer P, Bultz BD, Carlson LE, et al. Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology. 2000;9(3):203-213.
  9. Brown RF, Butow PN, Dunn SM, Tattersall MH. Promoting patient participation and shortening cancer consultations: a randomised trial. Br J Cancer. 2001;85(9):1273-1279.
  10. Wasson JH, Stukel TA, Weiss JE, Hays RD, Jette AM, Nelson EC. A randomized trial of the use of patient self-assessment data to improve community practices. Eff Clin Pract. 1999;2(1):1-10.
  11. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.
  12. Chen J. OL, Hollis S. J. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research. 2013;13(211).
  13. Dobscha SK, Gerrity MS, Ward MF. Effectiveness of an intervention to improve primary care provider recognition of depression. Eff Clin Pract. 2001;4(4):163-171.
  14. Berry DL, Blumenstein BA, Halpenny B, et al. Enhancing patient-provider communication with the electronic self-report assessment for cancer: a randomized trial. J Clin Oncol. 2011;29(8):1029-1035.
  15. Kjaer A, Rasmussen TA, Hjollund NH, Rodkjaer LO, Storgaard M. Patient-reported outcomes in daily clinical practise in HIV outpatient care. Int J Infect Dis. 2018;69:108-114.
  16. Ruland CM, Holte HH, Roislien J, et al. Effects of a computer-supported interactive tailored patient assessment tool on patient care, symptom distress, and patients' need for symptom management support: a randomized clinical trial. J Am Med Inform Assoc. 2010;17(4):403-410.
  17. Cleeland CS, Wang XS, Shi Q, et al. Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. J Clin Oncol. 2011;29(8):994-1000.
  18. Mark TL, Johnson G, Fortner B, Ryan K. The benefits and challenges of using computer-assisted symptom assessments in oncology clinics: results of a qualitative assessment. Technol Cancer Res Treat. 2008;7(5):401-406.
  19. Mazonson PD, Mathias SD, Fifer SK, Buesching DP, Malek P, Patrick DL. The mental health patient profile: does it change primary care physicians' practice patterns? J Am Board Fam Pract. 1996;9(5):336-345.
  20. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC health services research. 2013;13:211.
  21. Crane H TJ, Fredericksen R, Schmidt S, Brown T, Harrington R, Lober W, Kitahata M, Ralston J. Evaluating Integration of an HIV Medication Adherence Computer Assisted Self-Administered Interview (CASI) with Routine Patient Care. 5th International Conference on HIV Treatment Adherence; May 23-25, 2010, 2010; Miami, FL.
  22. Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2016;34(6):557-565.
  23. Basch E, Deal AM, Dueck AC, et al. Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. JAMA. 2017;318(2):197-198.
  24. Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract. 2006;12(5):559-568.
  25. Berry DL, Hong F, Halpenny B, et al. Electronic self-report assessment for cancer and self-care support: results of a multicenter randomized trial. J Clin Oncol. 2014;32(3):199-205.
  26. Fredericksen RJ SD, Fitzsimmons E, Musten A, Jacobs B, Suri S, Korlipara D, Hodge V, Gough K, Ramgopal M, Bacon J, McReynolds J, Lober W, Crane HM. . Provider perceptions of utility and impact of a same-day routine electronic patient reported outcomes (PRO) assessment in clinical HIV care. International Society for Quality of Life Research; 2020; Virtual conference.
  27. Fredericksen RJS, D. Fitzsimmons, E. Musten, A. Jacobs, B. Suri, S. Korlipara, D. Hodge, V. Gough, K. Ramgopal, M. Bacon, J. McReynolds, J. Crane, HM. Lober, W. . Patient perceptions of the utility and impact of a same-day self-administered routine electronic patient-reported outcomes (PRO) assessment in HIV care in two North American clinics. International Society of Quality of Life; October 21-24, 2020, 2020; online.
  28. HM. FRFEMJBSSLLWC. Patients' perceptions of value of a self-administered electronic patient-reported outcomes assessment on quality of their HIV care. . International Society of Quality of Life 26th Annual Conference; 2019; San Diego, CA.
  29. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. Jama. 1999;282(18):1737-1744.
  30. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613.
  31. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097.
  32. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med. 1998;158(16):1789-1795.
  33. Bradley KA, Bush KR, Epler AJ, et al. Two brief alcohol-screening tests From the Alcohol Use Disorders Identification Test (AUDIT): validation in a female Veterans Affairs patient population. Arch Intern Med. 2003;163(7):821-829.
  34. Newcombe DA, Humeniuk RE, Ali R. Validation of the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): report of results from the Australian site. Drug Alcohol Rev. 2005;24(3):217-226.
  35. WHO ASSIST Working Group. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): development, reliability and feasibility. Addiction. 2002;97(9):1183-1194.
  36. Kiechl S, Werner P, Egger G, et al. Active and passive smoking, chronic infections, and the risk of carotid atherosclerosis: prospective results from the Bruneck Study. Stroke. 2002;33(9):2170-2176.
  37. McLellan AT, Alterman AI, Cacciola J, Metzger D, O'Brien CP. A new measure of substance abuse treatment. Initial studies of the treatment services review. J Nerv Ment Dis. 1992;180(2):101-110.
  38. Simoni JM, Kurth AE, Pearson CR, Pantalone DW, Merrill JO, Frick PA. Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS Behav. 2006;10(3):227-245.
  39. Chesney MA, Ickovics JR, Chambers DB, et al. Self-reported adherence to antiretroviral medications among participants in HIV clinical trials: the AACTG adherence instruments. Patient Care Committee & Adherence Working Group of the Outcomes Committee of the Adult AIDS Clinical Trials Group (AACTG). AIDS Care. 2000;12(3):255-266.
  40. Lu M, Safren SA, Skolnik PR, et al. Optimal recall period and response task for self-reported HIV medication adherence. AIDS Behav. 2008;12(1):86-94.
  41. Fredericksen RJ, Mayer KH, Gibbons LE, et al. Development and Content Validation of a Patient-Reported Sexual Risk Measure for Use in Primary Care. J Gen Intern Med. 2018.
  42. Fredericksen RJ, Harding B, Nance RM, et al. Acceptability of electronic patient reported outcomes assessments (PROs) among patients in HIV care. International Society of Quality of Life 26th Annual Conference; 2019; San Diego, CA.
  43. Laporte M, Keller HH, Payette H, et al. Validity and reliability of the new Canadian Nutrition Screening Tool in the 'real-world' hospital setting. Eur J Clin Nutr. 2015;69(5):558-564.
  44. Fredericksen RJ, Fitzsimmons E, Gibbons LE, et al. Development and content validation of the Multifactoral assessment of perceived social support (MAPSS), a brief, patient-reported measure of social support for use in HIV care. AIDS Care. 2019:1-9.
  45. Earnshaw VA, Smith LR, Chaudoir SR, Amico KR, Copenhaver MM. HIV stigma mechanisms and well-being among PLWH: a test of the HIV stigma framework. AIDS Behav. 2013;17(5):1785-1795.
  46. EuroQol G. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199-208.
  47. Ainsworth BE, Jacobs DR, Jr., Leon AS. Validity and reliability of self-reported physical activity status: the Lipid Research Clinics questionnaire. Med Sci Sports Exerc. 1993;25(1):92-98.
  48. Fitzsimmons E, Loo S, Dougherty S, et al. Development and Content Validation of the IPV-4, a Brief Patient-Reported Measure of Intimate Partner Violence for Use in HIV Care. International Society of Quality of Life Research Conference; October 21, 20019, 2019; San Diego, California.
  49. World Health Organization. Adverse Childhood Experiences International Questionnaire. Adverse Childhood Experiences International Questionnaire (ACE-IQ) 2018; http://www.who.int/violence_injury_prevention/violence/activities/adverse_childhood_experiences/en/