Article Navigation
Article Contents
-
Lay Summary
-
Abstract
-
INTRODUCTION
-
MATERIALS AND METHODS
-
RESULTS
-
DISCUSSION
-
CONCLUSION
-
AUTHOR CONTRIBUTIONS
-
SUPPLEMENTARY MATERIAL
-
CONFLICT OF INTEREST STATEMENT
-
DATA AVAILABILITY
-
REFERENCES
Journal Article
, Adam M Johnson Department of Human Development & Family Science, East Carolina University , Greenville, North Carolina, USA Corresponding Author: Adam M. Johnson, MS, Department of Human Development & Family Science, East Carolina University, Greenville, North Carolina, USA; johnsonada19@students.ecu.edu Search for other works by this author on: Andrew S Brimhall Department of Human Development & Family Science, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on: Erica T Johnson Department of Human Development & Family Science, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on: Jennifer Hodgson Department of Human Development & Family Science, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on: Katharine Didericksen Department of Human Development & Family Science, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on: Joseph Pye Department of Family Medicine, ECU Health , Greenville, North Carolina, USA Search for other works by this author on: G J Corey Harmon Laupus Library, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on: Kerry B Sewell Laupus Library, East Carolina University , Greenville, North Carolina, USA Search for other works by this author on:
https://orcid.org/0000-0002-3620-4414 JAMIA Open, Volume 6, Issue 1, April 2023, ooac085, https://doi.org/10.1093/jamiaopen/ooac085
Published:
18 January 2023
Article history
Received:
30 November 2021
Revision received:
01 September 2022
Editorial decision:
04 October 2022
Accepted:
18 October 2022
Published:
18 January 2023
-
PDF - Split View
- Views
- Article contents
- Figures & tables
- Video
- Audio
- Supplementary Data
-
Cite
Cite
Adam M Johnson, Andrew S Brimhall, Erica T Johnson, Jennifer Hodgson, Katharine Didericksen, Joseph Pye, G J Corey Harmon, Kerry B Sewell, A systematic review of the effectiveness of patient education through patient portals, JAMIA Open, Volume 6, Issue 1, April 2023, ooac085, https://doi.org/10.1093/jamiaopen/ooac085
Close
Search
Search Menu
Abstract
Objective
The objective of this study was to systematically review all literature studying the effect of patient education on patient engagement through patient portals.
Introduction
Patient portals provide patients access to health records, lab results, medication refills, educational materials, secure messaging, appointment scheduling, and telehealth visits, allowing patients to take a more active role in their health care decisions and management. A debate remains around whether these additional aids actually improve patient engagement and increase their ability to manage their own health conditions. This systematic review looks specifically at the effect of educational materials included in patient portals.
Materials and Methods
In accordance with PRISMA guidelines, the literature search was mapped across 5 databases (PubMed, CINAHL, Scopus, PsychINFO, Embase), and implemented on June 2, 2020.
Results
Fifty-two studies were included in the review. Forty-six (88.5%) reported rates of patient utilization of educational resources in the patient portal. Thirty (57.9%) shared patients’ perceptions of the usefulness of the education materials. Twenty-one (40.4%) reported changes in health outcomes following educational interventions through the patient portal. This review found that efforts are indeed being made to raise awareness of educational resources in patient portals, that patients are increasingly utilizing these resources, that patients are finding them useful, and that they are improving health outcomes.
Conclusion
It seems that patient portals are becoming a powerful tool for patient education and engagement, and show promise as a means of achieving the quadruple aim of healthcare. Moving forward, research should establish more uniform methods of measurement in order to strengthen the literature surrounding the effectiveness of patient education through patient portals.
Lay Summary
Patient portals provide patients access to health records, lab results, medication refills, educational materials, secure messaging, appointment scheduling, and telehealth visits, allowing patients to take a more active role in their health care decisions and management. This article systematically reviewed all literature regarding patient education through patient portals to determine its effectiveness in improving patient engagement and ability to manage their health conditions. The results found that efforts are being made to raise awareness of educational resources in patient portals, that patients are increasingly utilizing these resources, that patients are finding them useful, and that they are improving health outcomes. It seems that patient portals are becoming a powerful tool for patient engagement and show promise as a means of improving patients’ health. Continued efforts need to be made to raise awareness of patient educational resources in patient portals so that patients utilize these resources to increase their knowledge, skills, and confidence for managing their own health and health care.
INTRODUCTION
Since the Institute of Medicine (IOM) established new aims for an improved healthcare system, efforts have been made to make healthcare safe, effective, patient-centered, timely, efficient, and equitable.1 Patient engagement has been shown to help achieve the quadruple aim of healthcare, which is to: (1) improve patient outcomes, (2) improve patient experience, (3) reduce costs, and (4) improve the providers’ experience in their work.2,Patient education is an effective way to increase patient engagement in their own health.3 Healthcare providers and organizations believe that using education to increase patient engagement will improve patients’ knowledge, skill, and confidence in managing their own health and health care.3,4
Current research shows that patients are interested in and want education and information about their health and how to manage it.5,6 They are increasingly turning to online sources for information about their conditions, especially when they feel providers fail to answer their questions.7–9 While the internet can be helpful10,11 sources such as YouTube, Wikipedia, Google, and social media platforms are unfiltered information sources and sometimes contain information that is either less accurate or inaccurate.7,12,13 This has problematic implications particularly for patients with lower health literacy, with research indicating they have a harder time distinguishing between reputable and unreputable sources.14
To address patients’ needs for health information and mitigate issues associated with poor quality information in many internet sites, healthcare providers, organizations and electronic health record (EHR) vendors created apps or digital resources to provide reputable education and tools to help patients manage their health.15 Some of these apps are standalone resources, while some called patient portals or personal health records (PHRs), are directly tethered or connected to the EHRs of providers and organizations.15 These tethered patient portals provide patients access to their health records, secure messaging, appointment scheduling, lab results, medication refills, educational materials, and telehealth visits, increasing patients’ ability to take an active role in their health care decisions and management.15 Healthcare providers can leverage patient portals to meet the rising demand for health education and information by providing reputable resources to patients online in a controlled and tailored manner.7,8,11 Currently, educational resources included in patient portals often include reading materials, videos, and links to reputable websites that are approved or created by the organization producing the patient portal.15
Previous reviews have studied patient portals generally, reviewing all the literature produced in recent years regarding their effect on patient empowerment,16,17 safety and quality of care,18 and utilization.19–22 In a meta-analysis, Neves et al18 found that patient portal use was associated with improved health outcomes for patients, as well as increased safety and quality of care. Alternatively, Ammenwerth et al16,17 found that patient portals had little to no effect on patient empowerment and health outcomes. Despite attempts to expand patient portals, patient use and adoption of these resources has been relatively low.19 To date, the percentage of patients who are registered to use their patient portal is typically less than 50% of the total patient population, and the number of patients who use it routinely is even lower.23 There appears to be a discrepancy between the resources provided and patients’ use of these resources. Zhao et al reviewed all literature surrounding barriers and facilitators of utilization and found that demographic factors such as age, race, and socioeconomic status can limit patients’ ability to access or use patient portals.19,24 A common barrier to accessibility is providing resources at a literacy level that the average patient can understand.19 For example, Chaet et al20 in their review, raised concern over the lack of patient portals that have resources in Spanish for Spanish speaking patients. Grossman et al21 reviewed all interventions used to increase utilization, and Beal et al22 reviewed the different ways researchers have used to track utilization. Current methods for increasing and measuring utilization have been widely varied and almost unique to the organization or research conducting the study, so there have been calls to analyze and standardize which practices are the most effective on both fronts in order to unify the field in a more common practice.21,22
While all these contributions have been valuable, no systematic reviews have been done to date to summarize the research regarding the use of patient education through patient portals. Just as with other aspects of patient portals, providers need to know about the effectiveness of education resources provided through patient portals, what factors determine patient use, and how to increase their effectiveness in the future. The objective of this study was to systematically review all literature studying the effectiveness of patient education through patient portals on patient engagement and ability to manage their health conditions.
Research question
What is the effectiveness of educational resources within patient portals?
Are patients aware of the educational resources within the patient portal?
Are patients utilizing the educational resources?
Do the patients find the content useful?
Does use of patient education information in patient portals increase patients’ understanding of their own health conditions and improve their self-management or health outcomes?
MATERIALS AND METHODS
The terms PHR and patient portal have become nearly synonymous in the research,15 so while both terms were included in our review’s search terms, we refer to these resources as patient portals throughout this review. The systematic review protocol and data underlying this article are available online via the Open Science Framework (https://osf.io/uey5c/?view_only=b3a792a77ab64fc8a5e5a39cec0cdfbd).
Search strategy and screening
One author (GJCH), who is an information specialist/librarian, iteratively designed the search strategy using recommended controlled vocabulary and keywords for patient education and patient portals (see Table1), in collaboration with 2 other authors (AMJ and KBS). The search was initially mapped in PubMed using MeSH terms and syntax and subsequently mapped to controlled vocabulary and syntax for the other identified databases (CINAHL, Scopus, PsychINFO, Embase). PubMed and Embase are standard databases for systematic reviews. The other 3 cover additional journals not indexed by PubMed and Embase but also focus on specific subject areas that were relevant to the research question. No limitations were placed on the search (ie, date, peer review, geography, or language). The date of the original search was June 2, 2020 All studies produced by the initial search were loaded into Endnote X9 (Clarivate Analytics, USA) for manual deduplication. After deduplication, the studies were loaded into Covidence (www.covidence.org), which identified additional duplicates missed during initial deduplication.
Table 1.
Reproducible search terms for PubMed
(“Medical Records”[mh] OR “Patient Access to Records”[mh] OR “health record”[tiab] OR “health records”[tiab] OR “Patient portal”[tiab] OR “Patient portals”[tiab] OR “Patient Web Portal”[tiab] OR “Patient Web Portals”[tiab] OR “Patient Internet Portal”[tiab] OR “Patient Internet Portals”[tiab] OR “Patient access to records”[tiab] OR ((electronic[tiab] OR automated[tiab] OR medical[tiab]) AND record*[tiab]) OR EHR[tiab] OR EMR[tiab] OR PHR[tiab] OR e-PHR[tiab]) AND (“Patient Participation”[mh] OR “health literacy”[mh] OR “consumer health information”[mh] OR “consumer health informatics”[mh] OR “patient education as topic”[mh] OR “Patient Involvement”[tiab] OR “Patient Empowerment”[tiab] OR “Patient Participation”[tiab] OR “Patient Activation”[tiab] OR “Patient Engagement”[tiab] OR “health literacy”[tiab] OR “patient education”[tiab] OR “patient guideline”[tiab] OR “patient guidelines”[tiab] OR “teaching material”[tiab] OR “teaching materials”[tiab] OR “instructional material”[tiab] OR “instructional materials”[tiab] OR “educational materials”[tiab] OR “educational material”[tiab] OR “consumer health information”[tiab] OR “consumer health informatics”[tiab] OR “consumer health materials”[tiab])
(“Medical Records”[mh] OR “Patient Access to Records”[mh] OR “health record”[tiab] OR “health records”[tiab] OR “Patient portal”[tiab] OR “Patient portals”[tiab] OR “Patient Web Portal”[tiab] OR “Patient Web Portals”[tiab] OR “Patient Internet Portal”[tiab] OR “Patient Internet Portals”[tiab] OR “Patient access to records”[tiab] OR ((electronic[tiab] OR automated[tiab] OR medical[tiab]) AND record*[tiab]) OR EHR[tiab] OR EMR[tiab] OR PHR[tiab] OR e-PHR[tiab]) AND (“Patient Participation”[mh] OR “health literacy”[mh] OR “consumer health information”[mh] OR “consumer health informatics”[mh] OR “patient education as topic”[mh] OR “Patient Involvement”[tiab] OR “Patient Empowerment”[tiab] OR “Patient Participation”[tiab] OR “Patient Activation”[tiab] OR “Patient Engagement”[tiab] OR “health literacy”[tiab] OR “patient education”[tiab] OR “patient guideline”[tiab] OR “patient guidelines”[tiab] OR “teaching material”[tiab] OR “teaching materials”[tiab] OR “instructional material”[tiab] OR “instructional materials”[tiab] OR “educational materials”[tiab] OR “educational material”[tiab] OR “consumer health information”[tiab] OR “consumer health informatics”[tiab] OR “consumer health materials”[tiab])
Table 1.
Reproducible search terms for PubMed
(“Medical Records”[mh] OR “Patient Access to Records”[mh] OR “health record”[tiab] OR “health records”[tiab] OR “Patient portal”[tiab] OR “Patient portals”[tiab] OR “Patient Web Portal”[tiab] OR “Patient Web Portals”[tiab] OR “Patient Internet Portal”[tiab] OR “Patient Internet Portals”[tiab] OR “Patient access to records”[tiab] OR ((electronic[tiab] OR automated[tiab] OR medical[tiab]) AND record*[tiab]) OR EHR[tiab] OR EMR[tiab] OR PHR[tiab] OR e-PHR[tiab]) AND (“Patient Participation”[mh] OR “health literacy”[mh] OR “consumer health information”[mh] OR “consumer health informatics”[mh] OR “patient education as topic”[mh] OR “Patient Involvement”[tiab] OR “Patient Empowerment”[tiab] OR “Patient Participation”[tiab] OR “Patient Activation”[tiab] OR “Patient Engagement”[tiab] OR “health literacy”[tiab] OR “patient education”[tiab] OR “patient guideline”[tiab] OR “patient guidelines”[tiab] OR “teaching material”[tiab] OR “teaching materials”[tiab] OR “instructional material”[tiab] OR “instructional materials”[tiab] OR “educational materials”[tiab] OR “educational material”[tiab] OR “consumer health information”[tiab] OR “consumer health informatics”[tiab] OR “consumer health materials”[tiab])
(“Medical Records”[mh] OR “Patient Access to Records”[mh] OR “health record”[tiab] OR “health records”[tiab] OR “Patient portal”[tiab] OR “Patient portals”[tiab] OR “Patient Web Portal”[tiab] OR “Patient Web Portals”[tiab] OR “Patient Internet Portal”[tiab] OR “Patient Internet Portals”[tiab] OR “Patient access to records”[tiab] OR ((electronic[tiab] OR automated[tiab] OR medical[tiab]) AND record*[tiab]) OR EHR[tiab] OR EMR[tiab] OR PHR[tiab] OR e-PHR[tiab]) AND (“Patient Participation”[mh] OR “health literacy”[mh] OR “consumer health information”[mh] OR “consumer health informatics”[mh] OR “patient education as topic”[mh] OR “Patient Involvement”[tiab] OR “Patient Empowerment”[tiab] OR “Patient Participation”[tiab] OR “Patient Activation”[tiab] OR “Patient Engagement”[tiab] OR “health literacy”[tiab] OR “patient education”[tiab] OR “patient guideline”[tiab] OR “patient guidelines”[tiab] OR “teaching material”[tiab] OR “teaching materials”[tiab] OR “instructional material”[tiab] OR “instructional materials”[tiab] OR “educational materials”[tiab] OR “educational material”[tiab] OR “consumer health information”[tiab] OR “consumer health informatics”[tiab] OR “consumer health materials”[tiab])
Two reviewers (AMJ and ETJ) independently screened all titles and abstracts against inclusion and exclusion criteria (see Table2). Conflicts were resolved by discussion until consensus was reached. Articles that passed the initial screening were included in a second round of screening, where studies underwent duplicate full-text review by 2 authors (AMJ and ETJ) to determine their inclusion to the final review. Any discrepancies between the reviewers were resolved by discussion.
Table 2.
Criteria for inclusion and exclusion
| Inclusion | Exclusion |
|---|---|
|
|
| Inclusion | Exclusion |
|---|---|
|
|
Table 2.
Criteria for inclusion and exclusion
| Inclusion | Exclusion |
|---|---|
|
|
| Inclusion | Exclusion |
|---|---|
|
|
Data extraction and synthesis
The PRISMA guideline for reporting systematic reviews was followed while conducting this study (https://prisma-statement.org/PRISMAStatement/Checklist). Data extraction was done using Covidence and Microsoft Excel. Extracted data elements included: author(s) last name(s), year of publication, country and language of publication, patient portal used, population size and characteristics, sampling method, study design, outcomes, and findings. Data extraction was performed by the primary investigator (AMJ) and confirmed by another author (ETJ). Data were synthesized using narrative synthesis methods.25 One author (AMJ) synthesized findings and one author (ASB) checked the results and provided recommendations for any needed changes.
Risk of bias and interrater reliability
Although risk of bias assessment is recommended for systematic reviews, study methods varied considerably among included studies. Since of risk of bias assessment instruments are created for specific study designs (see the variety of checklists here: https://casp-uk.net/casp-tools-checklists/), providing a unified risk of bias assessment using just 1 or even 2 risk of bias tools was not possible. Interrater reliability analysis using the Kappa statistic was performed to determine consistency among the screeners. Interrater reliability for the title and abstract screening was found to be Kappa = 0.275, and title abstract screening for the full text review was found to be Kappa = 0.499. These measurements are considered to be fair and moderate, respectively.26
RESULTS
The initial search retrieved 17,807 records across the 5 identified databases (PubMed—5347, CINAHL—3018, Scopus—2152, PsycInfo—3383, Embase—3018). Following manual (EndNote) and machine-driven deduplication (Covidence), 12,062 unique articles remained for title and abstract screening. Following title and abstract screening, 180 articles were retained for full text review. Fifty-two articles were retained for full inclusion in this review. The PRISMA flow diagram (Figure1) provides a record of the article selection process for the review.
Due to the large number of studies included (n = 52), a master table with information from all studies is included as Supplementary Material. Only a minority of studies included demographic information on participants, so a synthesis of that data was not included in this review. Results are organized by the research questions, with smaller tables describing the articles relevant to each of the 4 subquestions guiding the review. Of the studies included, 36 (69.2%) were based on US healthcare systems and 16 (30.8%) were based on other countries. Forty-four of the patient portal systems (84.6%) were utilized in outpatient settings, while 9 (17.4%) were for inpatient treatment (one was utilized in both, and so double counted). Forty-three (82.7%) provided education for the patient, 8 provided education for both the patient and the caregiver (15.2%), and one targeted only the caregivers (2%). The populations targeted in the included studies varied across several health conditions (ie, diabetes, cancer, pregnancy, etc.). Another important finding was that 40 of the studies (77%) were published between 2013 and 2020, with only one published before 2000,27 reflecting that patient portals are still a relatively new resource being developed.
Question 1—are patients aware of the educational resources within the patient portals?
The primary factor determining whether an article was included was its use of patient education within the patient portal. As a result, all 52 studies had some educational features as a portion of their patient portal and each made attempts to inform their patients about its availability (see Figure2). Of these studies, 15 (28.8%) had content pushing features, meaning that the patient portal automatically linked patients with relevant educational resources based on their medical information, such as diagnoses.28–42 While these studies confirm that patients were notified of the educational resources available, especially those who were enrolled in the 15 studies relying on push notifications, it does not necessarily provide insight into the awareness of the typical patient who is not enrolled in an ongoing study measuring the use of patient portals.
Figure 2.
Number of included articles relevant to each of the 4 research questions. Results are sectioned according to research question. “Applicable” represents the number of articles included in the review, which relate to each of the 4 questions. N/A indicates the number of included articles which did not include results relevant to each question.
Question 2—are patients utilizing the educational resources?
While all included patient portal studies had educational resources and made patients aware of them, utilization of these resources remained low. Of the 52 studies included in the review, 46 (88.5%) reported on patient’s utilization of the educational resources (see Figure2). This data was obtained in a variety of ways, including: (1) survey data relying on patient self-report (n = 16; 30.8%); (2) required utilization for pilot studies (n = 11; 21.2%), (3) qualitative statements simply saying that resources were used (n = 10; 19.2%); and (4) metadata audits of patient portal systems and how often resources were clicked on or accessed (n = 9; 17.3%).
The most common form of data collection (survey data) revealed that, according to patients’ self-report, utilization rates varied from as low as 20%43 to 95%42 with an overall average of 47% across 12 studies (see Table3).27,30,32,36,41–48
Table 3.
Rates of utilization
| Authors | Utilization |
|---|---|
| Arcia (2017) | 75% (12/16) |
| Cameron et al (2016) | 47% (55/116) |
| Cho et al (2019) | 42% (84/201) |
| Groen et al (2016) | 89% (33/37) |
| Jones et al (1992) | 24% (17/70) |
| Kelly et al (2017) | 61% (111/181) |
| Naveethan et al (2017) | 47% (99/209) |
| Roelofsen et al (2014) | 27% (110/405) |
| Steiner et al (2017) | 20% (43/216) |
| Warrington et al (2019) | 78% (276/354) |
| Woollen et al (2016) | 85% (12/14) |
| Zhang et al (2016) | 95% (19/20) |
| Authors | Utilization |
|---|---|
| Arcia (2017) | 75% (12/16) |
| Cameron et al (2016) | 47% (55/116) |
| Cho et al (2019) | 42% (84/201) |
| Groen et al (2016) | 89% (33/37) |
| Jones et al (1992) | 24% (17/70) |
| Kelly et al (2017) | 61% (111/181) |
| Naveethan et al (2017) | 47% (99/209) |
| Roelofsen et al (2014) | 27% (110/405) |
| Steiner et al (2017) | 20% (43/216) |
| Warrington et al (2019) | 78% (276/354) |
| Woollen et al (2016) | 85% (12/14) |
| Zhang et al (2016) | 95% (19/20) |
Note: Parentheses show the number of patients who reported utilizing the patient portal education over the total sample size. The total number of patients who utilized patient portal education in these studies was 871/1839 (47%).
Table 3.
Rates of utilization
| Authors | Utilization |
|---|---|
| Arcia (2017) | 75% (12/16) |
| Cameron et al (2016) | 47% (55/116) |
| Cho et al (2019) | 42% (84/201) |
| Groen et al (2016) | 89% (33/37) |
| Jones et al (1992) | 24% (17/70) |
| Kelly et al (2017) | 61% (111/181) |
| Naveethan et al (2017) | 47% (99/209) |
| Roelofsen et al (2014) | 27% (110/405) |
| Steiner et al (2017) | 20% (43/216) |
| Warrington et al (2019) | 78% (276/354) |
| Woollen et al (2016) | 85% (12/14) |
| Zhang et al (2016) | 95% (19/20) |
| Authors | Utilization |
|---|---|
| Arcia (2017) | 75% (12/16) |
| Cameron et al (2016) | 47% (55/116) |
| Cho et al (2019) | 42% (84/201) |
| Groen et al (2016) | 89% (33/37) |
| Jones et al (1992) | 24% (17/70) |
| Kelly et al (2017) | 61% (111/181) |
| Naveethan et al (2017) | 47% (99/209) |
| Roelofsen et al (2014) | 27% (110/405) |
| Steiner et al (2017) | 20% (43/216) |
| Warrington et al (2019) | 78% (276/354) |
| Woollen et al (2016) | 85% (12/14) |
| Zhang et al (2016) | 95% (19/20) |
Note: Parentheses show the number of patients who reported utilizing the patient portal education over the total sample size. The total number of patients who utilized patient portal education in these studies was 871/1839 (47%).
This finding excludes the 11 studies whose utilization rates were 100% because participants were required to use it as part of the study.39,49–56 The variability in utilization rates seems to be associated with sample size, as studies with smaller samples typically experienced higher utilization rates. There also appears to be some discrepancy between desire for resources and the actual rate at which participants use them. Patients reported wanting educational resources included in the patient portal between 84%5,6 and 89%57 of the time, and planned on using them “very often.”58 It is also worth noting that 78% of caregivers also indicated that they wanted educational resources.5 Although many participants reported wanting educational resources, and felt they would use them frequently, the data indicates that expressed desire for patient portal-based educational resources does not equate with an equally high level of use of those resources.
Metadata audits of the patient portal software reported the number of times educational resources were accessed over varying amounts of time23,31,59–65 but the nondescript nature of the audit made it hard to interpret how frequently educational resources were utilized by individual patients. However, one of the studies specifically reported that patients who accessed educational resources through the patient portal made up only 42% of registered patient portal users, and 10% of the entire patient population of that site.23
Question 3—do the patients find the content useful?
Only 30 of the 52 studies included in the review included some aspect that measured patients’ perception of the usefulness of educational content (see Figure2). Based on the review of those articles, it appears that patients find educational resources in patient portals to be extremely useful. Like utilization rates, these reports were received using a variety of methods, including: (1) surveying the study sample to see what percentage of participants perceived education resources as useful (n = 13, 43%), (2) qualitative statements simply stating that patients thought content was useful (n = 9, 30%), (3) having participants report level of usefulness using Likert scale measurements (n = 6, 20%), and (4) utilizing validated measures of usefulness (n = 2, 7%).
Of the 4 different styles of measurement, the 2 most informative were the survey studies reporting perceived usefulness by the percentage of participants and those which relied on Likert scales to quantify the degree of usefulness, both of which showed that patients thought the educational resources within the patient portal were very useful (see Table4).5,27,29,30,39,42,45,46,48–51,53,54,57,58,66–68 Two studies used the Net Promoter Score44 and the System Usability Scale65 to measure patient’s perceptions of usefulness, and both reported that patients found educational resources included in the patient to be very useful. While this only makes up 7% of the included studies measuring perceptions of usefulness, they are noteworthy because they represent a shift in the field to more standardized methods of measurement. The remaining 9 simply stated that patients thought the resources were useful, often with quotations from qualitative studies.40,43,55,59,61,63,69–71
Table 4.
Patient’s report of education resources’ usefulness
| Authors | Percentage | Authors | Likert |
|---|---|---|---|
| Benhamou (2011) | 74% (13/18) | Martinez et al (2018) | 5.8/7 |
| Warrington et al (2015) | 42% (5/12) | Baek et al (2018) | 4.23/5 |
| Groen et al (2016) | 69% (26/37) | Ahmed et al (2020) | 4.3/5 |
| Day et al (2019) | 78% (7/9) | Arcia (2017) | 4.7/5 |
| Yeh et al (2008) | 70% (35/50) | Hefner et al (2017) | 4.5/5 |
| Fricton et al (2008) | 70% (128/182) | Fiks et al (2014) | 8.4/9 |
| Hess et al (2006) | 71% (15/21) | ||
| Wiljer et al (2006) | 79% (37/46) | ||
| Wiljer et al (2010) | 98% (123/125) | ||
| Woollen et al (2016) | 86% (12/14) | ||
| Jones et al (1992) | 84% (59/70) | ||
| Kelly et al (2017) | 68% (61/90) | ||
| Zhang et al (2016) | 96% (19/20) |
| Authors | Percentage | Authors | Likert |
|---|---|---|---|
| Benhamou (2011) | 74% (13/18) | Martinez et al (2018) | 5.8/7 |
| Warrington et al (2015) | 42% (5/12) | Baek et al (2018) | 4.23/5 |
| Groen et al (2016) | 69% (26/37) | Ahmed et al (2020) | 4.3/5 |
| Day et al (2019) | 78% (7/9) | Arcia (2017) | 4.7/5 |
| Yeh et al (2008) | 70% (35/50) | Hefner et al (2017) | 4.5/5 |
| Fricton et al (2008) | 70% (128/182) | Fiks et al (2014) | 8.4/9 |
| Hess et al (2006) | 71% (15/21) | ||
| Wiljer et al (2006) | 79% (37/46) | ||
| Wiljer et al (2010) | 98% (123/125) | ||
| Woollen et al (2016) | 86% (12/14) | ||
| Jones et al (1992) | 84% (59/70) | ||
| Kelly et al (2017) | 68% (61/90) | ||
| Zhang et al (2016) | 96% (19/20) |
Note: Parentheses show the number of patients reporting patient portal education as useful over the total sample size. The total number of patients who said patient portal education was useful across these studies was 540/689 (78%).
Likert scale shows the average score of patient reports of usefulness of the patient portal education resources. A score above the midpoint of the scale (ie, <3/5, <4/7, <5/9) indicates patients reporting that the education was useful.
Table 4.
Patient’s report of education resources’ usefulness
| Authors | Percentage | Authors | Likert |
|---|---|---|---|
| Benhamou (2011) | 74% (13/18) | Martinez et al (2018) | 5.8/7 |
| Warrington et al (2015) | 42% (5/12) | Baek et al (2018) | 4.23/5 |
| Groen et al (2016) | 69% (26/37) | Ahmed et al (2020) | 4.3/5 |
| Day et al (2019) | 78% (7/9) | Arcia (2017) | 4.7/5 |
| Yeh et al (2008) | 70% (35/50) | Hefner et al (2017) | 4.5/5 |
| Fricton et al (2008) | 70% (128/182) | Fiks et al (2014) | 8.4/9 |
| Hess et al (2006) | 71% (15/21) | ||
| Wiljer et al (2006) | 79% (37/46) | ||
| Wiljer et al (2010) | 98% (123/125) | ||
| Woollen et al (2016) | 86% (12/14) | ||
| Jones et al (1992) | 84% (59/70) | ||
| Kelly et al (2017) | 68% (61/90) | ||
| Zhang et al (2016) | 96% (19/20) |
| Authors | Percentage | Authors | Likert |
|---|---|---|---|
| Benhamou (2011) | 74% (13/18) | Martinez et al (2018) | 5.8/7 |
| Warrington et al (2015) | 42% (5/12) | Baek et al (2018) | 4.23/5 |
| Groen et al (2016) | 69% (26/37) | Ahmed et al (2020) | 4.3/5 |
| Day et al (2019) | 78% (7/9) | Arcia (2017) | 4.7/5 |
| Yeh et al (2008) | 70% (35/50) | Hefner et al (2017) | 4.5/5 |
| Fricton et al (2008) | 70% (128/182) | Fiks et al (2014) | 8.4/9 |
| Hess et al (2006) | 71% (15/21) | ||
| Wiljer et al (2006) | 79% (37/46) | ||
| Wiljer et al (2010) | 98% (123/125) | ||
| Woollen et al (2016) | 86% (12/14) | ||
| Jones et al (1992) | 84% (59/70) | ||
| Kelly et al (2017) | 68% (61/90) | ||
| Zhang et al (2016) | 96% (19/20) |
Note: Parentheses show the number of patients reporting patient portal education as useful over the total sample size. The total number of patients who said patient portal education was useful across these studies was 540/689 (78%).
Likert scale shows the average score of patient reports of usefulness of the patient portal education resources. A score above the midpoint of the scale (ie, <3/5, <4/7, <5/9) indicates patients reporting that the education was useful.
Question 4—does use of patient education information in patient portals increase patients’ understanding of their own health conditions and improve their self-management or health outcomes?
The use of patient education resources provided through the patient portal had a positive effect on patient outcomes. Of the 52 articles included in the review, 21 (40.4%) reported on the association between patient portals and patient self-management and health outcomes (see Figure2). While each of these included studies used patient portals that featured patient education resources among other tools, they did not differentiate the individual contribution of education on patient outcomes, reporting results instead as the effects of the whole intervention. Patient outcomes were measured in 3 ways, including: (1) improvement in self-management of health conditions (n = 12, 57%), (2) changes in patient lab values (n = 5, 24%), and (3) adoption of desired health behaviors (n = 4, 19%).
The most common measure of patient outcomes, self-management of health, was determined in 2 different ways; quantitative measures that scored improvement (n = 6) and qualitative reports from patients (n = 6). The 6 studies that used quantitative outcomes found that use of educational resources in the patient portal were associated with significant increases in: (1) patient activation45; (2) knowledge of conditions33,56; (3) self-management42; (4) adoption of desired health behaviors33; (5) decision making55; and (6) decreased anxiety.68 The patients in the qualitative self-report studies indicated they felt that education through the patient portal: (1) increased their knowledge of their health30,46,48,61,63; (2) increased their confidence in their ability to take care of themselves or those in their care30,40,46,61,63; (3) improved safety during and after hospitalization46,48; (4) improved decision making and ability to communicate with healthcare providers46; and (5) decreased anxiety around treatment and prognosis.40,48 It should be noted that 3 studies also found improvements that, while positive, were nonsignificant in areas of self-efficacy,68,72 adherence,72 knowledge,4 and health behaviors such as physical exercise.45
As for improvements in health behaviors and lab values, 4 studies reported significant improvements in patients adopting positive health behaviors such as receipt of vaccination,32 use of effective contraception,37 medication compliance,33 and decision to undergo a preventative screening procedure.50 Changes in patient lab values (ie, HgA1c) following use of educational resources were positive, though not always significantly so. Two studies associated significant improvements66,70 in intervention groups who were given educational resources, while 3 found improvements that were not significantly better than control groups.4,36,44 In each of these cases, nonsignificant results still showed slight positive changes and were often presented alongside significant findings in other aspects of patient health, so it seems that use of educational resources in the patient portal is generally associated with improvements in patient outcomes.
DISCUSSION
This review sought to evaluate the effectiveness of educational resources within patient portal systems by determining availability and patients’ awareness of resources, their utilization and perceived usefulness of those resources, and the effect of those resources on patient outcomes. In response to the first 2 research questions of this study, many providers and healthcare organizations have developed or are utilizing patient portals to provide more accurate and reliable sources of information in response to patients’ increased interest in their own healthcare, and are making efforts to increase patient awareness of these resources.5,6 The main discussion point, however, is how to increase utilization of educational resources through patient portals in the general population. In response to the second two research questions of this study, educational resources provided through patient portals useful (see Table4), and when used, they had a significant impact on patient outcomes.32,33,37,50,66,70 The methods used to measure usefulness and patient outcomes varied widely, and creating more uniform methods of measurement will be the focus of discussion.
Questions 1 and 2
While this review indicates that patient education through patient portals improves patient experiences and patient outcomes, utilization of educational resources is low (see Table3)23 even after patients are made aware of them. These findings highlight a common finding in effectiveness research. Utilization rates of educational resources are potentially inflated because attempts to study patient portals require that participants are enrolled in studies where they are being asked, as part of the study, to use the resources. This was especially true in pilot studies where utilization rates were 100% because participants were required to use them. However, in studies which simply measured rates of use (ie, metadata studies) utilization rates of both patient portals and educational resources are relatively low. These findings present an interesting dilemma for professionals in this field that needs to be addressed. It may be that patients from the general population are still not aware of these features and therefore they remain underutilized. Strategies for increasing use by the average patient need to be developed. A couple of strategies that seem to make a difference, based on this review, is including additional resources, like patient navigators, or automating connection of patients with relevant resources. Considering that this technology is still relatively new, there is hope that patient utilization will increase as patients become more aware of its availability. Like many new technologies, utilization rates might continue to be low for the average American until a tipping point occurs that shifts use from early adopters (those who are either technologically savvy or heavily invested in their health) to part of the mainstream culture. Efforts to raise this awareness will be essential if patient portals are ever going to realize the full potential these educational resources have to increase patient health. Also, it is important to recognize that, given the rapid pace of technological advances, new technologies will continue to expand and develop, thus, requiring research to evolve with it.
Questions 3 and 4
Current research suggests that patients found educational resources provided through patient portals useful (see Table4), and when used, they had a significant impact on patient outcomes.32,33,37,50,66,70 While this review focuses on patient perceptions, it is also worth noting that Fricton et al5 found that 70% of caregivers thought it was useful too. Given that caregiver engagement can have enormous impacts on patient outcomes, this finding is notable.73 The variability of methods of measurement for utilization, usefulness, and impact of patient education included in this review may be due to the fact that research regarding patient education through patient portal systems is still in its infancy. Most of the articles included in the review (77%) were published between 2013 and 2020, because patient portals themselves are a relatively new technology. Up to this point, most research has focused on development, design, implementation, and beta testing new systems and resources, and so have used a wide variety of methods and measures of effectiveness. While these studies have added to the existing literature the fact that they are measuring different variables makes it hard to establish a consistent message regarding utilization rates and effectiveness. One benefit of a systematic review is it provides a comprehensive summary of the literature that exists around a specific topic and highlights some of the potential gaps. One finding this review highlights is that patients who are using the educational resources found in patient portals are finding them helpful. How helpful is hard to determine because researchers have typically used a variety of different methods to measure both utilization and effectiveness. Different measurements make it difficult to develop a cohesive argument regarding the effectiveness of these resources. Future research in this area should include a Delphi study to establish the best method of standardized measurement to help strengthen the argument surrounding patient portals and their effectiveness, as survey data does not seem to reliably measure use in the general poplulation23 and measures such as the Net Promoter Score44 do not specifically measure utilization or outcomes, but rather satisfaction. Until a specific study is done to provide a more definitive recommendation for standardize measurement of patient portal utilization, it seems that system audits of actual use, as used in the study done by Ancker et al23 provide the most accurate measure of utilization, and that Likert scales seem to be the most uniform way to measure perceived usefulness.
Limitations
This systematic review screened and included a large quantity of literature regarding broad topics of patient engagement, patient education, and health records, which may have contributed to the moderate interrater reliability and may limit the reproducibility of this study. While it had several strengths there is an important limitation to consider. The majority of articles that match our criteria are from the United States (69.2%). There are several possible explanations for this outcome. First, all of the authors on this project are from the United States and therefore may be more familiar searching systems based on literature published predominantly in the United States. Second, the search was limited to patient portals that were tethered. Other countries may have studies related to patient education that operate differently enough to not meet the inclusion criteria for this review (ie, Australia’s patient-owned record, which is not tethered to an organization). Lastly, it is also possible that most of the articles were from the United States because the regulatory and reimbursem*nt-based mechanisms noted above create an environment in which some healthcare systems provide patient portals that do not support the addition of features beyond access to the patient’s medical record.
Implications for future research
Now that patient portals are more common, future researchers studying the effect of education through patient portals should seek to use more uniform methods of measurement so that results can be easily compared between studies and systems. We are already witnessing the beginning of this trend, with some studies using validated measures of effectiveness rather than just participant self-report.44,65 What’s more, after beta testing educational resources in patient portals with small sample populations, studies should move to observe rates of utilization in the general population so that we can observe their impact in everyday contexts, rather than in controlled settings.23
In response to the issue of low utilization, future research should study the effect of interventions (ie, pushing education or patient navigators) on patient portal patient education utilization. Providers and organizations can raise awareness and utilization by programing patient portals to push, or connect, relevant education to patients. Relying on push notifications may help providers avoid the existing problem that is happening where a library of educational resources lies dormant while patients are seeking unreliable sources to answer many of their health care questions because they do not know where else to look.74,75 Personalized or patient-relevant education can be pushed to patients based on diagnosis,28,29,35,38 lifespan stage,30,76 prevention for at-risk populations,32,37 or any other personal health information.31,42 This can be done by providers, sometimes known as patient navigators,36 but can also be automated to reduce costs.39–41 It may be beneficial for future research to focus more on specific strategies that help increase utilization rates.
A word of caution. One of the problems often discussed about patient portals are barriers to adoption.24,77 Disparities have been found in patient use of patient portals and educational resources by age, race, and socioeconomic status.24 While patient portals present a low risk of harm to patients if misused, researchers should consider the risk of widening health disparities by providing yet another tool to privileged populations,78 and thus widening what has been dubbed “the digital divide”24 between privileged and underprivileged populations. Simple interventions such as providing visual summaries of personal health information, such as charts or picture descriptions,79 have been found to help neutralize disparities in health literacy.77 Future researchers need to be cognizant of issues of privilege and oppression that may be accentuated by interventions with patient education through patient portals and other rising technology in healthcare.
CONCLUSION
This review found that efforts are being made to raise awareness of educational resources in patient portals, that patients are increasingly utilizing these resources, that patients are finding them useful, and that they are improving health outcomes. It seems that patient portals are becoming a powerful tool for patient engagement and show promise as a means of achieving the quadruple aim of healthcare.2 Continued efforts need to be made to raise awareness of patient educational resources in patient portals so that patients utilize these resources to increase their knowledge, skills, and confidence for managing their own health and health care.3
AUTHOR CONTRIBUTIONS
All authors contributed to the study conception and design. Search terms and databases were selected by AMJ, GJCH, and KBS, searches were run and results uploaded to Covidence by GJCH. The screening of abstracts and full text review was carried out by AMJ and ETJ. AMJ extracted and organized the data, which was then confirmed by ETJ. AMJ and ASB drafted the manuscript. JH, KD, and JP served as committee members on AMJ’s dissertation committee, and reviewed and helped edit the manuscript as part of that role.
SUPPLEMENTARY MATERIAL
Supplementary material is available at JAMIA Open online.
CONFLICT OF INTEREST STATEMENT
None declared.
DATA AVAILABILITY
The data underlying this article are available in the Dryad Digital Repository.
REFERENCES
1
Institute of Medicine
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, DC
:
National Academy Press
;
2001
.
2
Bodenheimer T Sinsky C.
From triple to quadruple aim: care of the patient requires care of the provider
.
Ann Fam Med
2014
;
12
(
6
):
573
–
6
.
3
Carman KL Dardess P Maurer M
Patient and family engagement: a framework for understanding the elements and developing interventions and policies
.
Health Aff
2013
;
32
(
2
):
223
–
31
.
4
Gilliam DS.
Diabetic Patient Portal Use and Effects on HbA1c: A Doctor of Nursing Practice Project
.
2017
:
1
. https://search.ebscohost.com/login.aspx?direct=true&AuthType=ip,shib&db=ccm&AN=124653977&site=ehost-live&scope=site. Accessed October 17, 2022.
OpenURL Placeholder Text
5
Fricton JR Davies D.
2008
). Advances in patient safety personal health records to improve health information exchange and patient safety. In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds.
Advances in Patient Safety: New Directions and Alternative Approaches
. Vol.
4
. AHRQ Publication No. 08-0034-4.
Rockville, MD
: Agency for Healthcare Research and Quality; 2008.
OpenURL Placeholder Text
6
Kildea J Battista J Cabral B
Design and development of a person-centered patient portal using participatory stakeholder co-design
.
J Med Internet Res
2019
;
21
(
2
):
e11371
.
7
Aguirre PEA Coelho MM Rios D Machado MAAM Cruvinel AFP Cruvinel T.
Evaluating the dental caries-related information on Brazilian websites: qualitative study
.
J Med Internet Res
2017
;
19
(
12
):
e415
.
8
Delić D Polasek O Kern J.
Internet health-information seekers in Croatia—who, what, and why?
Med Inform Internet Med
2006
;
31
(
4
):
267
–
73
.
9
Kraschnewski JL Chuang CH Poole ES
Paging “Dr. Google”: does technology fill the gap created by the prenatal care visit structure? Qualitative focus group study with pregnant women
.
J Med Internet Res
2014
;
16
(
6
):
e147
.
10
Alhuwail D Abdulsalam Y.
Assessing electronic health literacy in the state of Kuwait: survey of internet users from an Arab state
.
J Med Internet Res
2019
;
21
(
5
):
e11174
.
11
Fiksdal AS Kumbamu A Jadhav AS
Evaluating the process of online health information searching: a qualitative approach to exploring consumer perspectives
.
J Med Internet Res
2014
;
16
(
10
):
e224
.
OpenURL Placeholder Text
12
Ferreira G
Traeger
AC
,Machado
G
,O’Keeffe
M
,Maher
CG.
Credibility, accuracy, and comprehensiveness of internet-based information about low back pain: a systematic review
.
J Med Internet Res
2019
;
21
(
5
):
e13357
.
13
Smith DA.
Situating Wikipedia as a health information resource in various contexts: a scoping review
.
PLoS One
2020
;
15
(
2
):
e0228786
.
14
Chen X Hay JL Waters EA
Health literacy and use and trust in health information
.
J Health Commun
2018
;
23
(
8
):
724
–
34
.
15
Hoyt RE Hersh WR Bernstam EV.
2018
).
Health Informatics: Practical Guide
.
Morrrisville
:
Lulu.com
.
16
Ammenwerth E Schnell-Inderst P ho*rbst A.
Patient empowerment by electronic health records: first results of a systematic review on the benefit of patient portals
.
Stud Health Technol Inform
2011
;
165
:
63
–
7
.
OpenURL Placeholder Text
17
Ammenwerth E ho*rbst A Lannig S Mueller G Siebert U Schnell-Inderst P.
Effects of adult patient portals on patient empowerment and health-related outcomes: a systematic review
.
Stud Health Technol Inform
2019
;
264
:
1106
–
10
.
OpenURL Placeholder Text
18
Neves AL Freise L Laranjo L Carter AW Darzi A Mayer E.
Impact of providing patients access to electronic health records on quality and safety of care: a systematic review and meta-analysis
.
BMJ Qual Saf
2020
;
29
(
12
):
1019
–
32
.
19
Zhao JY Song B Anand E
2018
:
1913
–
1922
.
20
Chaet AV Morshedi B Wells KJ Barnes LE Valdez R.
Spanish-language consumer health information technology interventions: a systematic review
.
J Med Internet Res
2016
;
18
(
8
):
e214
.
21
Grossman LV Masterson Creber RM Benda NC Wright D Vawdrey DK Ancker JS.
Interventions to increase patient portal use in vulnerable populations: a systematic review
.
J Am Med Inform Assoc
2019
;
26
(
8–9
):
855
–
70
.
OpenURL Placeholder Text
22
Beal LL Kolman JM Jones SL Khleif A Menser T.
Quantifying patient portal use: systematic review of utilization metrics
.
J Med Internet Res
2021
;
23
(
2
):
e23493
.
23
Ancker JS Mauer E Hauser D Calman N.
2016
:
277
–
284
.
24
Graetz I Gordon N Fung V Hamity C Reed ME.
The digital divide and patient portals: internet access explained differences in patient portal use for secure messaging by age, race, and income
.
Med Care
2016
;
54
(
8
):
772
–
9
.
25
Popay J Roberts H Sowden A
Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: A Product from the ESRC Methods Programme
.
2006
. https://www.researchgate.net/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme. Accessed June 2, 2020.
OpenURL Placeholder Text
26
McHugh ML.
Interrater reliability: the kappa statistic
.
Biochem Med
2012
;
22
(
3
):
276
–
82
.
OpenURL Placeholder Text
27
Jones RB McGhee SM McGhee D.
Patient on-line access to medical records in general practice
.
Health Bull (Edinb)
1992
;
50
(
2
):
143
–
50
.
OpenURL Placeholder Text
28
Abidi SS Goh A.
A personalised healthcare information delivery system: pushing customised healthcare information over the WWW
.
Stud Health Technol Inform
2000
;
77
:
663
–
667
.
OpenURL Placeholder Text
29
Ahmed R Drouin M Martin E Mirro M Ghahari RR Toscos T.
Perceptions on tailored educational messaging to patients with atrial fibrillation: towards increased patient engagement and medication adherence
.
Heart Lung
2020
;
49
(
2
):
216
.
30
Arcia A.
Time to push: use of gestational age in the electronic health record to support delivery of relevant prenatal education content
.
EGEMS (Wash, DC)
2017
;
5
(
2
):
5
.
OpenURL Placeholder Text
31
Borbolla D Del Fiol G Taliercio V
Integrating personalized health information from MedlinePlus in a patient portal
.
Stud Health Technol Inform
2014
;
205
:
348
–
52
.
OpenURL Placeholder Text
32
Cameron KA Brown T Lee JY
Delivering video patient education through the patient portal ahead of a scheduledclinic visit
.
J Gen Intern Med
2016
;
31
(
2
):
S174
.
OpenURL Placeholder Text
33
Chen Y Chemelil G Ersin Ö Mirro M.
An exploratory study to examine the impact of electronic personal health records on medication adherence and patient engagement among nonvalvular atrial fibrillation patients
.
J Am Pharm Assoc
2015
;
55
(
2
):
e249
.
OpenURL Placeholder Text
34
Hulse NC Long J Tao C.
2013
:
638
–
647
.
35
Ma C Warren J Staneck J Phillips P.
2005
:
139
36
Navaneethan SD Jolly SE Schold JD
Pragmatic randomized, controlled trial of patient navigators and enhanced personal health records in CKD
.
Clin J Am Soc Nephrol
2017
;
12
(
9
):
1418
–
27
.
37
Sridhar A Lambrechts S Kwan L Munnangi M Abutouk M Saigal C.
Impact on decisional quality with implementation of a shared decision aid for contraceptive choice
.
Obstet Gynecol
2017
;
129
(
1
):
62S
.
38
Wells S Rozenblum R Park A Dunn M Bates DW.
Personal health records for patients with chronic disease: a major opportunity
.
Appl Clin Inform
2014
;
5
(
2
):
416
–
29
.
OpenURL Placeholder Text
39
Warrington L Absolom K Velikova G.
Integrated care pathways for cancer survivors—a role for patient-reported outcome measures and health informatics
.
Acta Oncol
2015
;
54
(
5
):
600
–
8
.
40
Warrington L Absolom K Holmes M
It’s almost like a chemo buddy”. Cancer patient’s engagement with an online system to report symptoms and side-effects of treatment online (eRAPID)
.
Psycho-Oncology
2017
;
26
:
20
.
OpenURL Placeholder Text
41
Warrington L Absolom K Holmes M
You know it’s not just a waste of time, somebody’s looking at it. Patient engagement with online symptom reports during chemotherapy (eRAPID)—the importance of clinician use of data
.
Psycho-Oncology
2019
;
28
:
7
.
OpenURL Placeholder Text
42
Zhang D Jiang Y Zhou L
2016
:
159
–
166
. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85010952089&partnerID=40&md5=3a2ccee66d653903e937ae08816c9f2d. Accessed June 2, 2020.
43
Steiner M Wildhaber B Spahni S Geissbühler A McLin V.
Kidsetransplant: an innovative serious game for kids with liver disease and following liver transplantation
.
Pediatr Transplant
2017
;
21
:
103
.
OpenURL Placeholder Text
44
Cho AH Batch B Blalock DV Spratt S Dolor R.
Advancing real-world gains in efficacy through technology and skill building (targets). A pilot evaluation of the Time2Focus mobile application for diabetes education
.
J Gen Intern Med
2019
;
34
(
2
):
S125
.
45
Groen WG Kuijpers W Oldenburg HSA Wouters MWJM Aaronson NK Van Harten WH.
Feasibility and preliminary efficacy of an interactive portal for lung cancer patients
.
J Clin Oncol
2016
;
34 (3_Suppl)
:
201
.
OpenURL Placeholder Text
46
Kelly MM Hoonakker PL Dean SM.
Using an inpatient portal to engage families in pediatric hospital care
.
J Am Med Inform Assoc
2017
;
24
(
1
):
153
–
61
.
47
Roelofsen Y Hendriks SH Sieverink F
Differences between patients with type 2 diabetes mellitus interested and uninterested in the use of a patient platform (e-VitaDM-2/ZODIAC-41)
.
J Diabetes Sci Technol
2014
;
8
(
2
):
230
–
7
.
48
Woollen J Prey J Wilcox L
Patient experiences using an inpatient personal health record
.
Appl Clin Inform
2016
;
7
(
2
):
446
–
60
.
OpenURL Placeholder Text
49
Baek H Suh JW Kang SH
Enhancing user experience through user study: design of an mHealth tool for self-management and care engagement of cardiovascular disease patients
.
JMIR Cardio
2018
;
2
(
1
):
e3
.
50
Day FC Pourhomayoun M Keeves D
Feasibility study of an EHR-integrated mobile shared decision making application
.
Int J Med Inform
2019
;
124
:
24
–
30
.
51
Fiks AG Mayne S Karavite DJ DeBartolo E Grundmeier RW.
A shared e-decision support portal for pediatric asthma
.
J Ambul Care Manage
2014
;
37
(
2
):
120
–
6
.
52
Haggerty K Yadamsuren B Kibaru F Basic J.
Children’s information needs of a patient engagement system in a hospital room: insights from usability testing in a simulation laboratory
.
Proc Am Soc Info Sci Tech
2013
;
50
(
1
):
1
–
5
.
53
Hess R Bryce CL McTigue K
The diabetes patient portal: patient perspectives on structure and delivery
.
Diabetes Spectrum
2006
;
19
(
2
):
106
–
10
.
54
Martinez W Threatt AL Rosenbloom ST Wallston KA Hickson GB Elasy TA.
A patient-facing diabetes dashboard embedded in a patient web portal: design sprint and usability testing
.
JMIR Hum Factors
2018
;
5
(
3
):
e26
.
55
Smallwood AJ Schapira MM Fedders M Neuner JM.
A pilot randomized controlled trial of a decision aid with tailored fracture risk tool delivered via a patient portal
.
Osteoporos Int
2017
;
28
(
2
):
567
–
76
.
56
Zide M
57
Wiljer D Bogomilsky S Catton P Murray C Stewart J Minden M.
Getting results for hematology patients through access to the electronic health record
.
Can Oncol Nurs J
2006
;
16
(
3
):
154
–
8
.
58
Hefner JL Sieck CJ Walker DM Huerta TR McAlearney AS.
System-wide inpatient portal implementation: survey of health care team perceptions
.
JMIR Med Inform
2017
;
5
(
3
):
e31
.
59
Benson S St John A Carpenter K
Lab tests online Australia – a key partner in increasing pathology health literacy
.
Pathology
2019
;
51
:
S107
–
8
.
60
Cunningham SG Brillante M Allardice B
My diabetes my way: providing online support for diabetes self-care in Scotland
.
Diabetes Prim Care
2014
;
16
(
3
):
123
–
8
.
OpenURL Placeholder Text
61
Ector GI Westerweel PE Hermens RP
The development of a web-based, patient-centered intervention for patients with chronic myeloid leukemia (CMyLife): design thinking development approach
.
J Med Internet Res
2020
;
22
(
5
):
e15895
.
62
Huerta T Fareed N Hefner JL
Patient engagement as measured by inpatient portal use: methodology for log file analysis
.
J Med Internet Res
2019
;
21
(
3
):
e10957
.
63
Jordan SR Brungardt A Phimphasone-Brady P Lum HD.
Patient perspectives on advance care planning via a patient portal
.
Am J Hosp Palliat Care
2019
;
36
(
8
):
682
–
7
.
64
Lafreniere A Purgina B Wasserman JK.
Putting the patient at the centre of pathology: an innovative approach to patient education—MyPathologyReport.ca
.
J Clin Pathol
2020
;
73
(
8
):
454
–
5
.
65
Lum HD Brungardt A Jordan SR
Design and implementation of patient portal-based advance care planning tools
.
J Pain Symptom Manage
2019
;
57
(
1
):
112
–
117.e2
.
66
Benhamou PY.
Improving diabetes management with electronic health records and patients’ health records
.
Diabetes Metab
2011
;
37(Suppl 4)
:
S536
.
OpenURL Placeholder Text
67
Yeh YT Liu CT Wu SJ Lee TI.
2008
:
85
–
89
. https://doi.org/10.1109/HEALTH.2008.4600116. Accessed June 2, 2020.
68
Wiljer D Leonard KJ Urowitz S
The anxious wait: assessing the impact of patient accessible EHRs for breast cancer patients
.
BMC Med Inform Decis Mak
2010
;
10
:
46
.
69
Collins SA Rozenblum R Leung WY
Acute care patient portals: a qualitative study of stakeholder perspectives on current practices
.
J Am Med Inform Assoc
2017
;
24
(
e1
):
e9
–
17
.
70
Goldberg HI Ralston JD Hirsch IB Hoath JI Ahmed KI.
Using an Internet comanagement module to improve the quality of chronic disease care
.
Jt Comm J Qual Saf
2003
;
29
(
9
):
443
–
51
.
OpenURL Placeholder Text
71
McAlearney AS Walker DM Gaughan A Moffatt-Bruce S Huerta TR.
Helping patients be better patients: a qualitative study of perceptions about inpatient portal use
.
Telemed J E-Health
2020
; 26 (9): 1184–7. https://doi.org/10.1089/tmj.2019.019810.1089/tmj.2019.0198.
OpenURL Placeholder Text
72
Ross SE Moore LA Earnest MA Wittevrongel L Lin CT.
Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial
.
J Med Internet Res
2004
;
6
(
2
):
e12
.
73
Carr D Springer KW.
Advances in families and health research in the 21st century
.
J Marriage Fam
2010
;
72
(
3
):
743
–
61
.
74
Fomous C Miller N.
The role of National Library of Medicine web sites in newborn screening education
.
Ment Retard Dev Disabil Res Rev
2006
;
12
(
4
):
305
–
12
.
75
Fountain JS Slaughter RJ.
TOXINZ, the New Zealand Internet poisons information database: the first decade
.
Emerg Med Australas
2016
;
28
(
3
):
335
–
40
.
76
Timmer SMH.
Autism 24/7, self management 24/7
.
Int J Integr Care
2011; 11 (6): 1.
OpenURL Placeholder Text
77
Hemsley B Rollo M Georgiou A Balandin S Hill S.
The health literacy demands of electronic personal health records (e-PHRs): an integrative review to inform future inclusive research
.
Patient Educ Couns
2018
;
101
(
1
):
2
–
15
.
78
Veinot TC Mitchell H Ancker JS.
Good intentions are not enough: how informatics interventions can worsen inequality
.
J Am Med Inform Assoc
2018
;
25
(
8
):
1080
–
8
.
79
Morrow D Azevedo RFL Garcia-Retamero R
Contextualizing numeric clinical test results for gist comprehension: implications for EHR patient portals
.
J Exp Psychol Appl
2019
;
25
(
1
):
41
–
61
.
© The Author(s) 2023. Published by Oxford University Press on behalf of the American Medical Informatics Association.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Issue Section:
Advertisem*nt
Citations
Views
8,156
Altmetric
More metrics information
Metrics
Total Views 8,156
6,828 Pageviews
1,328 PDF Downloads
Since 1/1/2023
| Month: | Total Views: |
|---|---|
| January 2023 | 250 |
| February 2023 | 350 |
| March 2023 | 442 |
| April 2023 | 353 |
| May 2023 | 278 |
| June 2023 | 336 |
| July 2023 | 485 |
| August 2023 | 386 |
| September 2023 | 477 |
| October 2023 | 757 |
| November 2023 | 797 |
| December 2023 | 435 |
| January 2024 | 625 |
| February 2024 | 682 |
| March 2024 | 1,022 |
| April 2024 | 481 |
Altmetrics
Email alerts
Article activity alert
Advance article alerts
New issue alert
In progress issue alert
Receive exclusive offers and updates from Oxford Academic
Citing articles via
-
Most Read
-
Latest
Advertisem*nt
