Abstract
YouTube® is one of the leading platforms for health information. However, the lack of regulation of content and quality raises concerns about accuracy and reliability. CoMICs (Concise Medical Information Cines) are evidence-based short videos created by medical students and junior doctors and reviewed by experts to ensure clinical accuracy. We performed a systematic review to understand the impact of videos on knowledge and awareness about diabetes and PCOS. We then evaluated the quality of YouTube® videos about diabetes and PCOS using various validated quality assessment tools and compared these with CoMICs videos on the same topics. Quality assessment tools like DISCERN, JAMA benchmark criteria, and global quality scale (GQS) score were employed. Some of the authors of this study also co-authored the creation of some of the CoMICs evaluated. Our study revealed that while videos effectively improve understanding of diabetes and PCOS, there are notable differences in quality and reliability of the videos on YouTube®. For diabetes, CoMICs videos had higher DISCERN scores (CoMICs vs YouTube®: 2.4 vs 1.6), superior reliability (P < 0.01), and treatment quality (P < 0.01) and met JAMA criteria for authorship (100% vs 30.6%) and currency (100% vs 53.1%). For PCOS, CoMICs had higher DISCERN scores (2.9 vs 1.9), reliability (P < 0.01), and treatment quality (P < 0.01); met JAMA criteria for authorship (100% vs 34.0%) and currency (100% vs 54.0%); and had higher GQS scores (4.0 vs 3.0). In conclusion, CoMICs outperformed other similar sources on YouTube® in providing reliable evidence-based medical information which may be used for patient education.
Introduction
In the era of digital information dissemination, YouTube® has emerged as a prominent platform for individuals seeking health-related information. Its vast repository of user-generated content offers an accessible and convenient source for knowledge on various health topics. However, the unrestricted nature of this platform poses significant challenges, with concerns about the accuracy, quality, and reliability of health information presented in YouTube® videos (1). Among digital media platforms, YouTube®, the world's leading video-sharing website, has experienced explosive growth in the volume of health-related content. Notably, the accuracy of videos cannot be identified on YouTube® as most videos are not peer-reviewed (3, 3). This results in viewers being frequently exposed to unverified, poor quality, or misleading information (4). In addition, the search results on YouTube® are ranked by popularity, relevancy, and view history as opposed to content quality. This can be disadvantageous as most patients and healthcare professionals are not experienced in filtering this information.
To ensure the dissemination of peer-reviewed, evidence-based medicine on YouTube®, Concise Medical Information Cines (CoMICs) was started in August 2020 (5). One of the underlying principles of CoMICs is that illustrations help to remember information more than just text on its own (6). Thus, the videos mainly consist of visual aids with small amounts of text, so the complex information covered can be more quickly consumed and understood (7). All CoMICs are created by medical students or junior doctors based on national/international guidelines and are critically reviewed by experts in the field. This ensures the information in these videos is reliable, accurate, and up-to-date.
As the prevalence of diabetes and polycystic ovary syndrome (PCOS) continues to rise worldwide, ensuring the availability of accurate and dependable health information is crucial to empowering individuals to manage these conditions effectively. We performed a systematic review to understand the impact of videos on knowledge and awareness about diabetes and PCOS. We then evaluated the quality of YouTube® videos about diabetes and PCOS using various validated quality assessment tools and compared these with CoMICs videos on the same topics.
Methods
Systematic review
Authors SSB and FR conducted a comprehensive literature search to systematically review studies on Embase, Medline, and PubMed from 2005 to October 25, 2022, assessing the quality of YouTube® videos related to diabetes and PCOS. The start date of the search was 2005, as this was when YouTube® was launched. Original articles reporting novel data were included. Reviews, case reports, and other article types that did not present any original data were excluded. The search strategy is delineated in Supplementary Table 1 (see section on supplementary materials given at the end of this article), and the PRISMA framework is explained in Fig. 1. The risk of bias was evaluated using the Cochrane Risk of Bias tool with modifications to allow assessment of observational studies, as consistent with the current literature (8). Selection bias was defined as the inclusivity and adequacy of search terms and the relevance of inclusion and exclusion criteria used in selecting videos or developing the video intervention. Performance bias and detection bias were not assessed as there was a general lack of blinding due to the nature of the studies. Attrition bias was assessed by examining any videos, meeting the study’s predetermined search parameters and the inclusion and exclusion criteria but were not included in the study’s evaluation as well as any exclusions from evaluation or analysis. Reporting bias was assessed by examining the quality or effectiveness of the assessment tools used and the outcome measures reported by the study authors.
PRISMA chart for diabetes mellitus and polycystic ovary syndrome.
Citation: Endocrine Connections 13, 7; 10.1530/EC-24-0059
Video data collection
On October 26, 2022, authors VV and PVI performed a preliminary search on YouTube® to identify videos about diabetes and PCOS. Search terms encompassed ‘diabetic’, ‘blood sugar’, ‘diabetics’, ‘diabeties’, ‘diabetes educator’, ‘diabetes mellitus’, ‘diabetes health’, and ‘diabetes management’ for diabetes, and ‘polycystic ovary’, ‘sindrome de ovario poliquistico’, ‘pco-syndrom’, ‘hyperandrogenic anovulation’, ‘syndrome des ovaires polykystiques’, ‘polycystic ovarian syndrome’, ‘pos’, ‘stein-leventhal syndrome’, ‘síndrome dos ovários policísticos’, ‘pcos’, ‘多嚢胞性卵巣症候’, ‘sindrome delle ovaie policistiche’, ‘polycystic ovary disorder’, and ‘多囊卵巢综合征’ for PCOS. These search terms were retrieved from Cronycle. The authors have previously used the same search strategy to identify social media influencers (9). The search was conducted in incognito mode on Google Chrome in the UK to mitigate potential search result biases induced by browser cookies. The YouTube® relevance filter was employed for content relevance, and videos lacking audio, exceeding a duration of >10 min, or not in English were excluded from subsequent analysis to be in line with other similar studies (10). Duplicate videos were also eliminated from consideration.
Following a preliminary search, the 50 most relevant videos for diabetes and PCOS in English, <10 min video length and not part of CoMICs series from the YouTube® search results were selected for further scrutiny. YouTube determines the ranking of search results based on several key factors, including relevance, where it strives to present the most pertinent and valuable videos for a given search query; keyword optimisation, where alignment between the title, description, and tags of a video and the search query enhances its relevance and ranking; video quality, favoring high-quality, informative, and engaging content; user engagement, with videos receiving more views, likes, comments, and subscribers typically ranking higher; watch time, prioritising videos that maintain viewer engagement for longer durations, suggesting relevance and appeal; and freshness, where recently uploaded videos are given preference, reflecting YouTube’s emphasis on current and timely content. These factors collectively shape YouTube’s search ranking algorithm, which continues to evolve and may integrate additional variables over time (11). Among these, one video from the diabetes group was excluded due to an inoperative link. Additionally, 15 diabetes and 8 PCOS CoMICs videos that adhered to the inclusion criteria described for YouTube videos were included in the study. We also searched You and Your Hormones (Y&Y), a web-based project by the Society for Endocrinology to give patients and the public access to reliable online information on endocrine science. We found one relevant video for diabetes and PCOS, which is included in our comparison (12).
The length (the duration of the video), views (the number of times that a video was viewed), and likes (the number of times that users indicated that they liked a video by giving it a positive rating) of all the videos in their respective categories were taken into consideration. The likes-to-dislikes ratio was not collected as this function was removed from YouTube®. The metrics provide insights into the patterns of which video lengths attract the greatest number of views and/or likes. It also showed the ratio of views to likes (how many of those who viewed the video liked it), demonstrating how CoMICs videos compare to their counterparts (13).
Quality assessment tools
Three quality assessment tools, DISCERN, JAMA, and GQS (14, 15, 16) (Supplementary Tables 2, 3, and 4), were used to appraise the quality of included videos. Two independent authors completed each assessment tool. Each coder watched the whole video and inputted their results separately. This prevented the coders from conferring and blinding this aspect of data collection. All six authors were initially trained on how to score the videos coherently and were tested on a sample of ten videos for accuracy and inter-rater agreement.
DISCERN is a tool for patients and information providers to judge the quality of information on treatment choices and thus will also lead to production of high-quality, evidence-based consumer health information. It checks reliability, treatment information, and overall quality of the written information. This tool was conducted by MK and AJB, consisted of 16 questions, each scoring from 1 to 5 points. Questions were divided into three parts: reliability (questions 1–8), quality information about treatment options (questions 9–15), and overall score (question 16). The reliability scores for each of these videos on questions 1–8 were added up and the mean was calculated. The quality treatment option scores for question 9–15 were added up and a mean of the total scores for the videos was calculated. The total DISCERN score was calculated by summing up scores over questions 1–15. All videos were divided into five categories based on their total DISCERN score: very poor (<27), poor (27–38), fair (39–50), good (51–62), and excellent (63–75) (14).
The JAMA (Journal of the American Medical Association) system is a quality assessment tool that evaluates websites on the basis of four main aspects: authorship, attribution, disclosure, and currency. Authorship refers to authors, contributors, their affiliations, and credentials being clearly provided, in addition to a provision of substantial contribution to the area of study, drafting and critically reviewing the findings to ensure data of clinical relevance, reviewal of the final version with a mutual agreement from all the authors, and ensuring the accuracy and integrity of the work published with willingness to address any questions or concerns related to the content of the work. Authorship is the designation given to significant contributors of the published work. Attribution checks if references and sources are noted clearly along with all relevant copyright information. Disclosure refers to full disclosure of ownership of websites, along with sponsorships and other potential conflicts of interest. Currency verifies if the dates of upload and update of content are present. The JAMA tool was conducted by authors KK and MFS, and it is a quality scale used to evaluate health-related information from Internet websites. It consists of four criteria: Authorship, Attribution, Disclosure, and Currency. Each item is assessed and scored zero if it doesn’t meet the desired criteria or one point if the desired criteria are met, with a minimum score of zero and a maximum of four. There are three questions for Authorship, two questions for Attribution, three questions for Disclosure, and for a video to meet the criteria for these domains, they had to fulfil all the questions in each respective section. For the videos to achieve currency, they had to be published within the last 3 years (15).
The GQS (global quality scale) is used to evaluate the accuracy and dependability of shared information. The GQS tool was conducted by authors PVI and VV, consisted of a 5-point scale ranging from one to five (1 and 2 being low quality, 4 and 5 being high quality, and 3 being medium quality) (16).
Statistical analysis
The statistical analysis for all the quality assessment tools was conducted using ANOVA for parametric and Kruskal-Wallis testing for non-parametric analysis to test for significant differences between the video scores. This was followed by a Dunn Test to discern which exact groups differed significantly. All statistical tests were conducted using PRISM9 (version 9.5.1), and P < 0.05 was considered statistically significant.
For all the quality assessment tools, SAB was the third reviewer who assessed for any discrepancies in the scores of the two reviewers for each tool. This helped to reduce the likelihood of bias.
Results
Systematic review
A total of 18 studies meeting the specified criteria were incorporated into our review, with 17 focused on diabetes and 1 on PCOS (Table 1) (17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32). There was a low risk of bias in the included studies, as shown in Supplementary Table 5. The studies on diabetes found that most videos are effective in aiding glycaemic control and improving patient knowledge (18, 19, 20, 21, 23, 24, 25, 26, 29, 30). However, not all videos in these studies included evidence-based information (4, 22, 23, 29, 30). The study on PCOS videos found that an educational video programme for people with PCOS resulted in an increase in knowledge about the condition and increased awareness of self-management techniques (31).
Results of the systematic review.
| Reference | Objectives and methods | Main finding |
|---|---|---|
| Dyson et al. (20) | Objective: Develop a video-based lifestyle education program for newly diagnosed type 2 diabetes. Method: Randomised controlled trial with 42 participants. | Intervention group showed increased knowledge, improved A1c, cholesterol levels, and physical activity. |
| Heinrich E et al. (32) | Objective: Evaluate the quality and effectiveness of a video-based diabetes self-management program. Method: Study 1: Randomised controlled trial with 135 participants. Study 2: Pre- and post-test with 564 participants. | Study 1: Knowledge scores in the experimental group increased. Study 2: Participants were satisfied with video content and user-friendliness. |
| Liu et al. (28) | Objective: Determine factors associated with older adults' understanding of diabetes. Method: Cross-sectional study. | The videos were rated helpful and easy-to-follow. Older adults rely on prior knowledge for understanding. |
| Abedin et al. (22) | Objective: Evaluate YouTube videos on diabetes foot care. Method: Cross-sectional study of 89 videos using an 11-point checklist. | Only a small percentage (11.2%) of videos were very useful. Many videos were not useful. |
| Davis et al. (27) | Objective: Investigate a video's effect on patient self-medication efficacy. Method: Pre- and post-test with 51 participants. | Increased self-efficacy scores and medication adherence in the 3-month follow-up. |
| Zhang et al. (18) | Objective: Develop and test an electronic nutrition resource for pre- and type 2 diabetes. Method: Pre- and post-test with 156 participants. | Increased accuracy in identifying blood glucose-raising foods across all groups. |
| Leong et al. (19) | Objective: Assess YouTube videos on Type 2 diabetes for content and quality, focusing on South Asians. Method: Cross-sectional study of 71 videos using GQS and DISCERN. | 32.4% of videos were misleading; videos specific to South Asians often promoted alternative medicines. |
| McElfish et al. (26) | Objective: Develop a culturally appropriate blood glucose monitoring video for Marshallese patients. Method: Pre- and post-test with 50 participants. | Post-intervention knowledge scores and HbA1c levels significantly improved. |
| Paragas et al. (21) | Objective: Study effects of message-framed informational videos on diabetes management knowledge and self-efficacy. Method: Pre- and post-test with 165 participants. | Knowledge and self-efficacy were significantly higher in experimental groups compared to control. |
| Arna Abrar et al. (25) | Objective: Develop a diabetic foot care video in two Indonesian languages. Method: Cross-sectional study with 40 patients. | Videos increased knowledge about foot care and detecting risk factors for diabetic foot ulcers. |
| Gimenez-Perez et al. (17) | Objective: Evaluate YouTube videos for self-management in type 2 diabetes. Method: Cross-sectional study of 393 videos using AADE7 framework. | 40.2% of videos contained useful information; 25.7% were misleading. Alternative medicine videos were less useful and more misleading. |
| Kong et al. (29) | Objective: Evaluate the quality of information presented in diabetes-related videos on TikTok. Method: Cross-sectional study of 199 videos using DISCERN. | TikTok videos were about diabetes management. Non-profit accounts had higher video quality scores. |
| Gour et al. (31) | Objective: Evaluate a video-based educational program's efficacy for PCOS awareness. Method: Pre- and post-test with 41 participants. | Knowledge regarding PCOS and its self-management increased following the intervention. |
| Hu et al. (24) | Objective: Test a social media–based diabetes self-management education program for low-income Chinese immigrants. Method: Pre/post-test with 30 participants. | High satisfaction and a 0.5% decrease in HbA1c. Participants preferred video-based education. |
| Leong et al. (19) | Objective: Investigate a social media intervention's effect on HbA1c, knowledge, and self-care in diabetes. Method: Randomised controlled trial with 181 participants. | HbA1c change was not significant. Knowledge scores increased significantly in both groups. |
| Mahajan et al. (23) | Objective: Analyse YouTube videos on diabetic macular oedema. Method: Cross-sectional study of 104 videos using DISCERN, JAMA, and GQS. | Healthcare professional-targeted videos had higher quality scores. Overall video quality could be improved. |
Gimenez-Perez et al. (17) found that 40.2% of YouTube®️ videos on type 2 diabetes contained useful information, while 25.7% were misleading, particularly those from ‘alternative medicine‘ professionals. Limitations included the exclusion of videos without voiceover and joint evaluation for usefulness rather than accuracy. They concluded that healthcare professionals should only recommend specific YouTube®️ videos.
Zhang et al. (18) developed an electronic nutrition resource for New Zealand patients with pre/type 2 diabetes, significantly improving food identification for blood glucose control across patients (P = 0.013), health professionals (P = 0.003), and the general public (P < 0.001). Limitations included participant dropout, English-only presentation, and a predominantly young, educated female demographic.
Heinrich et al. (32) and Leong et al. (19) demonstrated that social media and video-based interventions improve diabetes knowledge and self-care activities but found no significant change in HbA1c levels. Leong et al. noted improved knowledge scores (68.3% to 76.7%, P < 0.001) with limitations of a single-centre study and self-reported outcomes. Heinrich et al. found high participant satisfaction with visuals and user-friendliness but did not assess long-term outcomes.
Dyson et al. (20) and McElfish et al. (26) found that video interventions improved diabetes knowledge and biomedical parameters like A1c (−0.7%, P = 0.024) and cholesterol (−0.5 mmol/L, P = 0.017). Dyson et al. concluded video interventions are effective for lifestyle advice for newly diagnosed patients, though limited by small sample size and varying treatments. McElfish et al. found significant improvement in knowledge and HbA1c among Marshallese patients, with limitations of a small sample and no control group.
Davis et al. (27) and Paragas et al. (21) reported that message-framed and general video interventions increased knowledge, self-efficacy, and medication adherence in diabetes patients. Paragas et al. was limited by non-randomised sampling. Davis et al. noted increased self-reported medication adherence and fewer concerns, limited by self-reported measures and high loss to follow-up.
Abedin et al. (22), Leong et al. (30), and Mahajan et al. (23) evaluated YouTube®️ videos on diabetic foot care and diabetic macular oedema, finding overall low quality and usefulness. Abedin et al. found only 11.2% of videos on diabetic foot care very useful. Mahajan et al. reported low-quality scores but better content in healthcare professional-targeted videos. Leong et al. observed variability in video quality on type 2 diabetes, with 32.4% of videos misleading. They concluded YouTube®️ is not currently a reliable source for diabetes education.
Hu et al. (24) tested social media-based diabetes education for low-income Chinese immigrants, finding high satisfaction (9.9/10) and decreased HbA1c after 6 months (0.5%, P = 0.003). Limitations included small sample size and self-reported outcomes.
Liu et al. (28) and Abrar et al. (25) developed educational videos on diabetic foot care and general diabetes management, respectively. Abrar et al. significantly increased patient knowledge in two Indonesian languages, limited by small sample size. Liu et al. found older adults rated a 3D informational diabetes video as helpful but slightly too fast.
Kong et al. (29) evaluated diabetes-related videos on TikTok, finding non-profit organisation videos had higher DISCERN scores. They concluded TikTok videos are not a reliable tool for diabetes education, limited by the focus on Chinese videos.
Gour et al. (31) found a video-based educational program increased awareness of PCOS, with limitations of a small single-centre sample without controls. They concluded video-based education effectively increases knowledge of PCOS.
In summary, video and social media interventions generally improve knowledge and self-care activities in diabetes and related conditions, though their impact on clinical outcomes like HbA1c is less consistent. Limitations across studies include small sample sizes, language restrictions, and reliance on self-reported data.
Video metrics
Overall, 124 videos were analysed, with 99 from YouTube®, 23 from CoMICs, and 2 from Y&Y. This included 65 diabetes videos and 59 PCOS videos (Table 2). While YouTube videos had the longest video length, views and likes, CoMICs videos had the best views-to-likes ratio for both diabetes and PCOS videos.
Video quality assessment.
| Video type | n | Length(s) | Views | Likes | Views/likes ratio | |
|---|---|---|---|---|---|---|
| Overall | All | 124 | 297 (240–491) | 25,480 (78–295,917) | 250 (10–3100) | 101.92 |
| YouTube (non-CoMICs) | 99 | 344 (240–519) | 81,444 (6364–360,641) | 1000 (60–4600) | 81.44 | |
| CoMICs | 23 | 286 (260–297) | 182 (111–332) | 5 (3–9) | 36.54 | |
| Y&Y | 2 | 119 (118–120) | 2013.5 (1756–2271) | 31 (26–35) | 64.95 | |
| Diabetes | YouTube (non-CoMICs) | 49 | 293 (207–493) | 215,914 (43,181–602,733) | 2200 (355–10,000) | 98.14 |
| CoMICs | 15 | 280 (243–297) | 219 (89–359) | 4 (2–6) | 54.75 | |
| Y&Y | 1 | 118 | 1756 | 26 | 67.54 | |
| PCOS | YouTube (non-CoMICs) | 50 | 388 (276–550) | 12,333 (2336–177,363) | 228 (19–2200) | 54.09 |
| CoMICs | 8 | 289 (278–297) | 179 (145–259) | 7 (6–10) | 25.57 | |
| Y&Y | 1 | 120 | 2271 | 35 | 64.88 | |
CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
DISCERN results.
| Video type | Very poor | Poor | Fair | Good | Excellent | Reliability | Quality of treatment options | Overall | |
|---|---|---|---|---|---|---|---|---|---|
| Proportion (%) | Mean (± standard deviation) | ||||||||
| Overall | All | 19.8 ± 3.0 | 10.3 ± 2.7 | 1.7 ± 0.4 | |||||
| YouTube (non-CoMICs) | 26 | 63 | 11 | 0 | 0 | 19.2 ± 3.4 | 11.5 ± 3.5 | 1.8 ± 0.5 | |
| CoMICs | 0 | 39.1 | 60.9 | 0 | 0 | 24.3 ± 2.8 | 14.8 ± 3.9 | 2.6 ± 0.6 | |
| Y & Y | 0 | 100 | 0 | 0 | 0 | 19.5 ± 0.7 | 9+/−0 | 1.5 ± 0 | |
| Diabetes | All | 20.1 ± 4.1 | 10.8 ± 3.6 | 1.8 ± 0.6 | |||||
| YouTube (non-CoMICs) | 28 | 66 | 6 | 0 | 0 | 19.0 ± 3.9 | 10.0 ± 3.2 | 1.6+/−0.4 | |
| CoMICs | 0 | 53.3 | 46.7 | 0 | 0 | 23.8 ± 2.7 | 13.5 ± 3.9 | 2.4 ± 0.6 | |
| Y & Y | 0 | 100 | 0 | 0 | 0 | 20.0 | 9.0 | 1.5 | |
| PCOS | All | 20.0 ± 4.3 | 13.5 ± 3.8 | 2.0 ± 0.6 | |||||
| YouTube (non-CoMICs) | 24 | 60 | 16 | 0 | 0 | 19.1 ± 3.9 | 13.0 ± 3.6 | 1.9 ± 0.5 | |
| CoMICs | 0 | 12.5 | 87.5 | 0 | 0 | 25.4 ± 2.6 | 17.3 ± 2.3 | 2.9 ± 0.64 | |
| Y & Y | 0 | 100 | 0 | 0 | 0 | 19.0 | 9.0 | 1.5 | |
CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Journal of the American Medical Association (JAMA) analysis.
| Video type | n | JAMA | ||||
|---|---|---|---|---|---|---|
| Authorship (%) | Attribution (%) | Disclosure (%) | Currency (%) | |||
| Overall | All | 124 | 45.2 | 0.0 | 0.8 | 62.9 |
| YouTube (non-CoMICs) | 99 | 32.3 | 0.0 | 1.0 | 53.5 | |
| CoMICs | 23 | 100.0 | 0.0 | 0.0 | 100.0 | |
| Y&Y | 2 | 50.0 | 0.0 | 0.0 | 100.0 | |
| Diabetes | All | 65 | 47.7 | 0.0 | 1.5 | 64.6 |
| YouTube (non-CoMICs) | 49 | 30.6 | 0.0 | 2.0 | 53.1 | |
| CoMICs | 15 | 100.0 | 0.0 | 0.0 | 100.0 | |
| Y&Y | 1 | 100.0 | 0.0 | 0.0 | 100.0 | |
| PCOS | All | 59 | 42.4 | 0.0 | 0.0 | 61.0 |
| YouTube (non-CoMICs) | 50 | 34.0 | 0.0 | 0.0 | 54.0 | |
| CoMICs | 8 | 100.0 | 0.0 | 0.0 | 100.0 | |
| Y&Y | 1 | 0.0 | 0.0 | 0.0 | 100.0 | |
CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Global Quality Scale (GQS) criteria results.
| Video type | n | GQS | |
|---|---|---|---|
| Median (IQR) | |||
| Diabetes | All | 65 | |
| YouTube (non-CoMICs) | 49 | 2 (2–3) | |
| CoMICs | 15 | 2 (1–3) | |
| Y&Y | 1 | 2.5 (2.25–2.75) | |
| PCOS | All | 59 | |
| YouTube (non-CoMICs) | 50 | 3 (2–4) | |
| CoMICs | 8 | 4 (4–4.25) | |
| Y&Y | 1 | 2.5 (2.25–2.75) | |
| Overall | All | 124 | 3 (2–4) |
| YouTube (non-CoMICs) | 99 | 3 (2–4) | |
| CoMICs | 23 | 3 (2–4) | |
| Y&Y | 2 | 3.5 (2.25–2.75) | |
CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Diabetes videos
The overall score for CoMICs video using DISCERN was highest compared to other videos (mean score; YouTube vs CoMICs vs Y&Y: 1.6 vs 2.4 vs 1.5) and significantly higher than YouTube® (CoMICs vs YouTube: P < 0.0001) (Fig. 2A, Table 3). Similarly, CoMICs videos were more reliable (19.0 vs 23.8 vs 20.0) and had better quality of treatment options (10.0 vs 13.8 vs 9.0) than other videos and significantly better than YouTube® (CoMICs vs YouTube: reliability P < 0.0001; quality of treatment options P = 0.0072) (Fig. 2B and C).
DISCERN analysis. CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Citation: Endocrine Connections 13, 7; 10.1530/EC-24-0059
About 30.6% of YouTube videos met the JAMA criteria for Authorship, 0% met the criteria for Attribution, 2% met the criteria for Disclosure, and 53.1% met the criteria for Currency. About 100% of CoMICs videos met the criteria for Authorship, 0% met the criteria for Attribution, 0% met the criteria for Disclosure and 100% met the criteria for Currency. The Y&Y video met the criteria for Authorship and Currency but not Attribution and Disclosure (Fig. 3A and B, Table 4).
Journal of the American Medical Association (JAMA) analysis. CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Citation: Endocrine Connections 13, 7; 10.1530/EC-24-0059
Y&Y videos scored highest on GQS (2.0 vs 2.0 vs 2.5). There was no significant difference in GQS scored for YouTube® and CoMICs videos (P = 0.1342) (Fig. 4A, Table 5).
Global Quality Scale (GQS) analysis. CoMICs, Concise Medical Info Cines; PCOS, Polycystic Ovary Syndrome; Y&Y, You and Your Hormones.
Citation: Endocrine Connections 13, 7; 10.1530/EC-24-0059
PCOS videos
The overall score for CoMICs video using DISCERN was highest compared to other videos (mean score; YouTube vs CoMICs vs Y&Y: 1.9 vs 2.9 vs 1.5) and significantly higher than YouTube® (CoMICs vs YouTube: P = 0.0026) (Fig. 2D, Table 3). Similarly, CoMICs videos were more reliable (19.1 vs 25.4 vs 19.0) and had better quality of treatment options (13.0 vs 17.3 vs 9.0) than other videos and significantly better than YouTube® (CoMICs vs YouTube: reliability P = 0.006; quality of treatment options P = 0.0104) (Fig. 2E and F).
About 34% of YouTube videos met the JAMA criteria for Authorship, and 54% met the criteria for Currency. About 100% of CoMICs videos met the criteria for Authorship and Currency. None of the videos from YouTube or CoMICs met the Attribution or Disclosure criteria. The Y&Y video met only the currency criteria (Fig. 3C and D, Table 4).
CoMICs videos scored highest on GQS (3.0 vs 4.0 vs 2.5). However, there was no significant difference in GQS scored for YouTube® and CoMICs videos (P = 0.1469) (Fig. 4B, Table 5).
Discussion
Our review indicated that videos effectively improve knowledge and awareness of both diabetes and PCOS. They also helped to improve glycaemic control. Nevertheless, it is noteworthy that not all videos demonstrated adherence to evidence-based principles, emphasising the need to assess the quality of publicly available videos and identify optimal information sources. However, there is a lack of research into the quality of educational videos accessible to the public.
When considering both diabetes and PCOS videos collectively, CoMICs consistently demonstrated superiority in multiple domains. However, videos from all sources in our study lacked disclosure and attribution, which are areas to improve in the future. While CoMICs videos consistently excelled in Authorship and Currency, the broader landscape indicates a pressing need for increased adherence to established criteria, particularly in Attribution and Disclosure, to ensure the reliability of health information accessible to the public.
Interestingly, there was no significant difference seen in the GQS analysis. This may be due to the limitation of the quality assessment tool, which relies on a single question for overall video scoring, making it challenging to differentiate nuances in video quality. This is important to note as viewers, akin to GQS analysis, tend to evaluate videos holistically, making it harder to discern varying levels of quality.
There is an evident demand to regulate the boom of digital medical knowledge to ensure that the quality and authenticity of the information are not compromised. Strategies to mitigate the misinformation, specifically regarding health literacy, should consider the level of knowledge that patients require. Studies have shown that information overload to patients can be detrimental, let alone misinformation (33). Quality assessment tools geared toward medical educational videos need to be developed to ensure a minimum uniform level of understandability, reliability, and actionability. Such tools can facilitate videos with an identifier indicating their accuracy and reliability. An area for further research would be to have patients also review the videos to better understand their perspective.
Strengths and limitations
Using three distinct quality assessment tools independently scored by two reviewers adds to the strength of our study findings. Although DISCERN and JAMA were initially created to assess the quality of publications and websites, respectively (14, 34), studies have used these tools with a similar methodology in video quality assessment (34, 35, 36, 37, 38, 39). However, there is a need for developing specific quality assessment tools for videos as there are inherent limitations to using tools like DISCERN and JAMA (40).
The YouTube® algorithm is designed to show videos that are personalised for each viewer to enhance their experience on the platform (41). Therefore, our sample data identified through incognito mode may not represent videos viewed by patients and healthcare professionals in real life. Although we attempted to mitigate this by using the ‘relevance’ function on YouTube®, the order of results will vary depending on the IP history of the computer. Hence, the results of the videos we sampled may differ compared to set preferences and computer history (42).
While medical students who created CoMICs may not have fully attained their medical degrees, the involvement of senior medical professionals in reviewing their content ensured that the CoMICs meet a certain standard of medical accuracy and reliability. This may have affected the comparison of CoMICs to other similar videos on YouTube®, especially if they are created by people from a non-medical background. Moreover, rebuttal of CoMICs as a comparator is important. The commonality of the name CoMICs, although make it easy to remember and share, reduces its visibility on YouTube. Moreover, the production quality of CoMICs is not up to the same standard as its counterparts on YouTube, including lack of narration, misjudgement of impact of background music and organisation of information on the screen for some of the videos. Furthermore, some of the CoMICs were reviewed by some of the authors in the study and are detailed in the Conflict of Interest Statement.
Conclusion
While videos effectively enhance knowledge and awareness of diabetes and PCOS, there is a significant difference in the quality, content, and reliability of videos available on public domains. CoMICs videos were consistently superior in multiple domains of three distinct quality assessment tools, suggesting they are better resources than other YouTube® videos. Future work on establishing reliable, in-built, and real-time quality assessment tools may facilitate videos with an identifier indicating their accuracy and reliability and reduce misinformation.
Supplementary materials
This is linked to the online version of the paper at https://doi.org/10.1530/EC-24-0059.
Declaration of interest
Some of the authors of the study also co-authored the creation of some of the CoMICs. PK supervised creation of the CoMICs lead by SAB. VV co-authored ‘CoMICsLite: Insulin centenary Ep.1 - history of diabetes until the discovery of insulin’ and ‘CoMICs Lite Polycystic Ovary Syndrome (PCOS) Ep.2 : Fertility.’ PVI co-authored ‘CoMICs Episode 73: Hospital management of hypoglycaemia in adults with diabetes’ and ‘CoMICs Episode 79: Management of adults with diabetes undergoing elective surgery,’ and KK co-authored ‘CoMICs Episode 78: Inpatient Care of the Older Frail Adult with Diabetes.’
Funding
This study received no grant from any funding agency in the public, commercial, or not-for-profit sectors.
Ethics statement
No human participants were involved. Only publicly accessible data were analysed. No patient data were collected.
Availability of data and materials
Data will be made available upon request to the corresponding author.
Author contribution statement
SAB and KM are the joint-first authors, contributing to all study aspects. SAB, KK, MK, AJB, PVI, VV, MFS SSB, and FR analysed and interpreted the data analysis. PK conceptualised and supervised the delivery of all study aspects and critically reviewed the manuscript. EM, VS, and DZ contributed substantially to delivering the CoMICs videos, developing the study and writing the manuscript. SIMBA and CoMICs team contributed towards all stages of this manuscript. All authors contributed substantially to drafting and approving the final draft of the manuscript. The final version has been reviewed and approved by all the authors.
Acknowledgements
We thank all the healthcare professionals who participated in this study. We thank all the CoMICs reviewers and endorsing organisations for their contribution and continued support. The SIMBA and CoMICs team members are Carina Pan, Pavithra Sakthivel, Anisah Ali, Isabel Allison, Tamzin Ogiliev, Haaziq Sheikh, Sung Yat Ng, Zahra Olateju, Maiar Elhariry, Meri Davitadze, Eka Melson, Sangamithra Ravi, Abby Radcliffe, Rachel Nirmal, Aditya Swaminathan, Shams Ali Baig, Dwi Delson, Soon Chee Yap, and Vardhan Venkatesh.
References
- 1↑
Suarez-Lledo V, & Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. Journal of Medical Internet Research 2021 23 e17187. (https://doi.org/10.2196/17187)
- 2
Khamsi R. Is YouTube just what the doctor ordered? New Scientist 2 Jan 2007. London, UK. (available at: https://www.newscientist.com/article/dn10859-is-youtube-just-what-the-doctor-ordered/)
- 3↑
Steinberg PL, Wason S, Stern JM, Deters L, Kowal B, & Seigne J. YouTube as source of prostate cancer information. Urology 2010 75 619–622. (https://doi.org/10.1016/J.UROLOGY.2008.07.059)
- 4↑
Nour MM, Nour MH, Tsatalou OM, & Barrera A. Schizophrenia on YouTube. Psychiatric Services 2017 68 70–74. (https://doi.org/10.1176/APPI.PS.201500541)
- 5↑
YouTube. SIMBA simulation. San Bruno, CA, USA: Google LLC, 2024. (available at: https://www.youtube.com/@simbasimulation8047)
- 6↑
Krum R. The science of infographics. In Cool Infographics: Effective Communication with Data Visualization and Design, pp1–55. Hoboken, NJ, USA: Wiley, 2013.
- 7↑
Henderson M, Selwyn N, & Aston R. What works and why? Student perceptions of ‘useful’ digital technology in university teaching and learning. Studies in Higher Education 2017 42 1567–1579. (https://doi.org/10.1080/03075079.2015.1007946)
- 8↑
Helming AG, Adler DS, Keltner C, Igelman AD, & Woodworth GE. The content quality of YouTube videos for professional medical education: a systematic review. Academic Medicine 2021 96 1484–1493. (https://doi.org/10.1097/ACM.0000000000004121)
- 9↑
Elhariry M, Malhotra K, Solomon M, Goyal K, & Kempegowda P. Top 100 #PCOS influencers: understanding who, why and how online content for PCOS is influenced. Frontiers in Endocrinology 2022 13 3213. (https://doi.org/10.3389/FENDO.2022.1084047/BIBTEX)
- 10↑
Tang K, Azhar U, Babar M, Ahmed A, Oh A, Day W, Harb H, & Chan FJ. Assessing the quality of YouTube videos on adhesive capsulitis. Cureus 2022 14 e27406. (https://doi.org/10.7759/CUREUS.27406)
- 11↑
Ahmed R. How does YouTube determine the ranking of search results? Medium. San Francisco, CA, USA: A Medium Corp., 2023. (available at: https://medium.com/@raheelytranker/how-does-youtube-determine-the-ranking-of-search-results-3d43dfb7b1e9)
- 12↑
You and You r Hormones. Bristol, UK: Society for Endocrinology, 2024. (available at: https://www.yourhormones.info/)
- 13↑
Oztermeli A, & Karahan N. Evaluation of YouTube video content about developmental dysplasia of the hip. Cureus 2020 12 e9557. (https://doi.org/10.7759/CUREUS.9557)
- 14↑
Charnock D, Shepperd S, Needham G, & Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. Journal of Epidemiology and Community Health 1999 53 105–111. (https://doi.org/10.1136/JECH.53.2.105)
- 15↑
Silberg WM, Lundberg GD, & Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor–let the reader and viewer beware. JAMA 1997 277 1244–1245. (https://doi.org/10.1001/jama.1997.03540390074039)
- 16↑
Li M, Yan S, Yang D, Li B, & Cui W. YouTubeTM as a source of information on food poisoning. BMC Public Health 2019 19 952. (https://doi.org/10.1186/S12889-019-7297-9)
- 17↑
Gimenez-Perez G, Robert-Vila N, Tomé-Guerreiro M, Castells I, & Mauricio D. Are YouTube videos useful for patient self-education in type 2 diabetes? Health Informatics Journal 2020 26 45–55. (https://doi.org/10.1177/1460458218813632)
- 18↑
Zhang Z, Monro J, & Venn BJ. Development and evaluation of an Internet-based diabetes nutrition education resource. Nutrients 2019 11. (https://doi.org/10.3390/NU11061217)
- 19↑
Leong CM, Lee TI, Chien YM, Kuo LN, Kuo YF, & Chen HY. Social media-delivered patient education to enhance self-management and attitudes of patients with type 2 diabetes during the COVID-19 Pandemic: randomized controlled trial. Journal of Medical Internet Research 2022 24 e31449. (https://doi.org/10.2196/31449)
- 20↑
Dyson PA, Beatty S, & Matthews DR. An assessment of lifestyle video education for people newly diagnosed with type 2 diabetes. Journal of Human Nutrition and Dietetics 2010 23 353–359. (https://doi.org/10.1111/j.1365-277X.2010.01077.x)
- 21↑
Paragas ED, & Barcelo TI. Effects of message-framed informational videos on diabetes management knowledge and self-efficacy. International Journal of Nursing Practice 2019 25 e12737. (https://doi.org/10.1111/IJN.12737)
- 22↑
Abedin T, Ahmed S, Al Mamun M, Ahmed SW, Newaz S, Rumana N, & Turin TC. YouTube as a source of useful information on diabetes foot care. Diabetes Research and Clinical Practice 2015 110 e1–e4. (https://doi.org/10.1016/j.diabres.2015.08.003)
- 23↑
Mahajan J, Zhu A, Aftab OM, Henry RK, Agi NYB, & Bhagat N. Educational quality and content of YouTube videos on diabetic macular edema. International Ophthalmology 2023 43 1093–1102. (https://doi.org/10.1007/S10792-022-02504-1)
- 24↑
Hu L, Islam N, Trinh-Shevrin C, Wu B, Feldman N, Tamura K, Jiang N, Lim S, Wang C, Bubu OM, et al.A social media-based diabetes intervention for low-income mandarin-speaking Chinese immigrants in the United States: feasibility study. JMIR Formative Research 2022 6 e37737. (https://doi.org/10.2196/37737)
- 25↑
Abrar EA, Yusuf S, Sjattar EL, & Rachmawaty R. Development and evaluation educational videos of diabetic foot care in traditional languages to enhance knowledge of patients diagnosed with diabetes and risk for diabetic foot ulcers. Primary Care Diabetes 2020 14 104–110. (https://doi.org/10.1016/J.PCD.2019.06.005)
- 26↑
McElfish PA, Rowland B, Riklon S, Aitaoto N, Sinclair KA, Ima S, Kadlubar SA, Goulden PA, Hudson JS, Mamis S, et al. Development and evaluation of a blood glucose monitoring YouTube video for Marshallese patients using a community-based participatory research approach. Policy, Politics and Nursing Practice 2019 20 205–215. (https://doi.org/10.1177/1527154419872834)
- 27↑
Davis SA, Carpenter D, Cummings DM, Lee C, Blalock SJ, Scott JE, Rodebaugh L, Ferreri SP, & Sleath B. Patient adoption of an internet based diabetes medication tool to improve adherence: a pilot study. Patient Education and Counseling 2017 100 174–178. (https://doi.org/10.1016/J.PEC.2016.07.024)
- 28↑
Liu CJ, & William A. What factors are related to understanding a stereoscopic 3D diabetes educational video in seniors? Journal of Visual Communication in Medicine 2014 37 65–73. (https://doi.org/10.3109/17453054.2014.981799)
- 29↑
Kong W, Song S, Zhao YC, Zhu Q, & Sha L. TikTok as a health information source: assessment of the quality of information in diabetes-related videos. Journal of Medical Internet Research 2021 23 e30409. (https://doi.org/10.2196/30409)
- 30↑
Leong AY, Sanghera R, Jhajj J, Desai N, Jammu BS, & Makowsky MJ. Is YouTube useful as a source of health information for adults with type 2 diabetes? A South Asian perspective. Canadian Journal of Diabetes 2018 42 395–403.e4. (https://doi.org/10.1016/J.JCJD.2017.10.056)
- 31↑
Gour A, Dubey P, Goel A, & Halder A. Remote assessment and reinforcement of patient awareness of role of lifestyle modification and treatment adherence in polycystic ovary syndrome using an online video based educational module. Journal of the Turkish German Gynecological Association 2022 23 1–7. (https://doi.org/10.4274/JTGGA.GALENOS.2021.2021-9-29)
- 32↑
Heinrich E, Nooijer de J, Schaper NC, Schoonus-Spit MHG, Janssen MAJ, & Vries de NK. Evaluation of the web-based Diabetes Interactive Education Programme (DIEP) for patients with type 2 diabetes. Patient Education and Counseling 2012 86 172–178. (https://doi.org/10.1016/J.PEC.2011.04.032)
- 33↑
Mayeaux EJ, Murphy PW, Arnold C, Davis TC, Jackson RH, & Sentell T. Improving patient education for patients with low literacy skills. American Family Physician 1996 53 205–211.
- 34↑
Azer SA. Are DISCERN and JAMA suitable instruments for assessing YouTube videos on thyroid cancer? Methodological concerns. Journal of Cancer Education 2020 35 1267–1277. (https://doi.org/10.1007/S13187-020-01763-9)
- 35↑
Garg N, Venkatraman A, Pandey A, & Kumar N. YouTube as a source of information on dialysis: a content analysis. Nephrology 2015 20 315–320. (https://doi.org/10.1111/NEP.12397)
- 36↑
Gorrepati PL, & Smith GP. A cross-sectional study analyzing the quality of YouTube videos as a source of patient education for treatments on psoriasis. Journal of Psoriasis and Psoriatic Arthritis 2021 6 8–1 1. (https://doi.org/10.1177/2475530320964782)
- 37↑
Szmuda T, Rosvall P, Hetzger TV, Ali S, & Słoniewski P. YouTube as a source of patient information for hydrocephalus: a content-quality and optimization analysis. World Neurosurgery 2020 138 e469–e477. (https://doi.org/10.1016/J.WNEU.2020.02.149)
- 38↑
Cassidy JT, Fitzgerald E, Cassidy ES, Cleary M, Byrne DP, Devitt BM, & Baker JF. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surgery, Sports Traumatology, Arthroscopy 2018 26 840–845. (https://doi.org/10.1007/S00167-017-4514-X)
- 39↑
Hohenleitner J, Barron K, Bostonian T, Demyan L, & Bonne S. Educational quality of YouTube videos for patients undergoing elective procedures. Journal of Surgical Research 2023 292 206–213. (https://doi.org/10.1016/J.JSS.2023.07.009)
- 40↑
Drozd B, Couvillon E, & Suarez A. Medical YouTube videos and methods of evaluation: literature review. JMIR Medical Education 2018 4 e3. (https://doi.org/10.2196/MEDEDU.8527)
- 41↑
Yesilada M, & Lewandowsky S. Systematic review: YouTube recommendations and problematic content. Internet Policy Review 2022 11 1652. (https://doi.org/10.14763/2022.1.1652)
- 42↑
Cetin A. Evaluation of YouTube video content related to the management of hypoglycemia. Cureus 2021 13 e12525. (https://doi.org/10.7759/CUREUS.12525)
