|Year : 2021 | Volume
| Issue : 1 | Page : 153-156
Critical appraisal of journal article by psychiatry PG residents using a new module: Impact analysis
Pookala Shivaram Bhat, Amit Chail, Kalpana Srivastava
Department of Psychiatry, AFMC, Pune, Maharashtra, India
|Date of Submission||09-Jul-2020|
|Date of Acceptance||09-May-2021|
|Date of Web Publication||24-Jun-2021|
Dr. Pookala Shivaram Bhat
Brig Med, HQ 3 Corps, C/O 99 APO
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: A gap exists from evidence-based medicine (EBM) to clinical practice and there is a felt need to bridge this. Critical appraisal of scientific articles during Journal club by postgraduate (PG) residents will help them to understand and apply the scientific evidence into best clinical practice. Hence, a new module of critical appraisal of journal articles was used for Psychiatry Residents in a Medical College and its impact was assessed. Materials and Methods: Nine psychiatry PG residents from three academic batches participated in the study after informed consent and Institutional ethical committee clearance. They were trained in the Colorado Psychiatry EBM Examination Test module. Subsequently, three consecutive journal article presentations of them were assessed under seven Subtests of the module. Results: There was gradual increase in the assessment scores of all PG residents with each journal article presentation. They also expressed satisfaction of the assessment method and felt confident of applying the principles of critical appraisal in their clinical practice in future. Conclusion: This study evaluated the impact of a new module of critical appraisal of journal article by psychiatry PG residents. It was found to be acceptable by residents and improved their competency to apply literature-based EBM into their clinical practice. It is recommended for further multicentric evaluation on a larger sample.
Keywords: Critical appraisal, evidence-based medicine, journal article, postgraduate teaching
|How to cite this article:|
Bhat PS, Chail A, Srivastava K. Critical appraisal of journal article by psychiatry PG residents using a new module: Impact analysis. Ind Psychiatry J 2021;30:153-6
Evidence-based medicine (EBM) is defined as “The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” EBM is a term used to describe a structure for practicing medicine. This structure incorporates three fundamental components: The best available clinical research, a clinician's judgment, and patient's values and beliefs.
There are five critical steps one needs to follow to practice EBM: (a) Construct a question from the clinical environment, (b) search the medical literature, (c) apprise the evidence, (d) integrate the evidence in clinical practice and (e) evaluate its application. Many studies have assessed the integration of EBM in medical curricula and have noted that practising EBM benefits medical students in improving their attitudes and skills.,,,
Medical students find the integration of EBM practice in medical education and clinical practice difficult due to challenges in correct and efficient search and retrieve of information to address the clinical question. The PICO method, which stands for P (population/people/patient/problem), I (interpretation), C (comparison), and O (outcome), has been found to be efficient and easy to use. It facilitates students to correctly transfer a clinical situation into an answerable question that fits smugly into literature search engines leading to efficient searches. This method has been used extensively since the days it was enunciated by Richardson et al. in 1995.
Journal club in medical teaching was started by Sir William Osler in 1887 for undergraduate students at MC Master University with an aim to secure training in the difficult science of debate. It is essentially an educational meeting in which a group of individuals discuss published articles, provide a forum for collective effort to keep up with the current literature and an objective to promote discussion and critique of research.
A gap exists from EBM to clinical practice and there is a felt need to bridge this gap. Critical appraisal of scientific articles during Journal club by postgraduate (PG) junior residents (JR) will help them to understand and apply the scientific evidence into best clinical practice. Presently, residents select journal articles randomly and discuss more the methodology rather than about relevance and clinical application of the content. Available studies from Indian Medical Colleges have shown that the current method of journal article appraisal by PG residents is unsatisfactory., Journal Club is the most common practice to teach the evidence-based practice to resident physician learners. This is a, highly variable experience with visible variable outcomes.
The authors in this study planned to train Psychiatry PG Residents in a Medical College using a newer module in their Journal club critical appraisal and to assess its impact on their competency to apply EBM into their clinical practice.
| Materials and Methods|| |
Rothberg et al. in 2013 developed a model of teaching EBM to psychiatric residents that addressed the essential skills. Their Colorado psychiatry EBM (CP-EBM) Examination tests knowledge of EBM through 14 open-ended questions. In question 1, residents' ability to assess and ask focussed clinical questions is evaluated. Similarly, questions 2–4 are for the ability to identify search strategy and acquire information, questions 5–9 are for appraisal skills, questions 10–11 are for knowledge of diagnostic and prognostic research designs, question 12 is for the ability to apply outcome measures in clinical practice and questions 13–14 are for the ability to assess a patient with an outcome measure over time. The scale was noted to have high interrater correlation (Cronbach's alpha of 0.95), intraclass correlation of 0.93, Internal Reliability (Cronbach's alpha of 0.84), Item difficulty (from 0.09 to 0.79), and Item discrimination (from 0.24 70 0.78). The CP-EBM Examination module was shared by Rothberg et al., with the authors for this study.
Nine Psychiatry PG Residents (three from each of three academic batches; 1st year-JR I, 2nd year-JR II and 3rd year-JR III) participated in the study after informed consent and Institutional ethical committee clearance. Initially, it was decided to assess the ability to frame the clinical question, search strategy and appraisal skills only, and hence, first seven questions of the module were utilized in the study. They were trained in the application of the CP-EBM Examination Test module for journal club presentations. Subsequently, three consecutive journal article presentations of them were assessed by faculty under seven subtests of the module, namely question construction (1), information source (2), study design (3), search strategy (4), relevance to practice (5), study validity (6) and significance of the study (7). Maximum possible score was 156. This was a qualitative longitudinal study. Progressive changes in their score over three consecutive journal club presentations were assessed. A feedback on the usefulness of the module was also obtained from the participant residents.
| Results|| |
In the first journal article presentation after training, the mean score of JR I was 59.66 which increased to 121.00 by the end of the third presentation [Figure 1], while the mean score of JR II increased from 59.00 to 127.00 [Figure 2] and mean score of JR III improved from 68.66 to 137.00 [Figure 3] at the end of the third presentation. However, there was the difference in their scores in the subtests. In JR I maximum increment was seen in subtest relevance to practice followed by subtest study validity. JR II maximum increment was seen in subtest study validity followed by subtest search strategy whereas in JR III maximum increment was seen in subtest question construction followed by the significance of the study.
| Discussion|| |
This study showed the usefulness of a newer assessment module in improving the journal article critical appraisal ability of Psychiatry PG residents. Though there was the improvement in the performance all batches of Residents, there was the difference in their performance in various subtests. Interestingly JR I showed higher improvement in the subtest of Relevance to practice whereas JR III showed maximum increase in the Question construction subtest. These may be due to the changes in appraisal capability over the course of training.
Residents expressed satisfaction of the assessment method and felt confident of applying the principles of critical appraisal in their clinical practice in future. The limitation of this method is the requirement of more time to assess (30 min) each participant as compared to the present method of assessment (10 min) using single page Likert scale pro forma. Furthermore, the participant size was nine PG residents only and hence, suggested that more such studies on the usefulness of this tool, with a cumulative large sample will be required for validation before generalizing for regular use.
The medical doctor is required to engage in patient care, teaching, and research. The learning during PG training enhances the ability to develop the research inclination and deeper understanding of publication. The Department/Unit of Medical Education emphasizes educational research and the implication of training modality as per standards. The added knowledge of the module and training on the defined module improves the analytical skills of PGs.
| Conclusion|| |
This study evaluated the impact of a new module of critical appraisal of journal articles by Psychiatry PG Residents. It was found to be acceptable by Residents and effective in making them more competent to apply literature-based EBM into their future clinical practice. It is recommended for further multicentric evaluation of this tool on a larger sample.
The authors express their gratitude for the support rendered by Dr. Brian Rothberg, Robert E F and Dr. Gretchen G of the University of Colorado Denver School of Medicine for sharing their CP-EBM Examination for this study.
This study was part of the research project of the First author for his Advance Course in Medical Education at G S Seth Medical College Mumbai in 2019.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Lévesque M, Gauthier-Boudreau J, Gagnon P, Bertulies-Esposito B, Hatcher S, Gagnon L. Evaluation of an evidence-based medicine educational intervention in a regional medical campus. Can Med Educ J 2018;9:E51-8.
Bhargava K, Bhargava D. Evidence based health care: A scientific approach to health care. Sultan Qaboos Univ Med J 2007;7:105-7.
Ilic D, Tepper K, Misso M. Teaching evidence based medicine literature searching skills to medical students during the clinical years – A protocol for a randomised controlled trial. Bmc Med Educ 2011;11:49.
Barghouti FF, Yassein NA, Jaber RM, Khader NJ, AL Shokhaibi S, Almohtaseb A, et al
. Short course in evidence-based medicine improves knowledge and skills of undergraduate medical students: A before-and-after study. Teach Learn Med 2013;25:191-4.
West CP, Jaeger TM, Mcdonald FS. Extended evaluation of a longitudinal medical school evidence-based medicine curriculum. J Gen Intern Med 2011;26:611-5.
Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ 2003;326:319-21.
Lai N. Teaching evidence-based medicine: A clinician's perspective. Malays Fam Med 2013:8:7.
Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: A key to evidence-based decisions (EDITORIAL). ACP J Club 1995;123:A12-3.
Bhattacharya S. Journal club and postgraduate medical education. Indian J Plast Surg 2017:50:302-5.
Misra UK, Kalita J, Nair PP. Traditional journal club: A continuing problem. J Assoc Physicians India 2007;55:343-6.
Herur A, Kolagi S, Ramadurg U, Hiremath CS, Hadimani CP, Goudar SS. Refining the journal club presentations of postgraduate residents in seven clinical departments for better evidence-based practice. Ann Med Health Sci Res 2016;6:185-9.
] [Full text]
Ahmadi N, Mckenzie ME, Maclean A, Brown CJ, Mastracci T, Mcleod RS, et al.
Teaching evidence based medicine to surgery residents-is journal club the best format? A systematic review of the literature. J Surg Educ 2012;69:91-100.
Rothberg B, Feinstein RE, Guiton G. Validation of the Colorado psychiatry evidence-based medicine test. J Grad Med Educ 2013;5:412-6.
Thomas K, Basheer A, Kumar DA, Kuruvilla S, Nagaraj N. Improving medical education: Need for educational research. J Curr Res Sci Med 2015;1:12-7. [Full text]
[Figure 1], [Figure 2], [Figure 3]