Saeed Sadiq Hamid,1 Tabassum Zehra,2 Muhammad Tariq,3 Azam Saeed Afzal,4 Hashir Majid,5 Erfan Hussain6
Abstract
Implementing work place based assessment: The modified direct observation of procedural skills (DOPS) across medical specialties — An experience from a developing country
Objective: To assess the skill level of residents regarding central venous catheterisation insertion, and to assess the reliability of scores in a simulated situation.
Methods: The quasi-experimental study with pre- and post-test design was conducted from February to June 2013 at the Aga Khan University, Karachi, and comprised four workshops attended by residents. The workshops were video-recorded for feedback and self-assessment. At the end of the workshops, knowledge and procedural skills were assessed using a self-generated 38-item, task-specific instrument after ensuring its content validity. Data were analyzed using SPSS 19.
Results: There were 40 residents in the sample. The self-generated instrument was reliable with Cronbach’s alpha value of 0.83 and inter-rater coefficient of 0.79. There was a significant improvement in the skills level post-intervention compared to the baseline mean values (p=0.001). The subjects were satisfied with the workshops, as indicated by a mean score of 8.83±1.367.
Conclusions: The workshops appeared to improve the central venous catheterization insertion skills of the residents.
Keywords: Direct observation of procedural skills, Workplace-based assessment, Postgraduate medical education, Resident, Central venous catheterization. (JPMA 72: 620; 2022)
DOI: https://doi.org/10.47391/JPMA.970
Introduction
Workplace-based assessment (WPA) has been used to increase the clinical confidence of medical residents by increasing the level of realism of the assessment and providing a mechanism for formative feedback.1
Direct observation of procedural skills (DOPS) of residents by the faculty remains a vital component of assessment across specialties. Assessment through observation provides ongoing data on resident performance with actual patients.1 Similarly, maintenance of professional competence is a lifelong process, and is motivated by a number of factors, including curiosity, self-identified gaps
in knowledge, and the desire to provide the very best care to patients.2
DOPS focusses on observing and evaluating the procedural skills of postgraduate residents in a workplace setting. Central venous catheter (CVC) insertion is one of the commonly performed bedside procedures3 by Internal Medicine (IM), Nephrology, Critical Care, Pulmonology,
Cardiology, Anaesthesia and Emergency Medicine residents who are part of the Aga Khan University (AKU) postgraduate medical education residency programmes.Literature reports a 15% rate of complications associated with the procedure that include arterial puncture, blood steam infection and pneumothorax.3 However, attention towards the insertion technique and appropriate training
with increased procedural experience have shown to reduce errors.3
The importance of appropriate CVC insertion training is recognised by professional societies, accreditation bodies and hospitals with residency training programmes across the globe with competence requirements and standards in place. The Accreditation Council for Graduate Medical
Education (ACGME) has also stipulated that IM residents must show basic proficiency in CVC insertion.4 The American Board of Internal Medicine stipulates that residents should demonstrate competence and safe performance of CVC insertion by means of assessments performed during residency training.5
