|Year : 2020 | Volume
| Issue : 2 | Page : 200-204
Workplace based assessment: A review of available tools and their relevance
Jyoti Prakash, K Chatterjee, K Srivastava, VS Chauhan, R Sharma
Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
|Date of Submission||30-Nov-2020|
|Date of Acceptance||28-Dec-2020|
|Date of Web Publication||15-Mar-2021|
Prof. Jyoti Prakash
Department of Psychiatry, Armed Forces Medical College, Pune - 441 040, Maharashtra
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Workplace-based assessment (WPBA) appears to be a promising tool for more comprehensive assessment of the learners. Relevant literature in this direction was collated and analyzed for its relevance, salience, and merit. Many WPBA tools are being used by various educational institutions which spans across multiple domains and over the entire duration of the workplace learning. It caters to holistic assessment with structured measures, real-time feedback, and continuous professional development. It is being used more for formative assessment and has limited utility in summative assessment as on date. WPBA tools have promising prospect in bringing novelty, objectivity, and holistic approach in assessment.
Keywords: Direct observation of procedural skill, mini-clinical evaluation exercise, workplace based assessment
|How to cite this article:|
Prakash J, Chatterjee K, Srivastava K, Chauhan V S, Sharma R. Workplace based assessment: A review of available tools and their relevance. Ind Psychiatry J 2020;29:200-4
Science is an alibi to many researches/endeavors which gave excellent result at inception in ideal test scenario; but failed to deliver or shimmer to expectation in the real-time working environment. Any person or entity is understood holistically in the context of its complex/dynamic interaction with the environment. In fact, often mere observation of a person' s naturally occurring learning in complex interplay with its environment, give the fair assessment of the shape the clay is going to take under guided hands of the potter and the shimmering heat of the furnace. Workplace based assessment (WPBA) is borne out of such desire to assess the true and holistic competence of learner in its working milieu which might not be assessed by a discipline-specific classroom setting paper-based assessment.
| Scope|| |
It is quite applicable to post graduate residents working at various clinical and paraclinical steam who are shaped under supervision in real-time working environment from a novice to a competent practitioner. On any routine working day a medical or surgical resident would go through many a learning event/opportunity. At an outpatient department, he/she would be conducting targeted physical examination of patients, making treatment plan and rendering it, explaining to individual and family about illness and nuances of treatment and interacting with fellow residents and staff in the team. They would revisit specific components of history and examination to ascertain the progress. If required, they would deliver bad news with desired diligence and empathy. Pending discharge of the patient, the resident would be addressing predischarge administrative and medical formalities and rendering advice with respect to various psychosocial and medical management during follow-up. During all this while he/she would be liaising or interacting with colleagues and faculty of other stream for various multidisciplinary component in management requiring a specific set of communication skills and respect for each other.
| Advantages|| |
There are many advantages of WPBA:,,,,
- The assessment of individual in its working environment. It brings the assessment closer to the true environment and reduces artificiality
- Assessment may be woven around the work without much disruption in work or any additional effort
- Assessment of multitude of skill over variety of cases and settings possible
- Holistic assessment of competencies are possible across preventive to therapeutic to rehabilitative spectrum; education, management to follow-up, patient orientation to family-community orientation, etc.,
- Easy to assess higher order thinking and echelons in Miller's pyramid
- Assessment of learners' addressal of ethical and legal issues in their work
- Assessment is continuous in nature through a course of time and complexity in issues of management
- Gives emphasis on other aspect of learning necessary for holistic management, i.e., record keeping, taking the team along, communicating with patients and peers, understanding larger view of health than addressing mere illness
- Gives opportunity of assessment and feedback my multiple assessors
- Gives opportunity for active involvement of learner in patient selection, decision making. Lerner-centric
- It also gives structure to “naturally occurring learning” in work environment for observation and assessments.
Things which are required to be kept in mind during the implementation of WPBA in an institution:,,,,,
- It should be able to address the desired competencies
- It should be able to yield data or feedback for reflection and appropriate plan for future
- It should be able to help in certification of certain competencies
- It should be able to identify gap in knowledge and skill for continuous development
- The practice environment should be safe and free of any possible litigation
- It should be easy to conduct and acceptable to broad range of participants (patients, learners, assessors)
- Designed tool should be adequately reliable and valid
- It ranges across a wide matrix of competencies.
| Utility Index|| |
As per Van der Vleuten, one should CARVE an effective assessment strategy. The mnemonics stand for following:
Cost-effectiveness - With regards to infrastructure, workforce training, etc.
Acceptability - To all involved in the process of assessment.
Reliability - It should yield consistent result across measure and time.
Validity - Ability of tool to measure what is intended to.
Educational impact - The assessment (formative/summative) should result in the promotion of learning or translation to competent practice.
C × A × R × V × E.
| Types of WPBA Tools|| |
There are various types of WPBA tools:,,,,,,,,,
- Mini clinical evaluation exercise (Mini CEX): A direct observation tool of a cross-sectional encounter; commonly used in clinical setting. Structured pro forma for assessment of skill, attitude, and behavior. Immediate feedback is given to the learner by the assessor,
- Direct observation of procedural skill: A direct observation tool of a cross-sectional encounter; commonly used in the assessment of procedures in clinical and paraclinical setting. Structured pro forma for the assessment of skill. Immediate feedback is given to the learner by the assessor
- Clinical encounter card: Cards to record salient aspects of examination and management of patients seen during the work for later perusal, assessment, and feedback by the faculty,
- Clinical work sampling: Direct observation, assessment and feedback by faculty on the learner's patient management across multiple domains of patient care (i.e., admission, management in the ward, and predischarge)
- Acute care assessment tool: Structured assessment of learner's ability during the period of work at acute ward, night shift. The observation is longitudinal in nature with feedback by the assessor given at the end,
- Consultation observation tool (COT): Similar to Mini CEX. Often used in primary care setting. Structured assessment and immediate feedback by assessor on the performance of a learner in a face-to-face consultation
- Audio-COT: It is COT conducted on leaner performing telephonic consultation
- Clinical examination and procedural skill: Direct observation and assessment of learner's interpretation of physical examination and conduct of procedural skill. Commonly used in the assessment of examination of breast, male/female genitalia, rectum, prostate, etc.
- Case-based discussion: Structured discussion of case management involving data collection, decision-making, administrative and team management, addressing ethic, and holistic approach,
- Multisource feedback (MSF): Sometime referred to as 360° feedback, in this, there is structured assessment of performance of learner by the peer, team member or even patient with regards to his/her professional conduct and relationship. Anonymized feedback is given for the learner to reflect on and make suitable improvement. Various other structured tool addressing this domain are mini peer assessment tool, team assessment of behavior, and patient satisfaction questionnaire,
- Quality improvement project/audit assessment tool: It measures learners' ability to successfully conduct a quality improvement project/audit. Various descriptors for the assessment of this competency could be topic selection, identification of right criteria/measures, methodology used, changes brought into implementation, evaluation of change and future implication, presentation skill and quality of project in general. Feedback is provided based on the given criteria,
- Leadership MSF: It can be used for the assessment of learner's/trainee's organizational skills, owning of responsibility, effective management of team, etc., With leadership MSF anonymized feedback may be taken from the peer and team for reflection and improvement on part of the learner
- Clinical supervisor report: A report given by clinical supervisor about a learner work in the hospital based on short structured pro forma. Four domain clusters which are addressed are relationship, diagnostic, management, and professionalism. Feedback on the observation is given
- Educational supervisor's review: Periodic review by educational supervisor to assess overall progress, identifies area of focused input and feedback; based on performance on various WPBA tools and “naturally occurring evidence”
- Learning logs: It is a comprehensive record of various works done in pursuance of learning competencies, reflection on the work/feedback given, evidence of continuous professional development, and relevant supportive document. It gives a longitudinal overview of quantity and quality of progress,
- Portfolio: It stores all documents generated through workplace-based assessments, record of clinical experiences, reflections, relevant meetings attended, informal of formal training events, critical incidents, etc., Structure and purpose of this tool are to show overall quality and quantity of learning and progress. Portfolio can be paper (diary, etc.,) or electronic (e-portfolio, etc.,) one. Based on the functionality, these may be classified into reflective, developmental, assessment, or showcase portfolio.,
Other WPBA tools:
- Nontechnical skill for surgeons: It is a rating of intraoperative nontechnical skill vital to safety of patients, i.e., situational awareness, decision-making, leadership, communication, and team work
- Anesthetist nontechnical skill: It is assessments of anesthetist's intraoperative nontechnical skill related to patient's safety. It emphasizes the importance of vigilance, clarity of communication, effective coordination, and timely anticipation
- Chart-stimulated recall: The use of charts of learner's patient encounters to explore clinical reasoning and knowledge application
- Objectively structured assessment of technical skills: Evaluator rated performance on a platform consisting of various technical skill sets.
The comparative overview of relevant WPBA tools are as brought out in [Table 1].
|Table 1: Overview of salient features of common workplace-based assessment tools|
Click here to view
| Diversity of Experience Group|| |
The workplace-based exposure and assessment may further be contextualized to various other groups as under:
- Pediatric group
- Geriatric group
- Chronic care group
- Vulnerable and disabled group
- Population health.
| Gradation|| |
Although the grading may be tailor made to the tools chosen, the learner may be graded under following broad heads:
- Much below the desired level of expectation
- Below the desired level of expectation
- Desired level of expectation
- Above the desired level of expectation
- Much above the desired level of expectation.
| Constraints and Challenges|| |
Following constraints and challenges exit in this paradigm:,,,,
- Inability of assessors to spare time for supervision and WPBA from their busy schedule
- Lack of adequate number of trained supervisors
- Inadequate alignment of assessment tools with workplace culture and setting
- Lack of desired standardization of tool for the given cultural or educational context
- Lack of communication of change in the method of assessment to the concerned party. Poor change management
- Conflicting evidence of effectiveness of reliable assessment tool and perceived unnecessary replacement of time-tested old methods of assessment
- Variability in assessment over time, domains, and multiple assessors
- Uneasiness of students and faculty toward acceptance of WPBA.
| Current Status of Workplace-Based Assessment|| |
WPBA is effective in day-to-day assessment of student's learning in naturally working environment. Significant benefit lies in the assessment of multiple domains including those which are not possible by traditional modalities.,,, The workplace-based approach in assessment is being used extensively by many educational institutes across the world including various Royal Colleges. The feedback by trainers, reflection by student and subsequent steps taken for onward learning gives the continuity in learning and has changed the concept form “assessment driven learning” to “learning driven assessment.” Although the WPBA tools have found its place in various formative assessments, the use of the same as summative assessment is still debated, limited to few situations only and yet not accepted by many an educational institution in its entirety. It is felt that its utility as a summative assessment tool will only be possible when these measures of assessments establishes their robustness reliability and validity; and the trainers are adequately competent in assessing students with these tools. Given the growing recognition of these tools and the benefits associated, that day does not seem too far.
| Conclusion|| |
WPBA tools are structured to semi-structured format to assess a range of knowledge and skills applications by a learner in its working environment. It gives emphasis to holistic concept of management, feedback-based reflection, continuous professional development, effective multidisciplinary integration, and wellness centric management of illness. It has shown promises as effective formative assessment tool but needs further time and validation for broader acceptance as summative assessment tool. Regardless of this debate, the WPBA tools has promising prospect in bringing novelty, structure, appreciation of finer nuances of concept while understanding holistic approach to management in a real work environment with true patients and ramification to the family/community at large. Addition of these tools in the current armamentarium of assessment tool is worthwhile.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Wilkinson JR, Crossley JG, Wragg A, Mills P, Cowan G, Wade W. Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ 2008;42:364-73.
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007;29:855-71.
Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7. Med Teach 2006;28:117-28.
Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65:S63-7.
Liu C. An introduction to workplace-based assessments. Gastroenterol Hepatol Bed Bench 2012;5:24-8.
Bodgener S. Royal College of General Practitioner WPBA handbook; 2020.
Swanwick T, Chana N. Workplace assessment for licensing in general practice. Br J Gen Pract 2005;55:461-7.
Swanwick T, Chana N. Workplace-based assessment. Br J Hosp Med (17508460) 2009;70:290-3.
Nesbitt A, Baird F, Canning B, Griffin A, Sturrock A. Student perception of workplace-based assessment. Clin Teach 2013;10:399-404.
van der Vleuten CP, Schuwirth LW. Assessing professional competence: From methods to programmes. Med Educ 2005;39:309-17.
Augustine K, McCoubrie P, Wilkinson JR, McKnight L. Workplace-based assessment in radiology-where to now? Clin Radiol 2010;65:325-32.
Pelgrim EA, Kramer AW, Mokkink HG, van den Elsen L, Grol RP, van der Vleuten CP. In-training assessment using direct observation of single-patient encounters: A literature review. Adv Health Sci Educ Theory Pract 2011;16:131-42.
Brown N, Holsgrove G, Teeluckdharry S. Case-based discussion. Adv Psych Treat 2011;17:85-90.
Burkill GJ. Work-based assessment for trainees – More than just a few new tools? Clin Radiol 2008;63:12-4.
Wall D, Singh D, Whitehouse A, Hassell A, Howes J. Self-assessment by trainees using self-TAB as part of the team assessment of behaviour multisource feedback tool. Med Teach 2012;34:165-7.
Campbell JL, Richards SH, Dickens A, Greco M, Narayanan A, Brearley S. Assessing the professional performance of UK doctors: An evaluation of the utility of the General Medical Council patient and colleague questionnaires. Qual Saf Health Care 2008;17:187-93.
Singh T, Modi JN. Workplace based assessment: A step to promote competency based postgraduate training. Indian Pediatr 2013;50:553-9.
Yule S, Paterson-Brown S. Surgeons' non-technical skills. Surg Clin North Am 2012;92:37-50.
Flin R, Patey R, Glavin R, Maran N. Anaesthetists' non-technical skills. Br J Anaesth 2010;105:38-44.
Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Taomoto J, Sugiyama Y, et al
. Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today 2013;43:271-5.
Recommendations for specialty trainee assessment and review incorporating lessons learnt from the workplace-based assessment pilot. Executive summary. Joint Royal Colleges of Physicians Training Board. August; 2014. http://www. jrcptb.org.uk/assessment
. [Last accessed 2020 May 07].
Young JQ, Rebekah S, Schwartz J, O'Sullivan PS. Faculty and resident engagement with a workplace-based assessment tool: Use of implementation science to explore enablers and barriers. Acad Med 2020;95:1937-44.
Prins SH, Brøndt SG, Malling B. Implementation of workplace-based assessment in general practice. Educ Prim Care 2019;30:133-44.
Franche RL, Cullen K, Clarke J, Irvin E, Sinclair S, Frank J, et al
. Workplace-based return-to-work interventions: A systematic review of the quantitative literature. J Occup Rehabil 2005;15:607-31.
Anderson DD, Long S, Thomas GW, Putnam MD, Bechtold JE, Karam MD. Objective structured assessments of technical skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res 2016;474:874-81.