Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation

Abstract

The use of checklists is recommended for the assessment of competency in central venous catheterization (CVC) insertion. To explore the use of a global rating scale in the assessment of CVC skills, this study seeks to compare its use with two checklists, within the context of a formative examination using simulation. Video-recorded performances of CVC insertion by 34 first-year medical residents were reviewed by two independent, trained evaluators. Each evaluator used three assessment tools: a ten-item checklist, a 21-item checklist, and a nine-item global rating scale. Exploratory principal component analysis of the global rating scale revealed two factors, accounting for 84.1% of the variance: technical ability and safety. The two checklist scores correlated positively with the weighted factor score on technical ability (0.49 [95% CI 0.17–0.71] for the 10-item checklist; 0.43 [95% CI 0.10–0.67] for the 21-item checklist) and negatively with the weighted factor score on safety (−0.17 [95% CI −0.48–0.18] for the 10-item checklist; −0.13 [95% CI −0.45–0.22] for the 21-item checklist). A checklist score of <80% was strong indication of incompetence. However, a high checklist score did not preclude incompetence. Ratings using the global rating scale identified an additional 11 candidates (32%) who were deemed incompetent despite scoring >80% on both checklists. All these candidates committed serious errors. In conclusion, the practice of universal adoption of checklists as the preferred method of assessment of procedural skills should be questioned. The inclusion of global rating scales should be considered.

Appendices

Appendix 1: Global rating scale

 

Appendix 2: Ten-item checklist

Date: _____________________

Evaluator: _________________

Resident: __________________

Procedural Checklist

 

STEP	YES	NO
Justified site selection	NA	NA
Justified catheter selection	NA	NA
Prepared site appropriately
Requested trendelenberg position	NA	NA
Identified proper landmarks
Inserted needle at correct angle
Aspirated while inserting needle
Inserted guidewire appropriately
Withdrew needle and incised skin
Inserted catheter and withdrew wire
Aspirated blood and flushed ports
Occluded ports
Secured catheter
Sealed site in sterile fashion	NA	NA

Appendix 3: Twenty-one item checklist

 

STEP	YES	NO
Flush the ports on the catheter with sterile saline
Clamp each port (ok to keep brown port open)
Remove brown port from end of catheter to accommodate wire
Area is cleaned with chlorhexadine
Resident gets in sterile gown, gloves, hat and mask
Area is draped in usual sterile fashion (must be full body drape)
The vein is localized using anatomical landmarks
The skin is anesthetized with 1% lidocaine in a small wheal
The deeper structures are anesthetized
Using the large needle or catheter-syringe complex, cannulate the vein while aspirating
Remove the syringe from the needle or advance the catheter into the vein removing both the syringe and the needle
Advance the guidewire into the vein no more than approximately 12–15 cm
Knick the skin with the scalpel to advance the dilator
Advance the dilator over the guidewire and dilate the vein
Advance the triple lumen over the guidewire
Never let go of the guidewire
Once the catheter is inserted remove the guidewire in its entirety
Advance the catheter to approx 14–16 cm on the right side
Ensure there is blood flow/flush each port
Secure the catheter in place (suture or staple)
Maintain sterile technique