Interrater Reliability dari Checklist OSCE Keterampilan Mencuci Tangan dan Memakai Sarung Tangan di Program Studi Ilmu Keperawatan UGM

https://doi.org/10.22146/jkkk.44249

Nur Fajriyah Rosyidah(1), Eri Yanuar Akhmad Budi Sunaryo(2), Totok Harjanto(3*)

(1) Program Studi Ilmu Keperawatan, Fakultas Kedokteran, Kesehatan Masyarakat, dan Keperawatan, Universitas Gadjah Mada
(2) Rumah Sakit Akademik Universitas Gadjah Mada
(3) Departemen Keperawatan Dasar Program Studi Ilmu Keperawatan Fakultas Kedokteran, Kesehatan Masyarakat, dan Keperawatan Universitas Gadjah Mada
(*) Corresponding Author

Abstract


Background: Objective Structured Clinical Examination (OSCE) is a method to evaluate students’ nursing skills. OSCE uses checklist as an instrument to test hand washing and gloving skills. While reliability values of both checklists are vital aspect for the instrument, they have not been measured in PSIK FKKMK UGM.
Objective: To identify the interrater reliability in the hand washing and gloving skills of OSCE checklists at PSIK FKKMK UGM.
Method: This research used non-experimental descriptive quantitative research type with cross-sectional design. The respondents were 92 first-year students at PSIK FKKMK UGM participating in the OSCE examination. Assessment on students performance were conducted using hand washing and gloving checklists by two raters. The scores were analyzed using Kappa and percent agreement (PA).
Result: Hand washing checklist had Kappa value of 0,146 (quite poor) and PA 80,40% (acceptable). Gloving checklist had Kappa value of 0,228 (poor) and PA 78,20% (acceptable). The results were divided into two categories, first Kappa not acceptable and PA acceptable or called Kappa paradox consisting of 14 checklist items. Furthermore, there were 11 checklist items in the second category namely Kappa not acceptable and PA not acceptable.
Conclusion: Hand washing and gloving skill checklists which are used by PSIK FKKMK UGM do not possess good interrater reliability in measuring the skills of nursing students. These checklists need revision and statistical test in order to improve education quality at School of Nursing at FKKMK UGM.

ABSTRAK

Latar belakang: Salah satu metode untuk mengevaluasi keterampilan keperawatan mahasiswa adalah menggunakan Objective Structured Clinical Examination (OSCE). Instrumen yang dapat digunakan dalam OSCE adalah checklist. Reliabilitas merupakan aspek penting dalam suatu instrumen. Pengujian reliabilitas dari checklist mencuci tangan dan memakai sarung tangan di PSIK FKKMK UGM belum pernah dilakukan.
Tujuan: Untuk mengetahui interrater reliability pada checklist OSCE keterampilan mencuci tangan dan memakai sarung tangan di PSIK FKKMK UGM.
Metode: Penelitian ini merupakan jenis penelitian deskriptif kuantitatif non-eksperimental dengan rancangan penelitian cross-sectional. Responden pada penelitian adalah 92 mahasiswa PSIK FKKMK UGM tahun pertama yang mengikuti ujian OSCE. Penilaian performa mahasiswa dilakukan menggunakan instrumen checklist mencuci tangan dan memakai sarung tangan oleh dua orang penguji. Hasil penelitian dianalisis menggunakan Kappa dan percent agreement (PA).
Hasil: Checklist mencuci tangan memiliki nilai Kappa 0,146 (cukup buruk) dan PA 80,40% (dapat diterima). Checklist memakai sarung tangan memiliki nilai Kappa 0,228 (buruk) dan PA 78,20% (dapat diterima). Terdapat 14-unit checklist yang masuk dalam kategori Kappa tidak dapat diterima, namun PA dapat diterima (paradoks Kappa). Terdapat 11-unit checklist yang masuk dalam kategori Kappa dan PA tidak dapat diterima.
Kesimpulan: Checklist keterampilan mencuci tangan dan memakai sarung tangan di PSIK FKKMK UGM belum memiliki interrater reliability yang baik dalam mengukur keterampilan mahasiswa. Checklist tersebut memerlukan perbaikan untuk menghasilkan reliabilitas baik demi kualitas pendidikan di Program Studi Ilmu Keperawatan FKKMK UGM yang lebih baik.


Keywords


checklist; cuci tangan; interrater reliability; OSCE; sarung tangan

Full Text:

PDF


References

  1. Nurhesti PO, & Kustanti A. Surgical Asepsis (Teknik Tanpa Singgung, Scrubbing, Gowning, Glovng). In Haryani, K. Aulawi, H. S. Pangastuti, P. O. Nurhesti, Purwanta, & A. Kustanti, Buku Skill Lab Sensasi dan Integritas Jaringan Blok 2.5 (p. 36). Yogyakarta: Fakultas Kedokteran UGM; 2017.
  2. Peyre SE, Peyre CG, Hagen JA, & Sullivan ME. Reliability of A Procedural Checklist as A High-Stakes Measurement of Advanced Technical Skill. The American Journal of Surgery. 2010: 110-114.
  3. Wimmers PF & Schauer GF. Validating OSCE Performance: The Impact of General Intelligence. Health Professions Education; 2017.
  4. Rush S, Ooms A, Marks-Maran D, Firth T. Students Perceptions of Practice Assessment in The Skills Laboratory: An Evaluation Study of OSCAs with Immediate Feedback. Nurse Education in Practice. 2014:6 27-634.
  5. Murti B. Validitas dan Reliabilitas Pengukuran. Matrikulasi Program Studi Doktoral Fakultas Kedokteran UNS. Surakatta; 2011.
  6. Sugiyono. Metode Penelitian Pendidikan: Pendekatan Kuantitatif, Kualitatif dan R&D. Bandung: Alfabeta; 2010.
  7. Landis JR & Koch G. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977: 159-174.
  8. Stemler SE, & Tsai J. Best Practice in Interrater Reliability: Three Common Approaches. Arizona: SAGE Publication; 2008.
  9. Osborne J. Best Practice in Quantitative Methods. California: SAGE Publishing; 2008.
  10. Graham M, Milanowski A, & Miller J. Measuring and Promoting Inter-rater Agreement of Teacher and Principal Performance Ratings. United State: Center for Educator Compensation Reform Westat; 2012.
  11. Cargo M, Stankov I, Thomas J, Saini M, Rogers P, Mayo-Wilson E, & Hannes K. Development, Inter-Reter Reliability and Feasibility of A Checklist to Asses Implementation in Systematic Reviews: the Case of Provider-Based Prevention and Treatment Programs Targeting Children and Youth. BMC Medical Research Methodology. 2015:15-37.
  12. Li, X. Kappa- a critical review. Uppsala: Department of Statistics Uppsala University. 2010.
  13. Bao S, Howard N, Spielholz P, Silverstein B, & Polissar N. Interrater Reliability of Posture Observation. Human Factors. 2009: 292-309.
  14. Clarke S, Rainey D, & Traynor M. Using The Objective Structured Clinical Examination (OSCE) to Assess Orthopaedic Clinical Skill for the Registered Nurse. International Journal of Orthopaedic and Trauma Nursing. 2011: 92-101.
  15. Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hroobjartsson A, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS). International Journal of Nursing Studies. 2011: 661-667.
  16. Kuppens S, Holden G, Barker K, Rosenberg G. A Kappa-related Decision: K,Y, G, or AC1. Social Work Research.2011;35(3):185-190.
  17. Morris R, Macneela P, Scott A., Treacy P, Hyde A, O'Brien J, Drennan J. Ambiguites and Conflicting Result: The Limitation of The Kappa Statistics in Estabilishing The Interrater Reliability of The Irish Nursing Minimum Data Set for Mental Health: A Discussion Paper. 2008:645-647.
  18. Bajpai R. Evaluation of Inter-Rater Agreement and Inter-Rater Reliability for Observational Data: An Overview of Concepts and Methods. Journal of the Indian Academy of Applied Psychology. 2015; 41(3): 20-27.



DOI: https://doi.org/10.22146/jkkk.44249

Article Metrics

Abstract views : 1145 | views : 1019

Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Nur Fajriyah Rosyidah, Eri Yanuar A.B.S, Totok Harjanto



Jurnal Keperawatan Klinis dan Komunitas (Clinical and Community Nursing Journal) 
collaborates with DPW PPNI DIY

Lisensi Creative Commons  

Jurnal Keperawatan Klinis dan Komunitas (Clinical and Community Nursing Journal) is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.